Mr Robertson was Labour's finance minister from 2017 until the party's election loss last year, taking in two huge disruptions to New Zealand's economy and fiscal outlook: the COVID-19 pandemic and Cyclone Gabrielle.
He also turned down the chance to succeed Ms Ardern, who left the role in January last year.
"I had observed that job up close and I knew exactly what it took and it takes more than 100 per cent of a person," he told AAP in an interview this week.
"I was at a point where that just wasn't where I was at. I could have pursued it and maybe I would get away with it, but I just don't think that's right."
The 52-year-old was central to Labour through its six years in office.
He applied the new 'wellbeing' approach to annual budgets, lifted minimum wages consistently above inflation, and deployed wage subsidies through the pandemic to prevent mass unemployment.
As sport minister, he also developed New Zealand's first women and girls in sport strategy, and hosted three women's World Cup events; for rugby, cricket, and together with Australia, football.
He is also the most senior LGBT politician in New Zealand history, serving as deputy prime minister from 2020-2023.
However, the Labour lion leaves politics without major tax reform, and particularly a capital gains tax, a long-time personal goal.
Labour lost elections in 2011 and 2014 campaigning on a capital gains tax, taking office in 2017 after Ms Ardern pledged not to introduce one, which she repeated in 2019.
After Ms Ardern left office in 2023, Mr Robertson developed tax reform that Labour intended to take to the election under leader Chris Hipkins, only for the new prime minister to get cold feet and scotch the plan.
"In politics, you've got to be prepared to accept that not every idea that you have is going to take off," he said.
"People will be aware that I think the tax system needs to change but you also live in a world where timing and politics matters."
In his valedictory speech on Wednesday night, he described New Zealand's tax system as "unfair and unbalanced".
"We are almost alone in the OECD in terms of not properly taxing assets and wealth in some form. Our current system entrenches inequality," he said.
"We need some political consensus about this to ensure we get it right and it sticks."
Mr Robertson missed out on the chance to become Labour leader three times, beaten twice in contests while in opposition, and then opting against succeeding Ms Ardern.
He was one of the few people Ms Ardern confided in before her shock resignation, mulling over her options before coming to the conclusion she had run out of gas to stand at the 2023 election.
Mr Hipkins confirmed he would not have challenged Mr Robertson if he had nominated.
"Of course I wouldn't have. Grant is one of my closest friends," Mr Hipkins, now opposition leader, said.
Mr Robertson also leaves with a sharply contested political legacy.
The government's COVID-19 policies - including mandates and lockdowns - were hugely contentious and drew out a radical fringe that occupied parliament for three weeks in early 2022.
Right-leaning parties also argue Mr Robertson's legacy will be ballooning government debt, as a result of New Zealand's loose monetary policy through the pandemic.
In his valedictory address, Mr Robertson included tributes to powerful women in his life: his predecessor as Wellington Central MP, Marian Hobbs, former leader Helen Clark, his mother and Ms Ardern.
"I was honoured every day that I worked with Jacinda, one of the most extraordinary leaders I've ever seen," he said.
