The announcement has triggered a by-election in Aston, less than a year after the Liberal MP narrowly held on to the outer Melbourne seat at the 2022 poll.
Mr Tudge has been in parliament since 2010, holding ministries in the education and human services portfolios under the coalition government.
He stepped down from cabinet after being accused of emotional and physical abuse by his former staffer Rachelle Miller, with whom he had an affair.
Mr Tudge denied the claims of abuse but opted not to return to cabinet once an investigation found he did not break cabinet rules.
Ms Miller was in parliament's public gallery when Mr Tudge announced his resignation.
In a statement, she said she tried to support the former minister through "the most stressful of times".
"Politics is brutal. I saw its impact on people and I live with that impact still," Ms Miller said on Twitter.
"I hope he seeks the help he needs for his health and more time with his family. I wish him peace."
In an emotional speech to parliament, Mr Tudge said his decision to resign was cemented following the death of his father earlier this year.
"It's not been an easy decision for me, but it is necessary for my health and for my family, amongst other reasons," he said.
Mr Tudge said he wanted to spend more time with his children.
"My daughters who are 18 and 16 know nothing other than their dad being away," he said.
"They've had to put up with things that no teenager should have to - including death threats, the most recent of which was last week.
"My son is a bit younger, but I equally want to be a good father to him."
The outgoing MP cited his work on cashless debit cards and in education among his proudest achievements.
"While this profession can be unforgiving at times, serving as a member of parliament, though, has been one of the greatest honours of my life," he said.
"I leave honoured to have served and proud of what we have achieved both for the people of Aston and across Australia, but particularly for Indigenous Australia.
"It is time to pass the baton to someone who can hopefully serve our local community better than anyone before them."
Last week, Mr Tudge gave evidence at the royal commission into the robodebt scheme, where he denied responsibility for checking whether the welfare program was lawful.
Prime Minister Anthony Albanese said it was a difficult decision for the Liberal MP, adding that serving in the parliament was an incredible honour.
"The member for Aston has been very lucky - one term in opposition and three terms as a minister. I'm in my third term as a minister and I've been here since last century," he said.
Opposition Leader Peter Dutton said Mr Tudge had a significant parliamentary career.
"He does have a great intellect and appetite for nothing more than seeing what is in our country's best interests," he said.
"He has been passionate about making sure that whether it's for local roads or projects in his local community, he's been a champion on behalf of those people. But his interest has extended nationally."
The by-election date for Aston has not yet been set. Mr Tudge held the seat by 2.8 per cent after suffering a swing of more than seven per cent at the election.
