Household contacts of positive cases are currently required to undergo seven days of quarantine, unless they are an essential worker without any symptoms.
However, Mr Hunt said there was the capacity for exemptions from isolation to include everyone, following the success of the measures for essential employees.
It comes after officials from the Australian Health Protection Principal Committee in NSW and Victoria are considering removing the household isolation rules in both states.
"NSW and Victoria, with the Commonwealth's support, are leading that work, and I strongly support that direction," Mr Hunt said on Tuesday.
"The next step is to consider expanding that definition (of household contacts) through the AHPPC ... and it has worked well with our nurses."
The health minister said officials would follow the medical advice when considering changes to the rules.
"It's always a balance of ensuring that we have decreasing case numbers, and that's a trend which I'm increasingly confident of," he said.
"As a result of that, that actually allows us to have greater freedoms."
Meanwhile, Treasurer Josh Frydenberg said despite falling case numbers, it was too soon to implement a UK-style scrapping of restrictions.
Mr Frydenberg said the workplace issues in cities, which had been exacerbated by the Omicron wave at the start of the year, were now beginning to ease.
"People are getting back to work and some of those workforce shortages are starting to abate," he told the Seven Network on Tuesday.
"This is the next step in the road, reducing restrictions even further, and we'll continue to monitor the situation in Australia."
It comes after Australia reopened its international borders earlier this week to overseas tourists, after two years of closures due to the pandemic.
Despite the new arrivals, the federal health department said it was unlikely there would be a spike in new COVID cases.
"Australia has very high levels of vaccination and case numbers, and hospitalisations due to COVID-19 have peaked in most states and territories," a health spokesman told AAP.
"There is little to no evidence to suggest that vaccinated international arrivals would lead to increased infection rates or a new wave of COVID-19."
Federal Labor has also finalised a report from its pandemic jobs and industry taskforce, which examined the effect of COVID-19 on workers.
Opposition Leader Anthony Albanese received the final report while in Launceston on Tuesday, with the party to consider the recommendations.
"The pandemic has brought real challenges, but Australians have been magnificent in their response," he said.
"At the same time, the government has let them down. The government let them down by not ordering enough vaccines. The government more recently let them down by not ordering enough rapid antigen tests."
There were a further 14 deaths in both NSW and Victoria on Tuesday, while Queensland had five fatalities, and the NT reported one.
Victoria registered 6786 new infections, with NSW reporting 8752, Queensland having 5583, while the ACT and Tasmania had 583 and 820 respectively and the NT 716.
15°C