The payments for workers who needed to spend time away from work while they isolated ended on June 30.
However, there have been calls for the payments to be reinstated as COVID case numbers and hospitalisations increase across the country, driven by new, infectious sub-variants.
Mr Albanese, who on Tuesday received his fourth COVID-19 vaccine dose, said he wouldn't bring back the payments due to the need for the government to rein back spending.
The new government had inherited the former's decision to end the payments and $1 trillion of debt, he said.
"They are circumstances which my government faces ... there are a range of things we would like to do, but we intend to be fiscally responsible in how we deal with issues," he told reporters in Sydney.
As case numbers rise across the country, Chief Medical Officer Paul Kelly said hospitals could hold back some elective surgery due to the strain on the system.
He said the emergence of influenza for the first time in three years in Australia was making the situation worse.
Health Minister Mark Butler said the rise in infections still had a long way to go.
Mr Butler indicated COVID-19 cases were not likely to peak nationally for at least four weeks during the third wave of infections.
"All of the modelling indicates that case numbers and hospitalisations have further to go over probably the next four to six weeks," he told Melbourne radio 3AW on Tuesday.
While case numbers have increased and calls have grown for mask mandates to return, the prime minister said he would follow health advice on the issue.
"Mandates on those issues, of course, are a matter for ... state governments around the country," he said.
"We will continue to take advice on these issues by health experts, and we've acted on all the advice that has been given during the pandemic."
But educating the public about the risks of exposure to COVID would be more helpful than reintroducing a mask mandate, epidemiologist Catherine Bennett said.
"If (people) know what the infection risk is ... that's the sort of information that can inform people's choices and might actually translate to better uptake of masks than simply putting rules in place," she told Sky News.
Prof Kelly said the rising number of reinfections had thrown a curve ball for handling the virus.
The Australian Health Protection Principal Committee has recommended the reinfection period be reduced from 12 weeks to 28 days following the rise of cases of the BA.4 and BA.5 Omicron variants.
This means people will be required to get tested and isolate if they redevelop symptoms 28 days after recovering from the virus, and could be reported and managed as new cases.
NSW and Western Australia have already followed suit.
Australia recorded more than 41,000 COVID cases and 58 deaths on Tuesday.
There are nearly 4500 patients in hospital with the virus.
LATEST 24-HOUR COVID-19 DATA:
NSW: 10,806 cases, 20 deaths, 2049 in hospital with 58 in ICU
Victoria: 10,627 cases, 16 deaths, 737 in hospital with 39 in ICU
Tasmania: 1812 cases, one death, 100 in hospital with two in ICU
Queensland: 6768 cases, 15 deaths, 860 in hospital with 12 in ICU
ACT: 1174 cases, no deaths, 140 in hospital with three in ICU
SA: 3668 cases, three deaths, 246 in hospital with six in ICU
WA: 6000 cases, three deaths, 297 in hospital with six in ICU
NT: 481 cases, no deaths, 39 in hospital with zero in ICU
12°C