The 34-year-old Serbian entered Australia late on Wednesday after declaring he had a medical reason not to get vaccinated against COVID-19.
Djokovic was taken to immigration detention after having his visa cancelled early Thursday morning.
But after several days at the Park Hotel he was allowed out by Judge Anthony Kelly on Monday for the duration of a Federal Circuit Court hearing reviewing the decision to cancel his visa.
The hearing has been plagued by technical issues, with live streams collapsing under the pressure of tens of thousands of people trying to watch.
Judge Kelly ordered Djokovic be taken from detention to a confidential location specified by his lawyers for the duration of Monday's hearing and and be returned there later.
He last week questioned the possibility of Djokovic being transferred somewhere where he could train for the Australian Open, should the challenge be successful.
Nick Wood SC says Djokovic declared before boarding his flight to Australia from Dubai that he had a medical contraindication and was able to provide evidence - a medical exemption by Tennis Australia.
Judge Kelly asked: "What more could this man have done?"
"Here, a professor and an eminently qualified physician have produced and provided to the applicant a medical exemption," he said.
"Further to that, that medical exemption and the basis on which it was given, was separately given by a further independent expert specialist panel established by the Victorian state government."
The judge said those documents were in the hands of the immigration department delegate who made the decision to cancel his visa on behalf of Home Affairs Minister Karen Andrews.
Mr Wood said Djokovic was "utterly confused" because he had done everything asked of him.
His lawyers argue Djokovic contracted COVID-19 on December 16 and was symptom-free before his arrival.
They say he met criteria for a temporary exemption under Australian Technical Advisory Group on Immunisation (ATAGI) guidelines and was denied procedural fairness during the decision to revoke his visa.
Djokovic was questioned by authorities through the night, between his arrival just before midnight Wednesday and the visa cancellation at 7.42am Thursday.
A partial transcript records Djokovic telling authorities he wasn't vaccinated against COVID-19. He has previously declined to confirm his vaccination status.
Government submissions say Djokovic is wrong to challenge the claim that previous infection is grounds for an exemption.
They say the ATAGI advice is clear that past infection is not a contraindication for infection and instead a person can defer vaccination for six months after acute illness.
"There is no suggestion that the applicant had 'acute major medical illness' in December 2021," the documents say.
"All he has said is that he tested positive for COVID-19. That is not the same."
The ATAGI advice also says a person who tests positive can receive a first or second dose of a COVID-19 vaccination once they are asymptomatic.
Chris Tran, representing the Commonwealth, told the court there was precedent for siding with his arguments in the case - poking fun at himself in the process.
"I think it will be useful for Your Honour to consider two cases - both of them losses of mine, so I'll be revisiting happy memories," he said.
He'll begin his detailed submission later on Monday.
Australian government lawyers have asked the judge to reject Djokovic's legal challenge and order he pay costs.
But should he win and the court order his immediate release, they want the judge to make it clear there's nothing stopping them detaining him again.
The hearing is continuing.
