An Office of AI will be formed in the department of prime minister and cabinet, Anthony Albanese will announce in an address at the University of Sydney on Wednesday.
The new office will help co-ordinate the design of new Australian Standards and to bring together work across the government.
The speech comes in response to concerns surrounding rules governing the use of AI, data centres and protections for Australian intellectual property.
AI company Anthropic has sought clarification over copyright settings, but hasn't reportedly sought an exemption.
The prime minister's speech isn't expected to detail changes to copyright laws.
In response to Anthropic's claim in released documents that investment in AI development and infrastructure will depend on copyright policy certainty, Mr Albanese will argue the right guardrails will attract business.
"Getting this right will enhance our appeal to international investors, by delivering greater clarity and speed for approvals, and a streamlined process for verifying compliance," he will say.
"But just as government developed co-ordinated approaches for other significant technologies: from civil aviation in the 1920s to genetics in the 1990s, we must do this with AI as well."
The prime minister will point to the attorney-general's consultation on copyright and artist protections where AI training is involved and Treasurer Jim Chalmers' responsibility for the "pivotal role" of AI in the government's productivity agenda.
Mr Albanese will say this comes in addition to work being done to mitigate risks that chatbots pose to children, to artificial intelligence in defence and national security.
"We know that both extremists and state actors already use AI to create propaganda aimed at young people - and to spread disinformation that targets democracies," he will say.
Labor MP Ed Husic, a former industry minister, said giving big tech social licence was a path "sadly doomed to failure".
"We tried self-regulation for ... a couple of decades, found out that it didn't work, and it won't work on a financial basis for these firms," he told Sky News on Tuesday.
"None of these firms will go one out from the other to bring in guardrails to limit the risks."
