Serbian Prime Minister Ana Brnabic said the government's decision came after requests by various green groups to halt the $US2.4 billion ($A3.3 billion) Jadar lithium project.
"All decisions (linked to the lithium project) and all licences have been annulled," Brnabic told reporters after a government session.
"As far as project Jadar is concerned, this is an end."
Earlier this week, Rio had pushed back the timeline for first production from Jadar by one year to 2027, citing delays in key approvals.
Rio Tinto said it was "extremely concerned" by Serbia's decision and was reviewing the legal basis for it.
The company committed to the project just last year as global miners pushed into the metals needed for the green energy transition, including lithium, which is used to make electric vehicle batteries.
Brnabic accused Rio Tinto of providing insufficient information to communities about the project.
In a statement, Rio said "it had always operated in compliance" with Serbian laws.
Thousands of people blocked roads last year in protest against the government's backing of the project, demanding Rio Tinto leave the country and forcing the local municipality to scrap a plan to allocate land for the facility.
Thursday's decision comes as Serbia approaches a general election in April and as relations between Belgrade and Australia have soured after the high-profile deportation of tennis star Novak Djokovic over the country's COVID-19 entry rules.
Djokovic himself spoke out in support of "clean air" in a December Instagram story post captioning a picture of the protests, which was published by digital sports platform The Bridge.
Serbia's populist ruling coalition, led by the Serbian Progressive Party (SNS), had initially showed support for lithium and copper mining, a stance that made it come under fire, helping erode the comfortable majority the party enjoyed in a 2020 vote.
"We are listening to our people and it is our job to protect their interests even when we think differently," Brnabic said on Thursday.
The SNS-led coalition is expected to hold parliamentary and presidential elections on April 3 although the date is yet to be officially confirmed by President Aleksandar Vucic.
Earlier this month, Brnabic said Rio's Jadar development would be likely paused at least until after the elections.
"A compromise will be probably reached after the elections, so that there could be a renegotiation of royalties or value-sharing," said a Rio Tinto shareholder, who declined to be named.
