Crews were called to the Bright Moon Buddhist Temple in Springvale South about 8pm on Sunday as large flames engulfed the building.
It took about 80 firefighters more than two hours to control the blaze.
About 30 residents in the surrounding area had to be evacuated, while others were told to close doors and windows and turn off heating and cooling systems.
No one was injured in the blaze, Fire Rescue Victoria said.
Firefighters were continuing to put out spot fires on Monday morning, assistant chief fire officer Paul Foster said.
"This was an extensive fire, very rapidly developing and quite spectacular to look at," he told ABC News Breakfast on Monday morning.
"It was seen for many kilometres around."
He said it was too early to determine the cause of the blaze.
It is understood up to 1000 members of the Springvale Buddhist community had their loved ones' ashes kept inside the temple.
Resident Jenny Tang, who had her father's ashes at the site, said she was devastated.
"I was really in shock and never thought it would happen," she told 3AW.
"It's a sanctuary. It's a place where I can go and chat to my dad because I lost him suddenly. It was also a place of worship. It means a lot to us."
It is still unsafe to enter the building so Ms Tang was unsure whether her father's ashes could be salvaged.
Fire Rescue Victoria will work with the local Buddhist community as they investigate the fire.
"There's a very culturally sensitive operation we've conducted here," Mr Foster said.
"Not only is it a place of worship, it's a place of gathering for the local Buddhist community."
Home Affairs Minister Clare O'Neil, the federal member for the Springvale region, said the community's devastation was palpable.
"This temple was like a second home for so many, built up by contributions from local volunteers and worshippers over decades into something beautiful," she tweeted on Monday.
"This is a devastating time for the Buddhist community in Melbourne. I know that, together, we can rally to support them in a time of real need."
Bright Moon Buddhist Society president Vinh Loi Ly was shattered to hear the news.
"I couldn't sleep last night. All night, no sleep. Why did it happen?" he told reporters.
Victorian Multicultural Affairs Minister Colin Brooks tweeted it was devastating to see the damage to the temple and it was an important part of the community.
9°C