The fire broke out about noon in the city of Jinjiang in Fujian province at footwear manufacturer Huiteng.
A total of 239 people were at the factory when the fire broke out, Xinhua said.
It said 213 were able to leave the factory, two of whom died after being taken to hospital for treatment.
Another 26 people previously reported missing at the scene were confirmed dead.
Videos released by state media in the afternoon showed flames ripping through the building and thick black smoke billowing into the sky.
A dozen or so people, apparently trapped by the blaze, could be seen on the rooftop as smoke swirled around them.
Chinese President Xi Jinping issued instructions urging an all-out rescue effort.
He said that the fire had "caused significant casualties" and that it came after a number of deadly accidents at production facilities in the country in recent months.
Xi called on authorities to hold those responsible strictly accountable.
Firefighting and rescue efforts were still under way, although open flames had largely been extinguished by about 4pm, a local firefighting official told state broadcaster CCTV.
Thick smoke could still be seen rising from the site, with multiple fire engines continuing to spray water by early Thursday evening, Xinhua said.
Jinjiang, in the coastal Fujian province, is a major footwear and clothing manufacturing hub and is often referred to as China's "shoe capital".
Preliminary findings indicate the fire started on the ground floor of the factory, a local official told CCTV.
Shoe-making materials stored in the building were highly flammable and could cause a blaze to spread rapidly, the official said.
Due to the materials and adhesives present, the smell at the scene was pungent, causing eye irritation, CCTV said.
Large amounts of items had also been piled up in the stairwells, hampering rescue and firefighting efforts, the state broadcaster quoted firefighters as saying.
People in charge of the company and "other relevant personnel" have been taken into custody, with the company's bank accounts frozen, Xinhua said.
The thousands of companies that make up Jinjiang's shoe sector manufactured more than 1.2 billion pairs of shoes in 2024, accounting for 20 per cent of global output, according to official data.
