The 46-year-old Italian, who has previously coached Valencia, Napoli and AC Milan, was hired by Marseille as Marcelino's replacement in September last year.
Marseille are ninth in the French top flight after a winless run of five matches, out of which they have lost two and drawn three.
"Olympique de Marseille announces the end of its collaboration with Gennaro Gattuso," Marseille said in a statement.
"Marseille would like to extend its special thanks to Gennaro and his entire staff for the unfailing commitment and professionalism they have shown on a daily basis, and wishes them all the best for the future."
Frenchman Jean-Louis Gasset is the leading candidate for the vacant managerial role, local media reports said.
Gasset, 70, most recently coached the Ivory Coast national team. He was sacked after a poor performance in the group stage of the Africa Cup of Nations, which they scraped through as the last lucky loser. Ivory Coast went on to win the tournament.
Marseille, on 30 points from 22 matches, next host Ukrainian side Shakhtar Donetsk in the Europa League knockout play-off second leg on Thursday, followed by another home game against 14th-placed Montpellier in Ligue 1 on Sunday.
17°C