Rubio spent two-and-a-half hours at the Vatican on Thursday before driving away in a convoy under tight security.
He met initially with Leo before sitting down with senior Vatican officials, including top diplomat Italian Cardinal Pietro Parolin.
The Vatican and the US State Department did not provide any immediate details about the encounters.
Rubio's meeting with Leo, the first between the Pope and a Trump cabinet official in nearly a year, appeared to have run longer than planned.
The Pope arrived 40 minutes late for a subsequent appointment with Vatican staffers, and thanked them for being patient.
Vatican photos of the meeting showed Leo and Rubio shaking hands before sitting down together at the Pope's official desk in the Vatican's Apostolic Palace.
Leo, the first US pope, drew Trump's ire after becoming a firm critic of the US-Israeli war on Iran and the Trump administration's hardline anti-immigration policies.
The president has kept up an unprecedented series of public attacks on the Pope in recent weeks, drawing a backlash from Christian leaders across the political spectrum.
On Monday, Trump falsely suggested the Pope believed it was OK for Iran to obtain nuclear weapons and said Leo was "endangering a lot of Catholics" by opposing the war.
Leo told journalists after the latest attack that he was spreading the Christian message of peace.
The Pope also firmly rejected the idea that he supported nuclear weapons, which the Catholic Church teaches are immoral.
"The mission of the church is to preach the Gospel, to preach peace," the Pope said.
"The church has spoken out for years against all nuclear arms, on that there is no doubt."
As Rubio arrived at the Vatican earlier on Thursday, Polish Prime Minister Donald Tusk was leaving from a meeting with Leo.
He told journalists he and the Pope discussed how to strengthen international co-operation and generate hope in the world.
"It is still possible that the world does not have to descend into chaos, if good people, people of goodwill, find one another and act in unity," Tusk said, speaking in Polish.
Leo, who on Friday marks his first year leading the 1.4-billion-member church, has grown more outspoken on the world stage in recent weeks.
During a four-nation African tour in April, he forcefully decried the direction of global leadership and said the world was "being ravaged by a handful of tyrants", in comments he later said were not aimed directly at Trump.
Rubio is Catholic, as is Vice-President JD Vance, and the two met Leo a year ago after attending the Pope's inaugural mass.
Rubio said at a White House briefing on Tuesday that he expected to discuss Cuba and concerns over religious freedom around the world with Leo.
Rubio is visiting Rome for two days, and is due to meet Italian Prime Minister Giorgia Meloni, who has defended the Pope from Trump, on Friday.
