The Brave Blossoms scored three tries to the Italians' one and threatened until the final whistle to finish 27-10 on Saturday night in a marked improvement on their recent run of form in international rugby.
The Azzurri's string of unforced errors hampered any chance to build momentum and belied the quality that saw them topple England and Scotland at the Six Nations this year.
Italy opened the scoring in the fifth minute when an angled run from outside centre Juan Ignacio Brex pierced the Japanese backline, but Japan fought back six minutes later, with towering skipper Warner Dearns forcing his way over the try line after a few phases of forward attacks.
Japan had most of the territory and possession throughout, aided by sharp kicking from the halfbacks that regularly put Italy on the back foot deep in their own half.
Japan scored again in the 17th minute when inside centre Yuya Hirose broke through the Italian line and offloaded to full back Takuro Matsunaga.
The third try came in the 47th minute after Japan plugged away over more than 15 phases until flanker Ben Gunter finally muscled over the line.
Italy's backs never found fluency, but the forwards brought their physicality to bear in some bullying mauls, pushing Japan to their five-metre line and coming away with a penalty conversion on the stroke of halftime, having forced two offside infractions.
Although both sides were scrappy in parts, Japan made fewer handling errors and were better able to push their advantage.
The Test was the second of the new Nations Championship, in which 12 sides from the northern and southern hemispheres face off for three fixtures this month and another three in November, before a finals weekend to be held in London.
Earlier on Saturday New Zealand prevailed over France in a 34-32 thriller to kick off the tournament.
Japan's head coach Eddie Jones, who also previously coached the Wallabies, was barred from attending the game as a penalty for having verbally abused officials in Australia earlier this year.
5°C