Taijul followed his 4-109 with 6-75 in the second innings to help dismiss the Kiwis on Saturday for 181 in the first session of the final day, with the tourists chasing a target of 332 runs.
Offspinner Nayeem Hasan complemented him with 2-40, while pacer Shoriful Islam (1-13) and offspinner Mehidy Hasan (1-44) contributed a wicket apiece.
Daryl Mitchell battled to top-score for the visitors with a patient 58 from 120 balls.
Resuming at 7-113, Mitchell's presence gave New Zealand a slim hope of an unlikely victory and he struck Taijul for a boundary from the third ball of the day before punching the same bowler for a single through cover to raise his ninth half-century off 99 balls.
Unlike many of his compatriots Mitchell looked confident against the host's spinners, but a mis-timed sweep shot against offspinner Nayeem Hasan eventually brought about his dismissal and effectively ended New Zealand's hopes of saving the game.
Skipper Tim Southee scored an attacking 34 off 24 balls, with one four and two sixes, before Taijul dismissed him to complete his 12th five-for.
Taijul then wrapped up the match when Ish Sodhi, who made a laborious 22 off 91, popped a simple catch to Zakir Hasan at silly mid-off.
A historic first! �— ICC (@ICC) Bangladesh record their maiden win in Tests at home against New Zealand 🔥#WTC25 | #BANvNZ | �: https://t.co/sKjcBFe2Df pic.twitter.com/U1fPrLvd6fDecember 2, 2023
Opener Mahmudul Hasan made 86 as Bangladesh were bowled out for 310 in their first innings but New Zealand compiled 317, for a seven-run lead, courtesy of a brilliant 104 from Kane Williamson.
Najmul Hossain Shanto then became the first Bangladesh cricketer to hit a century on debut as captain as the hosts made 338 in their second dig, and set a daunting 332-run target for the Black Caps on an increasingly deteriorating pitch.
Taijul then spun his way through the visitors' batting line-up late on day four and the final morning to lead Bangladesh to a first victory against New Zealand at home, and only their second overall.
The second and final Test of the series begins on December 6 at Mirpur.
14°C