Grimsby won a dramatic shootout 12-11 at the 9000-capacity Blundell Park on England's east coast on Wednesday, with Bryan Mbeumo missing the decisive penalty.
United needed late goals by Mbeumo and Harry Maguire just to salvage a 2-2 draw in regulation, having trailed 2-0 at halftime.
It will go down as one of the most embarrassing results in United's history and leaves the team winless in three games so far this campaign, after a woeful 15th-place finish in the Premier League last season.
Grimsby's jubilant fans stormed on to the field to celebrate with the players after Mbeumo's spot-kick rebounded off the crossbar to seal United's fate.
"It's an amazing feeling, one that will live on forever," said Grimsby's Charles Vernam, who opened his team's scoring in the 22nd minute.
Tyrell Warren added a second for the home side in the 30th minute.
Grimsby are currently fourth in League Two, and finished the match with a line-up containing several academy graduates and a Faeroe Islands international.
United, meanwhile, had $A415 million worth of new signings just in their attack in Mbeumo, Matheus Cunha and Benjamin Sesko.
Cunha also failed in the shootout by having his weak attempt saved, and Sesko took United's 10th penalty - only ahead of goalkeeper Andre Onana.
After Onana and Grimsby goalkeeper Christy Pym converted their penalties, the shootout reverted to the outfield players once again. Bruno Fernandes scored his for United, but Mbeumo couldn't.
"Going into the penalty shootout, the gaffer (manager) said to us, 'Lads, all the pressure is on them', and it was really," Grimsby midfielder Kieran Green said.
The result piles more pressure on Amorim, who came into the game with 16 wins from his 44 matches in charge of United after being hired in November.
He sat in the dugout during the shootout, not even watching the kicks.
A 1-0 loss to Arsenal in the opening round of the Premier League featured an encouraging performance by a new-look United team, but some worrying old habits returned when fading to a 1-1 draw at Fulham on Sunday.
The exit at Grimsby quickly brought back the gloom of last season.
Asked to explain what went wrong, the Portuguese manager did not hold back.
"Everything. The way we started the game, we were not even here,'' Amorim said.
"When everything is so important in our club, everything that happened, it's a problem in our club, we should do so much better. I just have to say sorry to our fans.''
United were only playing in the second round - for the first time since 2014 - after failing to qualify for European competition last season.
In other results on Wednesday, Brighton won 6-0 at second-tier Oxford, Everton beat third-tier Mansfield 2-0, and Fulham won 2-0 against second-tier Bristol City.
- with Reuters
