Spain looked in complete control after Alvaro Morata headed the 2010 champions in front after just 11 minutes and they finished the first half with almost 80% possession. But Japan, who knew they needed at least a draw to have a chance of progressing, roared to life at the start of the second half, scoring two goals in the first six minutes.
Half-time substitute Ritsu Doan smashed home the first at 48 minutes from outside the box as Spain were caught playing out of the back and goalkeeper Unai Simon failed to get enough on his attempted save to keep it out.
With Spain still reeling from that bolt from the blue, Japan took the lead in controversial circumstances.
The goal initially looked to have been disallowed with the ball appearing to roll out of play before Kaoru Mitoma cut it back for Ao Tanaka to bundle in from close range. But after a VAR review, the goal was given, turning the complexion of the group on its head.
At that point a win for Costa Rica over Germany in the group's other game would have knocked Spain out of the competition. But a Germany win meant that Spain progressed, despite failing to find an equaliser against Japan.
"Spain are one of the best teams in the world and we knew before the game that this was going to be very difficult, and indeed it was," Japan coach Hajime Moriyasu told reporters.
"But the players persisted and they persevered and they turned the tide of the game. Everybody was excited that we were able to play as one again.
"There were many fans who came all the way from Japan, as well as those that stayed at home. We are gifting this win to the people of Japan, and we are very happy about it."
Already Japan have made history in Qatar. They are the first team out of Asia, Africa or Oceania with multiple wins against former champions at the World Cup, and have done so in the same tournament after beating Germany in their opener.
And for the first time, Japan have reached the knockout stages in consecutive World Cups. They will look to make more history by going beyond the round of 16 for the first time.
Information from Reuters was used in this report.