Nervous Germany increased the pressure after the break and poured forward but despite dominating the second half, they could not find the back of the net before South Korea's last-gasp brace.
The result means both teams depart the tournament while Sweden win Group F and are joined by Mexico in the last 16.
This was South Korea's first win at the 2018 World Cup after losing 1-0 to Sweden and falling 2-1 to Mexico.
Mexico fans were hoping for a shocking upset - and the South Koreans delivered that with two goals in stoppage time for a 2-0 victory.
Germany fans were left devastated as the holders exited the World Cup in the group stage.
"I think that all of us weren't up for it and did not have the absolute will to show that we wanted to tear something down here at the World Cup", Neuer said. But Germany came out with an uninspired effort against South Korea, and its failure came in especially gut-wrenching fashion. France in 2002, Italy in 2010 and Spain in 2014 all suffered the same fate as the Germans who went into the tournament with the nucleus of the side that triumphed in Brazil four years ago.
Germany needed victory against the lightly regarded Koreans to give themselves a chance of progressing, but became the fourth champions in five World Cups to fall at the first hurdle during their title defence.
But it was Kim Young-gwon who broke the deadlock in added time when he found himself alone in front of goal after a corner and he poked the ball home from close range.
He said: "Until quite recently, we have probably been the most consistent, in terms of performance, team in the past 10, 12 years". South Korea, on three, finished third, ahead of Germany on goal difference.
Mats Hummels headed over the top when unmarked six yards out in the closing stages, before South Korea's two late goals.
South Korea's win was a huge boost to Mexico, advancing it through to the knockout stage.