"The final decision will probably not come until Sunday night", said Germany's head coach Loew on Friday.
It was a chance for the fringe players in Low's 27-man squad to earn their spot in the final 23 and Freiburg striker Nils Petersen was handed his debut.
"The defeat is annoying, we did well in the first half, but we lost a lot of ball in the second half and made life hard for ourselves".
Kick off was delayed by almost 100 minutes due to heavy rain, meaning Germany captain Manuel Neuer had to wait even longer to make his long-awaited return to the pitch.
Germany led early on through Mesut Ozil's strike but second-half goals from Martin Hinteregger and Alessandro Schopf turned the match on its head and gave Austria, who will not be at the World Cup in Russian Federation this month, a seventh successive victory.
It was Germany's fifth match without a win - their first such stretch since 1987-88 - and their first defeat to neighbors Austria for 32 years.
The match was delayed for over an hour due to inclement weather at the stadium.
Germany had no trouble adapting to the conditions and went ahead after 11 minutes.
Germany really should have doubled the advantage 10 minutes later, but Julian Brandt's low effort after a flowing move was diverted agonisingly past the far post by the legs of Siebenhandl.
Neuer then made a key save by pushing Florian Grillitsch's shot around the post on 32 minutes as it was 1-0 to Germany at the break.
Loew's crowning moment was in holding aloft the 2014 World Cup trophy in Rio de Janeiro.
Austria's midfielder Alessandro Schoepf celebrates scoring with his team-mates defender Aleksandar Dragovic and defender Stefan Lainer during the global friendly footbal match Austria v Germany in Klagenfurt, Austria, on June 2, 2018.
Austria turned the match around with goals from defender Martin Hinteregger in the 53rd and midfielder Alessandro Schoepf 16 minutes later.
Germany play their final friendly before Russian Federation 2018 when they host Saudi Arabia in Leverkusen next Friday with their opening World Cup match against Mexico in Moscow on June 17 having also drawn South Korea and Sweden in Group F.