The former Austrian midfielder played 103 international matches and scored 26 goals between 1998-2003, the most among Austrian players.
He left his post as coach of Austria’s under-21 team on Friday, the federation said on Saturday.
No details about Herzog’s contract with the U.S. soccer federation were revealed.
After stints with Rapid Vienna, Werder Bremen and Bayern Munich, Herzog played for Los Angeles Galaxy in 2004. He was assistant to Austria coaches Josef Hickersberger and Karel Brueckner before taking over the under-21 team in March 2009.
His first match as assistant to Klinsmann will be a Jan. 21 exhibition against Venezuela, followed by an exhibition against Panama four days later.