In 2002, Williams became manager of the Houston Astros. While serving as a National League coach at the 2004 All-Star Game in Houston — at a time when the Astros were struggling at the .500 mark — the crowd at Houston's Minute Maid Park responded to the introduction of Williams with a decidedly mixed reaction, in contrast to the ovations generally given members of the home team who are introduced at an All-Star game. The following day, the Astros fired Williams and two principal coaches, having likely waited until after the festivities to avoid a public embarrassment. Williams was replaced by Phil Garner, who led the Astros to the 2004 National League Championship Series but fell one game short of going to Houston's first ever World Series. (The following year, Garner led the Astros to the World Series.) Williams finished with a record of 215 wins and 197 losses.