By the late-1970s, Simmons was throwing out less than one-third of potential basestealers. During the 1980 season, Cardinals manager Whitey Herzog wanted Simmons moved to first base and Keith Hernandez to left field for the first six innings, with the latter often reverting to his original position at the end of games. Simmons' refusal and the ensuing feud with Herzog led to him being traded along with Rollie Fingers and Pete Vuckovich to the Milwaukee Brewers for Sixto Lezcano, Lary Sorensen, Dave LaPoint and David Green at the 1980 Winter Meetings on December 12. His batting average fell to .216 in his first season in the American League but, he rebounded in 1982 with a .269 batting average with 23 home runs and 97 RBIs and led American League catchers with a .995 fielding percentage in 121 games. The Brewers clinched the American League Eastern Division title, then defeated the California Angels in the 1982 American League Championship Series. Simmons met his old team, the St. Louis Cardinals in the 1982 World Series, hitting two home runs before the Brewers lost in a seven-game series. Simmons had one more good year in 1983 when he hit for a .308 batting average with 13 home runs and 108 RBIs and earned his eighth and final All-Star berth. His batting average fell to .221 in 1984, though Simmons rebounded in 1985 with a .273 average and 76 RBI's, and in March 1986 he was traded to the Atlanta Braves. He spent three seasons with the Braves as a utility player and pinch hitter before retiring as a player in 1988 at the age of 38.