In 1994, he signed as a free agent with the Yankees, hitting .311 in the strike-shortened season. He stole 20 of 32 bases that year, but tied Kenny Lofton and Felix Jose with 12 times caught stealing. When baseball resumed in 1995, he hit .261 before being traded to the Atlanta Braves and provided a boost down the stretch when he was 2 for 5 with 3 RBIs in the NLDS and NLCS, despite playing in a platoon role. In the first game of the 1995 World Series, Polonia drove in Fred McGriff for the game-winning RBI. Overall, he was 4-for-14 with a walk, a double, 3 runs scored, and 4 RBIs as he earned his first World Series ring.