Peña is best known for his late season success with the Atlanta Braves in 1991. Acquired from the Mets at the end of August (he needed to be on Atlanta's roster on August 31 to pitch in the playoffs), Pena went 13-for-13 in save opportunities, including saving Games 2 and 6 for Steve Avery in the 1991 NLCS. He blew a save in Game 3 of the 1991 World Series, but the Braves later won that game. He was on the mound for the final pitch of that classic series, giving up the base hit to Gene Larkin that won the title for the Minnesota Twins. Peña was the losing pitcher of Game 7. On September 11, 1991, Peña teamed with fellow Braves pitchers Kent Mercker and Mark Wohlers for a combined no-hitter against the San Diego Padres; Peña pitched the final inning of the 1–0 victory for a save. The 13th no-hitter in Braves franchise history, attendance was 20,477 at Fulton-County Stadium in the 1-0 shutout win. In 1992 he was hampered by tendinitis in his pitching elbow, leading the Braves to leave him off their World Series roster and causing him to miss the 1993 season. In 1995 he was reacquired by the Braves for their World Series run and was the winning pitcher of record for the first two games of their NLDS series.