Beginning in 1982, Shaffer served as musical director for David Letterman's late night talk shows: as leader of "The World's Most Dangerous Band" for "Late Night with David Letterman" (1982–1993) on NBC, for which he also composed the theme song, and as leader of the CBS Orchestra for the "Late Show with David Letterman" (1993–2015) on CBS. Letterman consistently maintained that the show's switch to CBS was because NBC "fired Paul for stealing pens" or some other trivial reason. Shaffer guest-hosted the show four times when Letterman was unavailable: February 9 and 11, 2000, during Letterman's recovery from his quintuple heart bypass surgery; March 24, 2003, when Letterman was suffering from shingles; and January 19, 2005, when Letterman went to receive an award for his racing team's victory in the 2004 Indianapolis 500.