In August 2001, Guppy re-joined O'Neill at Celtic for £700,000, where he spent two-and-a-half years. During his time at Celtic, they won the Scottish Premier League in his first season at the club. The following season, 2002-03, Guppy had a number of injury problems, meaning he missed the 2003 UEFA Cup Final. In 2003-04 Guppy only made one appearance for Celtic, in a Scottish League Cup tie against Partick Thistle, before leaving Celtic to re-join Leicester City in January 2004. Guppy left Leicester in the summer of 2004, to try to get to the US to play. He then spent a brief spell at Leeds United in August 2004, scoring once against Nottingham Forest, before gaining a short-term deal at Stoke City followed by a short spell back at Wycombe, scoring once against Swansea City. He moved to the United States but his short stint in Major League Soccer with Washington-based D.C. United was ended prematurely due to injury, and he was released by the club after playing in just five league games in which he was featured in a starting line-up alongside Freddy Adu in midfield.