Born in Boise, Idaho, Spencer grew up skiing and racing at Bogus Basin. He made the U.S. Ski Team at age 16, and debuted in World Cup competition at age 18 in a giant slalom in Park City. He attended the University of Colorado in Boulder and was a member of the 2002 U.S. Olympic team and finished 16th in the giant slalom, held in Park City. His best finish in international competition was 7th in the GS at the 2005 World Championships in Bormio, Italy.