Whiteside attended Delta State University before being drafted in the sixth round of the 2001 Major League Baseball (MLB) Draft by the Baltimore Orioles. He played in their organization through 2007, though he only played nine games in the major leagues with the Orioles, all coming in 2005. He signed with the Minnesota Twins in 2008 but was released after playing for their Triple-A team for a month. The San Francisco Giants then signed him, assigning him to the minor leagues. He was called up to be their backup catcher in May 2009, and he caught Jonathan Sánchez's no-hitter on July 10. In 2010, he remained the backup catcher and was on the Giants' roster when they won the World Series, despite not playing any playoff games.