Smith was selected by the New Orleans Hornets with the 18th overall pick in the 2004 NBA draft. As a rookie, Smith was named Western Conference Rookie of the Month three times (January, February, March) and participated in the 2005 NBA Slam Dunk Contest, finishing third. After averaging 10.3 points in 76 games (56 starts) as a rookie, he averaged 7.7 points in 55 games (25 starts) in 2005–06. Smith fell out of favor with Hornets coach Byron Scott due to a poor work ethic. He began the 2005–06 season as the starting shooting guard and ended it out of the rotation.