 While Smith was attending junior high school, his parents divorced. Continuing to pursue his interest in baseball, he would ride the bus for nearly an hour to reach Dodger Stadium, cheering for the Los Angeles Dodgers at about 25 games a year. Upon becoming a student at Locke High School, Smith played on the basketball and baseball teams. Smith was a teammate of future National Basketball Association player Marques Johnson on the basketball team, and a teammate of future fellow Hall-of-Fame player Eddie Murray on the baseball side. After high school Smith attended Cal Poly San Luis Obispo in 1974 on a partial academic scholarship, and managed to walk-on to the baseball team. In addition to his academic education, he learned to switch-hit from Cal Poly coach Berdy Harr. When Cal Poly's starting shortstop broke his leg midway through the 1974 season, Smith subsequently took over the starting role. Later named an All-American athlete, he established school records in career at bats (754) and career stolen bases (110) before graduating in 1977.
