Drafted in the 13th round by the defending world champion Baltimore Colts, Richardson played two seasons in the NFL, earning Colt Rookie of the Year honors in 1959. He caught a touchdown pass in the 1959 NFL Championship Game from quarterback Johnny Unitas.