Smith played for Aston Villa in the 1957 FA Cup Final against Manchester United's Busby Babes, where Villa won the game 2–1 to lift the cup. However his career was cut short after an achilles injury forced him into retirement in 1960. In total, he made 130 appearances, scoring 27 goals for Villa.