Crowther was born in Bilston, Staffordshire. A wing half, he signed for Aston Villa from non-league club Bilston Town for a fee of £750 in 1955. He was part of the Villa team that beat Manchester United 2–1 to win the 1957 FA Cup Final. Less than a year later, in February 1958, Crowther was hastily signed by United for £18,000 in the wake of the Munich air disaster. The transfer was completed around an hour before their match against Sheffield Wednesday in the FA Cup was due to kick off.