Crowe was born in Abercynon, South Wales but moved to Handsworth, Birmingham with his family when he was two years old. On leaving school, he played for Erdington Albion, the West Bromwich Albion nursery team, but signed for Aston Villa in 1951 and would spend most of his career with the Birmingham club. He established himself in the Villa team when Danny Blanchflower vacated the right-half berth in 1954. He missed the 1957 FA Cup Final due to injury but captained the side to the Second Division title in 1960 and League Cup Final success in 1961. He was capped 16 times by Wales.