Burley was born in Cumnock, East Ayrshire. He joined Ipswich Town in 1972 as an apprentice and made his senior debut against Manchester United at Old Trafford in 1973, being given the job of marking George Best. In 1978, he was a member of the Ipswich side which upset the odds to defeat Arsenal 1–0 in the FA Cup final. However, in 1981 injury forced him out of Ipswich's UEFA Cup final triumph over AZ Alkmaar. Town missed out on the First Division title on the last day of the season, finishing runners-up to Aston Villa.