Connor scored on his senior debut for Leeds United aged 17, in a 1–0 win over West Bromwich Albion on 17 November 1979. He made 108 appearances in total for Leeds over four seasons, scoring 22 goals. He joined Brighton &amp; Hove Albion in exchange for Andy Ritchie, in March 1983. However, he was unable to appear in their FA Cup Final appearance just months later as he was already cup-tied. The club ended the season relegated. The majority of Connor's games for Brighton came in the Second Division. His form here won him an England under-21 cap in November 1986, when he played and scored against Yugoslavia under-21. He scored 51 goals in 156 appearances before leaving Brighton as they dropped into the third tier in 1987. One of his most memorable goals for Brighton came when they knocked Liverpool out of the 1983-84 FA Cup, a season in which Liverpool won the Football League Cup, European Cup and were crowned English champions. He moved along the South Coast to sign for Portsmouth in a £200,000 deal.