Phillips began his career at Birmingham City as an apprentice in 1979. He was capped by England at youth level, and made his first-team debut for Birmingham in a 2–2 draw at West Ham United in February 1982. In all he played 44 league games for Birmingham, scoring 3 goals, before he was transferred to Oxford United in March 1984. Phillips enjoyed a long spell at Oxford, remaining with the club until 1993, and he featured in to date the club's only major honour, the 1986 Football League Cup Final. His final league season was with Northampton Town during the 1993-94 season.