Palmer started his career at West Bromwich Albion, joining as an apprentice in July 1983 before turning professional in December 1984. He made his league debut in September 1985, as a substitute against Newcastle United. He earned a move to Sheffield Wednesday in February 1989 for £750,000, where he made his name in the old First Division. He missed Wednesday's victory in the 1991 Football League Cup Final due to suspension.