Palmer joined Williams as a test driver for the 1982 and 1983 seasons whilst racing in F2, and made his Formula One debut at Brands Hatch on 25 September 1983, driving a Williams in the European Grand Prix. This drive was a 'thank you' from Frank Williams and Patrick Head. He finished 13th out of 26 starters. Moving to the Skoal Bandit RAM March team in 1984, his six finishes yielded one 8th place, three 9th, one 10th, and one 13th. He joined Zakspeed in 1985, starting in eight races and retiring from all except the 1985 Detroit Grand Prix, where he finished 11th. Sixteen starts with the same team in 1986 resulted in eight retirements and a best finish of 8th in Detroit.