In 1791, he voted in favor of the unsuccessful attempt to repeal the Test Act in Scotland. Also in 1791 and again in 1792, he voted with the opposition in the Oczakov debates concerning the Russian occupation of the Turkish port of Ochakiv on the Black Sea. In 1793, he joined the Friends of the People and voted for Grey's motion for inquiry into parliamentary reform, which did not achieve success until 1832. In December 1792, he voted against his fellow Whig, and party leader, Charles Fox's Libel amendment, but opposed the French war. Thereafter, Church regularly voted with the Foxite minority for the rest of his parliamentary career.