Charles James Fox
In his years out of office, Grenville became close to the opposition Whig leader Charles James Fox, and when Pitt returned to office in 1804, Grenville sided with Fox and did not take part.