In his years out of office, Grenville became close to the opposition Whigs leader Charles James Fox, and when Pitt returned to office in 1804, Grenville did not take part. Following Pitt's death in 1806, Grenville became the head of the "Ministry of All the Talents", a coalition between Grenville's supporters, the Foxite Whigs, and the supporters of former Prime Minister Lord Sidmouth, with Grenville as First Lord of the Treasury and Fox as Foreign Secretary as joint leaders. Grenville's cousin William Windham served as Secretary of State for War and the Colonies, and his younger brother, Thomas Grenville, served briefly as First Lord of the Admiralty. The Ministry ultimately accomplished little, failing either to make peace with France or to accomplish Catholic emancipation (the later attempt resulting in the ministry's dismissal in March, 1807). It did have one significant achievement, however, in the abolition of the slave trade in 1807.
In the years after the fall of the ministry, Grenville continued in opposition, maintaining his alliance with Lord Grey and the Whigs, criticizing the Peninsular War and, with Grey, refusing to join Lord Liverpool's government in 1812. In the post-war years, Grenville gradually moved back closer to the Tories, but never again returned to the cabinet. His political career was ended by a stroke in 1823. Grenville also served as Chancellor of the University of Oxford from 1810 until his death in 1834.
The Ministry of All the Talents, February 1806 - March 1807
September, 1806 - On Fox's death, Lord Howick succeeds him as Foreign Secretary and Leader of the House of Commons. Thomas Grenville succeeds Howick at the Admiralty. Lord Fitzwilliam becomes Minister without Portfolio, and Lord Sidmouth succeeds him as Lord President. Lord Holland succeeds Sidmouth as Lord Privy Seal.
