Shortly after leaving the university, he served in 20th Foot regiment commanded by James Wolfe during the Seven Years' War. He became friends with one of his fellow officers Charles Grey whose career he later assisted. In 1757 he took part in the amphibious Raid on Rochefort which withdrew without making any serious attempt on the town. The following year he was sent to serve in Germany and distinguished himself at Minden and Kloster-Kampen. For his services he was appointed aide-de-camp to the new King, George III, with the rank of colonel. This brought protests from several members of the cabinet as it meant he was promoted ahead of much more senior officers. In response to the appointment the Duke of Richmond resigned a post in the royal household. Though he had no active military career after this, his early promotion as colonel meant that he would be further promoted through seniority to major-general in 1765, lieutenant-general in 1772 and general in 1783.