Murray was born in Taymouth, Scotland, the eldest son of William Murray, 3rd Earl of Dunmore, by his marriage to Catherine Nairne; he was a nephew of John Murray, 2nd Earl of Dunmore. In 1745, both Murray, then only 15, and his father joined the ill-fated Rising of "Bonnie Prince Charlie" (Charles Edward Stuart), and the young Murray was appointed as a page to Prince Charles. The second Earl, his uncle, remained loyal to the Hanoverians.