Duke of Montrose
Graham was a younger son of James Graham, 3rd Duke of Montrose, by his second wife Lady Caroline Maria, daughter of George Montagu, 4th Duke of Manchester. James Graham, 4th Duke of Montrose, was his elder brother.