One of her brothers, Sir John Milley Doyle (1781–1856) was a commander of British and Portuguese forces in the Peninsular War and the War of the Two Brothers.