Born the fourth son of John Kerr, 7th Marquess of Lothian and Lady Cecil Chetwynd Talbot. Kerr was educated at Radley College and joined the first-rate HMS "Prince Regent" as a naval cadet in August 1853. He saw action during the Crimean War serving in the first-rate HMS "Neptune" in March 1854 and then in the third-rate HMS "Cornwallis" in May 1855. Promoted to midshipman in August 1855, he joined the frigate HMS "Shannon" on the East Indies and China Station in August 1856. He saw action during the Indian mutiny when the ship's crew landed as a naval brigade in December 1857. Kerr was wounded near Cawnpore but still took part in the capture of Lucknow in February 1858. After being mentioned in despatches on 31 March 1858 and promoted to mate on 28 September 1858, he transferred to the royal yacht HMY "Victoria and Albert" in June 1859.