Born the son of George Milne and Williamina Milne (née Panton) and educated at MacMillan's School in Aberdeen and the Royal Military Academy, Woolwich, Milne was commissioned into the Royal Artillery on 16 September 1885. He was initially posted to a battery at Trimulgherry in India and then joined a battery at Aldershot in 1889 before being posted back to India to a battery at Meerut in 1891. Promoted to captain on 4 July 1895, he joined the garrison artillery in Malta and then took part in the Suakin Expedition in 1896. Next he was appointed battery captain at Hilsea and then attended the Staff College, Camberley in 1897. There he became a friend of his classmate William Robertson. He took part in the Nile Expedition in 1898, seeing action at Omdurman and scoring a direct hit on the Mahdi's tomb with his battery. He served in the Second Boer War in South Africa, where he was appointed Deputy Assistant Adjutant-General on 18 February 1900, and was promoted to major on 1 November 1900. He was mentioned in despatches on 2 April 1901, and awarded the Distinguished Service Order (DSO) in the South Africa Honours list published on 26 June 1902. Following the end of the war in June 1902, Milne received the brevet rank of lieutenant-colonel on 22 August 1902 (the honour was gazetted in the October 1902 South Africa honours list), and returned to the United Kingdom on the SS "Orotava" which arrived at Southampton in early September.