Burges was educated at Winchester College. He was commissioned on 21 October 1893 as a second lieutenant into the Gloucestershire Regiment, and was promoted to lieutenant on 8 July 1897. Following the outbreak of the Second Boer War in late 1899, Burges was with the 2nd battalion of his regiment as they were sent to South Africa in January 1900. He participated in the Relief of Kimberley, operations in the Orange Free State, actions at Poplar Grove, Driefontein, Vet River, Zand River, the operations in the Transvaal, Rhenoster Kop, operations in Orange River Colony and in Cape Colony. For his service in the war, he received the Queen's South Africa Medal with four claps, and the King's South Africa Medal with two clasps. Burges stayed in South Africa until the end of the war in June 1902, and returned to Southampton on the SS "Orcana" in November 1902.