He joined the army on 4 January 1954 as a candidate officer. Geldenhuys obtained a BMil from the University of Pretoria in 1956 before joining 1 Special Service Battalion. In 1965, he was based at the South African Embassy in Luanda, Angola as a Vice-Consul, a position he held until 1968. He was appointed as Army Chief of Staff Intelligence and then Army Chief of Staff Operations. He went on to high command in the South African Army, serving as commander of South West Africa Command from 1977 until 1980, when he was briefly appointed as General Officer Commanding the South West African Territorial Force. Later in 1980, he became Chief of the Army. Geldenhuys was promoted to general and assumed the position of Chief of the South African Defence Force on 31 October 1985. In this role, he took part in negotiations that brought the Border War to an end in 1989, after 23 years of fighting.