Born in Victoria of German origins, Scherger graduated from the Royal Military College, Duntroon, before transferring to the Air Force in 1925. He was considered one of the top aviators between the wars, serving as a fighter pilot, test pilot, and flying instructor. He held senior training posts in the late 1930s and the early years of World War II, earning the Air Force Cross in June 1940. Promoted to group captain, Scherger was acting commander of North-Western Area when Darwin suffered its first air raid in February 1942. Praised for his actions in the aftermath of the attack, he went on to lead the RAAF's major mobile strike force in the South West Pacific, No. 10 Operational Group (later the Australian First Tactical Air Force), and was awarded the Distinguished Service Order in September 1944 for his actions during the assaults on Aitape and Noemfoor in Western New Guinea.