Born in Staunton, Virginia to then-Commonwealth's attorney (prosecutor) William Henry Harman and his wife, the former Margaret Susan Garber. His family had a tradition of military service, from his great-great grandfathers John Cunningham and George Moffett during the American Revolutionary War. His father served in the Virginia militia and U.S. Army during the Mexican-American War. During the American Civil War his father became an officer in the Virginia militia and Confederate States Army, helped to seize the Harpers Ferry arsenal and later died at the Battle of Waynesboro. Three uncles also served as Confederate officers, two under CSA General Stonewall Jackson; his uncle CSA Lt. L.T. Harman died of typhoid fever in the conflict. His other uncles became prominent in rebuilding Virginia's railroads. Harman would graduate from the Virginia Military Institute in 1874.