During Latimer's second year of studies, the Civil War broke out and Latimer left VMI to serve the Southern Confederacy. He first served as cadet drillmaster for the Richmond Hampden Artillery during the spring and summer of 1861. In the fall, he was commissioned as a first lieutenant and saw action with Maj. Gen. Richard S. Ewell's division in the Shenandoah Valley Campaign of 1862. After distinguishing himself with his battery at such battles as First Winchester and Cedar Mountain, Latimer was promoted to command of the battalion that had previously belonged to A. R. Courtney. After commanding the battalion at Fredericksburg in December 1862, Latimer was promoted to major in March 1863. Ewell referred to him as the "Young Napoleon", but his contemporaries noted his youth and small, slight stature by calling the "Boy Major." He became an officer in Maj. Richard Snowden Andrews's Battalion of Maj. Gen. Edward "Allegheny" Johnson's Division.