After the start of the Civil War, Semmes was appointed colonel of the 2nd Georgia Infantry. He was promoted to brigadier general on March 11, 1862. During the Peninsula Campaign, he was a brigade commander in Brig. Gen. John B. Magruder's Corps in the defense of Richmond. Rushed northward at the start of the Maryland Campaign, Semmes's brigade rejoined the Army of Northern Virginia in the division of Maj. Gen. Lafayette McLaws just as it was entering Maryland. His men participated in the holding action at Crampton's Gap during the Battle of South Mountain. At Sharpsburg, Semmes's brigade was a key part of General McLaws's strong counterattack that stunned the Union II Corps. In early November, his brigade was reorganized so that it only contained Georgia regiments. Held in reserve at the Battle of Fredericksburg, Semmes's reconstituted brigade served well at Chancellorsville, where it blunted the advance of an entire VI Corps division, and at Salem Church.