With Louisiana's secession in 1861, Stafford formed the "Stafford Guards", a militia unit that was soon mustered into Confederate service as Company B, 9th Louisiana Infantry with Stafford as its first captain. Stafford rose through the ranks and became the regiment's colonel in early 1862 when Richard Taylor was promoted. With the death of Brig. Gen. William E. Starke early in the Battle of Antietam, Stafford was elevated temporarily to command of the brigade. He suffered a minor wound in his foot during the battle, but soon recovered.