When South Carolina seceded in 1861, he joined Hamptons Legion as a captain of infantry. At the First Battle of Manassas, he was given control of the Legion after his superior officers were disabled. By 1862 Gary had been elected as lieutenant colonel of the infantry battalion in the Legion and was promoted to colonel when given control of a regiment. Hampton's Legion came under the command of General Longstreet and was active in the battles of Virginia through mid-1863 before being transferred to help the Army of Tennessee in the latter part of the year.