With the outbreak of the Civil War, Hillyer raised a Walton County company known as the "Hillyer Rifles" in the late spring of 1861. The men were mustered into the Confederate Army on June 13, with Hillyer elected as the captain of what became Company C of the 9th Georgia Volunteer Infantry. He and the regiment were sent by train to Virginia and assigned to the newly created brigade of George "Tige" Anderson in the newly organized Army of Northern Virginia. Hillyer saw extensive fighting at Fredericksburg (briefly commanding the regiment) and Gettysburg, where he fought at the famed "Wheatfield" on July 2, 1863. The 9th Georgia lost half of its 340 men in the fight, and Hillyer's company suffered considerable losses. With all the senior officers wounded or killed, Hillyer assumed command of the regiment for the rest of the Gettysburg Campaign, and wrote the official report of the 9th Georgia's service in the battle. Union General Joseph Hooker remarked that Confederate Captain Hillyer was "a most able, efficient &amp; gallant officer".