Following the secession of several Southern states and the outbreak of hostilities in early 1861, Davidson resigned from his recent commission as a captain in the U.S. Army (he had been promoted to that rank on May 13 and was formally dropped from the army for being AWOL on July 30, 1861). Meanwhile, he traveled to the South and entered the service of the Confederate States of America as a captain on March 16, 1861 and as a major in April 1861. He served in the Adjutant General's office and then on the Inspector General's staff. He was a staff officer for several leading generals including John B. Floyd, Albert S. Johnston, Simon Bolivar Buckner and William W. Mackall. He was promoted to colonel and was with Brigadier General Mackall among the forces which surrendered following the Battle of Island Number Ten on April 7, 1862. After being exchanged on August 27, 1862, he was appointed as the colonel in command of the military post at Staunton, Virginia, in the Shenandoah Valley.