With the outbreak of the Civil War, Smith joined the Union army and became the colonel of the 73rd Ohio Infantry, a regiment that was raised in Chillicothe in November 1861 and trained at nearby Camp Logan. Among his volunteer soldiers was Pvt. George Nixon III, the great-grandfather of future President Richard Nixon. Smith and his regiment saw action in western Virginia, fighting at the Battle of McDowell and the Battle of Cross Keys. During the late summer, as a part of the Army of Virginia, the 73rd OVI fought at the Second Battle of Bull Run near Manassas, Virginia.