Randall enlisted as a private in the 4th Pennsylvania Volunteer Infantry in the April 1861. He then accepted appointment to the 4th U.S. Infantry as second lieutenant in October. He served in New York and Washington, D.C., and in the field with the Army of the Potomac. He was a second lieutenant at the Siege of Yorktown, Battle of Gaines' Mill, Battle of Malvern Hill, Second Bull Run (where he commanded Company C), and Fredericksburg; and a first lieutenant at the Battle of the Wilderness. He was appointed a brevet captain for gallant service in the Battle of Antietam. He served as a major in the 14th New York Heavy Artillery Regiment, part of Major General Ambrose Burnside's IX Corps on the Richmond-Petersburg Front in 1864, and was commander of the regiment at the Battle of the Crater. The Dansville Advertiser reported on 7 September that he assumed command of the regiment on 18 August. This article also reports that he was wounded in the battle on 19 August. He was appointed a brevet lieutenant colonel, and then colonel for gallantry in the Battle of Fort Stedman. He mustered out of the Volunteer service on 26 August, and was appointed captain in the 4th US Infantry on 23 September 1865. Subsequently, he served at Davids' Island Military Reservation, New York Harbor, where he was in December 1867.