Harriman joined the Union Army in 1862 during the American Civil War and was assigned to the 30th Wisconsin Volunteer Infantry Regiment. In 1864, he helped to organize 37th Wisconsin Volunteer Infantry Regiment and was then promoted to Colonel and given command of the regiment. The regiment was later attached to the IX Corps under the command of future U.S. Senator Ambrose Burnside. Harriman and the 37th would later take part in the Battle of the Crater during the Siege of Petersburg. He was mustered out of the volunteers on July 17, 1865.