When the Civil War erupted, Indiana began organizing similar militia groups and integrating them with companies of new recruits into full regiments. Despite his age (approaching 40), Harrow enlisted in the Union Army. Through his political connections, Harrow was named as major of the 14th Indiana Infantry, serving under the command of Col. Nathan Kimball. He served in what became West Virginia at the battles of Cheat Mountain and Greenbrier River in late 1861, then participated in several fights against troops of Stonewall Jackson during the Valley Campaign as lieutenant colonel of the 14th. Following the Battle of Kernstown, where Harrow lost 54 men in action, he was promoted to full colonel.