In September 1862, Wheeler enrolled in the 150th New York Volunteer Infantry Regiment, where he was mustered in as first lieutenant in Company E. In December 1863, he was promoted to captain of his company. He was also made brevet major. He was mustered out with his company in June 1865. His regiment's first battle was the Battle of Gettysburg, and he fought in a number of other battles, including Sherman's March to the Sea.