James enlisted in the 16th Wisconsin Volunteer Infantry Regiment upon the outbreak of the American Civil War on October 13, 1861 at the age of 13. In 1864 he was captured and taken to Andersonville Prison. He survived Andersonville, and when he was mustered out July 18, 1865, it was with the brevet rank of Captain, backdated October 3, 1861, "for conspicuous bravery manifested by him at the battles of Corinth and Atlanta."