With his little brother David, James entered the United States Army in 1861, joining Company F of 16th Wisconsin Volunteer Infantry Regiment, upon the outbreak of the American Civil War. He participated in the Battle of Pittsburg Landing, and was discharged in 1862.