Born in Bridgton, Maine, Carter moved to Portland with his family in 1847, and again in 1857, to Massachusetts. He was preparing to enter Phillips Academy when Carter enlisted as a private in the 22nd Massachusetts Infantry at the start of the American Civil War and remained with the Army of the Potomac from August 5, 1862, until October 4, 1864. During the war, he took part in the Battle of Antietam, Gettysburg, the Wilderness and the Siege of Petersburg as well as other minor engagements.