Gordon was born in Charlestown, Massachusetts. He moved to Framingham, Massachusetts, at the age of five with his widowed mother. He graduated from the United States Military Academy in 1846, 43rd in a class of 59 cadets. He served under Lt. Gen. Winfield Scott in the Mexican–American War, earning the brevet of first lieutenant for gallantry at Cerro Gordo. He resigned from the army in 1854. After taking a course in the Harvard Law School, he practiced law in Boston.