A native of New York, Belknap graduated from Princeton University in 1848, studied law with a Georgetown attorney, and passed the bar in 1851. He moved to Iowa, where he practiced law in partnership with Ralph P. Lowe. Belknap entered politics as a Democrat and served one term in the Iowa House of Representatives. When the American Civil War broke out in 1861, Belknap joined the Union Army. A tall burly man, Belknap was a natural Union Army leader and recruiter. A veteran of the Iowa Militia who had attained the rank of captain, he was commissioned as a major in the 15th Iowa Volunteer Infantry. He took part in numerous engagements, including Shiloh and Corinth, served as a regimental, brigade, division, and corps commander, and served in high-level staff positions. By the end of the war, Belknap had been promoted to brigadier general of volunteers and received a brevet promotion to major general.