After graduating from college, Knapp enlisted as a private in the Company I, 16th Vermont Infantry and was quickly promoted to captain of Company F, 17th Vermont Infantry. During his military service, he was wounded at Gettysburg, Spotsylvania, and Petersburg. Knapp rose to the rank of lieutenant colonel and was brevetted a colonel for his gallantry during the Siege of Petersburg.