Upon the outbreak of the Civil War, Zimmerman left the Academy and tried to join the Union Navy, but was refused because he was too young. After some modifications, he was accepted, becoming a Master's mate and was assigned to the Mississippi River Squadron. While in the Navy, his ship was struck by torpedo which killed several seamen and threw him into the Yazoo River. He participated in the Battle of Saint Charles and Duvall's Bluffs and served with distinction during the capture of Arkansas Post in 1863, after which he was promoted to Ensign. During the Vicksburg campaign in his hometown, Zimmerman was placed in charge of a mortar boat and sent up the Yazoo River, making a successful run against the Confederate blockade at Vicksburg in April 1863. During the campaign, he was involved in several missions and was promoted to acting Master and Officer of the side wheel steamer USS "Ouachita" after the surrender of Vicksburg which he held until the end of the War. During the siege, his former home and his father's business were destroyed.