On May 19, 1863 and May 22, 1863, Rutherford led his regiment in two charges at the Vicksburg defenses at the start of the Siege of Vicksburg. The Union forces suffered significant casualties so Grant decided to besiege the city and proceed by maneuver instead of making further frontal attacks. The 97th Illinois Infantry Regiment then took part in the siege, culminating with the Confederate surrender on July 4, 1863.