Rufus Herring attended Davidson College, Davidson, North Carolina where he was initiated into Pi Kappa Phi fraternity on February 3, 1939. After graduating in the spring of 1942, Rufus Herring enlisted in the U.S. Naval Reserve and subsequently attended the Naval Reserve Midshipman School in New York City. After being commissioned in the rank of ensign in December 1942, he received diesel engine instruction at the University of Illinois, followed by orders to the Amphibious Training Base at Solomons, Maryland. In August 1943 he assumed command of the newly completed infantry landing craft and remained her commanding officer during the next year and a half. Herring was promoted to lieutenant, junior grade, in March 1944 and between February and July of that year he participated in the invasions of Kwajalein, Saipan, Tinian and Guam.