On his return to active duty, Sweet served as first sergeant of the 4th Recruit Training Battalion at Parris Island; on the Marine detachment at the Naval Ordnance Plant, Macon, Georgia; and on the Marine Detachment of the . He also served two separate tours at Marine Corps Base Camp Lejeune, first as an infantry chief, and following the Korean War, as regimental sergeant major of the 2nd Marine Regiment and field sergeant major of the 2nd Marine Division, respectively. He requested duty in Korea on the outbreak of hostilities there, and in 1951 saw combat as rifle company first sergeant with the 5th Marine Regiment. He was wounded in action that October and earned the Navy Commendation Medal with valor device.