Saragih began his military career during the Japanese Occupation of the Dutch East Indies. He supported the Indonesian government during the period of the State of East Sumatra and the Revolutionary Government of the Republic of Indonesia. In 1965, he became the Commander of the Section 1 in the Military District Commander, and participated in the Indonesian mass killings of 1965–66.