Born into a peasant family in the Byelorussian SFSR, Golubev grew up during the war years, and spent his early career in the navy, and combining working as a machinist with political activity with the Komsomol. He joined the KGB in 1959 and studied English and Persian, graduating from domestic intelligence work to foreign intelligence as part of the First Chief Directorate. Golubev developed particular expertise in Afghani affairs, and had a role in planning and then carrying out Operation Storm-333, the overthrow of incumbent Afghan president Hafizullah Amin in 1979. As part of the operation Golubev commanded a detachment of Spetsnaz troops during the military assault on Amin's headquarters at the Tajbeg Palace. He then served for a time as KGB resident in Istanbul, before rising through several posts in the KGB's command prior to the dissolution of the Soviet Union.