Born in St Petersburg, in the Russian Empire in 1912, Zakharov began his naval career in 1930 with service with Coastal Defence Artillery Brigades, before moving into the political branch. He was military commissar at the Pacific Fleet's base at Nikolayevsk-on-Amur during the German invasion of the Soviet Union, and during the war served on the military councils of the Pacific Fleet, the Volga Military District and the Northern Fleet. He followed this with work as a propagandist and as a senior instructor, and later as inspector of the fleet's political administration.