Pugo was born in Kalinin, Russian SFSR (now Tver, Russia) into a family of Latvian communists who had left Latvia after Latvia was proclaimed an independent country in 1918 and the Communist side was defeated in the war that followed. His family returned to Latvia after the Soviet Union occupied and annexed it in 1940.