Berzin was born into a Latvian peasant family in the Russian Empire. He completed the pedagogical seminar, and then worked as a village teacher, and began spreading revolutionary propaganda among the peasants. He joined the Russian Social Democratic Labour Party in 1902. In 1904, he was arrested and exiled to Olonets Governorate, but he escaped in 1905, and worked as a political agitator in the Baltic region during the 1905 Russian Revolution. In December 1905, he was arrested again, by the punitive expedition led by General Orlov. Released from prison in 1907, he settled in St Petersburg and took on the job of secretary of the St Petersburg committee of the Russian Social Democratic Labour Party (RSDLP). In 1907 he was a delegate of the 5th Congress of the Russian Social Democratic Labour Party, in London. He emigrated in 1908, and lived Switzerland, France, Belgium, Britain and the USA.