Levin was born in Vilna, then part of the Second Polish Republic. He was the son of Chaim Levin, a local merchant and gourmet food shop owner. After the Nazi invasion of the Soviet Union in June 1941, the family initially fled to the village of in present-day Belarus. When it became unsafe, the family returned to Vilna and lived in Vilna Ghetto. In 1943, when he was 16 years old, Levin and his older brother Shmuel joined the Avengers (Nokmim) partisan group of Abba Kovner. During the war, the Avengers impaired railways, destroyed train cars, cut communication wires, bombed bridges, and injured and killed Nazis. His small stature allowed him to hide, spy, scout, and escape detection easily. His brother was killed during one of the raids though circumstances are not known.