Vogel was born in Vienna, Austria. He fled Austria with his parents after the Nazi Anschluß in 1938 and at first studied animal husbandry at the University of Georgia. In the American South, he noted, the racism was as bad as the anti-semitism he witnessed in Europe. Later he received a bachelor's degree from The New School for Social Research in New York.