Banner was born to Jewish parents in Stanislau, Austria-Hungary (now Ivano-Frankivsk, Ukraine). He studied for a law degree at the University of Vienna, but decided instead to become an actor. In 1938, when he was performing with an acting troupe in Switzerland, Adolf Hitler annexed Austria to Nazi Germany. Banner emigrated to the United States, where he rapidly learned English.