Stern was born in Schweizer-Reneke, a small town in the Transvaal, of German-Jewish parents. Her father was interned in a concentration camp by the British during the South African War because of his pro-Boer leanings.[1] Irma and her younger brother, Rudi, were thus taken to Cape Town by their mother. After the war, the family returned to Germany and would constantly travel. This travel would influence Irma's work.