Born in Vienna in 1937, Jaray grew up in rural Worcestershire, England, where her parents emigrated in 1938 after the annexation of Austria by Nazi Germany made it unsafe for people of Jewish descent to live there. Jaray's father Franz Ferdinand Jaray was a chemical engineer and industrial inventor. Her mother, Pauline Arndt, attended Art School in Vienna. Jaray's great aunt was the gallerist Lea Bondi Jaray who was responsible for bringing many of the German Expressionists to London. Noting the influence of Gustav Klimt, leader of the Vienna Secessionists, Jaray has written that ‘He was one of the very first artists I learned about as a teenager.’