Teller was born in Vienna, Austria, to Shlomo Meir Teller and his wife Edna (née Lichtenstein). His father had fled to the United States after the Nazi annexation of Austria in 1938, but returned to Vienna after World War II to recover the family business. When Teller was a young boy, he and his parents moved to Stamford, Connecticut.