Stephens was born in New York City, the son of Xenia and Charles J. Stephens, a former vice president of General Products, a chemical company in Mexico. Both his parents were secular Jews. His mother was born in Italy at the start of World War II to Jewish parents who had fled Nazi Germany. His paternal grandfather, Louis Ehrlich, was born in Kishinev (today Chișinău, Moldova) in 1901. He fled with his family to New York after a pogrom and changed the family surname to Stephens (after poet James Stephens). Louis Stephens moved to Mexico City, where he founded General Products and built his fortune. He married Annette Margolis and had two sons, Charles and Luis. Charles married Xenia. They moved to Mexico City with their newborn son, Bret, to help run the chemical company, inherited from Louis. Bret was raised in Mexico City and is fluent in Spanish. As a teenager, he attended boarding school at Middlesex School in Concord, Massachusetts.