Erlanger was born on January 5, 1874, at San Francisco, California. His family was Jewish and his parents both immigrated from the Kingdom of Württemberg, Germany and met in California during the Gold Rush. Joseph was the sixth of seven children born to the couple. He completed his Bachelor of Science in Chemistry from the University of California, Berkeley in 1895. He then completed his M.D. in 1899 from the Johns Hopkins School of Medicine in Baltimore, Maryland, where he finished second in his class.