Johnson was born in Cleveland, Ohio, on July 8, 1906, the son of a lawyer, Homer Hosea Johnson (1862–1960), and the former Louisa Osborn Pope (1869–1957), a niece of Alfred Atmore Pope and a first cousin of Theodate Pope Riddle. He had an older sister, Jeannette, and a younger sister, Theodate. He was descended from the Jansen family of New Amsterdam, and included among his ancestors the Huguenot Jacques Cortelyou, who laid out the first town plan of New Amsterdam for Peter Stuyvesant. He grew up in New London, Ohio and attended the Hackley School, in Tarrytown, New York, and then studied as an undergraduate at Harvard University where he focused on learning Greek, philology, history and philosophy, particularly the work of the Pre-Socratic philosophers. Upon completing his studies in 1927, he made a series of trips to Europe, visiting the landmarks of classical and Gothic architecture, and joined Henry-Russell Hitchcock, a prominent architectural historian, who was introducing Americans to the work of Le Corbusier, Walter Gropius, and other modernists. In 1928 he met German architect Ludwig Mies van der Rohe, who was at the time designing the German Pavilion for the 1929 Barcelona International Exposition. The meeting formed the basis for a lifelong relationship of both collaboration and competition.