Madison Square Park, a green space diagonally across from the building to the southwest, was opened in 1844. The space was used extensively for cultural and military events, and Madison Avenue on the park's eastern border became an upscale residential row. The site of the New York Life Building was used between 1837 and 1871 as the Union Depot of the New York &amp; Harlem and the New York &amp; New Haven Railroads (now part of the Park Avenue main line). The location then hosted a concert garden named Gilmore's Garden, as well as P.T. Barnum's Hippodrome. The first Madison Square Garden (MSG) was built in 1879 on the northeast corner of Madison Avenue and 26th Street, and was replaced in 1890 by the second Madison Square Garden.