The church was designed in the early English Neo-Gothic style; the architect has not been identified. The sanctuary is set back from the street behind a garden which creates a facsimile of the English countryside and which has long been an oasis for New Yorkers, who relax in the garden, pray in the chapel, or enjoy free weekday concerts in the main church. The complex has grown somewhat haphazardly over the years, and for this reason it is sometimes called the "Holy Cucumber Vine". The sanctuary had a guildhall, transepts, and a tower added to it in 1852, and the lych-gate, designed by Frederick C. Withers, was built in 1896. Chapels were added in 1906 (lady chapel) and 1908 (mortuary chapel).