Downers Grove, 1832-1982