The heart of the city was fortified according to a 1704 plan by Governor Johnson