The walls were raised over the centuries with first stones and then the less expensive brick