Must remove the cache in order to reconfigure it [IMMUTANT-423]