Revert "Add test for removing cache and using instance store instead"

This reverts commit 624bc2f855e96d16445f0fa0c0e46c0efa895cbc.