Only clean cache entries on non-misses