Fix bugs in the cache clearing.

Any entry being used elsewhere in the program will be kept into the
cache.