More tolerant caches

This change give us a chance to clean up an internal cache after its input (e.g. the BlockShape) was set to an invalid value.