Only count files that actually go in the cache in the cache size check. Fixes #1653.