Make sure we don't OOM when caching a lot of values