Revert "fix #3759: OOM while loading caches"

This reverts commit 2f3e8ddd18303123a1b81995357f27461fa1d586, as the
symptom of the out of memory error might not be its cause, and it
slows down the loading of lists.