Do not setDocsWithField when retrieving doc values from cache