When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories (2023)

Abstract: Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the difficulty of encoding a wealth of world knowledge in their parameters. This paper aims to understand LMs’ strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments on two open-domain entity-centric QA datasets: PopQA, our new dataset with 14k questions about long-tail entities, and EntityQuestions, a widely used open-domain QA dataset. We find that LMs struggle with less popular factual knowledge, and that retrieval augmentation helps significantly in these cases. Scaling, on the other hand, mainly improves memorization of popular knowledge, and fails to appreciably improve memorization of factual knowledge in the tail. Based on those findings, we devise a new method for retrieval-augmentation that improves performance and reduces inference costs by only retrieving non-parametric memories when necessary.

## introduction large language models (lms; @xcite ) have been shown to be competitive on diverse nlp tasks, including knowledgeintensive tasks that require fine-grained memorization of factual knowledge @xcite . meanwhile, lms have also been shown to have limited memorization for less frequent entities @xcite , are prone to hallucinations @xcite , and suffer from temporal degradation @xcite . incorporating non-parametric knowledge (i.e., retrieved text chunks) largely helps address those issues stemming from reliance on lms' parametric knowledge-knowledge stored 1 our code and data are available at https://github. com/alextmallen/adaptive-retrieval . in their parameters @xcite )-but it is unclear whether it is strictly superior or complementary to parametric knowledge. understanding when we should not trust lms' outputs is also crucial to safely deploying them in real-world applications @xcite . this work conducts a large-scale knowledge probing of lms on factual knowledge memorization, to understand when we should and should not rely on lms' parametric knowledge, and how scaling and non-parametric memories (e.g., retrievalaugmented lms) can help. in particular, we aim to address the following research questions: we hypothesize that factual knowledge frequently discussed on the web is easily memorized by lms, while the knowledge that is less discussed may not be well captured and thus they require retrieving external non-parametric memories. we evaluate ten large lms of three families (i.e., gpt-neo, opt, and gpt-3) with varying scales on the open-domain question answering (qa) task in a zero-or few-shot prompting manner. we construct a new dataset, popqa, consisting of 14k questions to cover factual information in the long tail that might have been missed in popular qa datasets @xcite . we use wikipedia page views as a measure of popularity and convert knowledge triples from wikidata, with diverse levels of popularity, into natural language questions, anchored to the original entities and relationship types. we also use entityquestions @xcite , an open-domain qa dataset with a long-tail distribution. on both datasets, lms' memorization (rq 1 ) is often limited to the popular factual knowledge and even gpt-3 davinci-003 fails to answer the majority of the long-tail questions. moreover, on such questions, scaling up models does not significantly improve the performance (e.g., for the 4,000 least popular questions in popqa, gpt-j 6b has 16% accuracy and gpt-3 davinci-003 has 19% accuracy). this also suggests that we can predict if lms memorize certain knowledge based on the information presented in the input question only. we next investigate whether a semi-parametric approach that augments lms with retrieved evidence can mitigate the low performance on questions about less popular entities (rq 2 ). nonparametric memories largely improve performance on long-tail distributions across models. specifically, we found that retrieval-augmented lms are particularly competitive when subject entities are not popular: a neural dense retriever (izacard et al., 2022a)-augmented gpt-neo 2.7b outperforms gpt-3 davinci-003 on the 4,000 least popular questions. surprisingly, we also find that retrieval augmentation can hurt the performance of large lms on questions about popular entities as the retrieved context can be misleading. as a result, we devise a simple-yet-effective retrieval-augmented lm method, adaptive retrieval, which adaptively combines parametric and non-parametric memories based on popularity (rq 3 ). this method further improves performance on popqa by up to 10%, while significantly reducing the inference costs, especially with larger lms (e.g., reducing gpt-3 api costs by half), indicating the potential for future research in more efficient and powerful retrieval-augmented lms. ## evaluation setup we evaluate lms' ability to memorize factual knowledge through closed-book qa tasks with fewshot samples. we evaluate lms on our new dataset, popqa (figure 2 ), and entityquestions, both of which have long-tail distributions (figure 3 ). ## memorization depends on popularity and relationship type we evaluate a range of lms with varying numbers of parameters, to quantify how much factual knowledge they memorize and how different factors affect those memorization behaviors (rq 1 ). ## non-parametric memory complements parametric memory our analysis indicates that even the current stateof-the-art lms struggle with less popular subjects or certain relationship types, and increasing the model size does not lead to further performance improvements. in light of this, we extend our analysis to non-parametric sources of knowledge, as outlined in section (rq 2 ). specifically, we investigate the effectiveness of retrieval-augmented lms @xcite , which leverage non-parametric memories (i.e., retrieved text) to improve performance. ## adaptive retrieval: using retrieval only where it helps while incorporating non-parametric memories helps in long-tail distributions, powerful lms have already memorized factual knowledge for popular entities, and retrieval augmentation can be harmful. as outlined in (rq 3 ), can we achieve the best of both worlds? we propose a simple-yeteffective method, adaptive retrieval, which decides when to retrieve passages only based on input query information and augments the input with retrieved non-parametric memories only when necessary. we show that this is not only more powerful than lms or retrieval-augmented lms always retrieving context, but also more efficient than the standard retrieval-augmented setup. ## discussion and conclusions this work conducts large-scale knowledge probing to examine the effectiveness and limitations of relying on lms' parameters to memorize factual knowledge and to understand what factors affect factual knowledge memorization. our results show that memorization has a strong correlation with entity popularity and that scaling up models on long-tail distributions may only provide marginal improvements. we also demonstrate that non-parametric memories can greatly aid lms on these long-tail distributions, but can also mislead lms on questions about well-known entities, as powerful lms have already memorized them in their parameters. based on those findings, we devise simple-yet-effective adaptive retrieval, which only retrieves when necessary, using a heuristic based on entity popularity and relationship types. our experimental results show that this method is not only more powerful than lms or previous retrieval-augmented lms but also more efficient.