When Context Leads but Parametric Memory Follows in Large Language Models (2024)

Abstract: Large language models (LLMs) have demonstrated remarkable progress in leveraging diverse knowledge sources. This study investigates how nine widely used LLMs allocate knowledge between local context and global parameters when answering open-ended questions in knowledge-consistent scenarios. We introduce a novel dataset, WikiAtomic, and systematically vary context sizes to analyze how LLMs prioritize and utilize the provided information and their parametric knowledge in knowledge-consistent scenarios. Additionally, we also study their tendency to hallucinate under varying context sizes. Our findings reveal consistent patterns across models, including a consistent reliance on both contextual (around 70%) and parametric (around 30%) knowledge, and a decrease in hallucinations with increasing context. These insights highlight the importance of more effective context organization and developing models that use input more deterministically for robust performance.

## introduction large language models (llms) have significantly advanced the capabilities of natural language processing. when generating responses, llms can use the contextual information provided in a prompt along with or instead of the parametric knowledge embedded during pretraining @xcite @xcite . in order to generate accurate and coherent responses, llms need to effectively combine their parametric knowledge with provided contextual information, and understanding the balance between these two sources of information is crucial @xcite . most prior work has explored this question through the lens of counterfactual data where the parametric and contextual knowledge are in conflict @xcite . some studies suggest that the models prefer contextual knowledge, while others suggest they prioritize parametric knowledge @xcite . in many real-world scenarios such as question answering or summarization, however, contextual knowledge may complement rather than conflict with parametric knowledge. understanding how models integrate different sources of knowledge in knowledge-consistent scenarios is critical. moreover, while parametric knowledge enables llms to generate coherent text, it also contributes to the risk of hallucinations where responses are coherent and confident but factually incorrect or irrelevant @xcite @xcite . as such, our work also explores the hallucination tendency in knowledgeconsistent scenarios. real-world applications often present llms with varying amounts of context. building robust models requires understanding how effectively they leverage this information. this research investigates how the volume of context influences knowledge preference and hallucination tendencies in llms within a question answering (qa) framework. we study nine widely used llms to determine how different models navigate varying context sizes in knowledge-consistent settings, aiming to answer the main question: how do llms prioritize contextual and parametric knowledge when generating responses? additionally, we investigate the likelihood of generating hallucinations with increasing context, which parts of contexts are used, how similar are various types of knowledge, and further analyses exploring potentially unseen knowledge. figure 1 provides an overview of our investigation pipeline. we create a new dataset, wikiatomic, of atomic sentences serving as our context. we systematically adjust the amount of factual context provided to the models and prompt them in an open-ended qa task to study the responses generated by various models. then, we measure the amount of contextual knowledge models recall, the additional parametric knowledge they rely on, and the extent of hallucinations found in their responses. our results suggest that all models behave very similarly, integrating up to 30% parametric knowledge in their responses. for smaller contexts, models recall information from all parts of the context. however, for longer contexts, they predominantly focus on the first half, potentially missing key information from other parts of the input. the sequence of information in the responses largely mirrors the context and the additional parametric knowledge included is moderately similar to the contextual knowledge. lastly, the hallucination score decreases as more context is added. these findings highlight the importance of effective context organization and models that utilize input more predictably. our key contributions include: we structure our investigation as an open-ended question answering task to gain deeper insights into how llms utilize and integrate contextual knowledge with their parametric knowledge. by incrementally increasing the amount of context, we analyze the models' behavior, focusing on their ability to prioritize different types of information and their tendency toward factual hallucination in knowledge-consistent scenarios. we next introduce some key terminology used throughout the paper. topic: this is the title of the wikipedia article. the wikiatomic dataset includes 200 distinct articles, and questions are asked about these topics. atomic sentence: this is a sentence that contains a single piece of information that cannot be broken down into simpler components without losing its meaning @xcite . context: for each topic, a context consists of k atomic sentences provided in a prompt. in our experiments, 2 ≤ @xmath0, where contexts consist of increments of 2 sentences from 2 to 30 (e.g., 2, 4, 6, etc.), followed by increments of 5 sentences until reaching a total of 50 sentences. for each topic, this results in contexts of 20 different sizes, creating a total of 4,000 topic-context instances. response: given the context, the model generates a response that includes some degree of both contextual and parametric knowledge. this response is then atomized. a sentence directly derived from the provided context is considered as contextual (local) knowledge. in contrast, a sentence that is not entailed from the context is consid-ered as parametric (global) knowledge. following prior work @xcite , contextual knowledge comes from external sources provided during inference, while parametric knowledge is knowledge encoded (or "memorized") in the model parameters. as such, if a sentence introduces information not present in the context, it is considered a parametric knowledge sentence. ## wikiatomic dataset we created a novel dataset -wikiatomic -consisting of 200 articles from wikipedia. we selected wikipedia as the basis for creating our knowledgeconsistent dataset because its extensive data has historically been integral to the training of numerous llms, explicitly in models like gpt-2, gpt-3, bert, t5, and bloom, and implicitly in largescale aggregate corpora such as the pile @xcite and common crawl (luccioni and @xcite . this extensive use allows us to reasonably assume that information from wikipedia, particularly from older articles, is present in the pretraining data of several llms we study. ## results and analysis now, we present and discuss the results of our experiments. figure 4 shows how four models prioritize sources of knowledge (similar results were obtained from other models, plots included in appendix d). the plots display changes in the average number of local, global, and total atomic sentences in responses as context increases. somewhat surprisingly, we found that all models (except phi-3 in certain cases) show consistently similar patterns. this uniformity among the models suggests a shared underlying mechanism in processing and responding to contextual information. with no context provided (@xmath1, models tend to be most verbose as expected from an initially open prompt. they also peak in parametric knowledge, which drops drastically with the first four contexts. interestingly, total response lengths generally match context lengths. from @xmath2, the total sentences and local contextual knowledge steadily increase as context increases. more contexts were utilized but none of these models utilized 100% of them in their responses. global facts show a consistent trend: after the initial open-ended prompt, the global knowledge drops sharply and remains low, though never to zero, with some models slowly increasing or decreasing but always including some global knowledge in their responses. ## discussion understanding how llms utilize contextual and parametric knowledge is important for real-world knowledge-consistent use cases. we summarize our key findings and their implications: ## related work contextual grounding a persistent challenge for llms lies in reconciling contradictory or superseded information between the provided context and their internal knowledge base @xcite , with most recent work exploring this interplay between local and global knowledge in knowledge-conflict scenarios using counterfactual datasets @xcite @xcite . the preference for contextual or parametric knowledge in counterfactual settings is not straightforward. larger models seem better at adapting to counterfactual context, while smaller ones often prioritize their learned knowledge @xcite . when faced with counterfactual prompts, llms adjust their responses to align with the given context, even if it contradicts their pretrained parametric knowledge @xcite . in other cases, however, steering llms away from generating outputs aligned with their vast but potentially inaccurate pretrained knowledge, even when contradicted by the context, has been challenging @xcite . our work contributes to contextual grounding studies by prompting llms in a knowledge-consistent scenario and is the first to systematically analyze how the amount of atomic information in a prompt affects context utilization. hallucination there has been extensive work on hallucination detection with methods falling into two general categories: internal parameter based methods @xcite @xcite and external response based methods @xcite @xcite @xcite . similar to our work, @xcite analyze the tendency of llms to generate factual hallucinations in the presence of varying local context in question answering. unlike our approach, they study the effects of entirely switching out local context, whereas our approach incrementally increases the amount of context while comparing hallucination tendency between these responses. ## conclusion understanding how llms handle different context sizes is crucial for developing robust models. our evaluation of nine widely used llms with our wikiatomic dataset showed that all models process context similarly, balancing contextual and parametric knowledge, while adjusting response lengths consistently. they never use all provided contexts, always include some parametric knowledge, and prioritize information sequentially. as context increases, while contextual knowledge remains similar, parametric knowledge becomes more aligned with the context, and hallucination rates decrease. these insights highlight the importance of organizing context effectively and developing models that utilize input more deterministically.