Investigating Context Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style (2025)

Abstract: Retrieval-augmented generation (RAG) improves Large Language Models (LLMs) by incorporating external information into the response generation process. However, how context-faithful LLMs are and what factors influence LLMs’ context faithfulness remain largely unexplored. In this study, we investigate the impact of memory strength and evidence presentation on LLMs’ receptiveness to external evidence. We quantify the memory strength of LLMs by measuring the divergence in LLMs’ responses to different paraphrases of the same question, which is not considered by previous works. We also generate evidence in various styles to examine LLMs’ behavior. Our results show that for questions with high memory strength, LLMs are more likely to rely on internal memory. Furthermore, presenting paraphrased evidence significantly increases LLMs’ receptiveness compared to simple repetition or adding details. These findings provide key insights for improving retrieval-augmented generation and context-aware LLMs. Our code is available at https://github.com/liyp0095/ContextFaithful.

## introduction retrieval-augmented generation (rag) @xcite has gained increasing popularity as it improves the performance of large language models (llms) by integrating external information during the generation process, particularly when the model's internal knowledge is insufficient or outdated @xcite @xcite @xcite . it raises the importance of the study of how context-faithful llms are. in this study, we explore whether llms are context-faithful when encountering external information, particularly when that information conflicts with the llms' internal memory. to investigate the issue of context faithfulness, there are two main approaches to creating knowledge conflict contexts. one approach @xcite is entity substitution, which replaces the gold entity in context with a similar one. another approach @xcite involves generating counter-memory evidence with llms, and these studies have shown that llms are generally receptive to external evidence as long as it is coherent. these methods, however, overlook some important aspects of the task. first, previous work @xcite provides long contexts to llms, which can be challenging for llms to understand @xcite . it makes the test difficult to distinguish whether the cause of llm behavior is due to knowledge conflicts or lacking comprehension ability. for example, in figure 1 (a), the llm may overlook both contexts, but the reason why context 1 is ignored could be a lack of comprehension rather than knowledge conflicts. second, different llms are trained with different data and are likely to obtain different knowledge. thus, testing llms on the same dataset may be unfair. the test may favor llms with less knowledge. as shown in figure 1 (b), llms with strong memory are less likely to be correct. to address these issues, we introduce a method to quantify the memory strength of llms and generate short evidence in various styles to examine llms' behavior. inspired by @xcite , we assess memory strength by measuring the divergence in llms' responses to different paraphrases of the same question. intuitively, an llm demonstrates high memory strength when it consistently provides the same answer across all paraphrased versions of a question. for evidence styles, we classify them into direct and indirect forms: direct evidence provides a straightforward answer to the question, while indirect evidence incorporates additional details to support the answer. unlike @xcite that focuses on how different types of evidence influence llms' behavior, we study how the presentation of evidence affects its convincing availability. through these methods, we analyze the relationship between context faithfulness and llm memory strength, and we explore the impact of different evidence styles on context faithfulness. our conclusions are as follows: ## experiments in this study, we aim to investigate two key research questions. 1) does memory strength have an impact on the context faithfulness of llms? 2) does the style of evidence affect the context faithfulness of llms? these research questions are explored in section 4.2 and 4.3, respectively. we also provide additional studies in appendix b, which includes a study about the impact of option order and a case study. ## conclusion we investigate how context-faithful llms are to external evidence across two datasets, popqa and nq datasets, using llama2-7b, llama2-70b, chatgpt, gpt-4, llama3.2-3b and claude3.5. our findings highlight the critical role of memory strength in shaping llm behavior. there is a clear positive correlation between memory strength and memory answer ratio. furthermore, we demonstrate that paraphrasing significantly enhances the context faithfulness of llms across various models and datasets. these findings offer valuable insights for advancing research in retrieval-augmented generation and context-based llm applications.