Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models (2024)

Abstract: Generative language models often struggle with specialized or less-discussed knowledge. A potential solution is found in Retrieval-Augmented Generation (RAG) models which act like retrieving information before generating responses. In this study, we explore how the Atlas approach, a RAG model, decides between what it already knows (parametric) and what it retrieves (non-parametric). We use causal mediation analysis and controlled experiments to examine how internal representations influence information processing. Our findings disentangle the effects of parametric knowledge and the retrieved context. They indicate that in cases where the model can choose between both types of information (parametric and non-parametric), it relies more on the context than the parametric knowledge. Furthermore, the analysis investigates the computations involved in how the model uses the information from the context. We find that multiple mechanisms are active within the model and can be detected with mediation analysis: first, the decision of whether the context is relevant , and second, how the encoder computes output representations to support copying when relevant.

## introduction natural language processing (nlp) has made significant progress in recent years, mostly because of the development of large language models (llms). these models can perform a variety of tasks with minimal supervision. while pure generative llms are often capable of answering basic factual questions @xcite , simply by reciting information memorized from their training sets and stored in the model parameters, they are much less reliable in scenarios that require more specialized knowledge that is discussed less frequently on the web @xcite . 1 the code used in this project is available at our github repository: github.com/m3hrdadfi/rag-memory-interplay. the table shows the predicted outputs (o) and the corresponding probability p o assigned by atlas. the first row represents the baseline context. when the counterfactual "milan" is added, if the model answers "milan," this shows that the model relies on its non-parametric mechanism rather than its parametric memory. rag models combine retrieval-based and generative approaches and have been proposed as a way to address some of the drawbacks of basic generative language models in information-seeking scenarios. they improve the factual accuracy of answers to low-frequency queries @xcite as well as prediction consistency @xcite . these models employ a retriever to gather relevant external information and a generator to produce responses. this duality helps models generate text using both internal knowledge stored in the model's parameters (parametric memory) and external information (non-parametric memory), as shown by @xcite . an example of this is the atlas model @xcite , which connects a language model to an external source of information and allows atlas to handle tasks that need up-to-date or specialized knowledge. despite the success of rag systems in knowledge-intensive tasks, several aspects of these systems remain poorly explored in the research community. the most important mechanism rag models apply is to retrieve relevant passages from which information can be extracted, similar to classical open-book question-answering systems @xcite . on the other hand, rag systems must still produce sensible answers even when the retrieved context is less useful. in such cases, the rag systems generate answers based on the knowledge stored in their parameters similar to pure language models. the model's use of these two fundamental mechanisms -the non-parametric mechanism, where the model copies the answer from a relevant retrieved context and the parametric mechanism of recalling an answer from memorized knowledge -leads to questions about how these two mechanisms interact. which mechanism is more important, and how does the model determine which to rely on when given a context? as shown in in this study, we address two main research questions about how parametric and non-parametric memory interact within the atlas model. through two series of experiments, we aim to answer these two research questions and identify the factors that influence a model's dependence on its parametric memory versus its non-parametric memory. this understanding will help improve the way these models integrate and update information. the primary contributions of this paper are: ## method this study is inspired by previous work that applied causal mediation analysis to elucidate how language models process memorized knowledge stored in their parameters @xcite . however, we revise and enhance our method to better suit our research questions about the interplay between parametric knowledge and contextual information. the following sections describe the theoretical framework we built upon, the experimental setup, and the details of the datasets and preprocessing. ## p740 where was [subj] founded ? in these experiments, we work with retrieved context documents where the true answers have been replaced with a counterfactual. similarly to the setup by @xcite , the control variable x in this case corresponds to whether the embeddings of the context subject tokens or relationship tokens have been affected by noise (x ← 0) or are set to their original values (x ←1), as shown in figure 1 . in this second set of experiments, the average of te (ate) measures the shift towards the counterfactual when providing the model with uncorrupted subject or relation embeddings. the purpose of this measurement is to quantify the general impact of context subject tokens or context relation tokens on the model's decision to rely on the context or its learned parameters. the aie in this experiment shows how a selected representation m contributes to this decision, and again we carry out causal tracing over the model to find the most impactful model components. ## results and discussion we discuss the interpretation of the ate, aie, and pse (via causal tracing) for the two experiments. ## conclusion the study offers valuable insights into the inner workings of the atlas model -a fine-tuned rag model -and how this model processes information from external sources (non-parametric memory) and learned parameters (parametric memory) in different queries. to clarify this phenomenon, we conducted two sets of experiments to understand how the model decides to choose copying from external sources over recalling from learned parameters. in the first experiment, we replaced the object tokens with counterfactuals to separate the effects of context from parametric knowledge. the results revealed that the model relies heavily on the context and tends to copy from it. the second experiment revealed how the model decides to rely on non-parametric knowledge and which mechanism causes the model to choose to copy over recall. our results suggested that the model performs a relevance evaluation to ensure that the context is useful. if that is the case, it shifts focus towards identifying entities that might be replicated as the answer (the object tokens). the subject and relation tokens are essential here in determining context relevance. furthermore, both experiments' results indicated that the mlp in the early to middle layers is involved in contextualizing the relevance of the tokens, while attention in the later layers helps the model focus on integrating this information to extract object tokens as the final answer. these findings explain how these models behave and open doors to controlling this behavior in future research. temporal relevance: there is a potential limitation when dealing with outdated or rapidly changing information based on non-parametric memory. it is necessary to understand how the model adjusts to the temporal changes in knowledge and how effectively it can choose parametric versus nonparametric memory when these changes occur.