DYNAMICQA : Tracing Internal Knowledge Conflicts in Language Models (2024)

Abstract: Knowledge-intensive language understanding tasks require Language Models (LMs) to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. However, conflicting knowledge can be present in the LM’s parameters, termed intra-memory conflict, which can affect a model’s propensity to accept contextual knowledge. To study the effect of intra-memory conflict on LM’s ability to accept the relevant context, we utilise two knowledge conflict measures and a novel dataset containing inherently conflicting data, DYNAMICQA. This dataset includes facts with a temporal dynamic nature where facts can change over time and disputable dynamic facts, which can change depending on the viewpoint. DYNAMICQA is the first to include real-world knowledge conflicts and provide context to study the link between the different types of knowledge conflicts. We also evaluate several measures on their ability to reflect the presence of intra-memory conflict: semantic entropy and a novel coherent persuasion score. With our extensive experiments, we verify that LMs show a greater degree of intra-memory conflict with dynamic facts compared to facts that have a single truth value. Further, we reveal that facts with intra-memory conflict are harder to update with context, suggesting that retrieval-augmented generation will struggle with the most commonly adapted facts

## introduction language models (lms) have been useful in a variety of downstream applications from summarization to fact-checking, often relying on the factual knowledge memorized during pre-training * equal contribution. corresponding authors. and stored in their parameters, known as parametric knowledge @xcite . however, this internal memory is not infallible; it may contain misinformation, biases or simply outdated data, causing lms to produce factually incorrect output, occasionally termed 'hallucinations' @xcite . furthermore, conflicting representations of a fact may exist within training data, given conflicting viewpoints ('disputable facts') or temporal changes ('temporal facts') (see figure 1 ). we refer to this superposition of fact representation as the fact's dynamicity. in contrast, a static fact has only one possible representation. this can lead to intramemory conflicts, which can contribute to lm's uncertainty and instability in factual recall. this intra-memory conflict can be likened to a semantic expression of aleatoric uncertainty, noise inherent in training data @xcite . one way to guide lms to reliable answers is via retrieval-augmented generation (rag) @xcite , which provides additional contextual knowledge, known as non-parametric knowledge, to a language model. however, this context may conflict with the model's parametric knowledge, resulting in a context-memory conflict. initial research has found a tendency of models to overrely on parametric knowledge in the face of these knowledge conflicts @xcite , thus ignoring the retrieved context. while fact popularity has been known to reduce context utilization @xcite , little is known of the impact of internal memory conflicts. we explore this issue for the first time by looking at the effect of fact dynamicity on context utilization. in this paper, we investigate the effect of intramemory conflicts on context adaptation using a novel dataset dynamicqa. this dataset contains inherently conflicting temporal and disputable facts, paired with realistic retrieved contexts in which they occur. furthermore, we manipulate these contexts to include similar, competing objects. using figure 1 : we present examples of our dataset here, consisting of static, temporal, and disputable facts. we show how model output distribution can vary due to the popularity (a,b) and dynamicity (c) of facts. fact dynamicity (c) causes intra-memory conflicts between the different fact representations seen during pretraining. in the bottom row, we show the change in output probability (purple area) that the context must enact on the initial output distribution (dotted line) to force a new output distribution (purple line). dynamicqa, we study the interaction of intramemory and context-memory conflicts by adapting two existing conflict measures. first, we approximate an lm's fact dynamicity ground truth via the number of recorded representations for a fact in wikidata, and sample the lm's expressed intramemory conflict using semantic entropy, which captures the semantic variation present in parametric memory @xcite . furthermore, we introduce a coherent persuasion score, based on @xcite to approximate an lm's semantic shift in output distribution given competing context. with our framework, we show the effect of intramemory conflict on dynamic fact question answering. we find that manipulated contexts of static facts and facts with low dynamicity show the greatest persuasiveness for lms, despite their limited variance in the training dataset and, presumably, low likelihood to change. furthermore, we find that semantic entropy alone does not reflect an instance's likelihood for persuasion given additional context, suggesting that retrieval-augmented generation language models struggle in low-certainty domains. these results underline the need for new measures of intra-memory conflict and the need for other indicators of context use in rags @xcite . ## dynamicqa to evaluate the intra-memory conflicts, we present a dataset of 11,378 question-answer pairs featuring facts with varying levels of dynamicity (see figure 1 ). the dynamicity of a fact is difficult to determine via the pretraining dataset, so we use easily measured proxy scores as a reflection of the property. the question and answer pairs, alongside their proxy scores, are sourced from wikidata and wikipedia and have not previously been employed in similar datasets nor for approximating the degree of knowledge conflicts in general. we approximate temporality via the number of edits ( §3.1) and disputability via the number of reversions ( §3.2). for each question and answer pair, we obtain answerspecific context snippets from wikipedia, alongside their popularity scores, estimated from wikipedia page views. for static and temporal facts, we also create 'counter-memory' from these snippets using similar object replacements, to simulate realistic knowledge conflict scenarios. ## measuring knowledge conflicts in the pretraining stage, lms learn a conditional distribution to predict the next tokens given a preceding context; in this process, they also obtain world knowledge into their parametric memory @xcite . for example, in figure 1 , given a question (e.g., 'who is the father of queen elizabeth ii?'), the conditional probability of the correct answer 'george iv' is expected to be greater than similar objects, owing to the frequent co-occurrence of the subject, object and relation in the training corpus. one important element impacting fact acquisition and recall is the frequency of the fact in the training data. it can be assumed that high frequency (often approximated via fact popularity @xcite ) leads to a greater strength of association between the subject and the object, resulting in greater probability being attributed to the relevant tokens (figure 1b ). however, as information evolves over time, multiple different representations of a factual triplet may appear in a training corpus; meaning, given a subject and relation, there can exist a superposition of object representations @xcite depending on the temporal and situational context. rather than a singular object being assigned a high probability, for such facts, we expect several competing answers in the output distribution (figure 1c ). to evaluate this hypothesis, we expect to see two properties: (1) higher entropy in the output distribution of dynamic facts and (2) greater change in the output distribution to update dynamic facts. to verify (1), we look at the semantic entropy of output distributions ( §4.2). to investigate (2), we analyse the amount of effort required to shift the output distribution using both loss and our novel coherent persuasion score ( §4.3). both measures look at semantic changes in lm output, thereby limiting confounds due to syntactic inconsistencies and other issues arising from over-reliance on firsttoken probabilities @xcite . ## conclusion we investigate, for the first time, the effect of intramemory conflict on context adapation, using multiple natural causes of intra-memory conflict (i.e., fact 'dynamicity'). in this, we introduce a coherent persuasion score to measure the persuasiveness of the given context with consideration of the entire semantic output. furthermore, we release dynam-icqa, a qa dataset with realistic dynamic and static questions, (i.e., facts that naturally vary and predictor β p-value intercept 0.87(±0.008) @xmath0.08(±0.008) @xmath1.05(±0.008) 5.07e-9* spop -0.03(±0.008) 2.04e-5* sec -0.04(±0.008) 1.43e-6( seq 0.002(±0.008) 0.843