Taming Knowledge Conflicts in Language Models
Authors: Gaotang Li
Yuzhong Chen
Hanghang Tong
Venue: ICML 2025 spotlightposter (2025)
Keywords: Knowledge Conflict
Mechanistic Interpretability
Science of Large Language Models
Primary area: deep_learning->large_language_models

Abstract: Language Models (LMs) often encounter knowledge conflicts when parametric memory contradicts contextual knowledge. 
Previous works attribute this conflict to the interplay between "memory heads" and "context heads", attention heads assumed to promote either memory or context exclusively. In this study, we go beyond this fundamental assumption by uncovering a critical phenomenon we term the *superposition of contextual information and parametric memory*, where highly influential attention heads simultaneously contribute to both memory and context. Building upon this insight, we propose Just Run Twice (JuICE), a test-time attention intervention method that steers LMs toward either parametric beliefs or contextual knowledge without requiring fine-tuning. JuICE identifies a set of reliable attention heads and leverages a dual-run approach to mitigate the superposition effects. Extensive experiments across 11 datasets and 6 model architectures demonstrate that JuICE sets the new state-of-the-art performance and robust generalization, achieving significant and consistent improvement across different domains under various conflict types. Finally, we theoretically analyze knowledge conflict and the superposition of contextual information and parametric memory in attention heads, which further elucidates the effectiveness of JuICE in these settings. Our code is available at https://github.com/GaotangLi/JUICE.

Taming Knowledge Conflicts in Language Models
Gaotang Li
Yuzhong Chen
Hanghang Tong'
Abstract
Prior Belief
Our Finding
Language Models (LMs) often encounter knowl- edge conflicts when parametric memory con- tradicts contextual knowledge. Previous works attribute this conflict to the interplay between "memory heads" and "context heads", attention heads assumed to promote either memory or con- text exclusively. In this study, we go beyond this fundamental assumption by uncovering a criti- cal phenomenon we term the superposition of contextual information and parametric memory. where highly influential attention heads simul- tancously contribute both memory and con- text. Building upon this insight, we propose Just Run Twice (JuICE), test-time attention intervention method that steers LMs toward ei- ther parametric beliefs or contextual knowledge without requiring fine-tuning. JUICE identifies of reliable attention heads and leverages dual-run approach mitigate the superposi- tion effects. Extensive experiments across 11 datasets and 6 model architectures demonstrate that JUICE sets the new state-of-the-art perfor- mance and robust generalization, achieving sig- nificant and consistent improvement across dif- ferent domains under various conflict types. Fi- nally, theoretically analyze knowledge conflict and the superposition of contextual information and parametric memory in attention heads, which further elucidates the effectiveness of JUICE in these settings. Our code is available at https: // //gi teittuu..a.. wwt.c/oo//cocoo wttp:///aaaaaaaan///or....o
Transformer Layer
Transformer Layer
Memory
Memory
Head
Head
Context
Context
Head
Head
parametric memory
contextual information
Figure I. Our finding goes beyond the prior notion exclusive "memory head" and "context head", where we show that memory and contexts are encoded in attention heads in superposition.
inference, they leverage this parametric memory alongside the provided context to generate the next token. However, confficts arise when parametric memory contradicts contextual information phantiomnooaeeemooooooooo phenomenon known as knowl-
edge conflict (Xu et al., 2024). In such cases, the model may become uncertain about which source of knowledge to trust. These conflicts are particularly prevalent in real-world applications, especially context-heavy Large Language Models (LLMs) systems like aarreval-augmmented genera- tion (RAG) (Gao et al., 2023), LLM agents (Xi et al., 2025), and augmenedeeteed LLMs (Qu et al., 2025). Depending on the application, user may require an LLM to either remain faithful to its parametric memory or prioritize contextual reliance for accurate and reliable outputs.
Prior works have explored the behavior of LMs under knowl- edge conflicts, either by treating the model as oracle analyze how different contexts influence its predictions (Xie et al., 2024) or by treating the context as an oracle to evaluate how effectively the model follows it (Longpre et al., 2021). While these studies provide valuable insights into knowl- edge conflicts, the intrinsic mechanisms underlying these conflicts and corresponding mitigation strategies largely re- main unexplored. Some studies have taken important steps to characterize (Yu al., 2023) and intervene (Jin et al., 2024b) in knowledge conflicts, primarily focusing sin- gle conflict type (e.g., substitution-bast conflicts). While pioneering, these efforts leave opportunities for more com- prehensive understanding diverse conflict types and the development of fine-grained approaches to address knowl-
1. Introduction
Language Models (LMs) store vast amounts of informa- tion during pretraining parametric knowlege. During
University of Illinois Urbana- Chammangn VVisa Research. Cor- respondence to: Gaotang Li <gaotang3 @ illinoiis.edu edu>>, Hanghang Tong <chong @ illinois.edu>>.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Taming Knowledge Conflicts in Language Models
Performance Comparison
rate steering directions through residual head activations.
We evaluate JuICE in two distinct settings: enhancing para- metric beliefs and enhancing contextual reliance. For the first setting, we use six factual association datasets covering diverse domains, each tested under three levels knowl- edge conflict. In the second setting, we evaluate five datasets spanning diverse fields and formats, including open-domain question answering and sentence completion. Extensive ex- perimental results demonstrate the consistent state-of-the-ar performance of JUICE. Fig. 2 illustrates the strong perfor- mance of JUICE under the Gemma-2b model, with detailed results provided in Tab. 3. We also show the robustness of JUICE against key hyperparameters and paraphrased input.
18
Performance
60
31
Original
20
PH3
JUICE
Clean
Subs-Conflict
Coh-Conflict
Figure 2. Performance of different methods with Gemma-2b under various conflict types. JUICE achieves consistently high perfor- mance in facing challenging knowledge conflicts.
Finally, we analyze our empirical observations from a theo- retical perspective, conceptualizing knowledge conflict as the result of conflicting tasks inference, which arise from distinct tasks during training. In succinct setup, we demon- strate the existence of attention heads that simultancously contribute to both parametric and contextual knowledge and show how standard training encourages the formation of such heads. We further provide theoretic justifications for the effectiveness of JUICE under these settings.
edge conflicts. In addition, much of the existing literature predominantly adopts a single-sided perspective on knowl- edge conflict, focusing on enhancing contextual reliance and addressing issues commonly referred to as "RAG hallu- cination" (Goyal et al.; Huang et al., 2023; Shi al., 2024b). In contrast, advocate for unified method capable of flexibly steering the model toward either parametric or con- textual knowledge, offering broader utility.
In this paper, we begin by treating LMs as an oracle and considering the setting of factual recall, a task requiring pure memorization. We then treat contexts as providing misleading information (Shi al., 2023) and systemati- cally explore various types of knowledge conflicts over di- verse domains, including sentence-level (substitution), and paragraph-level (coherent) conflicts (Sec. 2), to uncover their underlying mechanisms and design effective interven- tion strategies. Starting with empirical analysis, our findings go beyond the hypothesis posited in (Jin al., 2024b) that model components exclusively contribute either paramet- ric contextual knowledge, uncovering the phenomenon of "superposition of contextual information and para- metric memory" (CP superposition), as shown in Fig. 1. We revealed the inconsistent behaviors of model compo- nents under different degrees knowledge conflicts and the counteracting effects of multiple individually effective interventions.
Our main contributions can be summarize as follows:
We conduct systematic and principled study of knowledge conflicts in LMs, considering both parametric and contextual perspectives and covering various types of datasets over diverse domains.
Problem.
Mechanism. We reveal the limitations of naive inter- vention methods by uncovering a critical phenomenon we term the "superposition of contextual information and parametric memory", where the relative role of model component in parametric versus contextual knowledge is not exclusive.
Algorithm. We propose JUICE, a simple yet effective method to steer an LM toward parametric or contextual knowledge without finetuning, leveraging a dual-run approach to mitigate the superposition effects.
Experiment. Through extensive experiments across 11 datasets and 6 architectures, set the new state-of-the- art performance and robust generalization, achieving significant and consistent improvements.
Building on these insights, we propose Just Run Twice (JuICE), simple yet effective method for steering LMs towards either parametric or contextual knowledge without finetuning. JUICE operates in two stages: (1) a head identi- ficution stage, where two sets of attention heads that yield consistent improvements with positive or negative scaling are identified using a minimal number of samples, and (2) dual-run inference stage, where the model runs twice: first saving the outputs of the identified heads, and then using scaled versions of these saved outputs to intervene during the second run. Intuitively, this approach ensures that the identified components are consistently effective, mitigating the superposition effects, and therefore provide more accu-
Theory. We provide a theoretical analysis of knowl- edge conflicts, conceptualizing the superposition of contextual information and parametric memory. This analysis further justifies the effectiveness of JUICE under these conditions.
2. Problem Setup
In this paper, we study how language models respond to varying degrees of knowledge conflict and propose methods
2

---

Taming Knowledge Conflicts in Language Models
to regulate these behaviors. We identify two complemen- tary perspectives on knowledge conftict: (1) when the input context is irrelevant or potentially misleading, we treat the LM as an oracle, aiming to enhance its reliance on para- metric beliefs; (2) when the input context is accurate, but the LM's prior knowledge may be outdated or incorrect, we aim to increase the model's dependence on contextual knowledge. Both perspectives hold intrinsic value and merit further investigation.
curated dataset is available at https://uuiiiggaaee co/dataseta tttttttssssssgtttttae httpa/////ggtttaaaaaaaailiill
2.2. Contextual Datasets
In this setup, we treat the input context the desired target and consider the prior knowledge of the language model as unreliable source of information. This approach enables more unified and versatile evaluation of baseline methods.
Since this setup has been extensively studied, we adopt the dataset choice of a seminar work (Shi et al., 2024b) by using two context-oriented knowledge conflict benchmarks: Memo-Trap (Liu & Liu, 2023) and NQ-Swap (Longpre et al., 2021). The details of these datasets can be found in Appen. D. We evaluate performance using exact match (accuracy) with respect to the contextual answer.
2.1. Parametric Datasets
In this setup, we treat the input context as potentially mis- leading information and the language model as an oracle. For our study, we carefully curate six datasets encompass- ing distinct types of knowledge conflicts in factual recalls. Below, we detail the specific design choices differing from prior studies and the underlying rationales:
2.3. Models
We create six datasets span- ning various domains of factual knowledge: World Capital, Athlete Sport, Book Author, Official Language, Company Headquarter, and Company Founder. This setting will al- low us to investigate the transferrability across unrelated domains of intervention methods, a critical aspect that is missing in the prior work (Jin et al., 2024b; Yu et al., 2023).
Diverse Factual Domains:
We benchmark our studies using six existing open-sourced base language models: Gemma-2b (Team et al., 2024), Llama2-7B (Touvron et al., 2023), LJama3-8B (Dubey et al., 2024), Phi2-2.7b (Javaheripi et al., 2023), StableLm2 1.6b (Bellagente et al., 2024), and Olmo-7b (Groeneveld et al., 2024). We conduct our analysis in Sec. 3 mainly using Gemma and evaluate the effectiveness of the intervention methods using all backbone models.
This is the exclusive approach adopted in prior works (Yu al., 2023; Jin et al., 2024b). A typical input takes the form (e.g.. "The name of the capital city of (s} is {ac}- The name of the capital city of {s} is"), where ag represents the substituted contextual answer that conflicts with the parametric answer ay- In our experiment, we aim to enhance the model's 's ability to output Gp, despite the conflicting presence of ac-
Sentence-level Conflict (Substitution-baase
3. Interpreting and Resolving Knowledge Conflicts
In this section, we analyze how the internal structure of language models (LMs) influences their parametric versus contextual tendencies through causal analysis. We quan- tify these tendencies by measuring the expected change in the probability of the output token (parametric versus con- textual) when perturbations are applied to specific model components. These perturbations ing the activation outputs. Formally, given distribution over input triplets (X, Vp. Uc), where X = {= is the input prompt set, encompassing various conflicting forms (e.g., clean input, substitution conflicts, and coherent con- flicts), Up and ye represent the parametric and contextual answers, respectively, we measure:
Paragraph-level Conffict (Coherent Counterfactual); Re- cent work (Xie et al., 2024) demonstrates that language models rely more on context when it is coherent. In this scenario, the context extends beyond a single substitution, reinforced by coherent and persuasive evidence, often gener- ated by advanced models like GPT-4. This presents a highly challenging case, as models almost inevitably output the contextual answer dg over the parametric answer dp- In our experiment, we focus on enhancing the model's ability to output ap. despite these difficult conditions.
implemented by scal-
There is also trivial type of knowledge conflict: when no conflict is present, in which case we still expect the model to respond faithfully. Detailed examples are provided in Appen. C. Importantly. different from (Xie al., 2024), which focuses solely on altering the model's predictions regardless of their correctness, we explicitly ensure that conflicting contexts include factually incorrect answers. For evaluation, we primarily rely on the exact match (accuracy) metric with respect to the factually correct answer. Our
(( [PP
(p Iz, do( = aaM))) - -P P(ul:) (I)
Here, (11 refers specific model component with index i, and is set to either yp ye upon our needs. While (2, v) can be drawn from an arbitrary distribution, we use Gemma and World Capital as concrete example in this section.
Previous works analyzing model internals typically adhere to two "locate-and-edie principles (Xu et al., 2024):
3

---

Taming Knowledge Conflicts in Language Models
Influence of Knock Out Clean
Influence of Knock Out Substitution Conflict
Influence of Knock Out Coherent Conflict
Enkiew Layar Attention
Entire Layer
62
08
AttenBon
0.
MLP
MLP
Change in Probs
Change in Probs
Change in Probs
-02-
6.-
02-
-01
-0.6
Entire Layer
0.-
Attention
-0.6
MLP
15 Layer Number
Layer Number
Layer Number
Figure 3. Influence of Knock Out (Zero Out) Model Components in changing the probahility of outputting the parametric answer tokens (ap) on the World d Capi tta dataset. Three different scenarios are considered: clean inputs, substitution conffict inputs, and coherent conffict inputs. We find that (I) removing (nearly) all components leads to decreases in probability of outputting ap in clean prompts, (2) removing components leads both increase and decrease in outputting Op in substitution conflict prompts, and (3) removing (nearly) all components leads to increases in probability of outputting ap in coherent conflict prompts.
Identify a circuit (specific model components) that is exclusively responsible for a particular functionality.
ory heads" and "context heads", positing that there are atten- tion heads exclusively responsible for promoting parametric or contextual knowledge. Specifically, promoting contextual knowledge involves knocking out parametric heads, and vice versa. While this approach achieves success in single-typed conflicts, we find its limitations when extended multiple kinds of confficts. Tab. I ranks the top-4 memory heads based on their effectiveness in substitution conflicts and evaluates their influence in coherent conflicts. Surprisingly, half of the top-performing "memory heads" in substitution conflicts become "context heads" in coherent conflicts. This shows that even the most influential model component could have completely opposite functionality.
Apply targeted interventions to these circuits to achieve the desired control or behavior.
In our motivating experiments, we demonstrate the need for additional criteria when performing interventions to address the complexities of model internals and knowledge conflicts.
3.1. Analysis
Observation 1: Inconsistent Behaviors of Model Com- ponents Under Different Degrees of Knowledge Conflict. In our first set of experiments, we examine how model com- ponents exhibit significantly different functionalities when faced with varying degrees of knowledge conflict. We set M to represent either the entire MLP, attention module, or both within layer i. For the intervention method, we fix it to be knocking out (i.., zero-ablating). The goal is to pro- mote parametric knowledge, setting y = Up in Eq. Fig. 3 illustrates these findings, revealing the following trends: (1) removing (nearly) all components decreases the probability of outputting parametric answers for clean prompts; (2) re- moving components leads to both increase and decrease in outputting parametric answers for substitution conflicts; and (3) removing (nearly) all components increases the proha- bility of outputting parametric answers for coherent conflict prompts. Quantitatively, the number of components yielding consistent parametric gains across all three conflict types is 0 for the entire layer, 1 for the MLP module, and for the Attention module (out of 26 layers in Gemma). These results suggest that the same model component may exhibit different influences on parametric and contextual knowledge depending on residual streams received from prior layers.
Table 1. The top 4 heads ranked by the average proh increase of contextual knowledge in substitution- based confticts via knocking out. We find that half of the top-influential memory heads in substi- tution conflict lead to contrary effects coherent conflict. Green denotes the desired behavior (f context and parametric) and red denotes the undesired behavior (L context and 1 parametric).
Head
Subs-Coufliet
Coh-Conffict
Context Prob
Para Prob
Context Prob
PPrra Prob
+0.04
(15 6)
+0.15
.0004
+0.38
2004
(9.3)
+0.13
-0.8
0.17
-0.09
-0.03
-0.13
Observation 2: Counteracting Effects of Multiple Inter- ventions. Expanding on prior observations, we evaluate the impact of multiple interventions on parametric knowl- edge. We first identify attention heads that consistently increase parametric logits when individually knocked out, ranking them by their average contribution. A natural ap- proach is to apply these effective individual interventions simultaneously. as proposed by Jin et al. (2024b). However, Tab. 2 reveals that combining individually helpful interven- tions does not always yield additive benefits and can even
Prior work (Jin et al., 2024b) introduces the notion of "mem-

---

Taming Knowledge Conflicts in Language Models
Stage 1: Head Identification
C clesm input subu conflict
Stage 2: Dual-Run Inference
(I) Calculate Score Each Head
First Run: Outpul of Selected Hleads
0
cob-conffiitt
Parametric
head input
Context
(2) Second Run: Add the Scaled Saved Output from the First Run
Scale the
head output
Prohahility
(2) Filter out inconsistent
(3) Select Top
head i such that Apedd japur) <0 for some
heads based on Anaul
Σj Ausdd, iqpui
ladden state
unsellccted beads
selected
changed output
Figure 4. Overview of JUICE. In the first head identification stage (left), JUICE identifies set of attention heads that could consistently achieve the desired effect. In the second inference stage (right), JUICE first saves the outputs of the identified heads, and then adds the scaled version of those outputs to the corresponding modules.
reduce performance. This behavior likely arises from the dependence of a model component's functionalitty on input residual streams, as highlighted in Observation Modi- fied activations from earlier layers may alter downstream behavior, leading to counteracting effects.
Stage 2 (Dual-run Inference). To mitigate counteracting effects from multiple interventions, the model runs twice. In the first run, the outputs of the identified heads are saved. In the second run, scaled versions of these saved outputs are added to the corresponding activations. Intuitively, the first- run activations serve as more reliable steering directions. We validate this intuition through experiments in Sec. 4.4 and analyses in Sec. 5.
Table Target probability value using multiple interventions der coherent conflicts. Top-i denotes combining O to i-th ranked individual intervention performances. This shows that different modules can "counteract" cach other, even though individual inter- vention contributes substantial performance gains.
Practical Implementation. The key hyperparameters of JUICE include the size of the head selection dataset D, the number of intervened heads K; and the scaling factors in- ference. In practice, we fix K to be constant number (e.g., 5) and determine the scaling factors using the validation set. We fix [D] to be 4 for all primary experiments. Addition- ally, we test the generalizability of JUICE by using a head identification set from a single domain and evaluating its performance across other domains.
Number of Intervened Components
'arget Proh Value
None (Original Model)
0.03
Top
0.12
Top
0.24
Top 10
0.14
Our findings collectively suggest a phenomenon we term the "superposition of contextual information and parametric memory" (CP Superposition), where the roles of "context" or "memory" of model components depend on the inputs they receive. Next, we discuss how we could propose effec- tive methods while acknowledging such superpositions.
4. Intervention Experiment
In this section, we analyze the intervention performance of JUICE and compare it against different baselines. Due to the page limit, we only present three models in the main paper. A more comprehensive experiment section with additional model results can be found Appen. D.
3.2. Our Approach: Just Run Twice (JUICE)
We introduce Just Run Twice (JUICE), test-time inter- vention method for addressing knowledge conflicts. Fig. 4 illustrates the core idea, and Alg. provides the detailed algorithm. JUICE operates in two stages.
4.1. Enhancing Parametric Beliefs
Setups. We use the datasets and evaluation metric detailed in Sec. 2.1. Notably, we have three different conflict types: No Conflict (Type 1), Substitution Conflict (Type 2). and Coherent Conflict (Type 3). For presentation clarity, we use the number represent these conflict types in Tab. 3.
Stage 1 (Head Identification). This stage identifies two sets attention heads that consistently achieve the desired effect with either positive or negative scaling. Each head is assigned score based on the expected change in the desired probability value under individual scaling, computed across small head selection dataset spanning multiple conflict types. To ensure consistency, only heads with non neg-negetive scores across all conflict types are selected. The top K, based on aggregated scores, are retained. This process en- sures reliability for individual head activations.
Baselines. We compare our methods against the following baselines: (1) Prompt: We instruct the LM to generate answers solely based on internal memory; (2) PH3: (Jin et al., 2024b) leverages patching-based methods identify and prune "context" and "memory" heads, demonstrating
5

---

Taming Knowledge Conflicts in Language Models
Table Results of intervention for enhancing parametric memory. All results are in accuracy (%). JUICE consistently achieves the
state-of of the-art performances in most cases. Bold denotes the best result. Additional model results can be found in Appen. D.2.
Athlete Sport
Book Author
Company
Company Headquarter
Official Language
World Capital
Dataset
Average
Confliet Type
3
1
3
3
2
3
2
1
2
1
2
3
Original
93.4
18.1
0.0
73.0
7.7
0.0
47.0
2.7
0.0
64.2
0.7
0.0
96.9
23.5
0.0
94.1
15.1
1.1
78.1
11.3
0.2
Prompt
93.4
44.5
22.4
47.0
3.8
61.2
0.0
96.9
22.2
94.1
35.7
78.1
29.6
10.5
PI13;
86.5
71.6
33.3
33.3
4.8
0.0
28.1
10.8
19.5
44.3
22.4
30.6
90.7
72.8
82.7
84.3
64.3
88.1
61.2
41.1
42.4
Gemma
PH3,
93.2
75.3
21.8
19.3
0.2
42.7
0.0
62.0
0.0
82.7
37.7
0.0
78.9
15.7
0.5
63.5
25.7
JUNE (Ours)
91.2
63.2
65.9
78.0
61.0
2.9
46.5
44,9
41.1
57.9
36.2
38,9
94.4
82.1
84.0
91.9
69.2
83.2
76.7
59.4
52.7
JUICE (Ours)
96.3
95.4
91.9
79.8
75.5
68.0
45.4
39.5
43.2
65.8
60.0
59.3
93.2
86.4
85.2
94.1
95.1
93.0
79.1
75.3
73.4
Original
90.4
9.0
47.0
57.5
29.3
0.0
75.2
0.7
95.7
46.9
0.0
95.1
22.5
82.5
25.9
Prompe
90.4
70.2
0.2
81.4
65,1
22.0
57.5
16.6
24.3
75.2
38.0
15.7
95.7
79.6
40.7
95,1
60.3
15.8
82.5
55.0
19.8
Ilama2
P113,
91.0
87.4
37.5
77.8
92.0
70.9
53.0
52.2
32.6
73.4
74.0
12.1
94.4
90.7
84.0
94,2
95.7
90.2
80.6
82.0
54.5
PH3,
89.0
88.1
10.5
80.2
86,1
64.5
52.7
50.0
34.0
73.4
72.9
18.5
94.4
85.5
80.7
94,0
91.3
85.3
80.6
79.0
48.9
JuNE (Ours)
89.9
61.6
50.4
77.1
85.6
79.8
53.6
47.0
40.9
72.2
66.3
64.0
93.8
92.0
95.7
94.6
94.0
95.7
80.2
74.4
71.1
JUICE (Ours)
88.6
91.0
88L5
53.0
51.9
74.1
74.3
96.1
96.2
83.0
Original
84.1
22.2
0.0
55.6
2.2
0.0
61.1
3.3
0.0
80.3
1,4
1.8
96.3
20.4
0.6
94.6
16.8
0.0
78.7
11.0
0.4
Prompe
84.1
87.4
4.1
55.6
77.7
0.0
61.1
38.3
0.6
80.3
48.2
0.0
96.3
85.2
5.6
94.6
83.8
11.9
78.7
70.1
3.7
Llama3
PH3,
86.4
86.5
14.1
87.4
48.9
78.0
55.3
9.4
96.3
%6.3
84.0
95.0
92.4
80.7
39.2
PH13,
86.5
86.3
12.5
61.1
84.8
6.8
58.3
51.7
27.8
70.0
56.2
26.8
96.3
95.8
87.0
91.4
87.6
90.3
77.3
77.1
41.9
Ju (Ours)
82.8
72.8
58.7
66.2
92.1
83.0
61.7
51.1
54.4
80.5
56.9
56.0
95.7
95.7
93.2
94.1
95.7
96.8
80.2
77.4
73.7
JUICE (Ours)
87,0
87.8
95,9
86.5
92.3
88.7
61.7
56.7
55.6
79.8
75,9
74.8
96.3
96.3
95.7
95.7
96.2
97.3
84.5
84.2
84.7
Table 4. Results of intervention for enhancing contextual knowl- edge, following the same convention as Tab. 3.
prompts, enhancing the factuality of the model.
3.
While PH3 and Prompt demonstrate notable improve- ments in substitution conflicts under certain conditions, they fail to effectively address coherent conflict scenar- ios. Importantly, there is a clear performance differ- ence when PH3 has a small set of head identification sets. JUICE can achieve better performance with a significantly smaller head identification set.
Model
Method
Swap
NQ
ech Ending
Hltte Spe-
Science qui
likstory
Proverb Ending
Proverb Translation
Average
Original
39.7
70.7
29.9
26.5
59.0
25.0
Promp:
38.0
26.6
58.4
CAD
56.9
37.1
Gemma
MD,,
$1.0
$2.8
45.5
57.8
62.0
50.0
PHE3,
50.2
50.2
35.2
50.1
65.2
53.8
JUNE (Ous)
38.7
79.3
50.1
26.8
67.1
52.4
JUICE (Ours)
$8.4
$4.1
47.0
74.6
66.8
66.2
Original
24.5
Prompt
39,6
21.3
52.5
CAD
29.8
05.4
20.2
28.6
$6.2
41.4
4. JUICE outperforms JUNE on average in almost all cases. In particular, the gap is about 20% with the Gemma model. This ablation further illustrates the effectiveness of the dual-run design of JUICE.
Llanz2
PH33,
48.2
63.4
20.4
66.7
58.8
$1.9
PH3
25,3
62.2
16.5
26.5
55.2
37.1
JUNE (Ous)
29.7
JUICE (Ouns)
49.5
Original
18.5
51.3
72.9
24.5
Pronpp
53.4
53.7
71.7
23.9
51.8
46.9
1.0m83
CAD
PEE3,
25.3
34.7
60.5
62.2
784
73.1
33.1
48.5
54.1
63.6
51.2
55.6
5. While PH3 bears an appealing ability to identify "cross- relation heads" (Jin et al., 2024b), its transferability is largely limited to closely related datasets (i.e., heads identified from the world capital dataset are effective for the official language dataset but not for the company headquarters dataset). In contrast, our method achieves high performance across diverse domains, with heads only being selected from the world capital domain.
PH3.
22.5
75.1
JuNE (Ous)
26.5
72.5
73.2
33.1
61.8
JUICE (Qus)
35.3
78.4
74.2
75.4
70.7
66.8
strong performance in substitution conflicts. We note that the original PH3 requires a development set of 200 samples for head identification. For a fair comparison, we include two versions of PH3: PH3,, the original version, and PH3., which uses the same amount of samples JUICE for head identifications (i.e., 4 samples). (3) JUNE (Just Run Once): an ablated variant of JUICE that only omits the dual-run design, whose details can be found in Appen. F.
4.2. Enhancing Contextual Reliance
Setups and Baselines. We use the datasets and evalua- tion metric detailed in Sec. 2.2. We compare our methods against the previously mentioned baselines and an additional one: CAD (Shi et al., 2024b), a decoding- based method that leverages contrastive decoding (Li et al., 2023b) to encour- age the language model to attend to its context.
Results. Tab. presents the results of these intervention methods across different models. Key observations include:
1. JUICE consistently and significantly outperforms all haselines in most cases. Experimental results indicate that JUICE can almost completely reverse the model's tendency to produce contextual knowledge, even in the most challenging (coherent conflict, Type 3) scenarios.
Results. Tab. 4 presents the results of these intervention methods across the models. The main conclusions from the prior subsection are still valid. JUICE consistently outper- forms all baselines on average and is versatile in promoting
2. JUICE achieves improvements on zero-shot clean
6

---

Taming Knowledge Conflicts in Language Models
Impact Head Identification Set Size on Performance
Impact of Number of Altention Heads on Performance
Impact Scaling Factor on Performance
Intervention Performance
Incoveraan Peffrrmnc
0.5
0.4
0.
Pos Stating (abr)
Clean
Sub-Canflet
Neg Scaling (abs)
Coh-Corflict
10
Head Set
Attention Heads Intervened
(a) Head Identification Set Size
(b) Number of Heads Intervened
(c) Scaling Factor
Figure 5. Robustness analysis of JuICE across key hyperparameters. We observe consistent intervention performance we vary the head identification set size, the number heads intervened, and the scaling factor magnitudes, underscoring the robustness and adaptability.
contextual knowledge as well.
Influence of Multiple Interventions
Ran Inic
4.3. Robustness of JUICE
In this section, examine the robustness of JUICE against variations in key hyperparameters and paraphrased prompts. Using Gemma as our backbone model, we systematically vary one hyperparameter at a time to isolate its effects on performance. Specifically, we evaluate the impact of three [D],
Average Lngit
.2
hyperparameters: the size of the head identification the number of intervened attention heads K, and the mag- nitude of the scaling factors. Additionally, we investigate robustness to paraphrased prompts by employing multiple curated templates for each conflict type, selecting one at random during evaluation. Detailed experimental setups and additional analyses are provided in Appendix D.3.
Number of Top Intervened Heads
Figure 6. Effect of Running Twice: Mitigating Counteracting Ef- fects of Multiple Interventions. All presented heads contribute to individual gains, starting from a baseline logit value of 0.03, The results show that naive single- pass interventions are unstable and prone to degradation. In contrast, the dual-run design ensures consistent and effective interventions.
Figure 5 illustrates the robustness of JUICE across these hyperparameters. The results demonstrate that JUICE main- tains consistently high performance across wide range of hyperparameter values, highlighting its stability and effec- tiveness.
individually effective interventions. As shown in Fig. single-pas -aeeeppssssss interventions are unstable and prone to perfor- mance degradation. In contrast, the dual-run design delivers consistently effective interventions.
Tab. 7 in Appendix D.3 presents the results of JUICE when applied paraphrased prompts. Our findings show that JUICE is highly robust to variations in input prompt formats, consistently maintaining its effectiveness across diverse tem- plates. Notably, JUICE continues to demonstrate superior performance, effectively shifting the model's 's reliance from context to parametric memory.
5. Theoretical Analysis
In the previous sections, we have conducted a comprehen- sive empirical analysis to identify the phenomenon of CP superposition and demonstrated the efffectiveness JUICE across variety of setups. In this section, we aim formal- ize our observations and understand the underlying mech- anisms behind both observations. Specifically, we concep- tualize knowledge conflicts as arising naturally within the weight matrices of the attention module, shaped through the training process via gradient descent. Under such condi-
4.4. JUNE vs. JUICE: Effeet of Running Twice
We conduct an additional experiment to demonstrate the effectiveness of the dual-run design. Following the same setup as in Tab. 2, we compare the intervened logit value of Run Once versus Run Twice when combining multiple

---

Taming Knowledge Conflicts in Language Models
tions, we elucidate that JUICE provides a superior approach compared to naive single-pass pass interventions. A more de- tailed theoretical analysis can be found in Appen. G. We first provide a brief overview of the model and task setup.
Set 2T = q and 2r+1 = 25+1-
In summary, the vocabulary space consists of V 8 SUAL AU {q} U.N. We remark that we use the same trigger token as the fixed query token in the factual recall task to induce knowledge conflicts.
Model Setup. We use a two-layer Transformer with one attention head per layer, absolute positional encoding, and residual connections. The input a sequence of tokens 21:T E [Av]", where T' is the sequence length, and N is the vocabulary size. Each token 2; is mapped to d-dimensional embedding ( and a positional embedding Pt E Rd is added. The input to the model is: 2T = x(z)) + Pt for t= 1, T. We denote x = [..,..., IT as the repre- sentation of the embeddings at layer I. These embeddings are updated through two layers as follows:
Assumption 5.1 (Near-orthogonal Initialization), All em- bedding, unembedding, and positional vectors are initialized randomly.
This ensures near-orthogoonlity among all embeddings and unembeddings, such that (6(z)))(z))) s 6,y(1[i ji) when the embedding dimension d is large. Our setting is similar to recent works (Bietti et al., 2024; Ghosal et al., 2024; Jiang et al., 2024b; Nichani et al., 2024).
x(++1= x() + w(lxx( x (MSK @ ko
5.1. CP Superposition
We first examine how knowledge conflict arises in our sim- plified model, starting by demonstrating its existence.
where ō is the column-wise softmax function. Finally, the embeddings are mapped back to the vocabulary space through a linear layer parameterized by Win E RdxN, The i-th column vector is denoted as p()).
Proposition 5.2 (Existence of Perfect Solver). There exists two-layer transformer that can solve both induction and factual recall tasks with the perfect accuracy.
Task Setup. We consider two tasks in parallel: Factual Recalls and Induction. They correspond to parametric and contextual tasks, respectively. A diagram illustration of the whole theoretical task setup can be found in Fig. 7.
The construction can be achieved as follows. By setting ov as a random matrix and defining
T-1 W = Σ PoPE+1.
In the factual recall task (Nichani et al., 2024), the goal is to learn associations between the subject token space and the answer token space A, based on bijective ground truth mapping G" S A. This models knowledge triples like (China, capital, Beijing), where the subject token (China, capital) maps to the answer token (Beijing). Non-critical tokens like "the" and "of" also constitute part of factual sentence, and we assume these tokens are from the noise token space N. Sequences 21:7+1 E [M]T+] are generated as follows:
(2)
1=1
w( KQ
C1 (W o(g) + +O2 Σ (s)))(1) (3)
SES
w C3 Σ (k)) + + Σ n0 (s)) d(s).
KEN
sES
(4)
where C1, C2 C2, C4 are appropriate scaling factors and C is large constant. In this setup, the first layer imple- ments "copy from previous embedding" behavior, while the second layer learns the critical tokens and associated memory required for the tasks. Notably, the construction of the second layer inherently forms superposition, which leads to knowledge conflicts.
1. Sample a fact 8 E S and index i E [T - 1] uniformly at random, and set 2; = 8.
2. For all & E [T - 1JM (i), sample mn uniformly from V without replacement.
3. Set žт = q. the query token and 2T+1 = (s)).
In the induction task (Olsson et al., 2022), the goal is to predict a token b E N following the second occurence of a trigger word g (e.g. ...gb... - b). Sequences #1.7+1 E [N]"+1 are generated as follows:
Next, we analyze how this construction could naturally emerge from training via gradient descent with a cross- entropy loss over the two tasks. We assume a perfectly learned first layer and focus on the dynamics of the second layer, as it suffices to illustrate the core idea. For simplicity, we assume a linear attention model and strictly orthogo- nal embeddings (i.e., all initialized vectors are orthogonal), which are common in the existing literature (Li et al., 2023c; Ahn et al., 2023; Zhang et al., 2024; Mahankali et al.).
1. Sample j E [T 21} uniformly, set 2j = q, and sample 2,+1 from N.
2. For all other token 24, sample uniformly at random from (2/4111 without replacement.
8

---

Taming Knowledge Conflicts in Language Models
2
2+1
s
1-1
q
a
2-1
q
2,1
q
2
Factual Recall (Parametric)
Induction (Context)
?
Z
S
1
2
q
2,1
q
a
vs.
2
Knowledge Conflict
Figure 7. Illustration of the theoretical task setup. The top row shows distinct tasks that two-layer transformer learns during training: the bottom row depicts the conflicting task encountered at inference. Here, denotes noisy tokens, is the subject token, a is the answer token associated with s, and q is the trigger and fixed query (EOS) token.
Proposition 5.3 (Learning the Second Superposition Layer via Gradient Descent, Informal). In a simplified setup using one-layer attention only transformer, the superposition head constructed in Eq.3 and E can be trained via gradient descent from zero initialization using the cross-entropy loss.
Finally, we manifest the effectiveness of the dual-run design over single-pass intervention.
Proposition 5.5 (Effectiveness of JUICE). Consider the model from Prop. 5.2 and the case when its inductive part dominates (i.e., exp(C1)CC >> exp(C2) (a)) then the in- tervention by JUNe/PH3 of deleting the two attention heads is not as effective as JUICE. In particular, in this case JUNE/PH3 does not result in the parametric answer, while JUICE does.
We defer the proof Appen. G. This proposition tells that the standard training objectives of language models encourages superposition. In practice, the first layer may also learn associative memories required by different tasks. Such formulation of the weight matrices naturally results in knowledge conflicts at the inference time.
Both attention heads from Prop. 5.2 can be identified as "influential context heads" in the above setting. However, when the first head is removed, the second head no longer functions for the induction task but instead transitions into a factual memorizer. A single- pasle--assssss intervention method may still remove the second head, as it was initially classified as a "context head". By instead deleting activations from the original arguably more reliable source-JuICE achieves precise control over the model's 's behavior and steers it as desired.
5.2. Knowledge Conflict
We now define and analyze the knowledge conffict task:
1. Sample an index j E [T - 22,(1,, set 2j g. and sample 1 from N.
2. Sample an index E [T 111 + +1 and E S. Set 24 s.
3. Set 2T = q.
Corollary 5.4 (Knowledge Conffict). Under the knowledge conftict inference setting, the model capable of solving both factual recall and induction from Proposition 5.2 may output either the inductive token or the factual token. More specifi- cally, if exp(Cii C3's < exp(C2) then the model outputs the factual recall answer G (s); otherwise, the model out- puts the induction answer 2j+1-
6. Conclusion
This work presents unified and principled study of knowl- edge conflicts in language models, revealing the phe- nomenon of superposition of contextual information and parametric memory. We propose Just Run Twice (JUICE), simple yet effective test-time intervention that reliably steers models toward either parametric beliefs or contextual information without requiring fine-tuning. JUICE consis- tently and significantly achieves effective intervention per- formance across different datasets under various conflict types. Our theoretical analysis further reveals the underly- ing mechanisms of knowledge conflict and the effectiveness of JUICE. These findings not only enhance our fundamental understanding of LMs' knowledge representation mecha- nism but also offer a practical method for improving model controllability in rorl-world applications. We discuss possi- ble limitations and future works in Appen.
This corollary highlights how distinct, well-defined training tasks can overlap at inference. The conflict arises naturally due to the associative memory structure of the weight matri- ces tied specific tokens. The model's output preference depends on the relative strengths of coefficients CI..... C4, which influenced by factors like the learning rate and the number of (task) samples. Notably, the coefficient C; should be sample-dependent in practice. (Yu et al., 2023) found that models are more likely to generate the parametric answer when the corresponding fact appears frequently in the pretraining data, aligning with our results.

---

Taming Knowledge Conflicts in Language Models
Acknowledgement
Schubert, L.,, Voss, C.,, Egan, B., and Lim, S. K. Thread: 2020.
circuits. Distill, 5(3):e24, 2020.
We appreciate Ruizhong Qiu for the early discussion about the work. This work is partially supported by NSF (2416070). The content of the information in this document does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred. The U.S. Government is authorized to reproduce and dis- tribute reprints for Government purposes notwithstanding any copyright notation here on. This research used the Delta advanced computing and data resource which is supported by the National Science Foundation (award OAC 2005572) and the State of Illinois. Delta is a joint effort of the Univer- sity of Illinois Urbana- Urbana-Champaign and its National Center for Supercomputing Applications. This work used the Delta system at the National Center for Supercomputing Applica- tions through allocation CIS250054 from the Advanced Cy- berinfrastructure Coordination Ecosystem: Services Sup- port (ACCESS) program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.
Chang, T. A. and Bergen, B. K. Language model behavior: A comprehensive survey. Computational Linguisties, 50 (1):293-350) 2024.
Chen, H.-T., Zhang, M., and Choi, E. Rich knowledge sources bring complex knowledge conflicts: Recalibrat- ing models to reflect conflicting evidence. Goldberg, Y., Kozareva, Z., and Zhang. Y. (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Lan- guage Processing, pp.
Abu Dhabi, United Arab Emirates, December 2022. Association for Compu- tational Linguistics. doi: 10. 10.18653/v1/2022..o 146. URL htt http:///laaaathhocco ogy .rrgy.org///// 2022. emnlp mmnlp-main.1.
main.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
Impact Statement
Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T. et al. A mathematical framework for transformer circuits. Transformer Circuits Thread, 1(1): 2021.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Elhage, N., Hume, T., Olsson, C.,, Schiefer, N., Henighan,
T., Kravec, S., Hatfield-Dodde 7., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv 2022.
References
preprint arXiv:2209.1065
Ahn, K., Cheng, X., Daneshmand, H., and Sra, S. Trans- formers learn to implement preconditioned gradient de- scent for in-contex: learning. Advances in Neural Infor- 2023.
Fang, T., Wang. Z., Zhou, W., Zhang, H., Song, Y., and Chen, M. Getting sick after seeing doctor? diagnos- ing and mitigating knowledge conflicts in event tempo- ral reasoning. In Duh, K., Gomez, H., and Bethard, S. (eds.), Findings of the Association for Computa- tional Linguistics: NAACI. 2024, pp. 3846-3868, Mex- ico City, Mexico, June 2024, Association for Computa- tional Linguistics. doi: 10.18653 fin.2i4.fiiiing--ee naacl. 244. URL hwtpt:ttt////. //asss///cca f Findings- http//alaaatttt ogy org/ 2024. 244/.
mation Processing Systems,
Arora, S., Li, Y., Liang, Y., Ma, T., and Risteski, A. Linear algebraic structure of word senses, with applications to polysemy. Transactions of the Association for Computa- fional Linguistics, 1-483-455, 2018
Bellagente, M., Tow, J., Mahan, D., Phung, D., Zhuravin- skyi, M., Adithyan, R., Baicoianu, J., Brooks, B., Cooper, N., Datta, A., et al. Stable Im 2 1.6 b technical report. arXiv preprint ar arxii:2402..78:22
Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang, H. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv: 11992.00099992 2023.
2024.
Bietti, A., Cabannes, V., Bouchacourt, D., Jegou, H., and Bottou, L. Birth of transformer: A memory viewpoint. Advances in Neural Information Processing Systems, 36, 2024.
Geva, M.. Schuster, R., Berant, J., and Levy, O. Trans- former feed-forward layers are key-value memories. In Moens, M.-F., Huang, X., Specia, L., and Yih, S. W.-t. (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 1484-5495, Online and Punta Cana, Dominican Republic, November 2021. Association for Computa- tional Linguistics. doi: 1/653//1/20211 1000021.mmlpmmainn.
Cabannes, V., Dohmatob, E., and Bietti, A. Scaling laws for associative memories. arXiv preprint 1rXiv:2310.02984, 2023.
Cammarata, N., Carter, S., Goh, G., Olah, C., Petrov, M.,
10

---

Taming Knowledge Conflicts in Language Models
446. URL https: /tte:://alllaii httpas////clammooooooorrrg emn monlp-mainee 446/.
the Association for Compulational Linguistics ACL 2024,
pp. 1193-1215, 2024b.
Ghosal, G. R., Hashimoto, T., and Raghunathan, A. Un- derstanding finetuning for factual knowledge extraction. In International Conference Machine Learning, pp. 15540-15558, PMLR, 2024.
K Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. Natural questions: a benchmark for ques- tion answering research. Transactions of the Association for Computational Linguisties, 7:453-466, 2019.
Goyal, S., Baek, C., Kolter, J. Z., and Raghunathan, A. Context-parametric inversion: Why instruction finetun- ing may not actually improve context reliance. In The Thirteenth International Conference on Learning Repre- sentations.
Li, J., Raheja, V., and Kumar, D. Contradoc: Understand- ing seff-contradictions in documents with large language models. arXiv preprint arXiv:2371.09122 2023a.
Li, X. L., Holtzman, A., Fried, D., Liang, P., Eisner, J., Hashimoto, T. B., Zettlemoyer, L.., and Lewis, M. Con- trastive decoding: Open- text generation as opti- mization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), Pp. 12286 72286-12312, 2023b.
Groeneveld, D., Beltagy, I., Walsh, P., Bhagia, A., Kinney, R., Tafjord, O., Jha, A. H., Ivison, H., Magnusson, I., Wang, Y., et al. Olmo: Accelerating the science of lan- guage models. arXiv preprint arXiv:2402.00888,, 2024.
Huang, L., Yu, W., Ma. W., Zhong, W., Feng, Z., Wang, H., Chen, Q. Peng, W., Feng, X., Qin, B., et al. A survey on hallucination in large language models: Principles, taxon- omy, challenges, and open questions. ACM Transactions on Information Systems, 2023.
Li, Y., Li, Y., and Risteski, A. How do transformers learn topic structure: Towards a mechanistic understanding. In International Conference on Machine Learning, pp. 19689-19729. PMLR, 2023c.
Javaheripi, M., Bubeck, S., Abdin, M., Aneja, J., Bubeck, S., Mendes, C. C. T., Chen, W., Del Giorno, A., Eldan, R., Gopi, S., et al. Phi-2: The surprising power of small language models. Microsoft Research Blog, 13):3, 2023.
Liu, A. and Liu, J. The memotrap dataset, 2023. URL https: /ttp::/gghhhbbbu memo-trap.
Longpre, S., Perisetla, K., Chen, A., Ramesh, N., DuBois, C., and Singh, S. Entity- -baed knowledge conflicts in question answering. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pp. 7052-7063, 2021.
Jiang, M., Huang, T., Guo, B., Lu, Y., and Zhang, F. En- hancing robustness in large language models: Prompting for mitigating the impact irrelevant information. arXiv preprint 4rXiv:2408.1 10615, 2024a.
Lv, A., Chen, Y., Zhang, K., Wang, Y., Liu, L., Wen, J.- R., Xie, J., and Yan, R. Interpreting key mechanisms of factual recall in transformer-based language models. arXiv preprint arXiv:2403. 19521, 2024.
Jiang, Y. Rajendran, G., Ravikumar, P., and Aragam, B. Do 11ms dream of clephants (when told not to)? latent con- cept association and associative memory in transformers. Advances in Neural Information Processing Systems, 37: 67712-67757, 2024b.
Mahankali, A. V., Hashimoto, T., and Ma, T. One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention. In The Twelfth International Conference on Learning Representations.
Jin, M., Mei, K., Xu, W., Sun, M., Tang, R., Du, M., Liu, Z and Zhang, Y. Massive values in self-attention modules the key to contextual knowledge understanding. arXiv preprint ar arXiv:2502.01556 2025.
Mahdavi, S., Liao, R., and Thrampoulidis, C. Memorization capacity of multi-head attention transformers. In The Twelfth International Conference on Learning Represen- tations.
Jin, Z., Cao, P., Chen, Y., Liu, K., Jiang. X., Xu, J., Qi- uxia, I.., and Zhao, J. Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval- augmented language models. In Proceedings of the 2024 Joint International Conference on Computational Lin- guistics, Language Resources and Evaluation (LREC- COLING 2024), pp. 16867-16878, 2024a.
McDougall, C., Conmy, A., Rushing, C., McGrath, T., and Nanda, N. Copy suppression: Comprehen- sively understanding an attention head. arXiv preprint arXiv:2310.0462. 2023.
Jin, Z., Cao, P., Yuan, H., Chen, Y., Xu, J., Li, H., Jiang, X., Liu, K., and Zhao, J. Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models. In Findings of
Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in gpt. Advances in Neu- ral Information Processing Systems, 35:17359-17372 2022a.

---

Taming Knowledge Conflicts in Language Models
Meng, K., Sharma, A. S., Andonian, A., Belinkov, Y., and Bau, D. Mass-editing memory in transformer. arXiv preprint arXiv:2210.07233 2022b.
Shi, W., Han, X., Lewis, M., Tsvetkov, Y., Zettlemoyer, L.,, and Yih, W.-t. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 783-791, 2024b.
Nanda, N., Rajamanoharan, S., Kramár, J., and Shah, R. Fact finding: Attempting to reverse-engineer factual re- call on the neuron level. In Al Alignment Forum, 2023c.,
pp. 19, 2023.
Tan, H., Sun, F., Yang, W., Wang, Y., Cao. Q. and Cheng, X. Blinded by generated contexts: How language models merge generated and retrieved contexts when knowledge conflicts? In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 6207-6227, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.166633/11/20244.c.oooooggo. aclanthology 2wwpr///2224...
Nichani, E., Lee, J. D., and Bietti, A. Understanding factual recall in transformers via associative memories. arXiv preprint orXiv:2412.065366 2024.
Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., et al. In-context learning and induction heads. arXiv
preprint arXiv:2209. 1895, 2022.
URL https:
Qian, C., Zhao, X., and Wu, S. T. merge conflicts!" exploring the impacts of external distractors parametric knowledge graphs. arXiv preprint arXiv:2309.08594, 2023.
Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L.., Rivière, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08225, 2024.
Qu, C., Dai, S., Wei, X., Cai, H., Wang, S., Yin, D., Xu, J., and Wen, J.-R. Tool learning with large language models: A survey. Frontiers of Computer Science, 1988):198343, 2025.
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Rabiza, M. A mechanistic explanatory strategy for xai. arXiv preprint urXiv:2411.01332 2024.
Roberts, A., Raffel, C., and Shazeer, N. How much knowl- edge can you pack into the parameters of language model? In Webber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Proceedings of the 2020 Conference on Empiri- cal Methods in Natural Language Processing (EMNLP), pp. 5418 5426, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/20200 emnlp- main
Wang. K. R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: circuit for in- direct object identification in gpt-2 small. In The Eleventh International Conference on Learning Representations,
Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., and Li, J. Knowledge editing for large language models: A survey. ACM Computing Surveys, 57(3):1-37, 2024.
Shi, D., Jin, R., Shen, T., Dong, W., Wu, X., and Xiong, D. Irean: Mitigating knowledge conflicts in I1m generation via identifying and reweighting context-aware neurons. In Globerson, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomezak, J., and Zhang, C. (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 4997-5024. Curran Associates, Inc., 2024a. URL https: wwwppreeeeiig...ui.........eeeee oc 2aa9e228996016dddd33333 (atpaaa00060603000b033e0.0000
Wang. Y., Feng, S., Wang. H., Shi, W., Balachandran, V., He, T., and Tsvetkov, Y. Resolving knowledge conflicts in large language models. In First Conference on Language Modeling. b.
Wu, S., Xie, J., Chen, J., Zhu, T., Zhang, K., and Xiao, Y. How easily do irrelevant inputs skew the responses of large language models? In First Conference on Language Modeling.
cc/papee paper_ htpp/app_rleeelleee fww.fflleepeeeee ptt.fflls///pppee///////e/e/
Sateboo--app---ooffeeeeee Conference.
pdf.
Xi, Z. Chen, W. Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. The rise and potential of large language model based agents: A survey. Science China Information Sciences, 68(2):12110) 2025.
Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E. H., Schärli, N., and Zhou, D. Large language models can be easily distracted by irrelevant context. In Inter- national Conference on Machine Learning, pp. 31210- 31227. PMLR, 2023.
Xie, J., Zhang, K., Chen, J., Lou, R., and Su, Y. Adaptive chameleon or stubborn sloth: Revealing the behavior of
12

---

Taming Knowledge Conflicts in Language Models
large language models in knowledge conflicts. In The Twelfth International Conference Learning Represen- tations, 2024. URL https: http:///ppeeeeeeeeeeeeeeeee net/ Darum?id=auKAUJzNG.
Findings of the Association for Computational Lin- guistics: EMNLP 2023, pp. 14544-14556, Singa- pore, December 2023. Association for Computational 118865//w12223.fnddinn
Linguistics. doi:
968. URL https: / / aeeelantholl orry.orrg/2233 emmin. 968/.
Xu, R., Qi, Z., Guo, Z., Wang, C., Wang, H., Zhang, Y., and Xu, W. Knowledge conflicts for llms: A survey. arXiv preprint 4rXiv:2403.08319, 2024.
Ying, J., Cao, Y. Xiong, K., Cui, L., He, Y., and Liu, Y. Intuitive or dependent? investigating Ilms' behavior style to conflicting prompts. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4221-4246 2024.
Yoran, O., Wolfson, T., Ram, O., and Berant, J. Making aetrieval-uggment language models robust to irrelevant context. In The Twelfth International Conference on Learning Representations.
Yu, Q., Merullo, J., and Pavlick, E. Characterizing mechanisms for factual recall language models. Bouamor, H., Pino, J., and Bali, K. (eds.), Proceed- ings of the 2023 Conference in Natural Language Processing, pp. 9924-9959, Sin- gapore, December 2023. Association for Computa- tional Linguistics. doi: 10. 10.18655//1/222333mmml / http://aaaaaattl....ooo/////////
Empirical Methods
615. URL emnlp-main 615/.
Yuan. X., Yang. Z. Wang, Y., Liu, S., Zhao, J., and Liu, K. Discerning and resolving knowledge conflicts through adaptive decoding with contextual information- entropy constraint. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.). Findings of the Association for Computational Linguistics: ACL 2024, pp. 3903-3922, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.18653/v1/2244 findings-acl.234. URL https: https://aclanne https///aaaathoogy.... org/2024
Zhang, M. and Choi, E. Mitigating temporal mis- alignment by discarding outdated facts. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Nat- ural Language Processing, pp. 14213-14226, Sin- gapore, December 2023. Association for Computa- tional Linguistics. doi: 10.18663/w1/2023.emmppp 879. URL https: hts///alaante //taaaaaalllooooo..e/o/.coooollt orgp emnlp-maing inm.min... 879/.
Zhang, R., Frei, S., and Bartlett, P. L. Trained transform- ers learn linear models in-context. Journal of Machine Learning Research, 15 2024.
Zhou, W. Zhang, S., Poon, H., and Chen, M. Context-faithful prompting for large language mod- els. In Bouamor, H., Pino, J., and Bali, K. (eds.),
13

---

Taming Knowledge Conflicts in Language Models
Contents
Related Works
15
Background
16
Conflict Examples
16
Expanded Experiment Section
17
D.1 Detailed Setups and Hyperparameters
17
D.2 Comprehensive Model Experiments
19
D.3 Details on Robustness Study
19
Algorithm Details
21
Limitations and Future Works
21
Theoretical Analysis
22
G.I Setups
23
G.2 Additional Notations
24
G.3 General Assumptions
24
G.4 Proofs
25
14

---

Taming Knowledge Conflicts in Language Models
A. Related Works
Knowledge Conflict. A considerable body of work has investigated the behavior of LMs in the presence of knowledge conflicts across various scenarios (Longpre et al., 2021; Chen et al., 2022; Wang et al., b; Tan et al., 2024; Jin al., 2024a; Xie et al., 2024; Ying et al., 2024; Qian et al., 2023). In general, these studies expose "context-parametric" conflicts, wherein LLMs exhibit ambiguity when contextual knowledge contradicts their parametric knowledge. However, these works do not delve into why these conflicts occur.
Two notable exceptions, Yu et al. (2023) and Jin et al. (2024b), take mechanistic perspective to analyze knowledge conflicts on narrow datasets, proposing "memory heads" versus "context heads." " In contrast, our work adopts a broader scope, covering multiple conflict types and diverse datasets. We go beyond their assumption by revealing the superposition of knowledge conflicts and attaining substantially improved performance over prior methods. Additionally, we shed light on the underlying causes of these conflicts, including the observation by Yu et al. (2023) that the frequency of fact in the pre-training corpus correlates with stronger tendency to produce parametric answers.
Beyond context-parament conflict, recent survey (Xu et al., 2024) identifies two additional forms of conflicts: inter- context conflicts (Li et al., 2023a), involving contradictory information within the provided context, and intra-memory conflicts (Chang & Bergen, 2024), arising when LLMs produce inconsistent responses to queries that are semantically identical but syntactically different. These two conflict types lie outside the scope of this paper, though they represent promising directions for future research.
RAG Hallucination and Irrelevant Contexts. "RAG Hallucination" and "Irrelevant Context" represent two contrasting perspectives on the knowledge conflicts studied in this paper. The former strives for models to rely exclusively on provided contexts, whereas the latter treats external context as potentially misleading source of information.
For RAG hallucination, many methods have been proposed improve faithfulness to context. These methods include two inference-time time categories: (1) Decoding- brcoddng-based approaches (Shi et al., 2023; Yuan et al., 2024) that amplify discrepancies in the output distribution with and without context, and (2) Prompt-bused approaches (Zhou et al., 2023; Zhang & Choi, 2023) that instruct the model to attend closely to contextual input. Additionally. finetuiin based methods reduce reliance on parametric knowledge through utilizing counterfactual knowledge conflict data (Longpre et al., 2021; Fang et al., 2024), although Goyal et al. reveals that certain instruction- finetuning can paradoxically increase the model's dependence on parametric knowledge. More recent work also leverages mechanistic insights (Shi et al., 2024a).
For irrelevant context, Shi et al. (2023); Wu et al. show how noisy or misleading contexts can negatively influence a model's ability to produce correct answers. Some works mitigate this effect through prompting (Jiang et al., 2024a) or finetuning ( (Yoran et al.).
Different from these works, our approach is more comprehensive and proposes lightweight, training-free techniques that allow steering an LLM toward either contextual or parametric knowledge on demand. We stress that both perspectives are valuable, and there is no absolute "correct" behavior. As demonstrated in this paper, knowledge conflicts arise at inference due distinet, well-defined (but contradictory) rules established during training. Our view aligns with Xu et al. (2024), leaving the choice of which knowledge source to prioritize up to the user and the application's 's needs.
Mechanistic Interpretability: Superposition and Intervention Mechanistic interpretability has garnered significant attention, with numerous works aiming to reverse engineer the hidden computational processes of large language mod- els (Cammarata et al., 2020; Elhage et al., 2021; Rabiza, 2024; Wang et al., a; Lv et al., 2024; Jin et al., 2025). Notably, (Arora et al.. 2018; Elhage et al., 2022) highlights the widespread phenomenon of polysemanticity, where neural networks often encode unrelated concepts within single neuron. Despite this recognition, popular intervention methods, such as Knowledge Editing (Wang et al., 2024), primarily modify model weights directly without accounting for the effects of superposition. In contrast, our work extends the concept of superposition to knowledge conflict and demonstrates how this understanding inspires our designs. We believe that our approach has the potential to be integrated with other intervention methods, such as knowledge editing or steering vectors, to enhance their effectiveness and interpretability. In addition, similar to our Observation 2, McDougall et al. (2023) shows the "Hydra Effect", where ablating one layer causes the other to compensate.
Associative Memory and Factual Recalls. Large language models are known to store vast amounts of knowledge in their weights (Geva et al., 2021; Roberts et al., 2020). Many existing studies adopt a mechanistic perspective on locating and
15

---

Taming Knowledge Conflicts in Language Models
editing the stored facts, primarily focusing on the feed-forward modules (Meng al., 2022a:b; Nanda et al., 2023; Wang al., 2024). More recently, attention modules have also been viewed associative memory (Bietti et al., 2024; Cabannes et al., 2023; Jiang et al., 2024b), and theoretical research further explores their capacity for memorization (Mahdavi et al.; Nichani et al., 2024). Nevertheless, these studies have yet to draw connection between associative memorization and knowledge conflicts. Our study also reveals that attention head could be vital for factual recall, aligning with the latter but less popular view of memorization.
B. Background
In this section, give brief overview of large language models. An autoregressive language model M learns probability distribution over vocabulary space V. Given an input sequence of tokens 21.t, the model first maps each token to corresponding embedding vector 2; via an embedding layer. These embeddings are subsequently passed through L decoder layers, each consisting of an attention module and an MLP module.
Let xt (1-1) denote the embedding of token at the previous layer (L - 1). Then, the update rule at the /-th layer can be
written as:
st It (1l-1) + Attn (9) + m
(5)
where Attn, and my are the outputs of the attention and MLP modules at layer respectively.
The attention module typically employs nı heads, each computing learned query, key, and value representations:
Qn = XWW, KA XWK, Vh = XWW
where X E Txx contains token embeddings (batch dimension omitted), and W W W. E Raxdi, Each head output
Qn Kn
(ead;(() softmax
Vh,
ddz
and ny heads are concatenated and projected back to IRdd
Attn [altiHead((((1] Concat(head) hhadd, headm,) w
where W E gm dx)xd.
After the attention module, the embeddings are fed into a position wise feed-forward network (often called MLP). parameterized by an up-weight matrix wilf) and a down-weight matrix w function Act (e.g,, GELU), The MLP output given by:
downn? combined with a non-linear activation
( m = Act (1((t)1) + Aun!") w w(l))
down
(6)
After all L. decoder layers, final unembedding layer projects the last hidden state back onto the vocabulary space V, producing a probability distribution over possible next tokens.
C. Conflict Examples
In Section 2, we outlined three types of conflicts we use for the parametric datasets. We provide some samples from them below.
16

---

Taming Knowledge Conflicts in Language Models
Parametric Dataset Examples
Athlete Sport.
Clean Input: Lebron James plays the sport of
Substitution- based (Sentence-level) Conflict: Lebron James plays the sport of tennis. Lebron James plays the sport of
Lebron James plays the sport of tennis. As a celebrated athlete, Lebron James has become synonymous with excellence in tennis, inspiring fans worldwide with their remarkable performances. Known for their dedication and unparalleled skill, Lebron James has dominated the world of tennis, earning accolades and admiration from peers and spectators alike. Tennis is not just a sport for Lebron James-it their passion, their craft, and the legacy they continue build. Question: What sport does Lebron James play? Answer: Lebron James plays the sport of
Coherent (Paragraph--evel) Conflict:
Company Headquarter.
Clean Input: The headquarters of Amazon are located in the city of
Substitution (Sentence-level) Conflict: The headquarters of Amazon are located in the city of Tokyo. The headquarters of Amazon are located the city of
The headquarters of Amazon are located in the city of Tokyo. As the central hub of operations, Tokyo serves as the strategic heart of Amazon, where key decisions are made and innovations are born. This vibrant city is synonymous with Amazon, symbolizing its commitment to excellence and progress. The connection between Amazon and Tokyo defining aspect the company's identity and global presence. Question: Where the headquarters of Amazon located? Answer: The headquarters of Amazon located in the city of
Coherent (Paragraph-level) Conflict:
World Capital.
Clean Input: The name of the capital city of France is
Substitution-based (Sentence-level) Conflict: The name of the capital city of France is Beijing. The name of the capital city of France is
The capital city France Beijing. Known for its vibrant culture and historical landmarks, Beijing is often seen as the heart France, attracting visitors from around the globe. a center for politics, arts, and commerce, Beijing perfectly encapsulates the spirit of France, making an essential destination for anyone exploring the country. Question: What the capital city of France? Answer: The capital city of France is
Coherent (Paragraph-level) Conflict:
We note that a well-trained LM expected to achieve high accuracy on clean inputs, moderate-to- to-low accuracy on substitution-bat conflicts, and near-zero performance on coherent conflict scenarios. The coherent conflict was proposed by (Xie et al., 2024).
D. Expanded Experiment Section
In Section 4, we illustrate the effectiveness of JUICE by demonstrating its strong intervention performance with three models. Due to the page limit, we omit many details and results. This appendix section serves as a complementary and expanded experiment section to the main paper.
D.I. Detailed Setups and Hyperparameters
Parametric Dataset Setups. While the general philosophy of the parametric dataset and detailed conflict examples are described in Section 2 and Appendix C, we provide additional details on the dataset curation process here. In general, we follow (Jin et al., 2024b) extracting common knowledge triplets from Wikidata. These extracted pairs are verified
17

---

Taming Knowledge Conflicts in Language Models
for correctness using GPT-4 and manual checks. Using the verified entities, we create specific instances (as shown in Appendix C) for clean, substiaution-conflie,, and coherent-conflict prompts by substituting key entities of a template. The coherent prompt template was generated by GPT-4o and verified manually for correctness and fluency. To ensure that our method does not overfit specific template. we conduct a robustness study detailed in Appendix D.3. The sizes of the dataset are around 200 for world capital, official language, and company founder, and around 500 for athlete sport, company headquarters, and book author.
Contextual Dataset Setups. Contextual datasets have been introduced in Section and we expand upon the two contextual datasets (NQ-Swap and MemoTrap) below:
Open-domain Question Answering: NQ-Swap is derived from the guestion-answering dataset NQ (Kwiatkowski et al., 2019), designed to test the ability to answer questions based on a reliable gold context. Unlike the factual recall tasks our parametric setup, this dataset offers more comprehensive coverage to evaluate the effectiveness of the proposed methods.
Diverse Context Types: Memo encompasses four distinct tasks: Hate Speech Ending, History of Science QA, Proverb Ending, and Proverb Translation. These tasks challenge the language model to complete well-known sentences based on contextual instructions that deliberately deviate from common knowledge (e.g., "Wrile a quote that ends in the word 'early': Better late than"), "). ). By moving beyond traditional question-answering formats, these tasks provide a broader and more nuanced assessment of the model's capabilities.
Detailed Experiment Setups in Sec. 3. For the experiments corresponding to Figure 3, we calculate the average probability value of the first (correct) token for each data sample and use that average as our final score. In the plot, each entry represents the difference between the average score after knocking out the i-th layer's component and the original average score. The shaded regions indicate the standard deviations across samples. All results are obtained where the model answers each clean imput prompt correctly (so the correct probability value is the parametric probability value). In the experiments corresponding to Table probability during substitution conflicts. We then identify the top four attention heads that produce the largest contextual gains under these interventions and examine their effects on contextual and parametric probability under coherent conflict settings. For the experiments related to Table 2, we use a small fraction of samples from the filtered World Capital dataset to identify attention heads that achieve the highest parametric probability gains under coherent conflicts when knocked out. We then evaluate the influence of knocking out these selected heads on the remaining dataset together according their ranks. This setup mimics realistic scenario where access test set information is unavailable.
filtered world-capital dataset,
the same dataset to measure the average change in context
Hypeeparaaetees. For JUNE, JUICE, PH3,, and РH3., the head identification set is fixed to be world capital for the parametric dataset, and proverb ending for the contextual dataset. PH3, leverages larger 200 development set and PH3, shares the same head identification set with JUNE and JUICE. For PH3, and PH3,, we follow their original setting of tuning the number pruned heads from 31, 5, 9, 15} based on validation. For JUICE and JUNE, we fix K 5 for smaller-scal models (Gemma, Phi2, Stablelm2) and K 10 for larger-sized models (Llama2, Llama3, Olmo). We choose the scaling factor + and based on validation, where a is tuned from {0, 1, 2, 3, 5) and a is tuned from {0. 1, 2, 33. For CAD, we follow their choice of setting = 1 the knowledge conflict dataset. For Prompt, apply the following instructions before the standard task prompt:
Prompt Instructions
Parametric Dataset, Substitution Conflict. Ignore the preceding statement and rely only your pre-trained knowledge. Complete the sentence accurately based on your memory of the world: Parametric Dataset, Coherent Conflict. The following passage contains misleading information. Ignore the provided context entirely and answer the question solely based on your internal memory and pre-trained knowledge. Contextual Dataset, Sentence Completion Type Dataset. Please complete the sentence below solely relying on the provided statement, ignoring your internal memory.
Contextual Dataset, Question Answering Type Dataset. Please answer the following question based the given context, ignoring your internal memory.
18

---

Taming Knowledge Conflicts in Language Models
D.2. Comprehensive Model Experiments
We provide additional model results, following the same setup as Section 4. Table 5 and Table show the result. The main conclusions from the main paper still hold.
Table Full Results of intervention for enhancing parametric memory. All results are in accuracy (%). Bold denotes the hest result.
Athlete Sport
Book
Company Founder
Company Headquarter
Official Language
World Capital
Dataset
Author
Averuge
Conflict Type
3
3
3
3
3
1
3
1
3
Original
93.4
18.1
0.0
73.0
7.7
0.0
47.0
2.7
0.0
64.2
0.7
0.0
96.9
23.5
0.0
94.1
15.1
1.1
78.1
11.3
0.2
Prompt
93.4
44.5
73.0
22.4
47.0
6.5
3.1
50.0
22.2
50.8
35.7
78.1
29.6
10.5
Gemma
PH3_1
86.6
71.6
33.3
33.3
4.8
0.0
28.1
10.8
19.5
44.3
22.4
30.6
90.7
72.8
82.7
84.3
64.3
88.1
61.2
41.1
42.4
PH3_s
93.2
75.3
0.0
21.8
19.3
0.2
42.7
5.4
0.0
62.0
0.7
0.0
82.7
37.7
0.0
78.9
15.7
0.5
63.5
25.7
JUNE (Ours)
91.2
63.2
65.9
78.0
61.0
2.9
46.5
44.9
41.1
57.9
36.2
38,9
94.4
82.1
84.0
91.9
69.2
83.2
76.7
59.4
52.7
JUICE (Ours)
96.3
95.4
91.9
79.8
75.5
68.0
45.4
39.5
43.2
65.8
60.0
59.3
93.2
86.4
85.2
94.1
95.1
93.0
79.1
75.3
73.4
Original
90.4
47.0
29.3
95.77
46.9
95.1
22.3
25.9
Prompt
90.4
70.2
0.2
81.4
65.1
22.0
$7.5
16.6
24.3
75.2
38.0
15.7
95.7
79.6
40.7
95.1
60.3
15.88
82.5
55.0
19.8
Llama2
PH3_I
91.0
87.4
37.5
77.8
92.0
70.9
53.0
52.2
32.6
73.4
74.0
12.1
94.4
90.7
84.0
94.2
95.7
90.2
80.6
82.0
54.5
PH3.s
89.0
88.1
10.5
80.2
86.1
64.5
52.7
50.0
34,0
73.4
72.9
18.5
94.4
85.5
80.7
94.0
91.3
85,3
80.6
79.0
48.9
JUNE (Ours)
89.9
61.6
50.4
77.1
85.6
79.8
53.6
47.0
40.9
72.2
66.3
64.0
93.8
92.0
95.7
94.6
94.0
95.7
80.2
74.4
71.1
JUICE (Ours)
91.5
88.6
91.0
82.8
91.1
88.5
53.0
51.9
54.1
74.3
74.3
73.6
96.11
93.8
94.4
95.4
95.4
96.2
82.2
82.5
83.0
Original
84.1
22.2
0.0
55.6
2.2
0.0
61.1
3.3
0.0
80.3
1.4
1.8
96.3
20.4
0.6
94.6
16.8
0.0
78.7
11.0
0.4
Prompt
87.4
55.6
77.7
38.3
48.2
96.3
85.2
83.8
11.9
78.7
70.1
Llama3
PH3_I
86.4
86.5
14.1
75.3
87.4
4.9
55.6
48.9
30.6
78.0
55.3
9.4
96.3
96.3
84.0
93.0
94.1
92,4
80.7
78.1
39.2
PH3.s
86.5
86.3
12.5
61.1
84.8
6.8
58.3
51.7
27.8
70.0
56.2
26.8
96.3
95.8
87.0
91.4
87.6
90.3
77.3
77.1
41.9
JUNE (Ours)
82.8
72.8
58.7
66.2
92.1
83.00
61.7
51.1
54.4
80.5
56.9
56.0
95.7
95.7
93.2
94.1
95.7
96.8
80.2
77.4
73.7
JOICE (Ours)
87.0
87.8
95.9
86.5
92.3
88.7
61.7
56.7
35.6
79.8
75.9
74.8
96.3
96.3
95.7
95.7
96.2
97.3
84.5
84.2
84.7
Original
56.1
68.9
10.8
73.6
21.1
75.9
29.0
Prompt
84.8
57.2
19.6
68.9
10.8
6.8
46.5
9.7
3,2
73.6
7.0
0.0
95.7
24.1
64.8
92.4
3.8
57.8
77.0
18.8
25.4
Olmo
PH3_I
85.0
82.1
35.7
70.3
84.0
70.5
44.9
50.3
34.1
68.4
64.1
53.9
95.1
92.0
93.0
95.1
87.6
76.2
78.4
62.3
PH3 s
83.0
78.2
1.1
64.9
83.8
34.0
36.2
36.2
9.7
70.5
52.3
5.0
94.4
93.8
62.3
91.9
91.4
34.1
73.5
72.6
24.4
JUNE (Ours)
67.4
66.5
39.1
72.6
83.6
57.2
45.4
44.9
38.7
68.6
55.7
61.6
94.4
92.6
92.6
93.0
94.6
91.4
73.7
73.0
63.4
JUICE (Ours)
82.4
75.2
48.3
73.2
85.8
72.3
47.6
48.6
41.3
72.0
68.55
56.4
95.
94.4
87.0
93.2
95.7
93.5
77.2
77.5
66.5
Original
61.8
15.3
0.0
$5.8
16.3
0.0
34.6
5.9
0.0
36.2
3.2
0.0
93.3
88.3
0.0
93.00
61.6
0.0
62.44
31.8
0.0
Prompt
61.8
11.7
55.8
11.5
0.0
34.6
5.3
36.2
2.4
93.3
72.4
49.2
25.4
Phi2
PH3JI
62.1
14.7
0.0
55.6
16.8
0.0
34.6
4.8
0.0
36.4
3.2
0.0
93.3
90.2
0.0
93.0
76.2
0.0
62.5
34.3
0.0
PH3 s
61.6
15.5
0.0
55,0
14,6
0.0
34.6
5.3
0.0
36.8
2.4
0.0
92.6
89.6
0.0
94.1
74.1
0.0
62.4
33.6
0.0
JUNE (Ours)
61.0
8.8
31.4
54.1
48.1
43.7
35.6
24.5
0.0
34.3
3.2
7.1
93.3
92.00
87.7
94.1
91.4
92.4
62.0
44.7
43.7
JOIC (Ours)
62.6
36.0
46.3
53.6
50.3
52.5
36.2
26.1
19.1
35.8
23.3
2.1
92.6
92.6
87.1
94.3
91.8
94.1
62.5
53.4
50.2
Original
47.5
6.3
30.2
0.0
14.2
58.7
18.8
14.1
Prompt
88.2
0.0
0.0
6:3
0.0
0.0
30.2
0.0
0.0
50.5
1.3
0.0
95.1
8.6
0.0
88.7
6.5
0.0
59.8
2.7
0.0
StableLm
PH3.I
89.3
68.7
21.4
5.1
70.5
20.2
30.7
30,9
9.0
49.5
40.9
31,3
95.7
85.8
88.3
80.6
90.3
89,2
58.5
64.5
43.2
PH3
88.8
66.3
19.0
2.4
42.4
17.7
27.0
28.0
1,6
47.9
39.4
8.1
94.4
80,9
61.1
81.7
82.8
76.9
57.1
56.6
30.7
JUNE (Ours)
89.9
$4.9
25.8
54.0
74.9
60.9
27.5
32.8
27.5
43.8
34.8
23.4
94.4
92.0
88.9
87.6
87.1
82.8
66.2
67.8
51.6
JUCC (Ours)
89.7
88.4
58.2
56.2
76.6
68.8
34.9
32.3
30.2
51.0
47.5
38,9
93.2
93.8
95.1
92.5
91.9
89.8
69.6
71.8
63.5
D.3. Details on Robustness Study
In this subsection, we detail the setup we briefly mentioned in Section 4.3. For robustness across the three hyperparameters, we vary the size of the head identification set [D] from to 10, the number of intervened head K from 1 to 30, and the scaling factor combination in {0, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 5.03 x (0. -0., 1.0, 1.5, - 2.0, - 23.0- 1.0.--3.5,4000 -4.5, -5.0. We fix Gemma to be the backbone model and World Capital as the test dataset. We only vary one variable at a time while keeping all other parts fixed. We measure the average accuracy across the three conflict types for the latter two plots. Figure 5a, Figure 5b, and Figure 5c plot the results respectively. It clearly demonstrates that JUICE maintains high performance across a wide range of hyperparameter values. For robustness against paraphrased prompts, we curate multiple prompt templates for each conflict type. During evaluation, a prompt template is randomly sampled generate the desired prompt. We provide (some) templates for the world capital dataset below
Paraphrased Prompts World Capital Clean Input
Clean Datasets. (1) It's crucial to know that the capital city of (subject} is (2) You are right to say that the capital city of (subject) is (3) According to the textbook, the capital city of (subject) is (4) In case you didn't know, the capital city of (subject) is (5) As we all know, the capital city of (subject) is
19

---

Taming Knowledge Conflicts in Language Models
Table Full results of intervention for enhancing contextual knowledge.
NQ
Hate Spe- Ending
History of Science qa
Proverh Ending
Proverb
Model
Method
Swap
Average
Original
38.7
70.7
29.9
26.5
59.0
45.0
Promgt
26.6
CAD
$6.9
81.7
16.9
37.1
62.9
51.1
PH13;
PH33,
50.2
80.2
35.2
50.1
63.2
55.8
(Ours)
JUICE (Ouns)
S8.4
84.1
47.0
74.6
66.8
66.2
Original
34,9
Prompt
39.6
58.5
21.3
25.7
52.5
39.5
CAD
29.8
65.4
20.2
28.6
54.2
41A4
Llama2
РH3
48.2
63.4
2014
68.7
58.8
51.9
PH3
25.3
62.2
16.5
26.5
55.2
37.1
JUNE (Ours)
29.7
76.8
49.3
34.3
52.8
48.6
JUICE (Ours)
49,5
93.9
50.2
77.1
62.6
66.6
Original
18.5
51.2
72.9
24.5
50.1
43.4
Prompi
33.4
53.7
71.7
23.9
51.8
46.9
CAD
34.7
60.8
73.1
33.1
54.1
51.2
Llama3
PH3
25.3
62.2
78.4
48.1
63.6
55.6
PH3,
22.5
51.2
75.1
25.0
51.8
45.1
JUNE (Ours)
26.5
72.5
73.2
33.1
61.8
53.4
(Ours)
Original
17.1
59.8
38.0
25.0
50.8
38.2
Promgt
27.1
51.3
CAD
41.0
62.2
25.5
27.1
51.3
41.4
Olmol
PH13
PH3
21.3
78.0
39.5
29.7
52.0
44.1
(Ours)
JUICE (Ous)
27.4
86.6
48.6
63.0
$6.9
56.5
Original
48.3
Prompt
22.7
85.4
49.0
32.0
41.7
46,2
CAD
41.1
91.5
48.6
34.1
44.0
51,9
Phi2
РH3
24.6
89.0
53.3
39.3
42.4
49.7
PH3
89.0
53.1
32.6
42.2
48.1
JUNE (Ours)
29.0
90.2
53.1
42.2
41.9
51.3
JUICE (Ours)
30.1
89.0
54.1
44.8
43.1
52.2
Original
10.4
69.5
36.1
32.3
52.8
40.2
Prompt
11.3
68.3
40.5
33.4
52.2
41.1
CAD
37.0
73.2
30.3
34.5
54.5
45,9
Stablel.a
PH3
11.5
77.1
39.2
42,1
72.1
48.4
PH3,
9.9
73.1
39.8
38.7
66.1
45.5
JUNB (Ours)
8.1
79.3
35.5
32.3
52.8
41.6
(Ouns)
Paraphrased Prompts World Capital Conflict Datasets
Substitution Conffict Datasets. (I): It's crucial know that (subject) plays the sport of (context answer). It's crucial know that (subject) plays the sport of (2): You right say that {subject) plays the sport of (context answer). You are right to say that {subject} plays the sport of (3); According to the textbook, {subject} plays the sport of (context answer). According to the textbook, (subject) plays the sport of (4): In case you didn't 't know, {subject} plays the sport of (context answer). In case you didn't know, (subject} plays the sport of (5): As we all know, {subject) plays the sport of (context answer). As we all know, {subject) plays the sport of Coherent Conflict Dataset. (I): The capital city of {subject) is (context answer). Known for its vibrant culture and historical landmarks, (context answer) is often seen as the heart of (subject). attracting visitors from around the globe. As center for politics, arts, and commerce, {context answer) perfectly encapsulates the spirit of (subject), making it an essential destination for anyone exploring the country. Question: What is the capital city of (subject)? Answer: The capital city of (subject} is
(2): The capital city of {subject} is (context answer). As hub of cultural heritage and modern development, (context answer) stands out as the most iconic city (subject). With its rich history, bustling streets, and vibrant atmosphere, (context answer] embodies the essence of (subject), drawing travelers and locals alike to its renowned landmarks and attractions. Question: What is the capital city of (subject)? Answer: The capital city of (subject) is
3 more coherent conflict templates are omitted due to presentation issues.
Table 7 presents the results of JuICE when applied to paraphrased prompts. Our findings show that JUICE is highly robust
20

---

Taming Knowledge Conflicts in Language Models
variations in input prompt formats, consistently maintaining its effectiveness across diverse templates. Notably, JUICE still demonstrates superior performance, effectively shifting the model's 's reliance from context parametric memory.
Table 7. Robustness of the proposed method (JvICE) against randomly selected paraphrased prompts. With the exact same intervention
procedure, the table demonstrates that JUICE remains highly robust across different prompt templates.
Athlete Sport
Book Author
Company Founder
Company Headquarter
Official Language
World Capital
Dataset
Average
Model
Method
1
3
2
3
2
3
2
1
2
1
3
1
2
3
Original
96.5
4.0
0.0
57.1
3.6
0.0
40.5
0.0
0.0
61.5
0.2
0.0
95.7
2.5
0.0
94.6
3.8
16.2
74.3
2.3
2.7
Gemma
JUICE
94.5
92.1
29.7
49.9
85.9
70.5
Nlama2
Original
95.6
1.5
0.2
60.1
10.1
0.0
47.5
0.6
0.0
72.9
0.2
1.4
93.8
4.9
0.5
95.1
0.0
77.5
3,4
0.4
JUICE
98.2
68.2
93.6
65.8
86.5
75.0
54.7
50.8
43.6
72.9
74.0
69.9
94.4
82.7
88.3
95.1
91.8
89.7
80.2
75.7
76.7
Llama3
Original
95.0
2.0
0.0
82.3
1.8
0.0
56.7
1.1
0.0
77.1
0.7
1.6
96.3
1.2
1.2
95.1
3.8
7.0
83.8
1.8
1.6
JUICE
95.4
61.6
47.7
95.7
97.3
70.7
E. Algorithm Details
In this section, we explain the algorithm of JUICE and JUNE in detail. Algorithm introduces JUICE.
In Stage 1, JUICE selects two sets of attention heads that consistently achieve the desired parametric-contex, change with either positive or negative scaling across different conflict types. To accomplish this, we use a small, well-designed dataset where the first output token reliably reflects the model's 's context versus parametric tendency. Each attention head is assigned a score, calculated by summing the changes in the probability values of the target tokens over this dataset. The dataset includes multiple forms of knowledge conflict, ensuring robustness against clean inputs, substitution-bast -bssd conflicts, and coherent conflicts, rather than focusing on single type. Each attention head scored separately for each conflict type. To ensure consistency, heads, we compute a final score by summing their scores across conflit types. The top K attention heads based on this final score are selected. Note that multiple scaling factors are applied for each attention head to ensure quasi-monotonicit In Algorithm 3, Scale( M, H., o;) means to scale that activation output of the head H. in model M by factor of o;,
retain only attention heads with positive scores across all conflict types. For these remaining
In Stage 2, JUICE executes a dual-run process: in the first run, saves the activation outputs of the identified attention heads. the second run, it adds the scaled versions of these saved outputs to the corresponding head activations. The scaling factors 8+ and B- are determined using the validation set.
As a meaningful baseline, propose alternative algorithm, JUNE (Just Run Once), which shares the same head identification stage JUICE but omits the dual-run design. Instead, JUNE directly scales the targeted head outputs during single inference run. This simplified design serves as an ablation study, highlighting the significance of JUICE's dual-run mechanism. Algorithm 4 presents the JUNE algorithm in detail.
F. Limitations and Future Works
This work mainly aims to illuminate the mechanisms underlying knowledge conflicts in language models and demonstrates how to leverage them. Our proposed method is designed to effectively prove the understanding of the discovered mechanism and may not best suit the applications where the efficiency requirement is paramount. JUICE requires caching first-run activations, which may slightly affect inference speed and increase memory overhead.
Real-world scenarios often involve partially irrelevant contexts, while we focus on irrelevant cases in this work, and the parametric and contextual answer may not be always distinct under more abstract domains as we discussed in this work. Extending our method these complex cases and settings remains an important direction for future research.
21

---

Taming Knowledge Conflicts in Language Models
Algorithm JUICE
Stage One: Head Identification
Input: model M, a small head slection dataset D, Scaling parameter at {ay) = {ay) n,),
Initialize 5 <- S, <- Dietf), H+ [l, H
fI....,nH!!
St,s t Record Head Score(St, ,,, a, .a) a
H+ H t Filter Inconsistent Head(S* ,
S + -t S + S-lliv
Output: TopKIndex ((5 her+), TopKIndex ({5 hien-)
Stage Two: Intervention
Input: input prompt z, model M, Intervened Heads S (5 ] 1i S2 (ST scaling factors ,+,
Step One: Save Important Streams
Feed into M, Initialize Aux 6 0
for Attention Head Output Hj (with Head Index 7) do
if E S₁ then Aux(1] = H end if
i E S2 then
Aux([] Hi
end if
end for Step two:Intervention
Feed x into M
for Attention Head Output Hy (with Head Index do if E S then
Hi + H1 S+ Aux([]
end if
if I E S2 then
H, + H1 + 8 * Aux[[]
end if
end for Output: Model Prediction
Algorithm 2 Record Head Score
Input: model M, small head slection dataset D, Scaling parameter a (a a :fay }3m Initilize Score Record Dict St S (with entries default to be zero)
for each sample (X, 3) D do for each conflict type j and the input E X do for each head H; E M do for each coefficient a; E a do S [>]l + S [ Py D( HH; = H, + H,) (x)) Py (M (x))
end for for each coefficient a; do
S; billi] S bi[ + P. ((M)Do (HI 11, + aalli) (z)) P, (x))
end for end for end for end for Output: St,S
G. Theoretical Analysis
We provide complete presentation of the theoretical analysis in this appendix section.
22

---

Taming Knowledge Conflicts in Language Models
Algorithm 3 Filter Inconsistent Head
Input: Score Record Diet S S Head Index Set H+ H
for each conflict type j do
for each head index do
if 1TSt []] then H + [+}
end if
if S [ < then
H < M+) end if end for end for Output: H H-
Algorithm 4 JUNE
Stage One: Head Identification
Input: model M, a small head slection dataset D, Scaling parameter a = (ay1ı=1.0 = {ay 1/ Initialize S+ + ,, S + Dictf), 1I+ + ".....,,H.... M H + [I....,,HH] nH)
St, t Record Head Score(S*, S M,, ,
H+ H + Filter Inconsistent Head(S*, "oaasstSS,... ,5, H 5t S + [)[V))
Output: TopKIndex, (St , TopKIndex hie
Stage Two: Intervention
Input: input prompt model M, Intervened Heads S {5 Feed into M
1K 1 S2 (8 scaling factors S 8-
for Attention Head Output Hr (with Head Index do i E S1 then
H Hi 8+ *
Hr
end if
if ffl.c.
S2 then
H H1 + 8 * Hi
end if end for Output: Model Prediction
G.1. Setups
Model Setup. We consider an attention-only Transformer model with two layers, where each layer has single attention head, uses absolute positional encoding, and employs residual connections. Suppose our input is sequence of tokens (21:r), each token 2n drawn from a vocabulary of size N. Our general model setup mimics Bietti et al. (2024). The model processes this sequence in the following way:
Token Embeddings:
Each token (originally one-hot encoded) is mapped into a d-dimensional space via an embedding function d(.)= N - Red We denote the embedded vector for token 2, by s; = ((z)).
Positional Embeddings: For each position / the sequence, there is corresponding positional embedding pt E Rd. We add P. to 2r, giving the full input representation:
2t ((2)) + Pt-
Attention Blocks: Let æ1. E RdxT be the input sequence to causal attention layer. This layer uses key (Wx), query (Wq), value (Wv), and output (Wo) matrices, each in ddxd For each position t, the layer computes
* WoWreLa of T W Wo x.) Wov Fua o T WRQ
23

---

Taming Knowledge Conflicts in Language Models
where is the softmax function and we use Wk W Wov WoWy. Writing this process collectively as Attn (11:7; ( WK, We, Wv, Wo) for the entire sequence, the f-th layer output is then combined with the input (via residual connection):
= FIT + Att((r1:; W W W W
Unembedding: After the second (final) Transformer layer, discrete probability distribution vector over the vocabulary is produced through linear layer Win. We denote Win [p( where (u(i) is the unembeddng vector of token in the vocabulary.
Task Data Setup We consider tasks trained on this two-layer transformer: factual recall and induction.
The objective of the factual recall task is to learn factual associations between the input factual token space S and the output answer token space A. We assume a bijective ground truth mapping G* S - A exists between these two spaces. This setup models real-world knowledge triples, such as (China, capital, Beijing), where (China, capital) is represented by a single factual token 8 and the answer (Beijing) by a single answer token a E The data distribution consists of length (21, .2,....; ,r, 21+1) [N]T+1, generated through the following process:
sequences 21:7+1
1. Sample a fact s and corresponding index i uniformly at random from S and [T 1/, respectively. Set =
2. For all remaining tokens 2ạ where k [ 1],(i), sample z uniformly at random from / without replacement.
3. Set 2/ q and 2141 g (s).
The objective of the induction task is to complete token sequences of the form [... , b, ... ,g] - [b]. where is the token following the second occurrence specific trigger word. For simplicity, we designate q as the sole trigger word (to induce knowledge conflict) and b E A.. The data distribution consists of length T sequences 21:7+1 == (21, 22, 27, 2T+1) [N]=+1, generated follows:
1. Sample an index j uniformly at random from [T' - 2]} and set 2j q. Sample 2j+1 from M.
2. For the remaining tokens, sample zy. uniformly at random from /o M1(z++1) without replacement.
3. Set by 9 and 2T+1 2++1-
In summary, the vocabulary space is defined as = S A (g) U.. We denote the factual dataset by Ds and the induction dataset by Dj.
G.2. Additional Notations
Suppose the embedding of a token is Φ(1), we use (i) to denote its remapped embedding W (i)) Similarly, we use pf to denote WovPs.
We use o; to denote (X (XTWkoZr) W in Proposition G.5. We acknowledge that we sometimes abuse the word usage of (pre-softmax) "logit" with token probability interchangeably.
We use N to denote the size of the vocabulary, and N, for the size W.. We use to denote the size of dataset, with nF to be the size of the factual dataset and ny to be the size of the induction dataset.
G.3. General Assumptions
Assumption G.1 (Near-orthogonal Embeddings). Every embedding, unembedding, and positional vector is i.i.d. random vectors drawn uniformly from the unit sphere S e Rd and the hidden dimension d is large.
This ensures the nearoorthygynality of initialized vectors.
G. ADDITIONAL ASSUMPTIONS IN TRAINING DYNAMICS
Assumption G.2 (Strictly Orthogonal Embeddings). (z4, kz)) = dıy where 2; can be arbitrary input vector (i.e., embedding d(i), unembeddinguu(), or remapped di(i) vector),
24

---

Taming Knowledge Conflicts in Language Models
Assumption G.3 (Dataset Properties). There does not contain any duplicates in the factual recall and induction dataset and each datapoint appears once. In particular, assume that each noisy token E N appears exactly once in the induction dataset the answer token.
We remark that Assumption G.2 is common assumption in existing literature for analyzing the learning dynamics of shallow transformers. Assumption G.3 is rather mild assumption which eases the analysis (avoiding repeated samples).
G.4. Proofs
Proposition G.4 (Existence of a Perfect Solver). There exists two-layer transformer that can solve both induction and factual recall tasks with perfect accuracy.
Proof. The optimal construction can be achieved by setting
T-1 Pt-1Pt
W kQ = C. Σ
T
and Wo to be a random matrix where C' is large constant. The first layer essentially achieves the "copy from previous embedding" effect. In the second layer, we set
Wio = Ci. (V-(WW))()) p()) +C2 Σ ((s))(() and Wov C3 Σ p(k))(k)) +Cs (s)) (8)
#ES
KEN
SES
where C1, C22 C3, C4 are appropriate scaling factors. Consider any input sequences 21,t after passing the embedding and positional encoding layer, have
(0(z)) + +PI...., .(z)) P
as the input. After the first layer, we have
[(0())) + p1) + (6' (6(2)) + P22 + (2)) + +1) + % (Ø(2z) + p3) + (0'(z2 + r22 + 1/- ..., (Ó(z) + p.) (0' (=+-x) + pl-1) +70
where yf is small negligible term due to large C' and d. Now it suffices to examine the last hidden state since only this is used for final prediction.
First, we show that such model can solve the task of factual recall perfectly. Note that with appropriate scaling C2, the attention weight concentrates on the (ø(s) + p.) + (e(()=1) + Pi-1) + 7/ terms. After transformation by Woy. this results in C4p (g* + O C... The logit of the correct answer will dominate OCC4), while other tokens will have smaller logit values O Ca or O(}).
Similarly, the model can also solve the task of induction perfectly. With appropriate scaling C₁ the attention weight concentrates the (ő(E)+1) + P++1)+ (d'(g) + +/)) +7 terms. After transfommation by W this results in Czp (cz+1). producing the correct answer.
Proposition G.5 (Restatement of Proposition 5.3, Learning of the Superposition Layer via Gradient Descent). Let X RIXT be the output of the first layer, which perfectly implements the "copy from previous token embedding" step. Ignoring positional encodings and under the assumptions in Appendix G.3.1, consider a one-layer attention model given by
fw (X) = WIT. XovX (xT (WWOTT)
99
where IT is the embedding of the final token and still freezing Win. to be a random matrix. Then the construction of the weight matrices Wov and W xo from Equation (8) can be learned via gradient descent on the cross-entropy loss from zero initialization to yield perfect accuracy on the training distribution expectation.
25

---

Taming Knowledge Conflicts in Language Models
Lemma G.6 (Gradient Derivations). The gradient Jw (X) in Equation (9) with respect W WxQ and Wov via the cross- entropy loss L can be expressed as following:
T
VWaf = Wiin (ey (ey-o(((X)))
Σ
(10)
-V x (WinhWoxX) (ey- (f (I(X))) oT
(11)
where = (XTWkoRr) and a denotes the foftmaxffunctinn
Proof. We first remark that we will slightly abuse the notation to omit inside & (.). First, let's write the loss function:
C v)) (of
(12)
Note that the model in Equation (9) can also be written as following:
f(X) Σ
W lin Woy
(13)
where = -x (xTTXKOO WKOFT), denotes the attention weight of the i-th toke. We first derive the gradient with respect to Wov:
Oc (x)
V C Of @Wov
(14)
(x)) oWov
((a (f (X) e,)),
(15)
which the first part is obtained from gradient of Cross-entropy loss wrt. pre-softmax logits. We now focus on the second part, which has that
(x)
T Σ o a,Wii Wovz,
T Σ 1=1
à aWov
σ.
W Wova,
(16)
aWov
aWov
Notice that each W. ov E R is a N x vector and therefore the differentiation result is tensor if we write in a compact form. Let's denote I, = WT W Wovw; then we have k-th component to 4. = =+())" Wova; which gives that
0t. aWov
u(k)aT
(17)
which means that 0f()), WWo
(b()) D 0g Revisiting Equation (15) results in that
26

---

Taming Knowledge Conflicts in Language Models
N
0C Ozk
azk aWov
Wo
V
L
let Z1. = / (X)
kkHL(k))
let 8z (f((()) -cy
k=
Skp(k)
0,34
(18)
T
Σ
(19)
Rewriting this in exact form gives the desired result.
For V Ww applying the chain rule iteratively yields the desired result.
Proof to Proposition G.5. We will show two steps: the first gradient step Iearn the desired Wov, and the second step learns the desired WKO. The training could converge with appropriate in two steps.
Before proceeding to the specific statement, we first rewrite the gradient wrt. Wov and WkQ of a single datapoint (d11:; y):
N Σ Bipl (i)
{
T
Wov C =
Bio (i)
N
T By (H
Σ
3,
og (n(1)
1=1
where we set = = (ey - (X)); At the same time, we have
-W XX WT Win (a (X()) e,)
n W Wun
2i (W T Bxy
T
Σ
se[is Wovar ... J WovarBBat
T
N
Σ
Base Wova; of
Σ
27

---

Taming Knowledge Conflicts in Language Models
where we set (Ex=1 SkmE Wovz,) We will show how this leads to the desired form of Wov and WKQ-
We have one additional simplification for the data setup, where we ignore the remapped embedding of the first position, and the remapped embedding in the last position. We further simplify the setting by ignoring the remapped embedding in the first and last token, so the last position is deterministically p(g) and the first position is Φ(Ez) for some L. We now taxonomize the different types of tokens and get their corresponding probability over the two types of tasks:
For factual recalls, we have
Noisy Tokens: Each (et;) has probability of O T to share the same position with (s)).
to be drawn for single datapoint and probability of O N.
N.
Remapped Noisy Tokens: Each (c;) has probability of O share the same position with ((s).
T N₂
to be drawn once and probability of O N(.)) to
Subject Token and Remapped Subject Token: By Assumption G.3, each (ss) and d'(s) must appear only once in full-batch gradient descent.
Query Token and Remapped Query Token: Φ(4) is deterministically fixed to be the last token for each datapoint. There are no o' (q) the factual recall task.
For induction, have
Selected Noisy Token and Remapped Selected Noisy Token: By Assumption G.3, each Φlez) will be selected as answer token only once a full-batch gradient descent; so does (e()).
Trigger Token and Remapped Trigger Token: o(q) is deterministically appear twice: one before the selected noisy token (,Cz), and the other to be the EOS token. o' (q) is guaranteed to share the same position with the answer token δle,).
(N, be Unselected Noisy Token and Remapped Unselected Noisy Token: Each token (b(x) has probability of O T
drawn for datapoint that it is not the answer and a probaiblity of O Nw
to share the same position with o' (cz). Their
Their
remapped embedding &' (EEx) has a probaiblity of N.. to be drawn for datapoint that ((EN. is not the answer.
Factual Token and Remapped Factual Token: ((s) and o' (s) will not appear in the induction dataset.
We will examine the signal of each token after the gradient steps. In the first step, since we initialize both weight matrices to be zero, we have
1 Vj and 3r Jj T
if + y l: =
N
and
Vwig
(20)
N
This means we are essentially only optimizing token and its answer are (s, 1/,, we have that
- Woy C. For each datapoint in the factual recall dataset, suppose the factual
28

---

Taming Knowledge Conflicts in Language Models
7
E
p (v) T (- V-
Won C) (s))
0
By 03) = 0
(
(21)
nT
! E
[1))] (ov Way d')(s))
0
(22)
7
n
2n E (v()) (o
nT
71
Nn
T
1
N
TiP
0
1
N
T
71
7
n
(23)
Fact incorrect terms
induction set
11
F
TJ N
0
-O
(24)
nN
N
T N
(25)
nN
B p (1) (- (-V ('((E))
0
(26)
nNa
N
7 E p (v) T (-Voor
0
7 NT
2n NT
(27)
n'T
Fact incorrect lerrs
induction
where we can see that the most signal is absored in ((s) with spurious correlations learned with ('(s). The Wov could act ussociative-menne memory module for the factual recall dataset essentially in single gradient step. For other umembedding vector other than p(y) to dot product with (- V Wox ())-)), remark that the gradient update from the factual recall dataset gives negative value.
Take an arbitrary point in the induction dataset, suppose the selected answer token is <j. we have
11 E (eg) (- (-Vwe, = nT
Y
(28)
n
NN,
fact and induction
B
[(())] (- Wor () (()) = =
(29)
nT
NN
fact induction
E [(g)]] V Win L) (4)
1)
mn NTn
(30)
n
nT
induction only
" [p((ee)
Won Won C) = 0
1
(31)
nf
NT
fact and induction
E (u(G) () = nN
7
o
(32)
n
NN,
fact and induction
7 " (cs)) (- L) = 0 n
"
(33)
nN
NN
Fact and induction
where we can see thaet the Wov terms learns the correct association between each ju(;) and db(), with spurious correlation learned with (E,)) We further remark that this Wov alone is able to make perfect predictions when loss is still high. However, as training progresses, the benign signal from V W L could also enable Wky to focus on the critical tokens.
Nowe we focus on the second gradient step. Since now WkO still zero matrix, we have
29

---

Taming Knowledge Conflicts in Language Models
1 oj T
jj
(34)
However, for now 9g doesn't have an order ((N) difference for l: = and k + Here the relative update signal for V Wow & still follows from the analysis in the first step, where the relative update of the correct signal still dominates, but by a smaller margin. With sufficiently large y in the second step, the training could converge. Now we focus on how the second step leads to the desired form of WKQ-
For the induction task, we show that the model will concentrate on the correct term ('(1) + ((G)). Let's recall the gradient with respect to Wxę:
T
V Wio
kapT T
I Balt Wovz:) = %
There are mainly "six types inputs" in a single datapoint with selected answer token ey: (1) desired focused term Φ(e,) (g)). (2) first occurrence of question Φ(g) + (c)-))). (3) last position @(g). (4) first position d(E1), (5) remapped
answer token with unrelated noise ((ej+1) d' (e(j), and (6) purely unrelated noise tokens ((Ex) + o (Ex-1).
We claim that
E hyy]
>> E hr]]
(35)
where j is the coefficient for the desired term (1) (ble) + d' (q) and k is any other types of terms. We can decompose
WovE, + (ByR(v) B
Wova;)
(36)
Large
Small
where now the subscript y refers to the token ej. We remark that the second term donimates the signal. From the analysis of the first gradient step, we know that
E [ply) Wovo' (4)] E [u(y)]
Wovo(es)] [ E [T(y)) Wovo'(ee) E [u(y)] T Wov(-)]
where (.) represents other terms (i.e., ((q), ((+))) o' (ex)). This means the term o(E,) + d'(2) has the largest signal (7y) in expectation. To see this, as we know 3, > 0, Bn < O. consider substitute ø' (q) with any other terms (e.g. @(g), ó(ex)) then Yi is guaranteed to decrease. The same reasoning applies to Φie,) as we fix o' (g). The only exception occurs with o' (ey). but we know that this term is guaranteed to not share the same position with o' (q) Therefore, we finish our claim.
similar statement can be made for the factual recall task where WKQ concentrates on the (())) (ei-1) and d' d'(s)+ terms. The second term could be regarded as "benign spurious correlation" under our setup. We can take sufficiently large v in the second step to enable the convergence in expectation. As such, the Wx kq also takes in the form of Equation (8).
Corollary G.7 (Knowledge Conflict). Under the knowledge conflict inference setting, the model capable of solving both factual recall and induction from Proposition 5.2 may output either if eep(Ci)Cs, exp(C2)Ca, then the model outputs the factual recall answer G* (s); otherwise, the model outputs the induction answer Ej-
the inductive token or the factual token. More specifically,
exp Ct) Proof. The attention weight on the d'(g) + Ō(E,) is approximately (xp(Ci)Hox((C))((TI ((s) (((-1) is approximately exp(Ci))exx((C)HH(
The attention weight on the
exp C2)
30

---

Taming Knowledge Conflicts in Language Models
The raw logit value of ej is C3s (xxp(C))+xxpl + and the raw logit value of (*(s) is C1 (xp(Ci)++xxe CC) (7 Other
(xp(C))
exp
terms have a small logit values. Therefore, if exp(Ci)C3s < exp(C2))C4, then the model outputs the factual recall answer (s). Otherwise, the model outputs the induction answer Ej
Proposition G.8 (Effectiveness of JUICE), Consider the model from Proposition 5.2 and the case when its inductive part dominates (i.e.. exp(Ci)Css >> exp(C2)Ca) then the intervention by JUNE/PH3 of deleting the two attention heads not as effective as JUICE, particular, in this case JUNe/PH3 does not result in the parametric answer, while JUICE does.
Proof. First, we remark that both attention heads (of the construction) are "highly influential" attention heads. if one scales up or down the activation output of the two heads, the logit value of the corresponding parametric answer decreases increases monotonically.
We now choose the intervention method to be knocking out for simplicity (which is exactly PH3; for JUICE, JUNE, it means adds a scaled version of the activation output by a factor of -1).
we were using a single-pass intervention method advocated by PH3 or JUNE, then this simply means we delete the activation output from both heads, which gives an answer of random guessing among elements the vocabulary space
If use the dual-run design of JUICE, then note that the activation outputs of the second layer from the first run has that
C4 exp(C2)
C3 (x1p(i))
Logit fact
Logit ind
(37)
(xp(C1) exp(C2) (T 2)
exp(C1) + exp(C2) exp(C)) + (T
In the second have
Logit
(2) fact
C exp(C(2 cxp(C2) (T 1)
Logit (2) ind
C33
(38)
exp(C22 -T-
By deleting the activation output from the first run, we have
Logit
(2)+ fact > 0
Logi
(2)+ and
<0 0
Logit (2)+
other
(39)
This shows that JUICE results in the correct parametric answer.
31


Reviews:
[{"review_id": "uMbjxYDQSz", "reviewer": "Reviewer_HLU5", "summary": "The authors contribute a new method and analysis framework for understanding knowledge conflicts in RAG applications. Specifically, they define the usual setting of knowledge conflict where the contextual (retrieved) knowledge may differ from the parametric (internal) knowledge of the LLM. In these cases, and active area of research is understanding how RAG agents choose between the contextual and the parametric knowledge.\n\nThe authors contribute the following:\n\n0. introduction of the concept of CP (contextual-parametric) superposition, which is the idea that depending on the input tokens, the LLM will lean more toward parameteric or external knowledge\n1. through a weight-masking study, a motivation for controlling knowledge conflict tradeoff via scaling attention head outputs\n2. a methodology that tunes an attention-head scaler via examining intermediate outputs of the model\n3. a comprehensive empirical study showing that the proposed method (JUICE) is singularly most effective at controlling knowledge conflict tradeoff\n4. theoretical characterization of the underpinnings of JUICE, showing that a LLM can indeed achieve CP superposition via pretraining (thus justifying the overall approach to this problem)", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful feedback and for recognizing the significance of our work. We agree with the reviewer that many of the ideas presented in our paper extend beyond the scope of knowledge conflicts, as superposition appears to be a widespread phenomenon. Gaining a deeper understanding of its underlying mechanism will ultimately lead to more robust and reliable use of language models.\n\nAdditionally, we really appreciate your careful checking and attention to detail in our manuscript. In the revised version, we will clarify the notations in Algorithm 1 according to your recommendations.", "claims_and_evidence": "Yes, I found all claims to be convincing.", "methods_and_evaluation_criteria": "Yes, the authors define the problem they study as CP superposition, namely the ability of models to choose between parametric and internal knowledge. The problem is therefore not to be the most factual, but rather to influence a certain outcome with high probability. The authors' empirical study is well-crafted around this objective.", "theoretical_claims": "I did not check the correctness of the proofs, but I read through the main results in detail, and they are quite coherent.", "experimental_designs_or_analyses": "I checked all experimental designs presented in the paper. No issues.", "relation_to_broader_scientific_literature": "In my opinion, this paper brings a strong empirical and theoretical understanding to the field of knowledge conflicts and more broadly LLM factuality. The vast majority of existing work has conducted only empirical studies that expose the behavior of RAG agents under knowledge conflict scenarios. This is one of the only works I'm aware of to actually propose a well-motivated and effective intervention.", "essential_references_not_discussed": "No essential references I could find that were missed", "other_strengths_and_weaknesses": "Nothing else to report here", "other_comments_or_suggestions": "Very minor nit: there may be some confusion with Algorithm 1, Step One: Save Important Streams: $H_l \\in S_1$ doesn't quite make sense, because $H_l$ appears to be an output, but $S_1$ is a parameter set of the model", "questions_for_authors": "No further questions", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tNFKuOhAL8", "reviewer": "Reviewer_bLHn", "summary": "The author proposed JUICE (Just Run Twice), a test-time intervention method to address knowledge conflicts in language models (LMs), where parametric memory (internal knowledge) contradicts contextual information. Extensive experiments across 11 datasets and 6 model architectures demonstrate that JUICE achieves state-of-the-art performance and robust generalization. For example, under the Gemma-2b model, JUICE consistently outperforms baselines like PH3 and Prompt, achieving high accuracy even in challenging coherent conflict scenarios. The method also shows strong robustness against hyperparameter variations and paraphrased inputs.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Effectiveness of JuICE on different architectures**\n\nThank you for the insightful question. Our main paper evaluates JuICE on a range of widely used dense attention-based open-source LMs. Following your suggestion, we additionally test JuICE on Mistral-v0.1-7B, a leading sparse attention-based model. Results below confirm that JuICE remains highly effective in this architecture, reinforcing the generality of our findings:\n\n| Dataset Conflict | World-Capital Clean | World-Capital Subs-conf | World-Capital Coh-conf |\n|------------------|:-------------------:|:-----------------------:|:----------------------:|\n| Original         |         96.2        |           76.8          |           2.7          |\n| Prompt           |         96.2        |           81.1          |           7.0          |\n| PH3$_l$           |         95.7        |           92.4          |          23.8          |\n| PH3$_s$            |         96.2        |           85.9          |          20.0          |\n| JuICE            |         96.1        |         **93.0**        |        **92.4**        |\n\n\n| Dataset Conflict | Official-Language Clean | Official-Language Subs-conf | Official-Language Coh-conf |\n|------------------|:-------------------:|:-----------------------:|:----------------------:|\n| Original         |         96.2        |           38.9          |           0.0          |\n| Prompt           |         96.2        |           54.3          |           0.6          |\n| PH3$_l$            |         95.0        |           72.8          |          0.6          |\n| PH3$_s$            |         94.4        |           43.8          |          0.0          |\n| JuICE            |         96.3       |         **90.1**        |        **82.5**        |\n\n| Dataset Conflict | Company-Founder Clean | Company-Founder Subs-conf | Company-Founder Coh-conf |\n|------------------|:-------------------:|:-----------------------:|:----------------------:|\n| Original         |         59.3        |           8.8          |           0.0          |\n| Prompt           |         59.3        |           23.1          |           1.6          |\n| PH3$_l$            |         54.9        |           20.3          |          0.0         |\n| PH3$_s$            |         59.3        |           8.2          |         0.0          |\n| JuICE            |        57.7       |         **44.5**        |        **17.3**        |\n\nThese results show that JuICE retains strong performance in sparse-attention models. We will include them in the revised version.\n\n**Q2: Insights on Model Design**\n\nThank you for bringing up this excellent point. At present, there may be no definitive answer to whether superposition is inherently \"harmful\", which would require a new, independent investigation. On the one hand, superposition can impede interpretability by making it difficult to disentangle the roles of individual parameters. On the other hand, we speculate it may enhance a model’s compactness and expressiveness, allowing the same parameters to accomplish multiple tasks. The reviewer's comment has inspired us to come up with a bolder speculation: this phenomenon might shed lights on understanding the success of some relatively small models (such as QwQ-32b), which has demonstrated remarkable reasoning performance on par with much bigger ones (such as Deepseek-r1-671b) with much more compact parameter size. \n\nIn our paper, we propose methods that recognize the inevitability of superposition, a perspective that has received little attention so far. We would like to leave further investigation to future works: how the LLM components accommodating superposition contribute to model expressiveness and whether increasing such components can lead to more compact models with equal expressiveness.\n\n**Weakness: Additional Runtime**\n\nWe agree that JuICE's strong performance comes with modest runtime overhead, as acknowledged in Appendix F. Future work may improve its efficiency. Additionally, users can trade off speed and performance by choosing JuNE, our lightweight variant, which runs faster and still performs on par with or better than other baselines.\n\n**Other Comment: Providing Visual Examples of Model Output Changes**\n\nThank you for the excellent suggestion! We now visualize how JuICE shifts model output probabilities across conflicting scenarios. For each case, we plot the average probability mass over the parametric token, contextual token, and other tokens (calculated as 1 - P(parametric) - P(contextual)) on the World Capital dataset using Gemma. The plot is available at this [anonymous link](https://anonymous.4open.science/r/ICML_25_Rebuttal_Plot-82C6) (see bLHn_comment). The results clearly show JuICE consistently promotes the desired parametric token across all conflict types. We plan to include this plot in the revised version.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "The reviewer did not check the proofs.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The proposed might be use in many domains since knowledge conflict is very common.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths\n1. Both experimental results and theoretical analysis are provided.\n\n2. Improvements are achieved, which shows the advantages of the proposed method.\n\n\nWeaknesses\n1. Dual-run inference will increase the running time to get the final results.", "other_comments_or_suggestions": "1. It might be helpful to provide visual examples of how model outputs change before and after applying JUICE, especially in challenging conflict scenarios.", "questions_for_authors": "1. JUICE's effectiveness  has been shown on several models. How do you expect it to perform on models with significantly different architectures, such as those using sparse attention?\n\n2.  The superposition of contextual information and parametric memory arises naturally from the training process. How can this insight be leveraged to design models that inherently avoid such conflicts, rather than relying on post-hoc interventions like JUICE?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YA0cVq09Li", "reviewer": "Reviewer_GwP9", "summary": "This paper proposes JuICE, a test-time intervention method to address knowledge conflicts in language models (LMs) by steering models toward either parametric or contextual knowledge. The method first identifies two sets of attention heads that consistently achieve the desired effect with either positive or negative scaling and then runs twice (saving outputs of the identified head and then scaling them).\nExtensive experiments on 11 datasets and 6 model architectures demonstrate state-of-the-art performance and robustness across conflict types. Theoretical analysis formalizes knowledge conflicts and justifies JuICE’s effectiveness.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the time and thoughtful feedback. Before addressing specific points, we emphasize that our primary goal is to understand the interplay between parametric (internal) and contextual (external) knowledge in LLMs. Our methodology primarily aims to validate the novel discovery, rather than exhaust engineering optimizations from all possible criteria, which, while having their own values, we see as orthogonal and suitable for future work.\n\n**Q1: General Abilities and Benchmarking LLMs with JuICE**\n\nJuICE is a targeted control tool that depends on whether the user intends to emphasize (a) parametric knowledge or (b) contextual information. Because to obtain the correct answer, such an intent varies across examples and remains an unknown prior in general benchmarks—even within the same dataset (i.e. question1 needs a while 2 needs b)—naively applying the same scaling factor to all examples can negatively affect performance. Consequently, JuICE is most effective when user intent is pre-defined, rather than for general-purpose benchmarking.\n\n**Q2: Sensitivity to Head Number and Scaling Factor**\n\nWe provide the result of our sensitivity analysis in this [anonymous link](https://anonymous.4open.science/r/ICML_25_Rebuttal_Plot-82C6) (see GwP9_Q2). We plot the number of heads $K$ (from 2 to 30) and scaling factors $\\alpha$ (grid search over 0-5) against the average test accuracy (of three conflict types) with Gemma on World Capital. The result shows that JUICE is highly robust against the choice of hyperparameters.\n\n**Q3: Scalability to Larger Models**\n\nJuICE runs linearly in the number of attention heads. Systems capable of running a 70B model can also accommodate JuICE.\n\n**Experimental Design 2: Real-world Conflict Complexity**\n\nWe agree that complex cases (e.g., partial conflicts) are important but currently unexplored. We have explicitly acknowledged this limitation in Appendix F of the original paper. Before diving into this more complex problem, the task of understanding conflicts between parametric memory and contextual information (which is the focus of this paper) poses a **standalone and substantial** challenge, which has not been fully addressed by existing works. Our study employs datasets drawn from diverse domains and explores both parametric and contextual perspectives, presenting the first rigorous, comprehensive analysis of memory-context knowledge conflict. Tackling mixed or partial conflicts likely requires a dedicated study, which we leave for future work.\n\n**Experimental Designs 3 & 1: Baseline Comparison & Prompting**\n\nThank you for highlighting this relevant work. IRCAN is indeed one of the best RAG-based hallucination methods focusing on one-sided knowledge conflicts. We will add it to the citation list. The problem setup of IRCAN and our work has many differences, and there exist multiple practical constraints (we will explain later on in detail). Accordingly, a point-to-point complete comparison between the two works is infeasible. However, we managed to identify a shared experimental setting between our paper and IRCAN (the Proverb-Ending dataset, referred to as MemoTrap in IRCAN). Using data reported in the IRCAN paper, we compare the two methods as follows:\n\n| Datasets | Proverb-Ending (MemoTrap in IRCAN) |\n| -------- | :-------: |\n| Gemma-2-IRCAN-CAD | 27.1 |\n| Gemma-2-JuICE | 74.6 |\n| LlaMA-2-IRCAN-CAD | 61.8 |\n| LlaMA-2-JuICE | 77.1 |\n| LlaMA-3-IRCAN-CAD | 54.4 |\n| LlaMA-3-JuICE | 75.4 |\n\nJuICE consistently outperforms IRCAN while requiring smaller identification sets. We will add this comparison to the revision.\n\nHere we further explain the infeasibility of a point-to-point comparison between these two methods:\n- Universal Component Discovery: IRCAN uses dataset-specific splits; it does not claim cross-domain generality. For our Table 4, we require either a domain-agnostic or no identification set to ensure fair comparison across tasks.\n- Heavy Tuning: IRCAN requires ~288 hyperparameter configurations (a 16×18 search grid), plus a large validation set (same size as test). Our benchmarks (e.g., NQ-swap, 5k samples) make this computationally infeasible within the rebuttal window.\n\nPrompting Baseline: The reviewer notes prompt baselines work better for large models (e.g., 70B). Unfortunately, our computing resources cannot even load such models. Our chosen model scales match or exceed those used in prior high-impact interpretability papers [1–4], including IRCAN. Consistent with prior work (e.g., PH3), we adopt prompting as our baseline.\n\n[1] Locating and Editing Factual Associations in GPT, NeurIPS'22 (1143 citations)\n\n[2] Inference-Time Intervention, NeurIPS'23 (409 citations)\n\n[3] Improving Alignment and Robustness with Circuit Breakers, NeurIPS'24 (61 citations)\n\n[4] Cutting Off the Head Ends the Conflict, ACL'24 (Baseline)\n\n\n**Typos**\n\nThank you for your careful checking. We will amend these typos and format issues in the revised version of our paper.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "The proposed method is sound. The evaluation is comprehensive and makes sense.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "1. The comparison to the Prompt baseline is unfair. Prompt works significantly better for larger models (e.g. 70b) while the models in the experiments are no larger than 8b.\n\n2. Limited Real-World Testing: Experiments focus on synthetic or curated conflicts; real-world noisy or partial conflicts (e.g., mixed relevant/irrelevant context) are not explored.\n\n3. There are not enough baselines. For instance, one important baseline is missing as mentioned below.", "relation_to_broader_scientific_literature": "The paper proposes a novel method to mitigate knowledge conflicts and demonstrate that highly influential attention heads may simultaneously contribute to both memory and context, which could be very useful for future studies on knowledge conflicts.", "essential_references_not_discussed": "The paper did not cite IRCAN [1], which proposed to identify and reweight context-aware neurons to mitigate knowledge conflicts. This is very relevant to the topic and method in this paper and should be considered as a baseline.\n\n[1] IRCAN: Mitigating Knowledge Conflicts in LLM Generation via Identifying and Reweighting Context-Aware Neurons.  NeurIPS 2024", "other_strengths_and_weaknesses": "Strengths: \n1. The paper provides novel insights: identifying CP superposition and challenging the prior assumption of mutually exclusive \"memory heads\" and \"context heads.\"\n2. JuICE requires no fine-tuning, no massive examples to identify target heads, and achieves consistent high performance across diverse domains, making it both lightweight and applicable.\n\nWeaknesses: see above", "other_comments_or_suggestions": "Typos: In Table 3, 4 and many other places, \"Llamma\" -> \"Llama\"\n\nThe titles on the top of every page is missing.", "questions_for_authors": "1. How does JuICE affect the general abilities of LLMs? Will it make the performance drop significantly on benchmarks such as MMLU, SuperGLUE?\n\n2. How sensitive is JuICE to the choice of K (number of intervened heads) and scaling factors?\n\n3. Can JuICE scale effectively to larger models (e.g., LLaMA 70B) with thousands of attention heads?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KE5ktNR2vd", "reviewer": "Reviewer_xf53", "summary": "This paper conducts a study of knowledge conflicts in LMs. The authors reveal that some attention heads contribute to both contextual information and parametric memory, instead of only contribute to one of them. They call this phenomenon \"superposition of contextual information and parametric memory.\"  \n\nBesides these findings, the authors propose a training-free method for steering LMs towards either parametric or contextual knowledge, called JUICE (Just Run Twice).  JUICE first identifies consistent influential attention-head, then saves activations of these heads and at last steers the model by adding these saved activations.\n\nExperimental results on many datasets show the effectiveness of JUICE. And the authors conduct theoretical analysis about the superposition phenomenon and the effectiveness of JUICE.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Claim and Evidence 1: The Effect of Scaling JuNE**\n\nThank you for the great suggestion. JuNE is an ablated variant of JuICE that only skips the second run but still incorporates and tunes its own scaling factor, as noted in Line 257 and Appendix E. We acknowledge that Proposition 5.5 may cause confusion, as JuNE is grouped with PH3 to emphasize its single-run nature, not the absence of scaling. We will clarify this distinction in the revised version of the paper.\n\n**Claim and Evidence 2: Further Analysis on Incorporating Activations from the First Run**\n\nThis is indeed a great point! Proposition 5.5 is a proof-of-concept showing how dual-run methods improve intervention under simplified assumptions. It remains valid when using scaling factors rather than mere removal. In a high-level proof sketch, as the second layer transitions into a \"factual memorizer\" (due to the role alternation via scaling), single-pass methods continue to treat it as a context component, thereby suppressing its influence—whereas JuICE correctly preserves it, leading to more effective control. \n\nBeyond the theoretical discussion in Section 5, our paper provides extensive empirical support for this dual-run strategy. In Section 3, we introduce the observations that motivate the approach; we then validate its effectiveness in Tables 3 and 4 by comparing JuICE and JuNE, and we conduct a more detailed analysis in Figure 4. Collectively, these experiments and analyses offer strong evidence of the proposed method’s efficacy.\n\nFollowing your suggestion, we have also performed an **ablation study examining the benefits of adding the activations obtained from the first inference to steer (without scaling)** using Gemma on all parametric datasets. The details are provided in this [anonymous link](https://anonymous.4open.science/r/ICML_25_Rebuttal_Plot-82C6) (see xf53_Claim_and_Evidence_2). Simply adding these activations already confers considerable performance gains, demonstrating JuICE’s effectiveness. Applying scaling with well-chosen coefficients further amplifies these improvements. This also connects to Q2 raised by reviewer GwP9, where we observed that JuICE exhibits strong robustness to different scaling coefficients.\n\n**Question: Comprehensiveness of the classified knowledge conflicts and effectiveness of the proposed method on new conflicts**\n\nA primary contribution of our work is the in-depth exploration of language model mechanisms, accompanied by methods grounded in these insights. We classify three key types of knowledge conflicts, providing a clean and controlled setting that illuminates the internal workings of LMs—an aspect largely overlooked by prior research, which tends to focus on high-level behavioral observations. While we do not claim to cover every possible knowledge conflict, our classification addresses a wide range of conflicts and yields actionable insights. Building on the well-defined and representative cases explored in our work, future research can extend these insights to more atypical and complex forms of knowledge conflict.", "claims_and_evidence": "The claim about the superposition phenomenon is well supported by the analysis and experiments.\nThe effectiveness of the proposed method JUICE can be better analyzed.\nAt first, will changing the scaling factor of JUNE can obtain similar or superior performance than JUICE?\nBesides, proposition 5.5 only analyzes the impact on JUNE after removing the attention head; however, in the actual experiment, a scaling method was used instead of actual removal. Therefore, compared to scaling, the benefits of adding the activations obtained from the first inference to steer still require further analysis.", "methods_and_evaluation_criteria": "Yes, the evaluation criteria make sense.\nThe effectiveness of proposed steering method need further analysis.", "theoretical_claims": "The theoretical claims about superposition phenomenon make sense.", "experimental_designs_or_analyses": "The experimental designs make sense.", "relation_to_broader_scientific_literature": "The article points out the limitations of previous articles regarding the incentive explanation of knowledge conflicts and proposes the superposition phenomenon of attention heads concerning knowledge conflicts.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "The paper is easy to understand. The proposed method achieved good performance in experiments. It would be better to add further analysis on why the steering approach is effective.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "Is the classification of these three types of knowledge conflicts comprehensive? If a new type of knowledge conflict emerges, can the proposed method be adapted to it?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper puts forward a superposition perspective on knowledge conflicts where model components use both parameteric and context information simultaneously. They also provide a new method to steer models by making two passes through the model, using the first pass to identify various categories of attention heads and get reliable directions for steering. The paper is generally well-written, has clear experiments, baselines, theoretical analyses. The paper is a clear accept!
Decision: Accept (spotlight poster)