Title: Understanding Parametric and Contextual Knowledge Reconciliation within Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Natural Language Processing

Information Retrieval

Network Analysis
Abstract: Retrieval-Augmented Generation (RAG) provides additional contextual knowledge to complement the parametric knowledge in Large Language Models (LLMs). These two knowledge interweave to enhance the accuracy and timeliness of LLM responses. However, 
  the internal mechanisms by which LLMs utilize these knowledge remain unclear. We propose modeling the forward propagation of knowledge as an entity flow, employing this framework to trace LLMs' internal behaviors when processing mixed-source knowledge. Linear probing utilizes a trainable linear classifier to detect specific attributes in hidden layers. However, once trained, a probe cannot adapt to dynamically specified entities. To address this challenge, we construct an entity-aware probe, which introduces special tokens to mark probing targets and employs a small trainable rank-8 lora update to process these special markers. We first verify this approach through an attribution experiment, demonstrating that it can accurately detect information about ad-hoc entities from complex hidden states.  Next, we trace entity flows across layers to understand how LLMs reconcile conflicting knowledge internally. Our probing results reveal that contextual and parametric knowledge are routed between tokens through distinct sets of attention heads, supporting attention competition only within knowledge types. While conflicting knowledge maintains a residual presence across layers, aligned knowledge from multiple sources gradually accumulates, with the magnitude of this accumulation directly determining its influence on final outputs.

Understanding Parametric and Contextual Knowledge Reconciliation within Large Language Models

Abstract

Retrieval-Augmented Generation (RAG) provides additional contextual knowledge to complement the parametric knowledge in Large Language Models (LLMs). These two knowledge interweave to enhance the accuracy and timeliness of LLM responses. However, the internal mechanisms by which LLMs utilize these knowledge remain unclear. We propose modeling the forward propagation of knowledge as an entity flow , employing this framework to trace LLMs' internal behaviors when processing mixed-source knowledge. Linear probing utilizes a trainable linear classifier to detect specific attributes in hidden layers. However, once trained, a probe cannot adapt to dynamically specified entities. To address this challenge, we construct an entity-aware probe, which introduces special tokens to mark probing targets and employs a small trainable rank-8 lora update to process these special markers. We first verify this approach through an attribution experiment, demonstrating that it can accurately detect information about ad-hoc entities from complex hidden states. Next, we trace entity flows across layers to understand how LLMs reconcile conflicting knowledge internally. Our probing results reveal that contextual and parametric knowledge are routed between tokens through distinct sets of attention heads, supporting attention competition only within knowledge types. While conflicting knowledge maintains a residual presence across layers, aligned knowledge from multiple sources gradually accumulates, with the magnitude of this accumulation directly determining its influence on final outputs.

1 Introduction

Through large-scale pre-training, Large Language Models (LLMs) have acquired vast amounts of knowledge implicitly encoded within their model parameters, forming the cornerstone of their general capabilities. However, this paradigm of knowledge storage faces twofold limitations: from a coverage perspective, parametric knowledge obtained through pre-training on public data struggles to extend to domain-specific private knowledge; from a temporal perspective, static parametric knowledge gradually loses its effectiveness as the real world evolves. To address these limitations, RAG techniques have been introduced to provide LLMs with additional contextual support [8, 45]. Such real-time contextual knowledge both expands the inherent boundaries of parametric knowledge and mitigates its temporal constraints.

This raises a crucial question: to what extent can LLMs accept and utilize external contextual knowledge, particularly when such external information conflicts with the model's inherent parametric

knowledge (which may be outdated or inaccurate)? Researchers have investigated this question through various approaches, including negation injection [11, 20] and entity substitution [23, 47], revealing that LLMs often stubbornly adhere to their original beliefs. Other studies [39, 16] have demonstrated that the fluency of external context and the strength of the model's prior beliefs significantly influence LLMs' receptiveness to external knowledge. While these studies have revealed behavioral characteristics of LLMs in knowledge conflict scenarios, our understanding of the underlying mechanisms remains limited. Research into this question is not only crucial for designing superior context injection strategies and enhancing models' ability to integrate external knowledge but also provides theoretical guidance for building more reliable language models, advancing the widespread application of retrieval-augmented LLMs.

To unravel these mechanisms, we propose entity-aware probing - a framework that traces knowledge propagation as entity flows through transformer layers. Traditional probes fail in dynamic settings due to fixed label constraints. We overcome this by introducing special token markers combined with low-rank adaptation (LoRA), enabling ad-hoc entity tracing without altering base model parameters. This allows us to quantify how contextual and parametric knowledge evolve across layers during conflict scenarios. Our analysis reveals three key insights: (1) Specialized routing : Contextual and parametric knowledge are processed by distinct attention heads, with minimal cross-type competition. (2) Asymmetric dynamics : Contextual knowledge emerges abruptly in early layers, while parametric knowledge accumulates gradually via MLPs. (3) Superpositional reconciliation : Conflicting knowledge persists in residual streams, but aligned signals from multiple sources reinforce each other's magnitude, directly influencing output selection.

Our contributions . Our main contributions are twofold: (1) An entity-aware probing framework for dynamic knowledge tracing - enables ad-hoc entity relevance detection without modifying base model parameters. (2) Empirical insights into RAG dynamics - offers a foundation for future studies on knowledge interaction mechanisms.

2 Preliminaries

To understand how parametric knowledge and contextual knowledge reconcile within the model, we first reformulate the standard transformer architecture [35] from a residual stream perspective [7, 26] (Section 2.1). This reformulation allows us to isolate and analyze the information written into the residual stream by each component (MLP, attention, and individual attention heads), providing a foundation for studying their functional roles and actual knowledge propagation dynamics. We then introduce probing techniques (Section 2.2) as diagnostic tools to empirically study how models dynamically reconcile conflicting knowledge sources during this accumulation process.

2.1 Residual stream analysis of transformer computation

Standard transformer models process tokens through a stack of identical layers, each containing self-attention and feed-forward sublayers. We begin by equivalently rewriting this computation as cumulative updates to a residual stream. Specifically, in an L -layer decoder-only transformer, the hidden state of t -th token evolves as follows:

where ∆ h attn l ( t ) and ∆ h mlp l ( t ) denote the contributions written to the residual stream by the selfattention and MLP sublayers, respectively. In retrieval-augmented generation, externally retrieved knowledge is introduced into the model's internal processing stream, where these sublayers form separate pathways for knowledge propagation and serve complementary roles.

Parametric knowledge access . The emergence of parametric knowledge has been observed to strongly correlate with MLP sublayer outputs [10, 25]. Specifically, denoting the MLP layer input as ˆ h ( t ) l , the parametric knowledge access can be viewed as querying non-linear key-value memories:

where the input ˆ h ( t ) l is first mapped to key space through matrix W up and non-linear activation function σ . Subsequently, the corresponding token-specific knowledge fragments ∆ h mlp l ( t ) are

US

Trump

Attention captured

'Trump'

from external

input. (

Eq. (3))

Attention captured

from MLP

memory.

Uncertainty remains in

how conflicting

residual signals are resolved.

？

Biden

MLP infers "

Biden

" from internal parameters.

(Eq. (2))

MLP

Attention

Document

president

Who is the

Query

The external document states that Trump was the president;

for brevity, only

Trump

is shown.

queried through W down and injected into the residual stream. This is a simplified description of parametric knowledge access; for more detailed modeling of memories querying, we refer readers to Geva et al. [10], Meng et al. [25]. We provide an illustrative example in Figure 1, where for a query 'Who is the US president?', the parametric knowledge 'Biden' gradually emerges in the residual stream of relevant tokens (e.g., 'US') through the injection of multiple MLP layers.

Knowledge routing by attention . The attention mechanism serves as a crucial pathway for transferring knowledge between tokens. The content written into the residual stream by a single attention head is given by:

where the four parameter matrices W ( k ) O,Q,K,V of the k -th attention head determine its attention preferences. Specialized heads (e.g., retrieval heads [27, 38] that attend to external contexts and memory heads [17] accessing parametric knowledge) are found to exhibit differentiated sensitivity to these dual knowledge sources. The outputs of N attention heads are summed to form the attention sublayer output ∆ h attn l ( t ) = ∑ N k =0 ∆ h attn , ( k ) l ( t ) . Taking Figure 1 as an example, both 'Trump' mentioned in the context and 'Biden' encoded in the model parameters influence the model's response to the query 'Who is the US president?' through the attention sublayer output ∆ h attn l ( t ) .

Based on this formulation, we analyze the model's behavioral mechanisms under the RAG setting by probing the accumulated information in each layer's residual stream h ( t ) l , MLP output ∆ h mlp l ( t ) , attention layer output ∆ h attn l ( t ) , and individual attention head outputs ∆ h attn , ( k ) l ( t ) .

2.2 Probing as mechanistic diagnostics

Probing techniques [1, 2, 12, 26] serve as diagnostic tools to analyze how linguistic properties and knowledge components are dynamically encoded in neural representations. A probe f θ : R d model →Y typically implemented as a lightweight classifier, is trained to decode target properties y ∈ Y (e.g., POS tags [34], named entities [15], or knowledge veracity labels [28]) directly from the residual stream, without modifying the base model's parameters.

Limitations of traditional probing . The classification head f θ of traditional probes is static-it is trained to recognize only a fixed set of labels Y (e.g., predefined POS tags or entity types) during training. This rigidity makes it brittle when handling ad-hoc probing targets at test time. For example, to probe whether a residual stream contains information about a specific entity, you cannot feasibly train an entity-specific probe for every possible entity.

3 Probing framework for knowledge conflict internal mechanisms

3.1 Competing hypotheses on knowledge conflict

Given a query q (e.g., 'Who is the current U.S. President?') and an external document D returned by retrievers, we define parametric knowledge as all entities encoded within the model parameters that are objectively relevant to answering q , while contextual knowledge comprises answer-relevant entities derived from D . These knowledge sources may conflict-for instance, parametric knowledge might encode 'Biden' whereas retrieved documents D might state 'After January 2025, Trump becomes the new U.S. President.' We investigate how language models process and reconcile competing knowledge sources through their residual streams. Specifically, we propose two competing hypotheses about the model's internal mechanisms for handling knowledge conflicts:

Attention competition hypothesis : Contextual and parametric knowledge compete for dominance through attention patterns, with the prevailing signal propagating through successive layers to determine the final output.

MLP arbitration hypothesis : Both knowledge types are initially aggregated in the residual stream, with MLP layers subsequently injecting modulation signals that suppress conflicting information before final prediction.

3.2 Entity-aware probing architecture

To investigate these two hypotheses, we design an entity relevance probing task. Given an external context D , a query q , and a specific entity e , the objective is to probe the base LLM's belief about 'whether entity e is relevant for answering query q .' We follow the approach of Ye et al. [42] to resolve the limitations of traditional probing outlined in Section 2.2, adapting the probe f θ for entity relevance probing.

Specifically, as illustrated in Figure 2, we append a triple ' [START] {target_entity} [END] ' after the external context D and query q . Here, [START] and [END] are reserved special tokens that explicitly indicate the target entity to the probe. To enable the model to process this structured input while preserving inherent processing mechanisms unchanged, we introduce a small trainable rank-8 LoRA update [13] exclusively to the input embedding layer. The base LLM parameters remain frozen during fine-tuning, with only the linear classification head and LoRA components updated to predict entity relevance at the [END] token position. This triple and LoRA update can be viewed as a form of 'soft prompt' designed to elicit the model's conflict reconciliation behavior. For a discussion of the intuition behind this approach and its limitations, please refer to Appendix B.

To validate that the entity relevance predictions primarily emerge from the model's inherent knowledge processing mechanisms rather than the introduced lightweight adaptation, we conduct an ablation study comparing LoRA-enhanced probes on both instruct and randomly initialized LLM variants. As shown in Table 1, the sharp performance gap between Probing and Probing(Random) demonstrates that the predictive patterns predominantly originate from knowledge pathways encoded in their parameters, with the LoRA update serving merely as an interface for observation rather than introducing substantive functionality. These findings align with the results of Ye et al. [42].

3.3 Probe training data construction

We construct probe training data by adapting the HotpotQA-distractor dataset [41], leveraging its sentence-level human-annotated evidence to precisely identify entities required for answering queries. Each sample contains a multi-hop query q and an external document D composed of 10 independent passages, where exactly two passages contain answer evidence while the remaining eight serve as distractors. We provide the two evidence passages and query q as input to Deepseek-v3-0324 , prompting it to generate a step-by-step answer to q . We then prompt the model to explicitly list entities involved in its generated answer (prompt template in Table 3). Entities appearing in both the evidence sentences and the model's answer are labeled as necessary (Yes). We then extract all entities mentioned in D using the prompt in Table 4, designating those not identified as necessary as irrelevant (No). From 6,200 processed HotpotQA samples, each generates training instances s i = < D i , q i , { e <j> i , y <j> i } N j =1 > , where y <j> i ∈ { Yes , No } indicates whether entity e <j> i is relevant to answering q i .

The US president is Trump.

Who is the US president

？

[START]

Trump

[END]

Document

(

D

)

Query (

q

)

Probing

Yes/NO

Multiple Transformer Layers

MLP

+

+

Linear

Layer

3.4 Knowledge dynamics tracing formulation

Let f θ ( h [END] ) : R d model →Y denote the probe constructed in entity relevance probing task, where Y ∈ { Yes , No } represents the model's belief about whether entity e is relevant for answering query q . To test the competing hypotheses outlined in Section 3.1, we systematically analyze the evolution of entity relevance beliefs by applying our probes at three critical points within each transformer layer's computation:

Attention-update entity probing : f θ (∆ h attn [END] ,l ) reveals the direct impact of attention mechanisms on model beliefs, where the attention module routes both knowledge types through the preceding tokens' KV cache.

MLP-update entity probing : f θ (∆ h mlp [END] ,l ) identifies the MLP module's direct influence on belief formation, corresponding to the model's integration of conflicting knowledge sources.

Layer output entity probing : f θ ( h [END] ,l ) tracks how entity e 's information aggregates across layers through successive attention and MLP updates up to layer l .

4 Probe validation via attribution task

To validate whether our entity-aware probe accurately reflects the model's internal beliefs about entity relevance, we conduct a controlled attribution task that evaluates two critical properties: (i) whether the probe identifies entities truly relevant to query q within retrieved documents D , and (ii) whether predictions derive from the base model's knowledge processing rather than the probe's parametric capacity.

4.1 Experimental setup

Task definition : Given a query q and an external document D containing N independent passages { p 1 , ..., p N } , the passage-level attribution task requires identifying passages containing answer evidence. For sentence-level attribution , we further pinpoint specific evidence-bearing sentences within each passage. We evaluate attribution performance on three established QA benchmarks-HotpotQA [41], TriviaQA [18], and SQuAD [30]-adapted for both passage-level and sentence-level attribution tasks. We measure attribution accuracy using standard information retrieval metrics: Precision , Recall and F1 . Implementation details for constructing the attribution benchmarks and formal metric definitions appear in Appendix C. Please refer to Appendix G for examples of the dataset.

Probing-based attribution : We address the attribution task through entity relevance probing. For each passage p i (or sentence s j ) in document D , we probe all mentioned entities E ( p i ) . Specifically, we apply the probe f θ to evaluate relevance for each entity e ∈ E ( p i ) , where relevance is defined by:

We mark the passage/sentence as an attribution target if it contains any entity with Relevant ( e ) = 1 .

Compared baselines : Three critical comparisons are established as follows: (1) Prompting : Instructs the base LLM to directly output attribution results through one-shot manual demonstrations, representing the simplest way to test its internal beliefs. The prompts we used are shown in Table 5. (2) Prompting(CoT) : Builds upon (1) by incorporating entity information extracted by the base LLM into the prompt and adding a CoT-style instruction that guides the model to reason through

entity-citation relations before producing the final attribution. The prompts are shown in Table 6. (3) Probing(Random) : Reinitializes the base LLM's parameters randomly (with parameters frozen during probe training) while maintaining identical probing architecture, isolating the probe's capacity to learn relevance patterns independently of LLM's knowledge processing.

4.2 Results analysis

As shown in Table 1, we evaluate mainstream open-source models (Qwen2.5, Qwen3, LLaMA3.1) and state-of-the-art proprietary models on the attribution task. Our analysis reveals two key findings:

LLMs possess intrinsic beliefs about entity relevance : For strong proprietary models, the simple Prompting method alone suffice to elicit accurate entity relevance judgments. Although smaller open-source models initially perform poorly, their entity relevance judgment significantly improves after more refined prompt design ( Prompting → Prompting(CoT) ), approaching or even surpassing some proprietary models. This suggests that these models internally possess knowledge of entity relevance, but this belief may require appropriate elicitation methods. The Probing approach achieves performance comparable to Prompting(CoT), indicating that probing serves as an alternative elicitation method that can effectively extract entity relevance information directly from hidden states.

Probing results stem from the LLM's own beliefs rather than probe parameters : Our critical baseline - probing a randomly reinitialized LLM - demonstrates that attribution performance collapses to near-random levels (average F1 of 20.3 vs. 73.0 for intact Qwen2.5). Precision metrics plummet to 10.2 on SQuAD and 7.7 on HotpotQA, comparable to random selection from candidate passages/sentences. This confirms that successful attribution depends on the base LLM's knowledge processing mechanisms rather than the probe's parametric capacity.

5 Understanding knowledge reconciliation within LLMs

5.1 Experimental setup

Evaluation Data : To investigate LLMs' knowledge reconciliation mechanisms, we construct knowledge conflict scenarios based on the HotpotQA dataset. Since HotpotQA is derived from Wikipedia-the primary pretraining corpus for LLMs-we assume its factual descriptions align with LLMs' parametric knowledge . For each query, we leverage sentence-level evidence from HotpotQA and instruct Deepseek-v3-0324 to replace original factual entities with fictional counterfactual entities, generating rewritten evidence and forming conflict-inducing external documents. Using prompt templates (Table 3), we create 786 samples, each formatted as s i = < D f , D cf , q, E > , where D f denotes the original document from HotpotQA, D cf denotes the counterfactual external document, q denotes query, E denotes all mentioned entities, categorized into:

0

10

20

30

Head ID

0

5

10

15

20

25

30

Layer ID

(a).

cf

cf

0

10

20

30

Head ID

(b).

cf

f

0

10

20

30

Head ID

(c).

cf

f

cf

0

10

20

30

Head ID

(d).

cf

f

f

0.02

0.04

0.06

0.08

0.10

parametric knowledge ( E f ): Factual entities from D f that align with the model's internal knowledge for answering q .

contextual knowledge ( E cf ): Counterfactual entities from D cf that conflict with parametric knowledge.

irrelevant knowledge ( E irr ): Remaining entities in E -( E f ∪E cf ) , unrelated to resolving q .

Metrics : We employ the PRAUC (Precision-Recall Area Under the Curve) [6] score to quantify the information intensity of parametric knowledge and contextual knowledge in residual stream. The metric robustly addresses class imbalance (relevant entities are rare compared to irrelevant ones). Specifically, query-relevant entities ( E f or E cf ) are treated as the positive class, while unrelated entities ( E irr ) form the negative class. PRAUC is computed using the probe f θ 's entity relevance scores P θ ( Yes | e, q, D ) . A higher PRAUC score indicates that the probe can more significantly detect the corresponding knowledge at the specified probing position.

5.2 Attention head analysis

We analyze the information intensity of all attention heads in LLaMA3.1, Qwen2.5, and Qwen3.0 models using probes. This section corresponds to the 'Attention-update entity probing' discussed in Section 3.4 formulation. Figure 3 presents the experimental results for LLaMA3.1, while results for Qwen2.5 and Qwen3.0 are shown in Appendix Figures 7 and 8 due to space constraints. All three models exhibit the following consistent patterns:

(i) Contextual and parametric knowledge are transferred by distinct attention heads : When feeding counterfactual context D cf (where contextual knowledge conflicts with the model's parametric knowledge) as external documents, subfigures (a) and (b) of Figure 3 measure the information intensity of contextual knowledge and parametric knowledge in individual attention heads, respectively. The results reveal that these two types of knowledge are routed through separate sets of attention heads, aligning with findings from existing mechanistic interpretability studies on attention heads [17, 27, 38]. Among the top-10 heads with the highest information intensity for each knowledge type, only 2 heads overlap in Qwen3.0, and 1 head overlaps in LLaMA3.1 and Qwen2.5. For clarity, we refer to these as contextual heads (routing contextual knowledge) and memory heads (routing parametric knowledge).

(ii) Fundamental differences in transfer mechanisms between knowledge types : Subfigures (a) and (b) of Figure 3 highlight a key distinction: the information intensity of contextual heads significantly exceeds that of memory heads. This suggests that contextual knowledge transfer resembles explicit retrieval, where complete knowledge units are captured and propagated by a small subset of attention heads. In contrast, parametric knowledge likely involves more intricate processing-each memory head may transfer fragmented knowledge pieces , which require multi-layer aggregation to form coherent knowledge units. This might explain why the information intensity of any single attention head remains relatively low. We provide further evidence supporting this hypothesis in Section 5.3.

0

5

10

15

20

25

30

Layer ID

0.002

0.004

0.006

0.008

0.010

0.012

Average prauc score

(a). Attention Scores across Layers

cf

cf

cf

f

0

5

10

15

20

25

30

Layer ID

0.00

0.05

0.10

0.15

0.20

Average prauc score

(b). MLP Scores across Layers

cf

cf

cf

f

0

5

10

15

20

25

30

Layer ID

0.00

0.05

0.10

0.15

0.20

Average prauc score

cf

cf

cf

f

(iii) Competition between same-type knowledge, but not across types : Subfigures (c) and (d) of Figure 3 present probing results under mixed contexts ( D f ∪D cf ), where factual (parametric-aligned) and counterfactual (conflicting) knowledge coexist. Here, contextual knowledge types (factual vs. counterfactual) compete for attention within contextual heads: the information intensity for both in subfigures (c) and (d) is substantially lower than in subfigure (a). However, the presence of factual knowledge in the context has minimal impact on memory heads (parametric knowledge routing). For LLaMA3.1 and Qwen3.0, the average information intensity of top-10 memory heads slightly increases ( ≤ 5% ), while Qwen2.5 shows a minor decrease ( ≤ 5% ). This indicates competition occurs only within the same knowledge type (contextual vs. contextual), not across distinct types (contextual vs. parametric).

5.3 Residual stream analysis

Building on the probing framework (Section 3.4), we analyze the information intensity of parametric knowledge ( E f ) and contextual knowledge ( E cf ) across transformer layers by probing Attention updates, MLP updates, and layer outputs. Key findings from experiments on LLaMA 3.1 under counterfactual documents D cf and mixed documents D cf ∪ D f (Figures 4-5, with Qwen2.5/3.0 results in Appendix Figures 9-12) reveal:

(i) Divergent propagation mechanisms for parametric vs. contextual knowledge :

Parametric knowledge ( E f ) exhibits low information intensity in early-layer attention and MLP updates but shows rapid amplification in later layers (Figure 4). This suggests fragmented parametric knowledge is gradually consolidated through multi-layer interactions, forming coherent units only near the output layer.

Contextual knowledge ( E cf ), in contrast, is immediately detectable in early attention updates, persists latently in residual streams, and is reactivated in later stages for answer generation. This reflects its direct encoding from external contexts.

(ii) Superposition as a Reconciliation Pathway : When replacing D cf with a mixed document D cf ∪ D f (Figure 5), parametric knowledge ( E f ) gains competitive advantage in output layers compared to scenarios without contextual support (Figure 4 (c)). This indicates that:

Factual knowledge from multiple sources (parametric + contextual) can superpositionally reinforce their information intensity in the residual stream, enhancing their influence on final outputs.

Conflicting knowledge (e.g., E cf ) coexists in the residual stream without erasure, suggesting LLMs preserve opposing evidence rather than discarding it.

5.4 Evidence from Native Generation Dynamics

Although we aim to minimize interference, probing may inadvertently influence the model's generation behavior. To further validate the knowledge reconciliation mechanisms identified through probing, we conduct two intervention experiments using LLaMA3.1 in its native generation setting.

0

5

10

15

20

25

30

Layer ID

0.002

0.003

0.004

0.005

0.006

0.007

0.008

Average prauc score

(a). Attention Scores across Layers

cf

f

cf

cf

f

f

0

5

10

15

20

25

30

Layer ID

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

Average prauc score

(b). MLP Scores across Layers

cf

f

cf

cf

f

f

0

5

10

15

20

25

30

Layer ID

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

Average prauc score

cf

f

cf

cf

f

f

For ease of evaluation, we formulate the test in a multiple-choice format, where factual_answer , counterfactual_answer , don't know , and other are randomly assigned to options A, B, C, D.

(1) Causal Intervention via Zeroing Attention Heads. Given a counterfactual document as context, we zero out the identified top-10 context heads and memory heads to observe the impact of this intervention on LLM response generation. The prompt used is shown in Table 10. As shown in Table 2, removing context heads reduces contextual influence (54.6% → 43.6%) and increases reliance on parametric knowledge (22.7% → 34.9%). Conversely, removing memory heads weakens parametric influence (22.7% → 9.2%) and strengthens contextual responses (54.6% → 65.5%). These results provide causal evidence that the identified heads perform the reconciliation function, as described in Section 5.2.

(2) Attention Redistribution via Prompt Manipulation. We design three different prompts to enhance the model's attention to counterfactual documents in the context: Duplicate and append , Insert prompt randomly , and Tag and append . Detailed descriptions of these strategies are provided in Appendix H. In the left panel of Figure 6, we provide both factual and counterfactual documents as context to the model. After applying the three strategies above, the proportion of counterfactual_answers increase significantly, from 27 . 7% to 45 . 3% , 40 . 9% , and 41 . 2% , respectively. This indicates that there exists competition between factual_answer and counterfactual_answer from the context, where strengthening one weakens the other. In the right panel of Figure 6, the context contains only counterfactual documents. After applying the three strategies above, the proportion of counterfactual_answer show no significant change. This suggests that there is no direct competition between counterfactual_answer from the context and factual_answer from the model's parameters; strengthening one does not notably affect the other.

5.5 Discussions

Our experimental findings reveal nuanced interactions between the two hypothesized mechanisms in Section 3.1, suggesting a hybrid processing strategy rather than exclusive adherence to either hypothesis.

Attention Competition Hypothesis Revisited . The hypothesis posits that contextual and parametric knowledge compete through attention patterns. Our results partially support this view but with critical

Factual + Counterfactual

Duplicate and append

Insert prompt randomly

Tag and append

0

10

20

30

40

50

60

70

Percentage (%)

56.5

40.2

51.1

44.9

27.7

45.3

40.9

41.2

4.7

3.8

1.3

3.3

11.2

10.7

6.7

10.5

Only counterfactual

Duplicate and append

Insert prompt randomly

Tag and append

0

10

20

30

40

50

60

70

Percentage (%)

22.7

22.8

22.8

23.2

54.6

55.6

59.9

51.3

8.4

7.9

4.9

11.5

14.3

13.7

12.3

14.0

factual_answer

counterfactual_answer

don't know

other

qualifications: (1) Distinct routing pathways : The separation of contextual vs. parametric knowledge into dedicated attention heads (Section 5.2(i)) supports the notion of parallel processing rather than direct competition. This suggests attention mechanisms implement a "division of labor" strategy rather than winner-takes-all competition. (2) Within-type competition : The observed suppression between factual/counterfactual contextual knowledge in mixed contexts (Section 5.2(iii)) aligns with attention-level competition, but only within the same knowledge type. This implies attention heads implement intra-type competition rather than cross-type rivalry.

MLP Arbitration Hypothesis Revisited . The hypothesis suggests MLP layers resolve conflicts through suppression of contradictory signals. Our analysis reveals more complex dynamics: (1) Persistence of Conflicting Signals : Despite MLP operations on the residual stream, conflicting contextual and parametric knowledge maintain a residual presence across layers (Section 5.3(i)), contradicting the suppression hypothesis. (2) Superpositional reconciliation : The enhanced parametric knowledge intensity under mixed contexts (Section 5.3(ii)) indicates MLPs may implement additive superposition of aligned evidence from multiple sources, with the accumulation magnitude directly determining its influence on final outputs.

6 Conclusions

This study presents a mechanistic understanding of how LLMs reconcile conflicting parametric and contextual knowledge in RAG frameworks. By introducing an entity-aware probing methodology, we uncover three fundamental principles governing knowledge dynamics: (1) Attention head specialization enables parallel processing of distinct knowledge types through dedicated routing pathways; (2) Asymmetric propagation mechanisms reveal contextual knowledge emerges abruptly via attention while parametric knowledge accumulates gradually through MLP layers; and (3) Superpositional reconciliation demonstrates that conflicting knowledge persists in residual streams, with aligned multi-source signals reinforcing each other to determine output selection. This work provides both theoretical foundations and practical guidance for building more reliable and interpretable retrieval-augmented language models.

References

Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings . OpenReview.net, 2017. URL https://openreview.net/forum? id=HJ4-rAVtl .

Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens, 2023. URL https://arxiv.org/abs/2303.08112 .

Baolong Bi, Shenghua Liu, Yiwei Wang, Yilong Xu, Junfeng Fang, Lingrui Mei, and Xueqi Cheng. Parameters vs. context: Fine-grained control of knowledge reliance in language models, 2025. URL https://arxiv.org/abs/2503.15888 .

Hung-Ting Chen, Michael J. Q. Zhang, and Eunsol Choi. Rich knowledge sources bring complex knowledge conflicts: Recalibrating models to reflect conflicting evidence, 2022. URL https://arxiv. org/abs/2210.13701 .

Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models, 2024. URL https://arxiv.org/ abs/2309.03883 .

Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proceedings of the 23rd International Conference on Machine Learning , ICML '06, page 233-240, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595933832. doi: 10.1145/1143844.1143874. URL https://doi.org/10.1145/1143844.1143874 .

Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread , 2021. https://transformercircuits.pub/2021/framework/index.html.

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey, 2024. URL https://arxiv.org/abs/2312.10997 .

Zorik Gekhman, Jonathan Herzig, Roee Aharoni, Chen Elkind, and Idan Szpektor. Trueteacher: Learning factual consistency evaluation with large language models, 2023. URL https://arxiv.org/abs/2305. 11171 .

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 5484-5495, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.446. URL https://aclanthology.org/2021.emnlp-main. 446/ .

Reto Gubelmann and Siegfried Handschuh. Context matters: A pragmatic study of PLMs' negation understanding. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 4602-4621, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/ 2022.acl-long.315. URL https://aclanthology.org/2022.acl-long.315/ .

John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representations. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages 4129-4138, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1419. URL https://aclanthology.org/ N19-1419/ .

Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations , 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9 .

Yizheng Huang and Jimmy Huang. A survey on retrieval-augmented text generation for large language models, 2024. URL https://arxiv.org/abs/2404.10981 .

Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. What does BERT learn about the structure of language? In Anna Korhonen, David Traum, and Lluís Màrquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 3651-3657, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1356. URL https://aclanthology. org/P19-1356/ .

Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, Xiaojian Jiang, Jiexin Xu, Qiuxia Li, and Jun Zhao. Tug-of-war between knowledge: Exploring and resolving knowledge conflicts in retrieval-augmented language models. In Nicoletta Calzolari, Min-Yen Kan, Véronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy , pages 16867-16878. ELRA and ICCL, 2024. URL https://aclanthology.org/2024.lrec-main. 1466 .

Zhuoran Jin, Pengfei Cao, Hongbang Yuan, Yubo Chen, Jiexin Xu, Huaijun Li, Xiaojian Jiang, Kang Liu, and Jun Zhao. Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics: ACL 2024 , pages 1193-1215, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.findings-acl.70. URL https://aclanthology.org/2024.findings-acl.70/ .

Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension, 2017. URL https://arxiv.org/abs/1705. 03551 .

Mintong Kang, Nezihe Merve Gürel, Ning Yu, Dawn Song, and Bo Li. C-rag: Certified generation risks for retrieval-augmented language models, 2024. URL https://arxiv.org/abs/2402.03181 .

Nora Kassner, Oyvind Tafjord, Hinrich Schütze, and Peter Clark. BeliefBank: Adding memory to a pre-trained language model for a systematic notion of belief. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 8849-8861, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.697. URL https://aclanthology.org/2021.emnlp-main.697/ .

Shahar Katz, Yonatan Belinkov, Mor Geva, and Lior Wolf. Backward lens: Projecting language model gradients into the vocabulary space, 2024. URL https://arxiv.org/abs/2402.12865 .

Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. Large language models with controllable working memory, 2022. URL https: //arxiv.org/abs/2211.05110 .

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 7052-7063, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.565. URL https: //aclanthology.org/2021.emnlp-main.565/ .

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entitybased knowledge conflicts in question answering, 2022. URL https://arxiv.org/abs/2109.05052 .

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 17359-17372. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ 6f1d43d5a82a37e89b0665b33bf3a182-Paper-Conference.pdf .

nostalgebraist. Interpreting GPT: The logit lens. https://www.lesswrong.com/posts/ AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens , 2020. LessWrong.

Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac HatfieldDodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. In-context learning and induction heads, 2022. URL https://arxiv.org/abs/2209.11895 .

Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. Language models as knowledge bases? In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLPIJCNLP 2019, Hong Kong, China, November 3-7, 2019 , pages 2463-2473. Association for Computational Linguistics, 2019. doi: 10.18653/V1/D19-1250. URL https://doi.org/10.18653/v1/D19-1250 .

Quang Hieu Pham, Hoang Ngo, Anh Tuan Luu, and Dat Quoc Nguyen. Who's who: Large language models meet knowledge conflicts in practice, 2024. URL https://arxiv.org/abs/2410.15737 .

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text, 2016. URL https://arxiv.org/abs/1606.05250 .

Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model?, 2020. URL https://arxiv.org/abs/2002.08910 .

Mansi Sakarvadia, Arham Khan, Aswathy Ajith, Daniel Grzenda, Nathaniel Hudson, André Bauer, Kyle Chard, and Ian Foster. Attention lens: A tool for mechanistically interpreting the attention head information retrieval mechanism, 2023. URL https://arxiv.org/abs/2310.16270 .

Gaurang Sriramanan, Siddhant Bharti, Vinu Sankar Sadasivan, Shoumik Saha, Priyatham Kattakinda, and Soheil Feizi. LLM-check: Investigating detection of hallucinations in large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024. URL https: //openreview.net/forum?id=LYx4w3CAgy .

Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. What do you learn from context? probing for sentence structure in contextualized word representations. In International Conference on Learning Representations , 2019. URL https://openreview.net/forum?id=SJzSgnRcKX .

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https://arxiv.org/abs/1706. 03762 .

Fei Wang, Xingchen Wan, Ruoxi Sun, Jiefeng Chen, and Sercan Ö. Arık. Astute rag: Overcoming imperfect retrieval augmentation and knowledge conflicts for large language models, 2024. URL https: //arxiv.org/abs/2410.07176 .

Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. Retrieval head mechanistically explains long-context factuality, 2024. URL https://arxiv.org/abs/2404.15574 .

Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. Retrieval head mechanistically explains long-context factuality. In The Thirteenth International Conference on Learning Representations , 2025. URL https://openreview.net/forum?id=EytBpUGB1Z .

Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum?id=auKAUJZMO6 .

Rongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu. Knowledge conflicts for llms: A survey, 2024. URL https://arxiv.org/abs/2403.08319 .

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering, 2018. URL https://arxiv.org/abs/1809.09600 .

Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. Physics of language models: Part 2.1, gradeschool math and the hidden reasoning process. In The Thirteenth International Conference on Learning Representations , 2025. URL https://openreview.net/forum?id=Tn5B6Udq3E .

Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, and Furu Wei. Differential transformer, 2025. URL https://arxiv.org/abs/2410.05258 .

Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi, Ece Kamar, and Besmira Nushi. Attention satisfies: A constraint-satisfaction lens on factual errors of language models, 2024. URL https://arxiv.org/abs/2309.15098 .

Jun Zhao, Can Zu, Xu Hao, Yi Lu, Wei He, Yiwen Ding, Tao Gui, Qi Zhang, and Xuanjing Huang. LONGAGENT: Achieving question answering for 128k-token-long documents through multi-agent collaboration. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages 16310-16324, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.912. URL https://aclanthology.org/2024.emnlp-main.912/ .

Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, and Bin Cui. Retrieval-augmented generation for ai-generated content: A survey, 2024. URL https://arxiv.org/abs/2402.19473 .

Wenxuan Zhou, Sheng Zhang, Hoifung Poon, and Muhao Chen. Context-faithful prompting for large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023 , pages 14544-14556, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-emnlp.968. URL https://aclanthology.org/2023.findings-emnlp.968/ .

Technical Appendices and Supplementary Material

Limitations

Our study has several limitations that warrant careful consideration. First, while our entity-aware probing framework demonstrates effectiveness in tracing knowledge flows within decoder-only transformers like LLaMA and Qwen series, its generalizability to other architectures (e.g., encoderdecoder models) or significantly larger models remains untested. Second, our experiments use synthetic counterfactual documents constructed via LLM-based entity substitution. While this ensures controlled evaluation, real-world retrieval contexts may contain more subtle conflicts involving temporal reasoning, numerical discrepancies, or conflicting logical implications-not merely entitylevel replacements. Future work should validate our findings on naturally occurring knowledge conflicts from dynamic knowledge sources.

A Related work

Retrieval-augmented generation and knowledge conflicts Large language models often face challenges such as outdated knowledge and hallucinations [33]. Retrieval-Augmented Generation (RAG) offers a solution to these issues by retrieving and integrating external data as a supplement [46, 19, 14]. External knowledge introduces challenges, especially conflicts with model-parametric knowledge [40, 31]. Such conflicts introduce source-level uncertainty and may significantly degrade model performance [40, 29]. While some studies suggest a model preference for parametric memory [24], others highlight a greater reliance on external knowledge [4]. Naturally, various methods have been proposed to mitigate these conflicts [36, 3, 22, 9]. However, how models handle such conflicts remains an area for further research.

Internal Probing for Model Interpretability A common approach to interpretability is probing model internals [44, 37, 5, 43]. Yuksekgonul et al. [44], Wu et al. [37] investigated the behavior of attention mechanisms when the model is generating factual answers. There are also approaches known as lens methods. These include LL-based methods [26] that analyze information flow within the attention mechanism [32], and approaches that incorporate Logit Lens during backpropagation [21]. We effectively improve upon these probing methods and adapt them to QA tasks, showing that they yield both promising performance and effective interpretability.

B Intuition and Limitations of Entity-aware Probing

First, we consider a simple case: Document : The capital of France is London. Q : Where is the capital of France? A : The capital of France is ___? In this case, the candidate entity tokens include 'London' (contextual knowledge) and 'Paris' (parametric knowledge). Here, the answer prefix 'The capital of France is' essentially serves to elicit the model's conflict reconciliation behavior. In standard generation, not every token prediction involves knowledge-intensive reasoning or conflict resolution. Many tokens (e.g., articles, conjunctions, formatting) are generated through relatively shallow pattern matching. The LoRA updates and the entity triples appended after the document and question in our work can essentially be viewed as a trainable soft prompt. It serves a similar function to "The capital of France is" in eliciting the model's conflict reconciliation behavior. However, we must acknowledge that this approach increases the risk of altering the model's internal behavioral patterns. We believe that constructing more flexible probing methods with minimal intervention in model behavioral patterns represents an important open problem worthy of continued exploration.

C Attribution dataset adaptation and evaluation metrics

Dataset adaptation : We build our evaluation using three established QA benchmarks, adapting them to support passage- and sentence-level attribution tasks. HotpotQA serves as both the training set of the probe and a sentence-level attribution testing set (see Section 3.3 for training details), while SQuAD and TriviaQA are exclusively used for passage-level attribution evaluation (without dataset-specific probe fine-tuning).

Hotpotqa [41]: Designed for multi-hop reasoning, HotpotQA provides sentence-level attribution target through human-annotated evidence sentences. Each query is paired with 10 passages, of which exactly 2 contain gold evidence sentences. The remaining 8 passages act as natural distractors, enabling fine-grained evaluation of sentence attribution without requiring synthetic negative sampling.

TriviaQA [18]: For passage-level evaluation, we construct input documents by combining the gold passage from each test sample with 4 randomly selected passages from other test instances.

SQuAD [30]: Following a similar passage-level evaluation protocol to TriviaQA but with shorter passages, we aggregate each gold passage with 9 randomly sampled irrelevant passages to form evaluation documents.

Evaluation metrics : We evaluate the results using three standard metrics: Precision , Recall and F1 . Given that the ground truth passage/sentence indices are known, we compute the metrics as follows:

Precision measures the proportion of correctly identified relevant passages among all predicted as relevant, computed as | Predicted ∩ GroundTruth | / | Predicted | .

Recall measures the proportion of correctly identified relevant passages among all truly relevant ones, computed as | Predicted ∩ GroundTruth | / | GroundTruth | .

F1 score is the harmonic mean of precision and recall, calculated as (2 · Precision · Recall ) / ( Precision + Recall ) .

D Implementation Details

The experiments were conducted using the Openrlhf framework with DeepSpeed for distributed training. The model was trained with BF16 mixed precision and Flash Attention optimization to accelerate computation and reduce memory overhead. All model parameters frozen except for the LoRA adapter injected into the input embedding layer. Training utilized a global batch size of 256 with a micro-batch size of 4 for memory-efficient gradient accumulation. The AdamW optimizer was configured with a learning rate of 1e-5 and ZeRO Stage 2 optimization. The model was trained for 2 epochs. The input sequence length was capped at 3,000 tokens. Training the probe requires approximately 3 days on 8 A100 GPUs, and testing on a single dataset takes about 1 hour.

E Visualization of PRAUC Scores for Qwen2.5 and Qwen3

0

5

10

15

20

25

Head ID

0

5

10

15

20

25

Layer ID

(a).

cf

cf

0

5

10

15

20

25

Head ID

(b).

cf

f

0

5

10

15

20

25

Head ID

(c).

cf

f

cf

0

5

10

15

20

25

Head ID

(d).

cf

f

f

0.01

0.02

0.03

0.04

0.05

0.06

F Full List of Prompts Used in the Experiments

0

10

20

30

Head ID

0

5

10

15

20

25

30

35

Layer ID

(a).

cf

cf

0

10

20

30

Head ID

(b).

cf

f

0

10

20

30

Head ID

(c).

cf

f

cf

0

10

20

30

Head ID

(d).

cf

f

f

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0

5

10

15

20

25

Layer ID

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

Average prauc score

(a). Attention Scores across Layers

cf

cf

cf

f

0

5

10

15

20

25

Layer ID

0.000

0.005

0.010

0.015

0.020

0.025

0.030

Average prauc score

(b). MLP Scores across Layers

cf

cf

cf

f

0

5

10

15

20

25

Layer ID

0.00

0.05

0.10

0.15

0.20

0.25

0.30

Average prauc score

cf

cf

cf

f

Example 1:

Passage A : The 2015 Diamond Head Classic was a college basketball tournament ... Buddy Hield was named the tournament's MVP.

Passage B : Chavano Rainier 'Buddy' Hield is a Bahamian professional basketball player for the Sacramento Kings of the NBA...

0

5

10

15

20

25

30

35

Layer ID

0.001

0.002

0.003

0.004

0.005

0.006

0.007

0.008

0.009

Average prauc score

(a). Attention Scores across Layers

cf

cf

cf

f

0

5

10

15

20

25

30

35

Layer ID

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

Average prauc score

(b). MLP Scores across Layers

cf

cf

cf

f

0

5

10

15

20

25

30

35

Layer ID

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

Average prauc score

cf

cf

cf

f

Question : Which team does the player named 2015 Diamond Head Classic's MVP play for? Answer : Sacramento Kings

Analysis :

Step 1: Using the clue '2015 Diamond Head Classic,' identify in Passage A that the MVP is 'Buddy Hield.'

Step 2: Using the clue 'Buddy Hield,' find in Passage B that he currently plays for the 'Sacramento Kings.'

Therefore, the answer is the Sacramento Kings.

The entities involved in the aforementioned analysis, in logical order, are: ['2015 Diamond Head Classic', 'Buddy Hield', 'Sacramento Kings']

The entities that not reffered to in the questions are: ['Buddy Hield', 'Sacramento Kings'].

Counterfactual Data Synthesis :

Let's conduct a counterfactual analysis by replacing the entities that were mentioned in the analysis but did not appear in the Question.

As we mentioned above, these entities are: ['Buddy Hield', 'Sacramento Kings']

We will replace them with fictional entities: ['Alex Taylor', 'Vancouver Bears']

Revised Passage A : The 2015 Diamond Head Classic was a college basketball tournament ... Alex Taylor was named the tournament's MVP.

Revised Passage B : Alex Taylor is a Canadian professional basketball player for the Vancouver Bears of the NBA...

Revised Question : Which team does the player named 2015 Diamond Head Classic's MVP play for?

Revised Analysis:

Step 1: Using the clue '2015 Diamond Head Classic,' identify in Passage A that the MVP is 'Alex Taylor.'

Step 2: Using the clue 'Alex Taylor,' find in Passage B that he currently plays for the 'Vancouver Bears.'

Therefore, the answer is the Vancouver Bears.

The entities involved in the revised analysis, in logical order, are: ['2015 Diamond Head Classic', 'Alex Taylor', 'Vancouver Bears']

The entities that not reffered to in the questions are: ['Alex Taylor', 'Vancouver Bears']

Example 2:

Paragraph A : Several current and former members of the Pittsburgh Pirates - ... John Milner, Dave Parker, and Rod Scurry...

Paragraph B : David Gene Parker, nicknamed 'The Cobra', is an American former player in Major League Baseball...

Question : Which former member of the Pittsburgh Pirates was nicknamed 'The Cobra'? Answer : David Gene Parker

Analysis

Step 1: Using the clue 'Pittsburgh Pirates' from the Question, identify in Paragraph A the list of current and former members that includes 'John Milner,' 'Dave Parker,' and 'Rod Scurry.'

Step 2: Using the clue 'nickname 'The Cobra'' from the Question, identify in Paragraph B that 'David Gene Parker' is nicknamed 'The Cobra.'

Step 3: Connect the information that 'Dave Parker' from Paragraph A is the same individual as 'David Gene Parker' from Paragraph B who is nicknamed 'The Cobra.'

Since 'David Gene Parker' and 'Dave Parker' refer to the same entity, we will use 'David Gene Parker' to refer to it in subsequent analysis.

The entities involved in the aforementioned analysis, in logical order, are: ['Pittsburgh Pirates', 'David Gene Parker', 'The Cobra']

The entities that not reffered to in the questions are: ['David Gene Parker']

Counterfactual Data Synthesis:

Let's conduct a counterfactual analysis by replacing the entities that were mentioned in the analysis but did not appear in the Question.

As we mentioned above, these entities are: ['David Gene Parker']

We will replace them with fictional entities: ['Alexander T. Taylor']

Revised Paragraph A : Several current and former members of the Pittsburgh Pirates - ... John Milner, Alex Taylor, and Rod Scurry...

Revised Paragraph B : Alexander T. Taylor, nicknamed 'The Cobra,' is an American former player in Major League Baseball...

Revised Question : Which former member of the Pittsburgh Pirates was nicknamed 'The Cobra'?

Revised Analysis :

Step 1: Using the clue 'Pittsburgh Pirates' from the Question, identify in Revised Paragraph A the list of current and former members that includes 'John Milner,' 'Alex Taylor,' and 'Rod Scurry.'

Step 2: Using the clue 'nickname 'The Cobra'' from the Question, identify in Revised Paragraph B that 'Alexander T. Taylor' is nicknamed 'The Cobra.'

Step 3: Connect the information that 'Alex Taylor' from Paragraph A is the same individual as 'Alexander T. Taylor' from Paragraph B who is nicknamed 'The Cobra.' Therefore, the answer is Alexander T. Taylor.

Entities involved in the revised analysis, in logical order, are: ['Pittsburgh Pirates', 'Alexander T. Taylor', 'The Cobra']

Entities that are not referred to in the questions: ['Alexander T. Taylor']

You are a logical analysis expert. Please provide your analysis and counterfactual data synthesis of the following material in the manner described above. In your analysis, the entities should be presented in the form of a list.

Passage A : {Passage_Content} Passage B : {Passage_Content} Question : {Question_Content} Answer : {Answer_Content} Analysis :

Example 1:

Sentence: The thank you letter is an opportunity for you to remind the interviewer why you are the best fit for the position and to address any remaining issues.

Answer: [thank you letter, interviewer, best fit, position, remaining issues]

Analysis: The answer in every example is a list that contains the entities in the sentence. Entities refer to specific nouns or noun phrases with clear meanings that play a key role in the sentence. Therefore, these words are extracted as entities.

You are a language expert. Now, I will give you a sentence, and you need to output an entity list for the sentence in the format similar to the example. If there are no entities, return an empty list.

<sentence> {Sentence} </sentence>

Table 5: Prompt template for citation selection in reading comprehension tasks.

You are an expert in reading comprehension. You will be given a passage and a question, where the sentences or paragraphs in the passage are segmented by identifiers such as <c0>. Please read the document and question carefully, analyze which sentences or paragraphs need to be cited to answer this question, and then return a list of citations in the format: <cite>[citation index only numbers]</cite> .

Example:

[Document Start]

<C0>In the last two decades, several hundred planets have been detected beyond our Solar-system.

<C1>Most of these extra-solar planets orbit sun-like stars.

<C2>A small number have been detected around stars that are in their late evolutionary state.

<C3>The phase directly after the RGB stage, the Horizontal Branch (HB), however, is still unexplored;

<C4>Besides its evolutionary stage, a star's chemical composition appears to be a major indicator of its probability.

[Document End]

[Question]

How does the metallicity of PSR B1620 b, at around 1% of the Sun's, compare to other known exoplanet host stars?

[Remind]

Please read the document and question carefully, analyze which sentences or paragraphs need to be cited to answer this question, and then return a list of citations in the format: <cite>[citation index only numbers]</cite> .

<cite>[0,3]</cite>

Now get ready to handle the following test case.

[Remind]

Please read the document and question carefully, analyze which sentences or paragraphs need to be cited to answer this question, and then return a list of citations in the format: <cite>[citation index only numbers]</cite> .

[Answer Citation List]

0

5

10

15

20

25

Layer ID

0.0020

0.0025

0.0030

0.0035

0.0040

0.0045

0.0050

0.0055

0.0060

Average prauc score

(a). Attention Scores across Layers

cf

f

cf

cf

f

f

0

5

10

15

20

25

Layer ID

0.005

0.010

0.015

0.020

Average prauc score

(b). MLP Scores across Layers

cf

f

cf

cf

f

f

0

5

10

15

20

25

Layer ID

0.00

0.05

0.10

0.15

0.20

Average prauc score

cf

f

cf

cf

f

f

0

5

10

15

20

25

30

35

Layer ID

0.003

0.004

0.005

0.006

0.007

Average prauc score

(a). Attention Scores across Layers

cf

f

cf

cf

f

f

0

5

10

15

20

25

30

35

Layer ID

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

Average prauc score

(b). MLP Scores across Layers

cf

f

cf

cf

f

f

0

5

10

15

20

25

30

35

Layer ID

0.00

0.05

0.10

0.15

0.20

0.25

Average prauc score

cf

f

cf

cf

f

f

You are an expert in reading comprehension and entity-aware citation analysis. You will be given a passage with sentences segmented by identifiers such as <c0>, entity information for each sentence, and a question.

Please read the document, entity information, and question carefully. Use step-by-step reasoning to analyze which sentences or paragraphs need to be cited to answer this question, considering both the semantic content and the entities present in each sentence. Then return a list of citations in the format: <cite>[citation index only numbers]</cite> .

Example:

[Document Start]

<C0>In the last two decades, several hundred planets have been detected beyond our Solar-system. <C1>Most of these extra-solar planets orbit sun-like stars. <C2>A small number have been detected around stars that are in their late evolutionary state, such as Red Giant Branch (RGB) stars and pulsars. <C3>The phase directly after the RGB stage, the Horizontal Branch (HB), however, is still unexplored; therefore, there is no empirical evidence for whether close-in planets, i.e., those with semi-major axes less than 0.1 AU, survive the giant phase of their host stars.

<C4>Besides its evolutionary stage, a star's chemical composition appears to be a major indicator of its probability for hosting a planet. <C5>Previous studies, e.g.' showed that main-sequence (MS) stars that host giant planets are metal-rich. <C6>This finding is supported by the large exoplanet search surveys around MS stars reporting a connection between planet frequency and metallicity, and a survey of 160 metal-poor main-sequence stars finding no evidence for Jovian planets. <C7>Until now, only very few planets have been detected around stars with metallicities as low as [Fe/H]= $-1$, i.e. 10% of the sun's metallicity. <C8>The detection of PSR B1620 b, a Jovian planet orbiting a pulsar in the core of the metal-poor globular cluster M4 ([Fe/H]=$-1.2$), suggests, however, that planets may form around metal-poor stars, although the formation mechanism of this particular planet might be linked to the dense cluster environment. <C9>We used the Fibre-fed Extended Range Optical Spectrograph (FEROS), a high-resolution spectrograph ($R=48,000$) attached to the 2.2 meter Max-Planck Gesellschaft/European Southern Observatory (MPG/ESO) telescope1, to observe the star. <C10>This star is classified as a red HB (RHB) star (Fig. 1) and its metal content is [Fe/H]_ mean = -2 . 09 i.e. about 1% that of the Sun. <C11>So far, is not known as a binary system. <C12>Detailed stellar parameters can be found in Supporting Online Material, Section [text1]. <C13>Previous radial velocity (RV) measurements of showed a systematic velocity of about 300 (km s)ˆ{-1} with respect to the Sun, indicating that the star belongs to the stellar halo. <C14>Indeed, the star has been connected to the Helmi stream, a group of stars that share similar orbital parameters that stand apart from those of the bulk of other stars in the solar neighborhood. <C15>The Helmi stream stars move on prograde eccentric orbits (R_peri~7 kpc, R_apo~16 kpc) that reach distances up to \vert z\vert_max~13 kpc above and below the galactic plane. <C16>From that, it has been concluded that these stars were once bound to a satellite galaxy of the Milky Way that was tidally disrupted 6-9 Ga ago. <C17>, with a mean metallicity estimate of [Fe/H]=-2.1, is far more metal-poor than any previously known exoplanet hosting star (Fig. 3). <C18>For the existing theories of hot giant planet formation, metallicity is an important parameter: in particular, it is fundamental for the core-accretion planet formation model. <C19>It might be that initially, in the planet formation phase, had a higher metallicity, and that during its subsequent evolution, it lost its heavier elements. <C20>For example, during the giant phase, heavy elements could have had been incorporated into dust grains and then separated from the star's atmosphere. <C21>However, given the star's membership to the Helmi stream, in which the most metal-rich sub-dwarfs known so far have [Fe/H]$~-1.5$, we do not expect its initial Fe abundance to exceed this value. [Document End] "metallicity"]

,

G Dataset example

Continued on next page

Continued on next page

. . .

. . .

H Supplementary Experiments for Competing Hypotheses

We employ three methods to enhance the model's attention to specific information within the prompt, and observe how the model's responses change accordingly. To enhance the model's attention to counterfactual information, we adopt the following three prompt construction strategies:

Duplicate and append : Duplicate the counterfactual sentences and append them to the original context to reinforce their salience through repetition (see Table 12).

Insert prompt randomly : Extract all counterfactual sentences, concatenate them, prepend an emphatic instruction, and insert the resulting string at a random position in the original context (see Table 13).

Tag and append : Extract and concatenate counterfactual sentences, enclose them with <must pay attention to> tags for emphasis, and append this segment to the original context (see Table 14).

Continued on next page

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state the paper's contributions, including the design of adaptive probes, empirical validation of their effectiveness, and analysis of mechanisms that resolve conflicts between internal and external knowledge. These claims are consistently supported by both theoretical discussions and experimental results throughout the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of our approach in the Appendix 6, Including only decoder-only models and synthetic counterfactual data limits the scope of our study.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: Our paper does not include formal theoretical results or proofs. Instead, we focus on empirical hypotheses regarding knowledge conflict resolution, which are validated through carefully designed ablation studies and experiments (see Section 4).

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide detailed descriptions of our method in the main paper(see in Section 3). And data sources, preprocessing steps, and our experimental setup can be seen in Section 4. Additionally, all prompt constructions and relevant implementation details are included in the appendix to support reproducibility.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We are eager to release the code and data publicly; however, due to ongoing company approval processes, we are currently unable to share them immediately. We plan to make them available as soon as these procedures are completed.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We will include detailed training and testing settings in the Appendix D to ensure full transparency and reproducibility.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [No]

Justification: Training the probe requires substantial computational resources, and our cost budget does not support running the training process multiple times. However, we conducted tests on multiple models as well as zero-shot evaluations across different datasets. The consistent improvements observed provide evidence of the probe's effectiveness and accuracy.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: See Section D

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have carefully reviewed the NeurIPS Code of Ethics and confirm that our research fully complies with its principles. Our work does not involve human subjects, sensitive data, or applications with potential for harm, and we have ensured fairness, transparency, and reproducibility throughout.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This work focuses on understanding internal mechanisms of models and has not been applied in real-world production or societal contexts. Therefore, we believe it does not have immediate broader societal impacts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The work presented in this paper does not involve data or models that pose significant risks of misuse, and therefore no specific safeguards are necessary.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have carefully followed the license terms and conditions associated with all third-party assets used in this work, including datasets and code. All original sources, creators, and license information are properly cited in the paper. When applicable, we have also included version numbers and URLs for clarity and reproducibility.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: This paper does not release any new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing experiments or research with human subjects

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing experiments or research with human subjects, and therefore no IRB approval or equivalent review was required.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: We use large language models to generate training data. Detailed descriptions and usage are provided in Section 3.3. All prompts can be found in the appendix.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel entity-aware probing framework to investigate how LLMs reconcile potentially conflicting parametric and contextual knowledge in RAG settings. The authors reinterpret transformer computation through residual streams and design a LoRA-based probing method that can detect ad-hoc entity relevance without altering base model parameters. Through extensive experiments across QA benchmarks and counterfactual setups, the authors reveal that contextual and parametric knowledge are routed by distinct attention heads, exhibit asymmetric propagation dynamics, and interact through a mechanism of superpositional reconciliation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* The proposed Attention Competition Hypothesis and MLP Arbitration Hypothesis provide novel insights for interpreting conflicting knowledge processing. These hypotheses are empirically validated through a carefully designed probing framework that traces entity-specific signals across transformer layers. \n* The distinction between attention head specialization and MLP-based accumulation mechanisms provides a refined understanding of LLM internal processes. The discovery that conflicting knowledge can coexist in residual streams is particularly revealing.\n* The authors conduct extensive and controlled experiments to validate their hypotheses. Notably, they introduce a key ablation baseline to demonstrate that probe predictions depend on the internal knowledge processing of the base LLM, rather than the capacity of the LoRA adapter.\n\nWeaknesses:\n\n* While the entity-aware probing framework avoids modifying base model weights, it introduces LoRA adapters and injects special tokens (\\[START\\], \\[END\\]) into the input. These interventions may perturb the native inference pathways. Consequently, the behavior traced by the probe may deviate subtly from that of the vanilla LLM, raising concerns about the fidelity of the interpretability conclusions.\n* A key claim is that conflicting knowledge persists in residual streams and is superpositionally reconciled rather than being suppressed. However, this assertion is not sufficiently substantiated with targeted experiments. For example, the authors do not investigate whether zeroing out high-information-intensity attention heads would alter the final model prediction.\n\nQUESTIONS:\n* Have the authors considered how toxic contextual knowledge that might trigger LLMs' safety mechanisms affects internal knowledge reconciliation? Would similar patterns of attention routing still hold?\n* To what extent do the \\[START\\] and \\[END\\] tokens introduced for probing influence the internal computation of LLMs? Specifically, how much attention do LLMs allocate to these OOD tokens?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper provides a rigorous analysis and mechanistic understanding of how LLMs reconcile conflicting parametric knowledge (internalized in the model parameters during pretraining) and contextual knowledge (present as in context tokens) in retrieval-augmented generation/RAG settings.\n\nSpecifically the authors propose a novel entity based probing technique which leverages specially appended tokens and a finetuned LoRA adapter to track entity-level relevance across transformer layers, without fine-tuning the base model.\nUsing this framework they probe the information of a specified entity at 3 computation points within each transformer layer: MLP update, attention update and layer output.\n\nThey find several interesting patterns, consistent across different models, namely that \n1. parametric and contextual knowledge utilize distinct pathways revealing attention head specialization; \n2. contextual knowledge emerges more abruptly whereas parametric knowledge accumulates more gradually through MLP layers; \n3. super-positional reconciliation shows conflicting knowledge persists in residual streams with similar information from multiple sources reinforcing each other over contradicting information e.g. parametric + factual document versus counterfactual document information.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The approach described as far as I know is quite novel.\n2. The information conflict between parametric and contextual knowledge and controlling/understanding it is a topic of high importance for academic and industrial research.\n3. The experimental approach is very rigorous and well through out. \n4. Experiments are robust with comprehensive and well-motivated ablations and conclusions.\n5. Reveals extremely interesting and insightful patterns around where and how parametric and contextual information is aggregated in the LLM and how knowledge conflicts are handled.\n\nWeaknesses:\n1. I am still less convinced on the triple being appended to the model input not affecting the outputs and conclusions at all. The authors attempt to isolate the impact of the probe by comparing performance on a randomly initialized model (where performance collapses) however it seems counterintuitive that this would have no impact and there are not better ways e.g. probing around the token the entity organically appears in the context (at least for the cases where it does)\n\nQUESTIONS:\nCould authors more clearly explain around why the entity triple being appended does not affect the output? For example even with a randomly initialized LLM the mean F1 is around 20 (one would expect zero for complete independence)\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn RAG,  LLMs are provided with some information in their input context. This information may conflict with the LLM’s internal knowledge. This paper designs a probing method to understand how the contextual and parametric knowledge propagates through the layers and how the conflicts are resolved. What is the role of attention and  MLP layers? The probe consists of a fine-tuned classification head to identify if an entity is relevant to answer the given question. Input to the classifier is the attention, the MLP, or the entire layer residual embedding. If the entity is classified as relevant, then the corresponding source from which the input is taken (attention head, MLP, or the entire layer) is considered relevant for the entity. With counterfactual information in the context, they run the probe for factual and counterfactual entities that are assumed to represent the parametric and contextual knowledge, respectively. The sources (attention, MLP or entire layer) that identify factual entities as relevant are considered important for parametric knowledge, and sources that identify counter-factual entities as relevant are considered important for contextual knowledge.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1.    The paper aims to understand a pertinent issue that often arises when LLMs are used with dynamic external information. It is important to understand how and when the LLMs rely on contextual information and when they fall back on their parametric knowledge. It is also important from the perspective of designing guardrails for LLMs, as the input context may prompt the model not to follow ethics or may contain profanity, and instructions to override LLM’s internal alignment. Understanding how contextual and parametric knowledge reconcile can help us design better guardrails as well.\n\n2.   The design of counterfactual information is neat and clever. Creation of fictional entities ensures that the contextual information is not in the parametric knowledge. \n\n3.   Design of an experiment to validate the probe using the attribution task is also helpful. \n\n\n**Weaknesses**\n\nThe most crucial part of an analysis paper is the experimental design, and I think there are some issues with it. \n\n\n\n**Probe design:**\n\n1.  During the training of the probe, you are not using the counterfactual data. And you assume that the factual entities represent the parametric knowledge (line 219-220).  So are they representing the parametric knowledge or the contextual knowledge? \n\n2. When training the classification head, do you compute loss using the MLP residuals from all the layers and attention residuals from all the heads of all the layers? If not, then can you rely on its output when it is used to quantify information intensity in the intermediate residual streams? \n\n3. In section 4, eqn 4, which residual stream are you using to compute $P_\\theta$?\n\n4. You are training a LoRA for the embedding layer. Why not simply learn embeddings of the two new special tokens? Why impact all other embeddings via LoRA?\n\n5. In Probing-based attribution (section 4), you need to first identify the entities in the sentence.  I.e, you have additional information about the entities in the document. If those are extracted using a stronger model like GPT-4o, then that gives probing an unfair advantage.\nMaybe for a fair comparison, you could provide the extracted entities as part of the input to the baseline prompting method, and also design a prompt with CoT style one-shot demonstration (the way you do in Table 2 for data generation).\n\n**Using the probe for understanding parametric and contextual knowledge (section 5)** \n\n6. The probe is only identifying if an input entity is relevant to answer the given question or not. I am not sure if its usage to identify processing units (attention heads and MLP layers) for counterfactual (contextual) or parametric knowledge is reasonable or not.\n\n7.  Line 219: “we assume its factual descriptions align with LLMs’ parametric knowledge.”: It is a very crucial assumption, and the entire analysis rests on it. Unfortunately, it may not always be true. If it were always true, then LLMs would get close to 100% performance on HotpotQA even when prompted without any documents, as the required knowledge is in their parameters.\nFor this analysis, maybe you should select those questions where the LLM gives the correct answer out of the box without providing any external document in the context. This will ensure that documents are truly aligned with LLM's internal parametric knowledge.\n\n\n**Typos:**\n\n1. “Our analysis reveals three key insights: Our analysis reveals three key insights:” p. 2\n\n2. “individual attention head outputs $∆h_{l}^{attn}(t)$.” p. 3, line 97 - individual attention head output should have $(k)$ as superscript.\n\n3. “Probing-based attribution: We addresses the..” p. 5, line 186\n\n4. “we probing all mentioned entities” p. 5, line 187. \n\n5. “$E − (E_f ∪ E_f )$” : p. 6, line 230: it should be $E_{f} ∪  E_{cf}$\n\n6. “However, how models handle such conflicts remains an area for further research. research.\"” : p. 9, line 325: there is a closing quote without any opening quote. \n\n7. “The outputs of N attention heads are summed to form the attention sublayer output ...” p. 3, line 91:  $i$  should go from 1 to N and not $k$ to N. The superscript on RHS should be $(i)$ and not $(k)$.\n\n8. “We follow the approach of Ye et al. [41] to resolve the limitations of traditional probing outlined in Section 2.2, adapting the probe fθ for entity relevance probing” p. 4, line 128 to 130 -  Looks like an incorrect citation-- did you mean to cite Physics of Language Models (PoLM) -3.1? You have cited PoLM-2.1.\n\nQUESTIONS:\n1. “Here, contextual knowledge types (factual vs. counterfactual) compete for attention within contextual heads:” Line 265: Once you identify the contextual and memory heads in (i) from fig 3(a) and 3(b), do they remain fixed for (iii)?\nFor section 5.2 (iii), only the top 10 contextual and memory heads are important. Maybe a better representation would be a bar chart with 10 bars for $E_f$ and 10 bars for $ E_{cf}$; X axis representing top-10 contextual head ids (in descending order); Y axis representing information intensity. That will give a direct comparison between the contextual information intensity gap between $E_f$ and $E_{cf}$. You can repeat the same for memory heads.\n\n2. “However, the presence of factual knowledge in the context has minimal impact on memory heads” line 267: In the case of $E_f$, memory heads and contextual heads are corroborating with each other. Is there any analysis that demonstrates this?\n\n3. “the average information intensity of top-10 memory heads slightly increases” line 269: Are you identifying top-10 memory heads again, or are they the same as identified in section 5.2 (i). \n\n4. The range of prauc scores in Fig. 4a and 5a is not the same. Does that impact your analysis?\n\n5. In Fig. 4a and 5a, how are you aggregating the information intensity across heads?\n\n6. “Contextual knowledge (E_cf ), in contrast, is immediately detectable in early attention updates,” p. 8, line 284:  Isn’t the range on the y-axis for Fig. 4(a) too low to make this comment?\n\n7. Line 303 to 305:  Could you please elaborate on this?\n\n8. “Asymmetric propagation mechanisms reveal contextual knowledge emerges abruptly via attention while parametric knowledge accumulates gradually through MLP layers;” Line 338: Which observation supports the abrupt emergence of contextual knowledge via attention? Gradual accumulation via MLP layers seems to be common to both parametric and contextual knowledge.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an entity-aware probing framework for investigating how large language models (LLMs) reconcile conflicting parametric and contextual knowledge, particularly in retrieval-augmented generation (RAG) settings. The authors design a lightweight probing mechanism using LoRA to trace entity flow within transformer layers and evaluate its effectiveness in both attribution and knowledge conflict scenarios. They find that attention and MLP layers process parametric and contextual knowledge via distinct mechanisms, and that aligned multi-source knowledge accumulates in a superpositional manner.\n\nSTRENGTHS AND WEAKNESSES:\n### Strength\n\n**Timely and Relevant Topic:** The work addresses an important and relatively underexplored problem—how LLMs reconcile conflicting information from parameters vs. retrieval context.\n\n**Broad and Empirical Analysis:** The paper explores multiple probing dimensions (attention head, MLP, residual stream), across multiple model types and benchmarks, contributing a wealth of experimental insights.\n\n---\n\n### Weakness\n\n**Too Narrow Focus on Relevance Discrimination**: A large portion of the paper is dedicated to building and evaluating the entity relevance probe. However, relevance discrimination alone does not explain *how* reconciliation happens. There is a disconnect between the claim of studying reconciliation mechanisms and the design of experiments, which focus more on *detection* than reconciliation.\n\n**Lack of Grounded Real-World Applications**: The knowledge conflict scenarios are synthetic and overly simplified (e.g., single-entity substitutions). There’s little discussion of how this mechanism translates to more complex real-world retrieval settings with temporal or causal dependencies.\n\n**Weak Link Between Relevance and Parametric Knowledge**: It remains unclear how relevance probing reflects the role of **parametric knowledge**. The entity relevance task seems entirely context-driven (i.e., entities in `D` and their alignment with `q`). The paper does not clarify how parametric knowledge contributes to relevance judgments or whether it is required at all.\n\nQUESTIONS:\n1. **What is the conceptual connection between the relevance discrimination task and knowledge reconciliation?** Relevance seems like a surface property, whereas reconciliation implies *resolution* of conflicting information. Why is the former a good proxy for the latter?\n\n2. **How does parametric knowledge contribute to relevance judgments?** Given that all relevance predictions seem to depend on context (via `D`), is parametric knowledge playing any functional role in this task?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}