Expert Heads: Robust Evidence Identification for Large Language Models
Authors: Qi Wu
Jianfeng Qu
Ximing Li
Zhixu Li
Venue: ICLR 2026 Poster (2026)
Keywords: Large language model
Knowledge Integration
Attention Mechanisms
Primary area: interpretability and explainable AI
TL;DR: Expert Heads: Who Are Responsible for Finding Key Evidence from the Context?

Abstract: Large language models (LLMs) exhibit strong abilities in multi-document reasoning, yet their evidence identification is highly sensitive to input order. We trace this limitation to attention mechanisms, where many heads overemphasize sequence boundaries and neglect central content.
We systematically analyze attention distributions under document permutations and discover a small subset of heads that consistently prioritize task-relevant documents regardless of position. We formalize these as Expert Heads, identified via activation frequency and stability across permutations.
Experiments on LLaMA, Mistral, and Qwen reveal architecture-specific patterns: mid-layer heads in LLaMA and Mistral dominate semantic integration, while deeper-layer heads in Qwen specialize in evidence selection. Moreover, Expert Heads exhibit concentrated focus during understanding and more distributed engagement during generation. Their activation strongly correlates with answer correctness, providing diagnostic signals for hallucination detection.
Leveraging Expert Heads for document voting significantly improves retrieval and ranking on HotpotQA, 2WikiMultiHopQA, and MuSiQue, outperforming dense retrievers and LLM-based ranking with minimal overhead. Ablations confirm that even a small subset achieves robust gains.
Our findings establish Expert Heads as a stable and interpretable mechanism for evidence integration, offering new directions for context pruning, hallucination mitigation, and head-guided training of LLMs.

Published as a conference paper at ICLR 2026
EXPERT HEADS: ROBUST EVIDENCE IDENTIFICATION FOR LARGE LANGUAGE MODELS
Qi Wu
Jianfeng Qu
School Computer Science and Technology
School of Smart Governance
Soochow University
Renmin University of China Suzhou, China
Suzhou, China
wugi 7137 gma com
Suzhou Key Lab of Multi-modal Data
Fusion and Intelligent Healthcare
Suzhou City University
Suzhou, China jfquê edu.er.
Ximing Li
Zhixu Li
College of Computer Science and Technology Jilin University
School of Smart Governance
Renmin University of China Suzhou, China
Jilin, China
RIKEN Center for Advanced Intelligence Project
School of Information
Japan
Renmin University of China
x iiimminn mig g0 1. com
Beijing, China
zhi Chixuls @rruuua. edu.cn
ABSTRACT
Large language models (LLMs) exhibit strong abilities multi-document rea- soning, yet their evidence identification is highly sensitive to input order. We trace this limitation to attention mechanisms, where many heads overemphasize sequence boundaries and neglect central content. We systematically analyze at- tention distributions under document permutations and discover a small subset heads that consistently prioritize task-relevant documents regardless of position. We formalize these as Expert Heads, identified via activation frequency and sta- bility across permutations. Experiments on LLaMA, Mistral, and Qwen reveal architecture-specifi patterns: mid-layer heads in LLaMA and Mistral dominate semantic integration, while deeper-layer heads in Qwen specialize in evidence selection. Moreover, Expert Heads exhibit concentrated focus during understand- ing and more distributed engagement during generation. Their activation strongly correlates with answer correctness, providing diagnostic signals for hallucination detection. Leveraging Expert Heads for document voting significantly improves retrieval and ranking on HotpotQA, 2WikiMultHHopAA, and MuSiQue, outper- forming dense retrievers and LLM-based ranking with minimal overhead. Abla- tions confirm that even small subset achieves robust gains. Our findings establish Expert Heads as a stable and interpretable mechanism for evidence integration, offering new directions for context pruning, hallucination mitigation, and head- guided training of L.L.s.!
INTRODUCTION
Large language models (LLMs)) exhibit strong capabilities in aggregating information across mul- tiple documents (Lewis al., 2020; Wei et al., 2022). Yet, their ability to identify task-relevant evidence is highly sensitive to input order (Liu et al., 2023; Zheng et al., 2023; Pezeshkpour & Hr-
"Corresponding author
!Our code, dataset, and the models used in this work available at httpp::/ // thub. Xuan- Vaan- wwwaaaa////eeeaaadaeaaa

---

Published as a conference paper at ICLR 2026
uschka, 2023). This positional sensitivity largely stems from attention mechanisms: many heads overemphasize sequence boundaries and fail to consistently capture critical content in the middle of the context (Press et al., 2021; Guo et al., 2024; Wu et al., 2025). Addressing this limitation requires a deeper understanding of how attention distributes across documents and which components are truly responsible for robust evidence integration.
To this end, we systematically examine LLM attention under context permutations. As shown in Fig. in each model, small of attention heads consistently attend gold documents regardless of their position. These heads appear to play a unique role in identifying task-critical evidence and exhibit resilience to positional variations. This raises an important question: can we reliably identify such heads and leverage them to enhance both robustness and interpretability?
We build on this observation by introducing framework isolate these heads. We define Activated Heads as those that allocate more attention to all gold documents than to distractors, thereby avoid- ing the pitfalls of naive top-attention selection. We then quantify their stability across permutations and designate the most reliable and focused subset Expert Heads.
Our analysis reveals deeper insights into LLM internal organization. Expert Heads follow distinct layer-wise distributions across architectures: in LLaMA and Mistral, mid-layer heads dominate semantic integration, while Qwen, deeper-layer heads specialize in evidence selection, highlight- ing architecture-specifi alignment strategies. Moreover, under Euery-as-Soure and Response-as- Source attention, more heads participate during answer generation, and their focus is slightly more dispersed, indicating functional shifts across decoding stages.
Through extensive experiments, we show that Expert Heads are closely linked to model perfor- mance. When the model produces correct answers, these heads activate more frequently and focus on task-relevant evidence. Conversely, when the model makes errors, their activation weakens or spreads out, leading insufficient evidence integration and occasional hallucinattinss. Leveraging Expert Heads for document voting consistently improves identification and ranking across multi- ple benchmarks (Yang et al., 2018; Ho et al., 2020; Trivedi et al., 2022), outperforming standard retrieval pipelines and direct LLM-based ranking.
In summary, Expert Heads provide a stable, interpretable, and efficient mechanism for evidence integration, revealing LLM internal reasoning patterns, layer-specifie roles, and practical utility for improving document identification and ranking.
Lager
Layer
Ilead 1d
Hoad 14
Honad Md
Aurrage Mundiie Stare
Anrrage Mirili Scere
Figure 1: Head-wise distribution of average attention scores for gold documents across three LLMs. Scores computed by averaging input permutations where gold documents appear in different positions. Each subplot presents all heads across layers, with color intensity representing the average score assigned to gold documents. The heatmaps highlight small subset of heads that consistently allocate strong attention to gold documents, indicating their potential role as Expert Heads. Distinct color maps are used for clarity across models.
2

---

Published as a conference paper at ICLR 2026
2 ATTENTION HEAD ACTIVATION
To investigate how large language models (LLMs) dynamically allocate attention to task-relevant evidence during context understanding and answer generation, this section analyzes the activation patterns of attention heads under different document permutations.
2.1 PRELIMINARIES
Input Construction and Permutations. Given a query Q, a set of m distractor documents (D1, Dz...., ) and gold documents {G1, G Gn), we construct multiple input per- mutations to systematically evaluate the effect document order on attention head behavior. Each input sequence consists of task instruction, all distractor documents, and the query, with gold doc- uments inserted at different positions among the distractors. Instead of enumerating all possible permutations, only vary the insertion positions of gold documents relative distractors. This results in m + permutations per query, ensuring that every gold document appears all possible positions within the context.
Attention Sources. For any document E {GI," ,u, DI, "D We consider two types of attention sources:
Query-as-ooure: Attention from query tokens Q to document D, reflecting the model's estimation of relevance during context understanding:
AQ-D
(L.))
1
Σ dED
(
(1.A) q.d
(I)
Q
SEQ
Response-ce -ssusseee-- Attention from generated response tokens R to document D, capturing the evidence actually utilized during answer generation:
((A)) R
1
Σ PER dED
Σ
(1.A) r.d
R
Together, these two perspectives provide a comprehensive view: Query-as-Source indicates which content the model deems important, while Desponse-as-Soure reveals how that content is subse- quently used in generation.
Activated Heads. For each permutation π and attention source sre E Q, R. an attention head (L, h) considered activated if attends more strongly to all gold documents than to any distractor:
1,
(L,b) if A A A Vj (1....,, n), Vi e {1....,m. m),
Activated(I, h)
0, otherwise.
This stricter criterion avoids biases from naive top-attention selection, where some heads might appear highly activated due to positional effects rather than true relevance the task.
2.2 ATTENTION ANALYSIS SETUP
Models. We conduct experiments on three widely used instruction-tuned LLMs: LLaMA-3-8B- Instruct (Patterson al., 2022) (32 layers, 32 attention heads per layer), Mistral-7B-Inee In-ntutt v0.3 (AI, 2024) (32 layers, 32 heads per layer), and Qwen-2.5-7B In-strutt (Yang et al., 2024; Team, 2024) (28 layers, heads per layer). These models offer sufficiently rich attention structures to enable fine-grained head-level interpretability.
Dataset. For reproducibility, we randomly sample 5,000 instances from the HotpotQA (Yang al., 2018) train set, each containing two gold and eight distractor documents, yielding nine permutations per query. This results in 45,000 input instances per model, providing a comprehensive basis for analyzing attention behaviors under systematically varied document positions.
Attention Extraction. For each input instance, we extract attention maps from heads across all layers. Both Query-as-Source and Response- Response-as-Soure attention matrices computed using Eqs. and 2. The binary activation status each head is then determined using Eq. 3. This setup enables head-wise analysis of how attention distribution depends document ordering.
3

---

Published as a conference paper at ICLR 2026
2.3 ACTIVATION PATTERNS OF ATTENTION HEADS
Figure shows that gold documents placed boundary positions (beginning or end of the context) trigger larger number of activated heads, but with relatively weaker attention scores. In contrast, gold documents located in the middle elicit fewer activated heads, yet with stronger and more fo- cused attention.
This positional effect is also reflected in performance: as shown in Appendix Fig. 7, multi-hop QA accuracy is highest when gold documents appear at the start or end, but drops when they occur in the middle. This demonstrates that both attention allocation and downstream performance are sensitive to document position.
We further observe notable differences between attention sources. With Query-as-Souree fewer heads activated, but their average scores higher, suggesting that small set of heads suffices to capture task-critical semantics. With Response-as-Soune attention is more dispersed, indicating broader integration of evidence during answer generation.
more heads are engaged, but their
Overall, these findings highlight that LLMs dynamically adjust their attention allocation strategies depending the stage of processing and the position of critical evidence within the context.
Mistral-7B-Juttrrcceer
Qwen-
Neerinn Activation Number
1:00
1300
Responss
Auwruge Attention Score
0.3000
5.2015
6.0000
Permutatioo
Figure 2: Number of activated heads (top) and average attention scores (bottom) across different gold document permutations. Gold documents at boundary positions activate more heads but with weaker attention, whereas middle positions activate fewer heads but with stronger, more focused attention. The x-axis (1-9) denotes the gold document position within the context, from start (1) to end (9).
EXPERT HEAD IDENTIFICATION
Building on the activation patterns observed Sec. 2, this section analyzes the behavior of activated heads to identify those that consistently demonstrate stability and strong focus on gold documents across different document permutations.
3.1 DEFINITION
To quantify the reliability and importance of individual attention heads, we introduce two comple- mentary statistics:
4

---

Published as a conference paper at ICLR 2026
Llama Instruct
Mistral 7B Instruct-v0.3
2.5-7 5.57
Attention from Question
17
24
Layer ID
Attention from Response
12
24
20
12
32"32 beads
32"32 heads
28728 heads
Head
Figure 3: Distribution of Sensitive Heads across layers and head indices. Top: Question-as-Souree: Bottom: Response-as-Souree. Color intensity reflects the number of permutations in which head satisfies the Sensitive Head criteria, with darker shades indicating higher consistency. Expert Heads, which satisfy criteria across all permutations, appear in the darkest color.
Activation Frequency ((,,))))) measures the proportion of samples in which head (1, h) is activated under permutation reflecting its consistency:
((
Activated(I, Ar
(4)
S
SES
Here, S denotes the set all input samples.
Average Attention Score A((:)); measures the mean attention allocated by head h1, to gold doc- uments across activated samples, reflecting its ability to concentrate on task-relevant information:
AU E.es Activated(l, h)) Σj=1 Gr
(1,M).s
(5)
Exes Activated(1, h)
To systematically evaluate activated heads, adopt two thresholds: (1) Activation frequency threshold Ty 0.6, requiring that a head be activated in more than 60% samples under a given permutation. (2) Average attention
percentile TØ 0.9, retaining only the top 10% of activated heads ranked by average attention to gold documents. These thresholds strike a balance between reliability (frequent activation) and focus (strong attention evidence).
Sensitive and Expert Heads. An Activated head classified as a Sensitive Head it satisfies both thresholds for at least one permutation:
SensitiveHeads
(1,h)
En, ((lM) > A > (T
(6)
A Sensitive Head is further promoted to an Expert Head if it consistently meets both thresholds across all permutations:
ExpertHeads - -((..))) Vπ, f
p(l,)) ST A AQA) > PTp (Ap
The full selection procedure is summarized in Alg. in Appendix C.
5

---

Published as a conference paper at ICLR 2026
Owe SiB.trrn
18.33
1/8184
10:10
13000
06.1
10.13
Eacra
14.
61.
Average Aiteaiin Score
Figure 4: Head-wise distribution of Expert Heads and their average attention scores. Horizontal stacked bars show contributions from Question-ass assonnr--- (orange) and Lesponse-as-Soure (red). The concentration of Expert Heads differs across models, reflecting distinct layer-specific roles in semantic integration and evidence selection.
Liana Instract
Mistral 7B Instruct-v0.3
Qwen 3.5-7 Instrodt
Expert Hoad Activatio Frequency
8.55
3.0
033
3.83
8.1
0.20
0.65
k.
3.53
Expert Stedd Average Alfentten Score
60053
3.0045
8.003
51099
8.0051
1.003
Eries
Brigoeoo
Beigoene
Figure 5: Relationship between Expert Head activation and model answer correctness. Green curves denote correct answers, red curves incorrect ones; solid lines represent Duestion-as-Soure dashed lines Response-as-Son Soouree. Top: activation frequency; Bottom: average attention score. Expert Heads are more active and concentrated in correct cases, while weaker dispersed activation cor- relates with errors. The x-axis (1-9) indicates the gold document's position, from beginning (1) to end (9).
3.2 EXPERT HEAD ANALYSIS
Figure 3 illustrates the distribution of Sensitive Heads across model layers and head indices. For (LaMA-3-8BB 3----ttutttttt and layers, underscoring the role of these layers in semantic integration. In contrast, Qwen-2.5-7B- Instruct exhibits a greater concentration of Sensitive Heads deeper layers, suggesting that later layers specialize in evidence selection.
Sensitive Heads are concentrated in middle

---

Published as a conference paper at ICLR 2026
Among these Sensitive Heads, those that remain consistently active across all permutations are iden- tified as Expert Heads. Figure shows that, under Svery-as-Sourte Expert Heads form a smaller but more focused subset of the broader pool engaged under Response-as-Souue ao-Soee This indicates that, while answer generation involves a larger set of heads for evidence integration, Expert Heads retain sharper focus on task-relevant documents.
Figure further demonstrates the relationship between Expert Head activation and model answer correctness. When answers are correct, Expert Heads show both higher activation frequencies and stronger average attention scores on gold documents. Conversely, when answers are incorrect, Ex- pert Head activation is weaker and their attention becomes more dispersed, leading to poorer evi- dence integration and an increased risk of ballucination.
4 EVALUATION OF EXPERT HEADS
We evaluate the practical effectiveness of Expert Heads on document identification and ranking tasks, rather than directly QA accuracy. Conventional QA evaluation relies on metrics such as answer correctness or string matching, which may obscure whether a model actually attends task-relevant evidence. Performance can also be influenced by components beyond attention (e.g., feed-forward layers or decoding strategies), making it difficult to isolate the contribution of evidence integration. In contrast, document identification and ranking provide a more controlled and interpretable setting, allowing us to directly assess whether Expert Heads reliably focus on critical documents.
4.1 EXPERIMENTAL SETTING
Task Definition. Given a query Q, two gold documents (Gi, G2). and eight distractor documents (Di,..., Da). the model must identify and rank the gold documents among all candidates. We eval- uate performance using both identification-oriented metrics (Precision @ 2) and ranking-oriented metries (NDCG@2 and MAP) provide a comprehensive assessment.
Datasets. Experiments are conducted on the test sets of HotpotQA (2,269 samples). 2WikiMul- tiHopQA (Ho al., 2020) (2,471 samples), and MuSiQue (Trivedi et al., 2022) (2,486 samples), providing diverse scenarios for evaluating document identification and ranking.
Document Ranking with Expert Head Voting. We evaluate Expert Heads derived from both 3-Sttt-tt-t Mistral-7B. For each model and each attention source, we first select Expert Heads based on activation frequency Ty and average attention score percentile Tp. fixing the number Expert Heads at five per setting to ensure fairness.
Query-as-Soure and Response-as-Son Source attention across Instruct-v0.3, and Q.wen-2.5-7B-Innttuue
Given query and candidate documents, each Expert Head independently produces a ranking based on its attention scores from the query to each candidate. Final rankings are then aggregated through voting scheme actoss the five Expert Heads. Details of the thresholds and voting procedure provided in Appendix and C.
Baselines. We compare Expert Heads against widely used retrieval models: BM25 (Robertson et al., 2009), DPR (Karpukhin et al., 2020), Contriever (Izacard et al., 2021), MiniLM (Wang et al., 2020), GTR (Ni et al., 2021), ColBERTv2 (Santhanam al., 2021), BGE (Xiao al., 2024), Qwen3 (Zhang et al., 2025) and LLM Rank, which refers to direct document ranking by model generation.
4.2 MAIN RESULTS
As shown Table 1, experimental results demonstrate that Expert Heads substantially surpass all baseline retrieval models across all datasets, indicating their effectiveness in document ranking tasks. Importantly, they provide a principled and interpretable mechanism for document ranking. By fo- cusing small, consistently activated subset of heads, they highlight the model components most responsible for evidence integration, offering insights into reasoning patterns without relying on black-box full-model outputs.
7

---

Published as a conference paper at ICLR 2026
Table 1: Precision @2 (P@2), NDCG@2, and MAP performance three datasets. Each base model includes three variants: Expert Heads (Question), Expert Heads (Response), and LLM Rank. Baselines include BM25, DPR, Contriever, MiniLM, GTR, ColBERTv2, BGE, and Qwen3, Best results are shown in bold; second- are underlined.
HotpotQA
2WikiMuliHHoPAA
MuSiQue
Method
P@2
NDCG@2
MAP
P@2
NDCG@2
MAP
P@2
NDCG@2
MAP
BM25
57.41
50.23
60.58
52.77
45.64
56.80
49.30
43.27
54,66
DPR
60.14
52.97
63.17
61.27
54.85
64.79
58.61
50.82
61.50
Contriever
61.50
55.19
64.90
59.51
52.15
62.07
62.37
54.04
64.01
MiniLM
71.04
64.04
71.82
73.07
65.72
73.43
65.47
56.75
66.37
GTR
64.26
57.32
66.65
70.03
61.89
70.26
64.42
55.94
65.78
ColBERTv2
64.63
57.31
66.58
68.70
61.21
69.89
60.42
51.80
62.25
BGE
75.23
69.45
76.37
77.12
71.06
77.76
70.25
62.89
71.26
Qwen3
68.55
61.77
70.21
73.27
65.84
73.68
66.55
57.54
67.05
ALaMA-3-SB-Instuctt
Expert Heads (Q)
88.23
89.97
94.03
73.47
77.84
85.71
82.18
84.78
90.08
Expert Heads (R)
90.72
91.98
95.08
77.30
81.58
87.83
83.57
86.05
90.95
LLM Rank
66.31
70.06
78.22
76.49
79.65
86.60
69.63
73.70
80.93
Mistral-7B-Be 7eer--Brnnnt--llee
Expert Heads (Q)
88.08
89.73
93.60
70.72
75.27
83.21
80.03
82.98
88.76
Expert Heads (R)
88.5 89
90.50
94.13
73.27
77.62
84.83
80.53
83.40
88.85
LLM Rank
36.45
39.97
55.03
45.14
49.82
62.22
39.72
44.31
57.54
Expert Heads (Q)
86.93
88.69
93.19
74.69
78.47
85.72
81.30
83.85
89.44
Expert Heads (R)
88.45
90.02
93.87
77.94
81.88
88.29
82.70
85.02
90.11
LLM Rank
77.81
78.70
86.11
76.57
80.89
87.15
76.65
79.68
88.97
4.3
ABLATION STUDY
Layer-wise Ablation Study
Threshold Sensitivity Analysis
92.0
Pradeiin5
NDC
Number Bleads
Nur Depam
M
91.0
Score
00.0
Prechlazgg2
89.0
88.0
1645 bu5)
6.0) tea)
8.00
det 8.80)
e.t ext e.t))
8.8)
10:70 8.60)
10:70 1075 8,90
8,wu
0.78 8.8
(e7s, 0.70
Layer
(ers,
1975 18.78
ceuu
reaa
Threshold (a
to
Figure 6: Ablation study results. Left: Layer-wise ablation, where all heads within one layer are treated as Expert Heads. Results show that middle layers contribute most, while lower layers play limited role and final layers degrade performance. Right: Threshold sensitivity for Response- as-Source Expert Heads across varying activation thresholds. Performance improves under stricter criteria, even though the number of selected heads decreases sharply, indicating that tighter thresh- olds yield more specialized and effective subsets.
As shown in Fig. 6, further conduct ablation experiments analyze layer contributions and threshold sensitivity on LLaMA-3-8B. Instrrctt using the HotpotQA test set (2,269 samples), with Precision @2 and NDCG@2 @2 2 as evaluation metrics.
Layer-wise Ablation. We evaluate the contribution of different layers by treating all heads from a given layer expert heads. Results reveal that middle layers contribute the most to performance,
8

---

Published as a conference paper at ICLR 2026
confirming their critical role in semantic integration and evidence selection, while lower layers play a limited role. Interestingly, using the final layer for document identification and ranking leads to significant drop in performance, possibly because the model is already focused preparing to generate the next token.
Threshold Sensitivity. We further analyze the effect of varying activation thresholds Response- as-Source expert heads. As thresholds become stricter, the number of selected expert heads de- creases substantially. Interestingly, instead of degrading, performance gradually improves under stricter criteria. This indicates that higher thresholds effectively filter out less informative heads, leaving smaller yet more specialized subset that contributes more strongly to evidence identifica- tion and ranking. Thus, Expert Heads not only remain robust to threshold selection but can even benefit from tighter activation constraints.
5 DISCUSSIONS
Expert Heads provide natural signal for context reranking, pruning, and interpretability. Because they consistently highlight task-critical evidence, their attention patterns can guide context reranking by moving highly attended documents closer the end of the context, or context pruning by discarding distractors that receive negligible attention. This head-guided mechanism reduces computational cost in long-context settings while preserving reasoning quality. In addition, Expert Heads make the model's reasoning process more interpretable: instead of black-box ranking scores, users can directly observe which documents are prioritized by the most reliable heads, yielding transparent explanations for model outputs.
Expert Heads offer a principled approach hallucination mitigation and factuality detection. Our experiments show that correct answers are associated with stronger and more concentrated Ex- pert Head activations, while incorrect answers display weaker or dispersed activation. This suggests that activation strength and focus can be used as diagnostic signal to detect hallucinations in real time. Responses accompanied by weak or diffuse Expert Head patterns can be flagged as unreliable, enabling factuality- aware decoding strategies that enforce tighter grounding of answers to verifiable evidence, particularly in multi-document reasoning scenarios.
Expert Heads also serve guidance for model distillation and reinforcement learning. In knowledge distillation, the attention maps of Expert Heads a teacher model can be transferred to student model, ensuring that evidence-centric reasoning is preserved during compression. In rein- forcement learning or Reinforcement Learning from Human Feedback (RLHF), Expert Head acti- vation can be incorporated into the reward function, encouraging models to prioritize task-relevant documents during reasoning. This bridges interpretability and training, providing a way to align model optimization not only with human preferences but also with evidence fidelity, ultimately leading to more reliable and controllable language models.
RELATED WORK
Retrieval and Ranking Methods. Retrieval has evolved from sparse term-based approaches to dense neural representations (Johnson et al., 2019; Khattab Zaharia, 2020; Khattab al., 2021a;;; Santhanam et al., 2022) and embeddings derived from large language models (Zhang et al., 2023; Xiao et al., 2023). Modern encoders prioritize efficiency and generalization, while recent open- source models (Li et al., 2023a; Zhang et al., 2024) enhance multilingual capabilities, instruction- following, and long-context handling. In parallel, LLM-based ranking methods directly leverage language models as rerankers (Zhuang et al., 2023; 2024a;b; Drozdov et al., 2023), often achieving superior semantic matching and reasoning, albeit with higher computational cost.
Functional Specialization and Interpretability of Attention Heads. Early work (Jain & Wallace, 2019; Clark et al., 2019) questioned whether raw attention weights provide faithful explanations, showing they can be misleading without controlled analysis. Later studies(Kovaleva et al., 2019; Vig, 2019) demonstrated that attention becomes informative when paired with interventions e.g., head ablation (Michel et al., 2019; Voita al., 2019), attention-wett Smith, 2019; Wiegreffe & Pinter. 2019), causal probing (Pruthi et al., 2019), and counterfactual edits (Abnar & Zuidema, 2020)-which can identify heads that causally influence model behav-
manipulation (Serrano &

---

Published as a conference paper at ICLR 2026
ior. More recent research (Li et al., 2023b; Kumar et al., 2024; Zheng et al., 2024) highlights functional specialization of specific heads for example, induction heads (Ren et al., 2024), name- mover heads (Tigges al., 2023; Garefa-Carrasor et al., 2024). These findings support treating certain attention heads as mechanistic primitives that drive model behavior, enabling targeted inter- pretability and modification. Our work extends this line of inquiry by identifying Expert Heads as stable mechanisms for robust evidence integration.
7 CONCLUSION
In this work, we identify Expert Heads, a small subset of attention heads that consistently focus on task-critical evidence across permutations, models, and attention sources. Our findings reveal layer-specific functional specialization LLMs: mid-layer heads in LLaMA and Mistral domi- nate semantic integration, while deeper-layer heads in Qwen specialize in evidence selection. We demonstrate that Expert Heads provide interpretable signals for evidence integration, enabling im- provements in document identification, ranking, and hallucination detection. Beyond performance gains, they establish a transparent and stable mechanism that links internal model reasoning with external behavior. Overall, Expert Heads serve both interpretability tool and a practical mech- anism to enhance efficiency, reliability, and controllability in large language models.
ACKNOWLEDGMENTS
This work is supported by the National Natural Science Foundation of China (No. 62576236), the Key-Area Research and Development Program of Guangdong Province (202480101050005) the Natural Science Foundation of Jiangsu Province (BK20251823), Beijing Natural Science Foun- dation (4262049), Suzhou Key Laboratory of Artificial Intelligence and Social Governance Tech- nologies (SZS2023007), Smart Social Governance Technology and Innovative Application Platform (YZCXPT2023 the Innovation System of the Integration between Industry and Education for Smart Governance (CJRH2024101), the Leadership Talent Program (Science and Education) of SIP, the Fundamental Research Funds for the Central Universities, JLU, the Priority Academic Program Development of Jiangsu Higher Education Institutions, the open research fund of Suzhou Key Lab of Multi-modal Data Fusion and Intelligent Healthcare and the Science and Technology Development Plan of Suzhou City (Science and Technology Research Program (Medical and Health Innovation)) (Grant No. SYW2025065).
REFERENCES
Samira Abnar and Willem Zuidema. Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928, 2020.
Mistral AI. Mistral-7b-inssmcc-00..3 2024. URL httppss//hhggggggfcc.cooo////
mastral-700 Instruct--t.....
Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. What does bert look at? an analysis of bert's attention. arXiv preprint arXiv:1906.
04341, 47X4v:1906.04341, 2019.
Andrew Drozdov, Honglei Zhuang, Zhuyun Dai, Zhen Qin, Razieh Rahimi, Xuanhui Wang, Dana Alon, Mohit Iyyer, Andrew McCallum, Donald Metzler, al. Parade: Passage ranking using demonstrations with large language models. arXiv preprint 0rXiv:2310.14408, 2023.
Jorge Garefa-Carrasco, Alejandro Maté, and Juan Carlos Trujillo. How does gpt-2 predict acronyms? extracting and understanding circuit via mechanistic interpretability. In International Confer- ence on Artificial Intelligence and Statistics, pp. 3322-3330. PMLR, 2024.
Qi Guo, Leiyu Wang. Yidong Wang, Wei Ye. and Shikun Zhang. What makes good order of examples in in-context learning. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 14892-1444 14904, 2024.
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi-hop hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint (XXXx:0011.0066. 2020.
10

---

Published as a conference paper at ICLR 2026
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXhv:2112.09118, 2021.
Sarthak Jain and Byron Wallace. Attention is not explanation. arXiv preprint arXiv:1902.101. 2019.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. JEEE Transactions Big Data, 7(3):335-547, 2019.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In EMNLP (1), pp. 6769-6781, 2020.
Omar Khattab and Matei Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 48, 2020.
Omar Khattab, Christopher Potts, and Matei Zaharia. Baleen: Robust multi-hop reasoning at scale via condensed retrieval. Advances Neural Information Processing Systems, 34:27670-27682,
2021a.
Omar Khattab, Christopher Potts, and Matei Zaharia. Relevance-guided supervision for openqa with colbert. Transactions of the association for computational linguistics, 9:929-944, 202 lb.
Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky, Revealing the dark secrets 2019.
of bert. arXiv preprint
Sreejan Kumar, Theodore Sumers, Takateru Yamakoshi. Ariel Goldstein, Uri Hasson, Kenneth A Norman, Thomas L Griffiths, Robert D Hawkins, and Samuel A Nastase. Shared functional spe- cialization in transformer-baat based language models and the human brain. Nature communications, 15(1):5523, 2024.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. aerrival--augmentdd gener- ation for knowledge-intenste nlp tasks. Advances in neural information processing systems, 33: 9459-9474, 2020.
Chaofan Li, Zheng Liu, Shitao Xiao, and Yingxia Shao. Making large language models better foundation for dense retrieval. arXiv preprint arXiv:2312.15500 2023a.
Chong Li, Shaonan Wang, Yunhao Zhang, Jiajun Zhang, and Chengqing Zong. Interpreting and ex- ploiting functional specialization in multi-head attention under multi-task learning. arXiv preprint arXiv:2310. 18.33,, 2023b.
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. arXiv preprint
arXiv:2307.03172. 2023.
Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? Advances in neural information processing systems, 32, 2019.
Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Y Zhao, Yi Luan, Keith B Hall, Ming- Wei Chang, et al. Large dual encoders generalizable retrievers. 2021.
arXiv preprint
David Patterson, Joseph Gonzalez, Urs Hölzle, Quoe Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R So, Maud Texier, and Jeff Dean. The carbon footprint of machine learning training will plateau, then shrink. Computer, 55(7):18-28, 2022.
Pouya Pezeshkpour and Estevam Hruschka. Large language models sensitivity to the order of op- tions in multiple-choice questions, arXiv preprint arXiv:2308. /483, 2023.
11

---

Published as a conference paper at ICLR 2026
Ofir Press, Noah A Smith, and Mike Lewis. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprint inXiv22108.124000 2021.
Danish Pruthi, Mansi Gupta, Bhuwan Dhingra, Graham Neubig, and Zachary C Lipton. Learning to deceive with attention-based explanations. arXiv preprint arXiv:1909.07913, 2019.
Jie Ren, Qipeng Guo, Hang Yan, Dongrui Liu, Quanshi Zhang, Xipeng Qiu, and Dahua Lin. Identifying semantic induction heads to understand in-context learning. arXiv preprint arXiv:2402. 13055, 2024.
Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and be- yond. Foundations and Trends@ in Information Retrieval, 3(4):333-33)) 2009.
Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. Colbertv2: Effective and efficient retrieval via lightweight late interaction. arXiv preprint arXiv:2112.01488, 2021.
Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. Plaid: an efficient en- gine for late interaction retrieval. In Proceedings of the 31st ACM International Conference Information & Knowledge Munagement, pp. 1747-1756, 2022.
Sofia Serrano and Noah Smith. Is attention interpretable? arXiv preprint arXiv:1906.03731, 2019.
Qwen Team. Qwen2.5: A party of foundation models, September 2024. URLhttps: //tpt:pp:////////// github. hts/bllog/wwe22..
Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda. Linear representations of sentiment in large language models. arXiv preprint 1001::23/0.55144 2023.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-554, 2022.
Jesse Vig. A multiscale visualization of attention in the transformer model. MXXi::9906007/111 2019.
arXiv preprint
Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. Analyzing multi-head self-attention: Specialized heads heavy lifting, the rest can pruned. arXiv preprint arXiv:1905.09413, 2019.
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self- attention distillation for task-agnostic compression of pre-trained transformers. Advances neu- ral information processing systems, 33:5776-5788 2020.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 0124824-248311 2022.
Sarah Wiegreffe and Yuval Pinter.
Attention is not not explanation.
arXiv preprint
arXiv: 04626, 2019.
Xinyi Wu, Yifei Wang, Stefanie Jegelka, and Ali Jadbabaie. On the emergence of position bias in transformers. arXiv preprint arXhv:250.001951, 2025.
Shitao Xiao, Zheng Liu, Peitian Zhang, and Xingrun Xing. Lm-cocktail: Resilient tuning of lan- guage models via model merging. arXiv preprint arXiv:2311.13534, 2023.
Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian- Yun Nie. C-pack: Packed resources for general chinese embeddings. In Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, pp. 641-649, 2024.
12

---

Published as a conference paper at ICLR 2026
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang. Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang. Peng Wang. Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang. Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yugiong Liu, Zeyu Cui. Zhenru Zhang, and Zhihao Fan. Qwen2 technical report. arXiv preprint arXiv:2407.100 1007716611,. 2024.
Zhilin Yang, Peng Qi. Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint 18kiv:8809009000 2018.
Peitian Zhang, Shitao Xiao, Zheng Liu, Zhicheng Dou, and Jian-Yun Nie. Retrieve anything to augment large language models. arXiv preprint arXiv:2310.07554, 2023.
Peitian Zhang. Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou. Soaring from 4k to 400k: Extending Ilm's context with activation beacon. arXiv preprint arXiv:2401.03460 2 (3):5, 2024.
Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05) 17,, 2025.
Chujie Zheng, Hao Zhou, Fandong Meng. Jie Zhou, and Minlie Huang. Large language models are not robust multiple choice selectors. arXiv preprint wrXiv:230.038882 2023.
Zifan Zheng, Yezhaohui Wang, Yuxin Huang, Shichao Song, Mingehuan Yang, Bo Tang. Feiyu Xiong, and Zhiyu Li. Attention heads of large language models: A survey, arXiv preprint arXiv:2409.0377 03752, 2024.
Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Bendersky. Beyond yes and no: Improving zero-shot IIm rankers via scoring Ane-grained relevance labels. arXiv preprint arXiv:2310. 11l.2330..1122,, 2023.
Shengyao Zhuang, Xueguang Ma, Bevan Koopman, Jimmy Lin, and Guido Zuccon. Promptreps: Prompting large language models to generate dense and sparse representations for zero-shot doc- ument retrieval. arXiv preprint 240.:2404.18222. 2024a.
Shengyao Zhuang. Honglei Zhuang, Bevan Koopman, and Guido Zuccon. A setwise approach for effective and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International ACM SIGIR Conference Research and Development in Information Retrieval, pp. 38 47, 2024b.
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
In preparing this paper, GPT-S-mini was employed exclusively for grammar checking and minor language polishing. No scientific content, experimental design, or interpretation of results was generated by any LLM. All ideas, analyses, and conclusions presented are solely those of the authors.
B
IMPLEMENTATION DETAILS
80
78
Accuracy (%)
Mistral-7B-Insse
76
Llama-3-8 3-8B--88--- Instruct 3-su-nnruut
-wee-2.55-B-IImslle
74
72
70
1
2
4
5
6
7
9
Permutation
Figure 7: Effect of gold document position multi-hop QA accuracy. The x-axis (1-9) denotes the gold document's location, from start (1) to end (9). Accuracy peaks when gold documents are placed the beginning end of the context, and drops when they appear in the middle- demonstraannn strong positional sensitivity in model performance.
Table 2: Thresholds and selected Expert Heads for each model and attention source.
Model
Attention Source
(Ty, Tp)
Expert Heads (Layer Id, Head Id)
LLaMA-3-8B-Instuue
Question
(0.65, 0.9) (13,4), (14,13), (14,20), (14,22), (16.1)
LLaMA-3-8B -nstrtt
Response
(0.8, 0.9)
(13,18), (14,13), (16,1), (16,8), (17,24)
Dissrall77--Instuutt-v...
Question
(0.6, 0.9)
(15,1), (15,27), (16,12), (16,22), (18.3)
Mistral-7B Instruct-vO.3
Mistral-7B-Instruet--000
Response
(0.75, 0.9)
(15,1), (15,27), (16,12), (18,3), (19,9)
wwen-2.5-7B-IInstrue
Question
(0.6, 0.9)
(16,0), (19,17), (19,20), (19,22), (21.5)
Oween2.5-7B-Innttrus
Response
(0.8, 0.93) (19.15), (19,22), (21.5), (22.1), (22.7)
Experiments were conducted 8 NVIDIA V100 GPUs with 32GB memory each. For model responses, the maximum input length was set to 8,192 tokens and the maximum generation length to 4,096 tokens with greedy decoding. All retrieval models used maximum input length of 512 tokens, while LLM Rank was restricted to a maximum generation length of 30 tokens with greedy decoding. To ensure reproducibility, all random seeds for model inference, data sampling, and dataset splitting were fixed at 42.
For Expert Head performance evaluation, in order maintain five heads per category, the activation frequency and average attention score percentile thresholds were set as shown in Table 2.
For threshold sensitivity analysis, the number of selected Expert Heads ranged from 1 to 30.
14

---

Published as a conference paper at ICLR 2026
C ALGORITHMIC DETAILS
Algorithm Expert Head Selection
Require: Query Q. Gold Documents (Gi...., ) Distractor Documents (D..... Dm }, Atten-
tion Tensors A (1.h), Activation Frequency Threshold Ty, Attention Percentile Threshold Tp
Ensure: Set of Expert Heads Hespert
1: Initialize empty dictionary to store activated heads and their attention scores for all samples and permutations
2: for each input sample S do
3:
for each input permutation æ of documents do for each attention head (1, h) do for each source src {Q, R} do
4:
5:
6:
Compute attention A are-G, and A Dr using Eq. or Eq. 2
(L.),,
Determine activation using Eq. end for Store activated heads and their attention scores for later aggregation 10: end for
7:
8:
9:
end for
12:
end for
for each permutation x and activated head (1, h) do
14:
for each source src E {Q, R) do
15:
Compute Activation Frequency f(l, h)) using Eq. Compute Average Attention A(I, h)" using Eq. 5
16:
17:
end for
18: end for
19: Identify Sensitive Heads Hsensitive using Eq.
20: Identify Expert Heads Hexpent using Eq. 7
21:
return Hespen
Algorithm 2 Expert Head Voting for Document Identification and Ranking
Require: Query Q, Candidate Documents (Ci,...,Cy, C,]. Expert Heads Haxpens Attention Tensors
(L.A), Top-K voting threshold K
Ensure: Ranked list of documents {C())..... ..., C(k)
1: Initialize vote counts: V(C,) <- 0 for all i E [1,
2: for each Expert Head (1, h) E Hexpen do
3: Compute attention scores (C,) + AQ_c, a
4: Rank documents in descending order of (C,))(C))
5: Select top-K documents from this ranking
6: for each document C in Kop-K< do
7:
(CC:) + V(C.)
This head casts one vote for C;
8: end for
9:
end for
10: Rank all documents in descending order of V(C,)
11: return Ranked document list
For clarity and reproducibility, we provide the algorithmic details the proposed framework this appendix. Algorithm summarizes the procedure for identifying Expert Heads from raw attention patterns, including input permutations, activation statistics, and consistency checks across different document orderings. Algorithm 2 then illustrates how the selected Expert Heads leveraged for document identification and ranking. forming the basis of our evaluation experiments in Sec. 4. 'Toeether, these algorithms provide complete framework for Expert Head selection and evaluation.
15

---

Published as a conference paper at ICLR 2026
D PROMPT TEMPLATES
This appendix presents the prompt templates used in the experiments for multi-hop question answer- ing and document ranking. Templates are tailored each model family: LLaMA-3-8B-Inse Mistral-7B-Imsttuut-00.333 and Qwen-2.5-7B-Insstrucl
Multi-hop QA Templates: Specify structured document inputs followed by query. The assistant must generate an answer strictly grounded in the provided documents.
Document Ranking Templates: Present 10 candidate documents and a query. The assistant in- structed to output only a ranked list of document indices, ordered by relevance, without explana- tions.
These standardized templates ensure consistency across models and tasks, and facilitate repro- ducibility of our experimental results.
Table 3: Multi-hop Question Answering prompt template for -LLaMA---BB-Isstrre
Multi-hop Question Answering for LLaMA- --A-AA---BB--nttttee
Input: < ('begin.of.test | tt id >system< end www.heeadeiree Read the following documents and answer the question based ONLY the provided information. </ect_id| >< start _eadeerrerrr..re id >user< end header_id|>
Title:
Content:
Title:
Content:
Question: (question) <lect_id] >< baweeadddrddddassstttttssss.sss (eddeeeeee
Output:
(response)
Table 4: Multi-hop Question Answering prompt template for Disstal-7B-Instruct-e
Multi-hop Question Answering for Mistral- Histral-7B-Instruee--e In-truttuut--
Input: <a> 

Read the following documents and answer the question based ONLY the provided information.
Title:
Content:
Title:
Content:
Question: {question)


Output: (response}
16

---

Published as a conference paper at ICLR 2026
Table 5: Multi-hop Question Answering prompt template for Swwen225-7B-Innstree
Multi-hop Question Answering for Qwen-2.5n5 2-5-7B-Instruc
Input: < <lim_start >>system
Read the following documents and the question based ONLY the provided information. <lim end|>
< <lim.start/
| Puser
Title: ...
Content:
Title:
Content:
Question:
{question] <lim_end]
<|im start peart/>assi bassssssseeee stant Output: (response}
Table 6: Document ranking prompt template for LLaMA-3-8B-Inssmuct
Document ranking for LLaMA-3-8B-Instnn
Input: beeeginnoottteece >>< www.stttteeeeereeeeeeee >user< header.dd You expert document ranker. You are given 10 documents question. Rank all documents from most relevant to least relevant answering the question.
Document [1]:
Title:
Content:
Document [2]:
Title:
Content:
Question: {question)
Return ONLY a comma-scparan list of document numbers, sorted from most relevant to least rele- vant. Do NOT include any other text or explanation. Example output: 7.3,9,1,510.662...8 www.t.headdr....... passist istant cnd.header.id]
<|cot_id] ><
Output: (ranking list)
17

---

Published as a conference paper at ICLR 2026
Table 7: Document ranking prompt template for Cestsall-B--Istructt-vii
Document ranking for Mistral-7B- -B--Inttrutt--v)
Input: (INST]
You are an expert document ranker. You are given 10 documents a question. Rank documents most relevant to least relevant in answering the question.
Document [1]:
Title:
Content:
Document [2]:
Title:
Content:
Question: {question)
Return ONLY a comma-separated list of document numbers, sorted from most relevant to least rele- vant. Do NOT include any other text or explanation. Example output: 1,3,,,,,511,,,2,4,88 

Output: (ranking list}
Table Document ranking prompt template for Qwwen22.5-7BIInstruu
Document ranking for Qwen-2.5-7B. 2.5-7B-Instruet
Input: <| im_start J>system You are Qwen, created by Alibaba Cloud. You are helpful assistant. <I <lim.end|>
i start >user
You are an expert document ranker. You are given 10 documents and a question. Rank all documents from most relevant to least relevant in answering the question.
Document [1]: Title:
Content: ....
Document [2]:
Title:
Content: ...
Question: {question}
Return ONLY comma-separated list of document numbers, sorted from most relevant to least rele- vant. Do NOT include any other text explanation. Example output: 1.3.9.1.511.6.2...88
<lim_end|>
<lim.start bassistant Output: (ranking list)
18


Reviews:
[{"review_id": "mKJyZ9XXiW", "reviewer": "Reviewer_zSoH", "summary": "This paper investigates attention distributions in LLMs under document permutations in multi-hop question answering setting. They conduct an analysis of attention patterns with LLama3, Mistral and Qwen2.5 models, and then identify a small set of attention heads, named expert heads, that consistently attend to task-relevant documents across all permutations based on activation frequency and average attention score. Their experiments suggest that the layer-wise distribution of expert heads varies across different model architectures, and the activation of expert heads is correlated with answer correctness. Additionally, they leverage expert heads to rank candidate documents on HotpotQA, 2WikiMultiHopQA and MuSiQue and show that expert heads are effective for document retrieval and ranking.", "questions": "- Could you explain more details about the evaluation metrics used in the ranking experiments? ie. what do you measure by Precision@2, NDCG@2 and MAP?\n- In the left figure in Figure 6, what do the two dashed lines indicate? Why does the performance remain the same across different layers?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- They conduct extensive analysis of attention patterns under document permutations with three model families, providing interesting insights into the cause of position bias and attention behavior across architectures.\n- Their method to identify expert heads is clearly defined and reasonable.\n- The paper is clearly written and easy to follow. Figures are well presented and intuitive to understand.", "weaknesses": "Overall the paper is well-structured. It would be clearer to add some clarification about the experiment setup and evaluation metrics used in the document ranking experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EANuWuJy1t", "reviewer": "Reviewer_1F1i", "summary": "This paper presents an empirical investigation into the functional specialization of \"expert heads\" within a deep learning architecture (presumably a multi-headed attention mechanism). The authors identify emergent specialization patterns and analyze how these specialized components contribute to the model's performance. The core contribution lies in the detailed empirical observations demonstrating this specialization and the subsequent high performance achieved on specific benchmarks, primarily those focused on precision.", "questions": "- Could the authors provide a holistic evaluation by including Recall and F1-scores for all experiments? Furthermore, presenting the Precision-Recall curve (PR-AUC) would significantly strengthen the evaluation.\n- I would also recommend the authors include experiments on tasks specifically designed to test high-recall performance. How do the expert heads perform when the loss function or evaluation criteria are optimized for recall rather than precision?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper provides interesting and potentially valuable empirical data regarding how specialization emerges and functions. These observations offer a foundation for future theoretical work or architectural improvements.\n- The analysis clearly demonstrates the effectiveness of the expert heads in scenarios where minimizing false positives is the priority.", "weaknesses": "- The most significant weakness of this work is the narrow scope of the evaluation, which relies almost exclusively on precision. This provides an incomplete picture of the model's efficacy. In many critical applications (_e.g._, anomaly detection), recall (sensitivity) is equally or more important than precision. By neglecting recall or balanced metrics (e.g., F1-score, PR-AUC), the paper fails to demonstrate the overall utility of the expert heads.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZadldwJR5q", "reviewer": "Reviewer_D66K", "summary": "This paper studies the order sensitivity of evidence identification in multi-document reasoning with LLMs. The authors observe that a small subset of attention heads consistently focuses on gold documents across input permutations and define these as **Expert Heads**. They report architecture-specific layer patterns (e.g., LLaMA/Mistral peaking in mid layers, Qwen in deeper layers) and claim strong P@2/NDCG/MAP on HotpotQA/2Wiki/MuSiQue by using Expert Heads to identify and rank gold documents among 10 candidates.", "questions": "- Do the heads selected as Expert Heads change depending on factors such as document length or topic differences within the dataset?\n- How does accuracy vary with the size of the training data used for selecting Expert Heads?\n- If the candidate documents vary widely in length, does the proposed algorithm remain robust?\n- You evaluate 7B and 8B models; what differences do you observe for larger models?\n- Your experiments target retrieval rather than QA; if applying the proposal to QA, would it be used in a two-stage pipeline—retrieving supporting documents -> generating the answer with the LLM?\n- How do you explain that Expert Heads (R) generally outperform Expert Heads (Q)?\n- At line 361 the HotpotQA test size is reported as 2,269, whereas Figure 6 says “HotpotQA test (2,323 samples).” Which number is correct?\n- If the training data changes, do the selected Expert Heads change substantially?\n\n(Other Comment)\n- Line 108: Consider clarifying whether $D$ denotes ``document tokens''; the current notation is ambiguous.  \n- Line 108: The symbols for “document $D$” and “distractor document $D$” collide; please disambiguate.  \n- Line 111: Define $l$ (layer) and $h$ (head) explicitly.  \n- Line 138: Improve clarity by illustrating gold–distractor relatedness in HotpotQA (e.g., with a concrete example or similarity scores).  \n- Line 245: Add a definition for $P_{\\tau_p}(\\cdot)$.  \n- Figure 6 (right): The x-axis is discrete; a line plot may mislead—consider bars or points.  \n- Numerals: Add thousands separators to 4+ digit numbers (e.g., 2,269).  \n- References: Prefer final proceedings over arXiv where available (e.g., Ren et al., 2024 to ACL Findings). Unify citation formats for Mistral and Qwen.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Simple method validated across multiple architectures, with clear analysis and visualization.\n- Consistent improvements are shown across multiple families (e.g., LLaMA, Mistral, Qwen).  \n- Consistent improvements on HotpotQA/2Wiki/MuSiQue under the proposed evaluation.\n- Implementation details (code and prompts) aid reproducibility.", "weaknesses": "- The selection of Expert Heads relies heavily on supervision from a HotpotQA training subset. This raises a concern that the selected heads may overfit to HotpotQA-specific structure, style, and difficulty, potentially overstating cross-domain generalization. Although you report results on 2Wiki/MuSiQue, the paper lacks evidence that the head-selection procedure itself can be reproduced stably outside HotpotQA.\n- The authors fix the condition of having two adjacent gold documents, which is a strong assumption. Does the method still work when gold positions are more flexible (e.g., non-adjacent)?\n- The number of candidate documents is fixed at 10; ablations that vary this number are missing.\n- The hyperparameters are effectively retrofitted to match the target outcome (the number of selected heads), which weakens claims about generalization.\n- Please state the data sizes and the precise train/validation/test splits explicitly.\n- While the proposed method substantially outperforms the baselines, I worry that the baselines may be too weak.\n- Ranking based on Response→Document attention is only available after full-context generation, making it ill-suited for pre-filtering (cost reduction). Despite this, Expert(R) often yields the best results in the main tables; this is useful for offline analysis but offers limited benefit in practical deployments.\n- Several notation/formatting issues remain; please see the notes below and incorporate any that are helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ls3efqEBMu", "reviewer": "Reviewer_PouH", "summary": "This paper investigates how attention mechanisms in Large Language Models handle evidence identification across multiple documents, with particular focus on their sensitivity to input order. The authors identify a small subset of attention heads termed \"Expert Heads\" that consistently focus on task-relevant documents regardless of their position in the context. They demonstrate that these Expert Heads show different layer-wise distributions and can be leveraged to improve document retrieval and ranking tasks.", "questions": "How do Expert Heads identified on HotpotQA transfer to other datasets? Can Expert Heads be identified once and reused, or must they be recomputed for each new task/dataset?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The identification of architecture-specific patterns (mid-layer specialization in LLaMA/Mistral vs. deeper-layer specialization in Qwen) provides valuable insights into model behavior.", "weaknesses": "1. Threshold Selection Appears Ad-Hoc: While Table 2 shows different thresholds for different models/sources, the paper doesn't provide principled guidance for selecting these thresholds in new settings. The choice of τ_f = 0.6 and τ_p = 0.9 seems empirically driven without theoretical justification.\n2. Only using the HotpotQA training set may not be sufficient to establish robust patterns, and using multiple randomly sampled datasets may be better.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)