Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders
Authors: Guangzhi Xiong
Zhenghao He
Bohan Liu
Sanchit Sinha
Aidong Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Sparse Autoencoder
Model Interpretability
Retreival-augmented Generation
LLM Hallucination
RAG Faithfulness
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Retrieval-Augmented Generation (RAG) improves the factuality of large language models (LLMs) by grounding outputs in retrieved evidence, but faithfulness failures, where generations contradict or extend beyond the provided sources, remain a critical challenge. Existing hallucination detection methods for RAG often rely either on large-scale detector training, which requires substantial annotated data, or on querying external LLM judges, which leads to high inference costs. Although some approaches attempt to leverage internal representations of LLMs for hallucination detection, their accuracy remains limited. Motivated by recent advances in mechanistic interpretability, we employ sparse autoencoders (SAEs) to disentangle internal activations, successfully identifying features that are specifically triggered during RAG hallucinations. Building on a systematic pipeline of information-based feature selection and additive feature modeling, we introduce RAGLens, a lightweight hallucination detector that accurately flags unfaithful RAG outputs using LLM internal representations. RAGLens not only achieves superior detection performance compared to existing methods, but also provides interpretable rationales for its decisions, enabling effective post-hoc mitigation of unfaithful RAG. Finally, we justify our design choices and reveal new insights into the distribution of hallucination-related signals within LLMs. The code is available at https://github.com/Teddy-XiongGZ/RAGLens.

Published as a conference paper at ICLR 2026
TOWARD FAITHFUL RETRIEVAL-AUGMENTER GEN- ERATION WITH SPARSE AUTOENCODERS
Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Aidong Zhang Department of Computer Science University of Virginia {guangzhi, zhenghao, bohan, sanchit,, aignn edu
ABSTRACT
Retrieval-Augmentedd Generation (RAG) improves the factuality of large language models (LLMs) by grounding outputs in retrieved evidence, but faithfulness fail- ures, where generations contradict or extend beyond the provided sources, remain a critical challenge. Existing hallucination detection methods for RAG often rely either on large-scale detector training, which requires substantial annotated data, querying external LLM judges, which leads to high inference costs. Al- though some approaches attempt to leverage internal representations of LLMs for hallucination detection, their accuracy remains limited. Motivated by recent ad- vances in mechanistic interpretability, we employ sparse autoencoders (SAEs) to disentangle internal activations, successfully identifying features that are specif- ically triggered during RAG hallucinations. Building on systematic pipeline of information-basee feature selection and additive feature modeling, we intro- duce RAGLens, a lightweight hallucination detector that accurately flags unfaith- ful RAG outputs using LLM internal representations. RAGLens not only achieves superior detection performance compared to existing methods, but also provides interpretable rationales for its decisions, enabling effective post-hoe mitigation of unfaithful RAG. Finally, justify our design choices and reveal new insights into the distribution of ballucination signals within LLMs. The code is available htttps:: homdd///d
wtt.c/oo///T//a Xi ongGZ /RAGLens
INTRODUCTION
Generation (RAG) has emerged as a promising paradigm for improving the factuality of large language models (LLMs) (Lewis al. 2020). By conditioning generation on passages retrieved from external corpora, RAG systems aim to ground model outputs in verifiable evidence. However. in practice, grounding does not eliminate unfaithfulness (Magesh et al.) 2025
Gao et al.) 2023). Models may still contradict the retrieved content, introduce unsupported details, extrapolate beyond what the evidence justifies (Maynez al., 2020 Rahman et al. (0266). These faithfulness failures, commonly referred to as hallucinations in the RAG setting, undermine user trust and limit deployment domains where faithfulness source information critical (Huang ct al. 2025 Zakka et al. 2024).
Various approaches have been proposed to address this challenge. One direction fine-tune spe- cialized detectors to distinguish faithful from unfaithful generations (Bao et al., 2024 Tang et al.) 2024a). While this method provides direct supervision, its effectiveness is often constrained by the need for large amounts of high-quality annotated training data. Another line of work employs LLMs as judges, where an auxiliary L.M prompted to assess faithfulness given the retrieved passages and generated answers (Zheng et al.) [023] Li et al.) 2024). However, these approaches struggle to detect hallucinations produced by the same model and introduce significant computational overhead when relying on large-scale external LLMs. More recently, researchers have explored the use of the LLM's internal representations, such as hidden states or attention scores, to capture hallucina- tions directly (Han al.) 2024 Sun al.) 2025 Zhou et al. (025). While these methods show promise, the extraction of reliable hallucination related signals remains challenging, and detection performance is often insufficient for practical deployment.

---

Published as a conference paper at ICLR 2026
Mcanwhile, recent advances in mechanistic interpretability have shown that sparse autoencoders (SAEs) can disentangle specific, semantically meaningful features from the hidden states of LLMs (Huben al., 2024). By enforcing sparsity, SAEs identify features that correspond to concrete concepts, as evidenced by their consistent activation across similar cases (Bricken et al. 2023 Shu et al.; 2025). This property, known monosemanticity, provides a transparent link between inter- nal activations and model behaviors. While recent work has explored the use of SAEs to detect signals associated with generic LLM hallucinations (Ferrando al. 2025 Suresh al. 2025 Ab- daljalil et al. 2025| Tillman & Mossing 2025 Xinn et al.; hallucinations in RAG settings pose unique challenges due the complex interplay between retrieved evidence and generated content. It remains unclear whether SAE features can effectively capture these dynamics, In this work, we directly investigate whether SAEs can identify interpretable features that are predictive of halluci- nations in RAG, enabling both accurate detection and deeper insight into failure cases.
We present RAGLens, lightweight SAE-based detector that flags unfaithful RAG outputs by lever- aging LLM internal activations through systematic pipeline of information-based feature selection and additive feature modeling. Experimental results show that RAGLens identifies features highly relevant to RAG hallucinations and achieves superior detection performance compared to existing methods when evaluated on the same LLM. We further demonstrate the interpretability of RA- GLens, enabled by its additive model structure and transparent input features, and highlight how these interpretations facilitate effective post-hoc mitigation of unfaithfulness. Finally, our analy- ses examine the design choices underlying RAGLens, revealing that mid-layer SAE features with high mutual information about the labels are most informative for detecting RAG hallucinations, and that generalized additive models (GAMs) are particularly well-suited for mapping SAE features to hallucination predictions. To our knowledge, RAGLens is the first approach to systematically demonstrate the effectiveness of SAE features for detecting hallucinations in RAG, and compre- hensively investigate design principles for building accurate and interpretable detectors. Here is summary of our contributions:
We demonstrate that SAEs capture nuanced features specifically activated during RAG hal- lucinations, establishing a strong foundation for detecting RAG unfaithfulness from LLM internal representations.
Building on these SAE features, we introduce RAGLens, a lightweight hallucination de- tector that outperforms existing methods in detection accuracy while providing transparent and interpretable feedback to aid in hallucination mitigation.
Through detailed analyses, we justify the key design choices in RAGLens and offer new insights into the distribution of sallucination-relatee signals within LLMs.
2 RELATED WORK
Augmentte Generation (RAG) integrates retrieval modules with large language mod- els (LLMs) to ground responses in external knowledge sources (Lewis et al. 2020 Guu et al.) 2020). This design has improved factual accuracy in tasks such as open-domain question answer- ing, knowledge-intensive dialogue, and domain-specific search (Shuster et al. 2021 Siriwardhana jet al. 2023 Xiong et al.) 2024 Oche et al., 20255 Wei et al. (025). However, RAG systems remain vulnerable to faithfulness errors: even when relevant passages are retrieved, models may contradict
evidence, invent unsupported details, or extrapolate beyond the source (Niu et al. 2024 Sun et al.)
2025). These failures have been studied under terms such as hallucination, ungrounded generation, or source inconsistency, and are increasingly recognized as central obstacle to deploying RAG in
real-world applications (Zhang et al.) 2025 Gao al., 2023 Elchafei & Abbu-ilkheir 2025).
To address this challenge, a growing body of work has developed detectors to judge whether a generated response is faithful to the retrieved evidence (Manakul et al.) 2023 Sriramanan et al. 2024). Early approaches focused on fine-tuning specialized detectors, which can be effective but require large amounts of high-quality training data, particularly when adapting large models (Bao) et al. 2024| Tang et al. 2024a). With the rise of foundation models, researchers have begun using LLMs as evaluators, prompting an auxiliary LLM to compare generated answers against source passages and determine whether hallucination occurs (Zheng et al. 2023 Bui al. 2024). However, this often necessitates the use of larger LLMs, leading to high computational costs, sensitivity to prompt design (Wang al. 2024), and explanations that may be plausible but do not faithfully reflect
2

---

Published as a conference paper at ICLR 2026
the underlying decision process (Turpin et al. 2023). More recent studies have explored leveraging LLM internal representations for hallucination detection, but challenges such as the polysemanticity of neurons and the opacity of hidden states have limited the extraction of high-quality features,
resulting in insufficient detection performance (Elchafei & Abu-Fllhheir 2025
Sun et al. 2025).
Recent research has shown that sparse autoencoders (SAEs) can expose semantically meaningful features within the hidden representations of LLMs (Huben et al.) 2024 Shu et al. 2025). By
constraining activations through sparsity-inducing bottleneck, SAES learn dictionaries of features that often correspond to human-interpretable concepts such as syntactic roles, entities, or factual
attributes (Bricken al. 2023 Gujral et al. 2025). This capability has facilitated analysis of model internals, enhanced interpretability, and even enabled targeted control of generative behavior (Shi et al., 2025p. The interpretability of SAF-derived features makes them attractive for tasks where transparency is critical, such as hallucination detection.
3. RAGLENS: FAITHFUL
GENERATION VIA
SPARSE REPRESENTATION PROBING
3.1 PROBLEM SETTING
Following prior work (Niu et al. 2024 Song et al. 2024; Sun et al. 2025), we use "RAG" to denote any context-conditione) generation process in which an LLM produces an answer based on both user query/instruction and provided context. The faithfulness detection task is to determine whether the generated answer is consistent with the given context. In such tasks, each annotated instance consists of: (1)
query instruction q; (2) set of retrieved passages C; (3) an answer sequence V1. generated by an LLM, where T is the sequence length and V1st denotes the prefix up position t; and (4) binary label É E {0, 1} indicating whether the answer contains hallucination relative to C. We assume access to frozen LLM & and a corresponding SAE with encoder & trained on the hidden states in the L-th layer of 4. We denote by (-))) the mapping that returns layer-L L hidden states. Given a generation V1.7, we obtain
h & ( C),
t= I
(I)
and transform these via the SAE encoder into sparse features
E(h))
2, E R
where K is the size of the dictionary and only small number of features are active at each position.
Our goal to examine whether the SAE features contain signals that help detect hallucinations related to RAG. Section 3.2 presents our detection method, and Section 3.3 shows how the results
support explanation and mitigation. An overview shown in Figure] !
3.2 HALLLUCINATION DETECTION
Instance-level Feature Summary. Because target labels are instance- summarize token- level activations into an instance representation via channel-wise max pooling:
max Zt,k1 I5:T
A I,..., K,
(3)
where 2t,k is the k-th element of 2, g and collect (Z Ex) R
Information-based Feature Selection. We quantify the information of each pooled feature E. (k I,... K) about the hallucination label l using mutual information (MT):
/
p(ãx, E) dēg.
I(Zx: E)
Σ
p(ž,, loga p(žx) p(e)
(4)
(E())))
We rank features by MI and select the top K' dimensions:
S arg max
Σ
T(Es: 8),
(5)
|S|=
KES
3

---

Published as a conference paper at ICLR 2026
Hallucination Detection
LLM 4
[Twcn]
[oells)
Question
How do twins happen?
layor
[..] the split takes place after the ninth day after fertilization [..]
Context
Encodar
of SAE
max pooling
Mi-bosad seleccion
h1
h2
...
hot
Output
between eighth and ninth day of fertilization [..]
token-level features
instance-level featuras
features
prediction
Hallucination Explanation
Hallucination Mitigation
Local Explanation
Question + Context
Oulput
[..] place between the eighth and ninth day
S output
User
LLM
RAGLens
Global Explanation
Example: [..] could be high 600
Updated
Prediction Only
Output
instancs-lovel Feedback
factual inserts (e.g. numbers, dates/times) that the model fabricates
Undated
Prodiction Explanation
ouaput
Culpul
Token-level Feedbock
Figure 1: Overview of RAGLens for detecting, explaining, and mitigating hallucinations in retrieval- augmented generation using interpretable sparse features.
yielding 2 KK the subvector of z restricted to indices S. our experiments, MI is estimated with a binning- banning-based method applied to the pooled activations. While we do not explicitly utilize the hidden states of the retrieved passages C, our encoding of the model output y in Equation | is conditioned on C. This allows the SAE features to implicitly capture interactions between the gen- erated answer and the retrieved content. Empirical results in Appendix Show that the selected SAE features encode knowledge relevant to the retrieved passages, and their activations are dynamically influenced by counterfactual interventions on C.
Transparent Prediction with Generalized Additive Models. After selecting informative SAE features, we model the instance label from the pooled representation using generalized additive model (GAM) (Lou et al. 2012 Caruana al., 2015
Hastie 2017):
k'
g 2 30 + Σ f2
where g is the link function (c.g., logit for binary classification) and each univariate shape function f; learned using bagged gradient boosting (Nori al. 2019). By selecting only K" features (K" < K), the fitted GAM serves as a lightweight detector, requiring the encoding of only a small subset of SAE features. Our analysis in Section 5 further validates that GAM is well-suited for modeling hallucination signals from SAE features, outperforming more complex predictors such as MLP (Popescu et al. 2009) and XGBoost (Chen & Guestrin 2016).
Justification of Max Pooling on Sparse Activations. Beyond the practical advantage of stor- age efficiency, we provide a theoretical justification for using max pooling: in the sparse activa- tion regime, it can help distinguish ballucination-related features from random noise by amplifying signals associated with relevant targets. To facilitate the analysis, for any fixed feature index k suppressing the respective notation for clarity, model the token-level activation 2 0 as condi- tionally independent across tokens given the label E E {0, 1}, with a rare activation mechanism:
f0, with probability 1 - Pep Vz, with probability Pe. where the "active-value" random variable V, has a distribution F supported on (0, oo) that is inde- pendent of & and i...d. across tokens. Let 5 = max 2 and = Pr(( 1).
1 t= 1= T
Theorem (Max pooling in the sparse-activation regime). IfT x p < 1 with p (p: + Po), then
T(2: E) = TT(1 - T) T (Ap)2
{(TP)))
Ap P1 Po
(8)
2l
P
4

---

Published as a conference paper at ICLR 2026
where iff P1 * Po- The leading dependence is linear in T and quadratic in Ap.
Proof sketch. Let A 1 0}. Independence across tokens implies Pr(A=1 E) qe
1 1- (1- (1-pe) pe)". so [(A; = h + (1 -1-110 >)qu)= - [[Th((n) + -))((g)))) For pt < 1, qe = Tpp
and second-order expansion of h gives
()) x(1 - σ) ?
+ +0((T)))
(9)
2ln
p
Since A is a deterministic function of 2. I(A:E) S [(2: In the single-hit regime (Tý < 1), the extra information in beyond A occurs only on rare multi-activation events, contributing ((Tp))
Combining yields the claim. A full proof is in Appendix G
3.3 HALLUCINATION EXPLANATION AND MITIGATION
Since the detection results are computed from SAE features within an additive modeling frame- work, our approach naturally supports interpretability at both the local (instance-specific) and global (instance-invariant) levels, which can then be leveraged to improve faithful RAG generation.
Local Explanations via Sparse Feature Attribution. Because our GAM operates additively on a small set of selected SAE features, each prediction can be decomposed into a sum of feature contributions. For any given example, we can attribute the hallucination prediction to the specific sparse features that are most strongly activated. By aligning these activations with token positions, we obtain token-level feedback that highlights which parts the generation are likely ungrounded relative the retrieved passages. This fine-grained attribution enables users to directly pinpoint fabricated factual inserts such as numbers, dates, or named entities.
Global Explanations via Intrinsic Model Interpretability. Beyond instance-specift attributions, RAGLens also provides global, instance-invarian explanations. With the dictionary learning prop- erty of SAEs, each SAE feature corresponds to semantically coherent concept that be sum- marized as human-undee uannaannnnndrstdaablle knowledge. Furthermore, our use of GAMs in RAGLens enables visualization of the learned shape function for each feature, offering stable explanation of the map- ping from feature magnitude to predicted hallucination risk. Practitioners can therefore inspect how changes in a given feature systematically increase or decrease the prediction, enabling consistent feature-level auditing.
Mitigation through Multi-level Feedback. The interpretability of our framework enables expla- nation signals to be directly incorporated into mitigation strategies at inference time. Specifically, detection results can be provided to I.IMS as instance-level warnings, prompting the model re- consider and revise potentially hallucinated content. By aligning sparse activations with text spans identified by local explanations, we can further highlight problematic tokens that may require edit- ing, thereby guiding the model to refine its output.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
We conduct experiments on two RAG hallucination benchmarks with Llama2 backbones: RAGTruth NNu et al., (2024) and Dolly (Accurate Context) (Hu et al. 2024). both of which include human annotations for outputs generated by Llama2-7B/33D To further evaluate generalizability across
architectures, we also test our method on Llama3.2- 1B, Llama3.1-8B. and Owen3-0.6BB) 0.6B/4B using two additional datasets, AggreFact (Tang et al.] 2023) and TofuEval (Tang et al., (024b), which contain hallucinations produced by variety of L.LMs. For consistency with prior work (Sun et al.)
2025 Tamber et al. 2025), we report balanced accuracy (Acc) and macro F, (F1).
We compare RAGLens with representative detectors based on (i) prompt engineering (e.g., Friel & Sanyal 2023), (ii) model uncertainty (e.g., Manakul et al. 2023), or (iii) LLM internal representa- tions (e.g.. Sun et al., 2025). We also include a fine-tuning baseline, "Llama2- 1aamaa2-13BLL.. 13B()))", following existing work (Sun et al. 2025). For a fair comparison with methods that analyze internal signals during LLM generation,
evaluate all detectors Llama2-7B 7B and Llama2-13B using the cor- responding samples generated by these models in RAGTruth and Dolly. Table lists the specific baselines, with details in Appendix A Implementation details are provided in Appendix B
5

---

Published as a conference paper at ICLR 2026
4.2 PERFORMANCE ON RAG HALLUCINATION DETECTION
the performance of RAGLens on RAGTruth and Dolly, with comparisons to pre- vious methods using the same backbones. As the table shows, the SAE features of both Llama2-7B and Llama2-1333 13BB contain sufficient information to accurately detect hallucinations, achieving AUC scores greater than 80% on both datasets. More importantly, RAGL consistently outperforms existing baselines, demonstrating its effectiveness in identifying and leveraging internal knowledge for RAG hallucination detection. These results highlight the strong potential of SAEs to serve as powerful detectors by using knowledge already embedded within LLMs to identify hallucinations.
Table 1:
Performance comparison of different hallucination detection methods on RAGTruth and Dolly. The best results are highlighted in bold.
RAGTruth (Llama2-7B)
Dolly (Llama2-7B)
RAGTruth (Llama2-138)
Dolly (Llama2-13B)
Method
AUC
Acc
Fg
AUC
Acc
Fi
AUC
Fy
ALC
Fi
Prompt
0.6700
0.6720
0.6200
0,5476
0.7300
0.6899
0.6700
0,5823
Llama2-13R(E 13R(U)))
D.6350
0.6572
0.6043
0.7044
0.5545
0.6664
LwMLM
0.6940
0.7365
0.6550
0.7702
0.5956
0.7684
0,6800
0,7000
FAcTScore
0,5428
0.5333
0.6719
0.4813
0.5354
0.6849
0.5294
0.4646
0.5954
FactCC
0.4976
0,5022
0.4589
0.6169
0,5758
0,5882
0.4753
0,4800
0.4121
0.6496
0.6162
0.5250
ChainPoll
0.6738
0.6841
0.7006
0.6593
0.6200
0.7414
0.7378
0.7070
0.6004
RAGAS
0,7290
0.6822
0.6667
0.66488
0.6560
0.6392
0.7541
0,7080
0.6987
0.6412
0.6480
0.5306
0.6510
0.6821
0.6658
0.6264
0.6800
0.6567
0.7073
0.7063
0.3944
RelCheck
0.6912
0.6467
0.6736
0.6494
0.6100
0.5412
0.7897
0.7200
0.7823
0.6621
0.5700
0.3944
PCTrue)
0.7093
0.5648
0.6549
0.6191
0.5095
0.8496
0.6266
SellCheckGPP
0.5844
0.4642
0,5300
0,3188
0,5844
0.4642
0.5300
0,3188
LN-Entropy
U.5912
0.6850
0.5656
0.5912
0.6074
0.5656
0.6261
Energy
0,5619
0.5088
0.6657
0.6074
0.5656
0.6261
0,5619
0,5088
0.6657
0.6074
0.5656
0.6261
Focus
0.6233
0.6522
0.6783
0.6567
Perplexity
0,5091
0.5333
0.6749
0.6825
0.6363
0.7097
0.5091
0.5333
0.6749
0.6825
0.6363
0,7097
EigenScore
0.6045
0.5422
0.6682
0.6786
0.7241
0.6640
0,5267
0.7214
0,7200
SEP
0,7143
0.6187
0.7048
0.6067
0.6060
0.7023
0.8098
0.7288
0.7799
0.7093
0.6800
0.6923
SAPLMA
0.7107
0.5155
0.6502
0.6500
0.6084
0.6653
U.6605
ITI
0.6714
0.5667
0.6496
0.5494
0,5800
0.6281
0.8501
0.6177
0.6850
0.6530
0.5583
0.6712
ReDeFP
0,7458
0.6822
0.7190
0.7949
0.7373
0,7833
0,8244
0.7889
0.7587
0.8420
0.7070
0,7603
RAGLens (Ours)
0.8413
0.7576
0,7636
0,8764
0.7778
0,8070
0.8964
0,8333
0,8148
0,8568
0,7576
0.7895
4.3 CROSS-MODEL APPLICATION
While SAE features are not transferable across different LLMs, the RAGLens detector trained on one LLM can be flexibly applied to text outputs generated by other LLMs. To examine whether LLMs contain sufficient internal knowledge to detect hallucinations produced by other LLMs, we con- duct cross- moos- evaluations by training series of RAGLens detectors based SAEs of multiple open-source LLMs, and test their performance on RAG outputs from various LLMs in RAGTruth, AggreFact, and TofuEval. Specifically, we prompt each LLM to assess the faithfulness of the RAG output in a chain-of- th-iuuof- (Co'T) style (Wei et al. 2022), )) using the template from Luo et al.) (2023). and compare these results to those of the same model's SAE-based detector via RAGLens
Figure 2 shows the performance of all evaluated LLMs across different datasets. The SAE-based detector consistently outperforms each model's own CoT-style self-judgments. Larger LLMs ex- hibit stronger internal knowledge, achieving higher detection performance with SAE-based detec- tors. While earlier generation models such as Llama2-7B and Llama2-13B have lower CoT judg- ments on certain datasets, their SAF-based detectors perform comparably to newer models of similar size (e.g.. Llama3. 1-8B). Meanwhile, although Qwen3-0.66 achieves competitive CoT performance on AggreFact and ToluEval, its SAE- detector lags behind those of larger LLMs, suggesting that the informativeness of internal knowledge correlates more with model size than with training pipeline. Overall, these results indicate that models "know more than they tell" and that SAEs can reveal latent faithfulness signals that are not consistently captured by CoT reasoning.
4.4 GENERALIZATION ACROSS DOMAINS
Beyond cross-mode applications, we further assess whether the internal signals captured by RA- GLens generalize across domains. Specifically, we train the RAGLens predictor one domain and evaluate its performance on other domains. Table Sublee]repee reports the generalization performance (AUC) of RAGLens across different datasets and tasks.

---

Published as a conference paper at ICLR 2026
RAGTruth
AggreFact
TofuEval
1.0
CoT
0.8
SAE
Acc
0.6
0.4
0.2
0.0
0.6
0.4
0.2
0.0
Ulama2-88
Llama2-138
Llama3. -2-18
Llama3. 1mma3.1-86 1-8-
Owen3- 68
Owen3 448
Llama2-78 2-8
LlamaZ-198
Llama 2-198
Llama3. Llama3. Llama3.2-18 2-11
L--m 1-88
Owen3-0 Owen3-0.6B
Owen3-49
Llama2-78
Llama2-138
Llama3 22118
Llama3. 1-88
Qwen3 68
Owen3-48
Figure 2: Comparison of LLM CoT-style self-judgment versus internal knowledge revealed by SAE features for hallucination detection across datasets.
The left part of Table 2 presents cross-dataset results, showing that RAGLens generalizability de- pends on the diversity of its training data. For example, a detector trained on RAGTruth significantly outperforms the CoT baseline on AggreFact and TofuEval without retraining. In contrast, predictors trained on AggreFact and TofuEval, while still outperforming CoT most cases, do not generalize well those trained RAGTruth. This can be attributed dataset differences: RAGTruth ers multiple subtasks, whereas AggreFact and TofuEval focus on single tasks. The results indicate that RAGLens trained with diverse samples is robust and generalizable to domain shifts.
Table 2: Generalization across datasets and subtasks. Left: RAGLens trained on a single dataset and evaluated on others. Right: RAGLens trained on one RAGTruth subtask and evaluated on other subtasks. "None" indicates zero-shot performance with CoT prompting. All scores are AUROC.
Train Train)Test
RAGTruth
AggreFact
TofuEval
Train' Test
Summary
QA
Data2txt
Llama2-7B
Llama2-7B
None
0.4842
0.5741
0.5562
None
0.4924
0.4845
0,4949
RAGTruth
0,8806
0,8019
0.7637
Summary
0.8191
0,8253
0.6443
AggreFact
0.5330
0.8330
0.6123
QA
0.7081
0.8835
0.6609
ToluEval
0.7747
0.6161
0.7846
Data2Ixt
0.5386
0.6616
0.8454
Llama2-133
Llama2- 13B
None
0.4959
0.5285
0.5583
None
0.5196
0.5088
0.4765
RAGTruth
0,8674
0.7831
0.7319
Summary
0.7539
0.8330
0.6627
AggreFact
0.4669
0,8285
0.6239
QA Data2txt 0.5653
0.6619
0.8769
0.6669
TofuBBal 0.7342
0.5727
0.7883
0.7373
0.8491
Generalization across task types is shown in the right part of Table RAGLens, when trained on one task, consistently transfers its learned knowledge to other tasks and outperforms the CoT baseline. Among the three subtasks, the predictor trained summarization (Summary) exhibits the strongest generalizability, surpassing those trained on question answering (QA) or data-to-text generation (Data2txt). Additionally, knowledge transfer between Summary and QA is more effective than between Data2txt and the other tasks. These results suggest that RAGLens can capture common signals shared across different RAG tasks, while also revealing the presence of task-specific signals that limit generalization.
4.5 INTERPRETABILITY RAGLENS
In addition strong performance, RAGLens provides interpretability for the hallucination detection process. Since RAGLens uses SAE features that are disentangled and correspond to specific con- cepts, analyze which features are most indicative of hallucinations and what they represent.

---

Published as a conference paper at ICLR 2026
With the deployment of the GAM classifier, RAGLens can transparently illustrate how each feature contributes to the final prediction through learned shape functions. Table presents two represen- tative SAE features from Llama3.1-88 that most predictive of hallucinations, the GAM classifier trained on RAGTruth. For each feature, we show two example activations from RAGTruth outputs, accompanied by semantic explanation distilled by GPT-5 from 24 activation cases. We also visualize the learned shape function for each feature, where the y-axis (feature effect on hallucination prediction) is zero-centered, illustrating how the feature value (x-axis) influences the likelihood of hallucination.
identified by
Table 3: Interpretation of two SAE features in Llama3. 1-BBB. Each row shows the feature ID, brief explanation of its semantic role, and example text spans where the feature is activated. Top activated tokens in each example are shown bold, while the hallucinated tokens are highlighted in red. The shape functions learned by the GAM are visualized in the rightmost column, illustrating each feature's ' impact on hallucination prediction.
ID
Explanation
Examples
Shape Plot
Context: no mention of age Output: [...] at the age of 34 [...]
unsupported numeric/time
22790
specifics
Context: no mention of release schedule
Output: [...] be released in August [...]
Context: [...]] could be arrested on the spot [...]
grounded, high- high-salience
Output: [...] could be arrested on the spot [...]
17721
Context: [...] software can be licensed as a [...]
tokens
Output: [...] software can be licensed as a [...]
As shown in the table, Llama3. 1-8B contains various types of features that help detect hallucinations from different perspectives. For example, feature 22790 indicates potential hallucinations that related to unsupported numeric/time specifics. Its corresponding shape function (learned by GAM) exhibits a monotonic increase hallucination likelihood as activation strength rises. RAGLens also uncovers SAE features that are negatively correlated with hallucinations, such as feature 17721, which captures signals associated with grounded, high-salience tokens. This interpretability not only clarifies how RAGLens works, but also provides insights into the internal knowledge of LLMs. Additional examples from other LLMs are provided in Appendix H and Appendix ! presents case studies using counterfactual perturbations to validate that these features are specifically sensitive to hallucination patterns unique to RAG scenarios.
4.6 MITIGATION OF UNFAITHFULNESS WITH RAGLENS
Leveraging its detection and interpretation capabilities, RAGLens can provide post-hoc feedback to LLMs mitigate hallucinations. We evaluate this by applying Llama2-7B-basee RAGLens 450 outputs from RAGTruth, and prompting the same model to revise its original output using RAGLens feedback. Specifically, we compare the effectiveness of instance-level feed- back (detection results only) and token-level feedback (which includes additional explanations from RAGLens interpretation) for hallucination mitigation.
Table reports the resulting hallucination rates (lower is better) as judged by multiple automatic L.M judges. In addition, two human annotators evaluated a subset of 45 outputs, with an inter- annotator agreement of 78.3%. Although hallucination rates vary among different types of annota- tors, the results consistently show that both types of RAGLens feedback effectively reduce halluci- nations in the revised output. Notably, the more nuanced token-level feedback enabled by RAGLens interpretability leads to further reductions compared to instance-level feedback. We further applied trained RAGLens detector (Llama3. 1-BB based) to all 450 examples and found that instance-level feedback converted 29 outputs from hallucination to non- aon-hllucination, while token-level feedback achieved 36 such conversions, confirming the advantage of token-level feedback.
8

---

Published as a conference paper at ICLR 2026
Table 4: Mitigation of Llama2-7B hallucinations using SAE-based internal knowledge. Hallucina- tion rates (lower better) are reported original outputs and after applying instance- and token- level feedback, as judged by Llama3.3-70B, GPT-4o, GPT-o3, and human annotators.
Llama3.3-70B
GPT-4o
GPT-o3
Human
Original
43.78%
37.78%
64.44%
71.11%
Instance-level Feedback
42.22%
36.44%
60. 40.44%
62.22%
Token-level Feedback
39.11 %
34.22%
58.88%
55.56
DISCUSSIONS
Beyond the main results on hallucination detection, interpretation, and mitigation using SAE fea- tures, we further analyze several key SAE-specific design choices in RAGLens, including the se- lection of the LLM layer, the feature extractor, the number of selected features, and the predictor architecture.
LLM Layer Selection. We first vary the layer from which SAE features are extracted, covering the full depth of several LLMs (Llama3,2-1BB Llama3-8B, Qwen3-0.6BB and Qwen3-4B). Figure presents the heatmaps LLM performance on various subtasks in RAGTruth IRAGTruth-Summary, RAGTruth- -A,, and RAGTruth RAGTruth-Dataatxx) Data2txt), where layer depths are normalized for direct comparison. The results show that the performance trend in layers is consistent among LLMs but varies by task. In the Summary and QA tasks of RAGTruth, the performance peaks around the middle layers, whereas the Data2ixt task exhibits a comparatively flat performance pattern across layers.
116
IAGTruu S.Smmaryy
1.9
RAATTruth-A
2.00
6m
817
375
6.76
6.25
364
0.0
0.00
2.2
0.29
6.66
400 F1
Arr
11
F1
Por
0:00
Limma2
A77
Uamada
ACY 21
60"
Figure 3: Layer-wise analysis of Llama3.2- Llama3-88, Qwen3-0.6B, and Qwen3-4B on sub- tasks in RAGTruth IRAGTruth-Summary RAGTruth-QA, and RAGTTuth-DData2xxx)
Feature Extractor Comparison. We further compare different feature extractors, specifically SAE and Transcoder (Dunefsky et al., 2024), as well as pre-activation versus post-activation signals (i.e., features extracted before or after applying the activation function). Table [5] shows that pre- activation features consistently outperform post-activation features for both extractors. Transcoder and SAE achieve similar accuracy, indicating no clear advantage for either architecture. These re- sults suggest that while both extractors are effective, the choice of activation point is more critical, with pre-activations retaining more informative signals about RAG hallucinations.
Table 5: Comparison of SAE and Transcoder feature extractors, using and post-activation sig-
nals, for hallucination detection with Llama3.2-1B across three datasets.
RAGTruth
AggreFact
TofuEval
Architecture
Activation
Acc
Fi
Acc
Acc
F;
Pre-activation
0.7810
0.7892
0.7308
0.7388
0.6865
0.6876
SAE
Post-activation
0.7606
0.7700
0.6939
0.7091
0.5637
0.5642
Pre-activation
0.7778
0.7830
0.7468
0.7586
0.6652
0.6666
Transcoder
Post-activation
0.7594
0.7684
0.7373
0.7525
0.6195
0.6178
We also examine how varying the number of selected features (K') affects performance, using mutual information (MI) ranking to identify the most informative fea-
Analysis of Feature Count.

---

Published as a conference paper at ICLR 2026
tures. Figure 4 shows the performance of Llama2-7B-bun -B-ddsed RAGLens K' decreases from 1024 to 1, comparing MI-based selection to random selection (Rand.) starting with the same set of fea- tures. As expected, performance drops fewer features used, but the decline is much more gradual with MI-based selection, demonstrating that MI effectively prioritizes informative features for hallucination detection. Differences in trends between datasets further highlight the varying complexity of hallucination detection tasks.
RAGTruth
RAGTruth
0.7 TofuEval Acc
TofuEval
Aggrefact Acc
AggreFact
0.60
0.80
0.7
0.7
Score
0.75
0.75
0.6
0.6
0.6
0.70
0.70
0.5
0.65
0.65
0.5
0.5
Rand.
10
log (K))
log (K')
log2
log2
log2
log2(k) (K))
Figure 4: Effect of varying the number selected features (K') hallucination detection perfor- mance, comparing mutual information (MI) ranking and random selection (Rand.).
Predictor Ablation. Lastly, we compare logistic regression (LR), generalized additive model (GAM), multilayer perceptron (MLP), and eXtreme Gradient Boosting (XGBoost) as predictors for hallucination detection using the selected SAE features. Figure [ that GAM consistently outperforms LR and also surpasses more complex models such as MLP and XGBoost, despite its additive structure. This suggests that while the effect of individual features on the output is of- ten nonlinear, the overall contribution of SAE features can be effectively captured an additive manner. Consequently, GAM is particularly well-suited for leveraging SAE features, offering both strong performance and interpretability.
RAGTruth
AggreFact
TofuEval
Gwen3 11400F1
Qwe13 48 Acx
Qwen3
Qwne3
48 ACC
48-Acc
Owen3
Qward
Ownr3
Owan3
Swesa 45--1
Ape
SB Fl asg
Lau775- Acc 6BFINIT
31208 Acc 48-53
2xh
2003
Lamad 2
ammz Uammar 35-Acc
Jaamz
Llamad
Acd.
-1.43
-B5-Ac
20-11
18
138 Acc
18F1
138 Acc
1B F1
138 Acc
L
Ubmaa322 ACC
Upma32 18 Acc
18 ACC
158 F1
15
138
LR
GAM
MLP
XGBoost
Figure 5: Comparison of logistic regression (LR) and generalized additive model (GAM), multi- layer perceptron (MLP), and eXtreme Gradient Boosting (XGBoost) as predictors for RAGLens, evaluated across multiple models and datasets.
CONCLUSION
In summary. this work demonstrates that SAEs can serve as powerful and interpretable tools for detecting RAG hallucinations. By leveraging internal representations of LLMS, our proposed RA- GLens framework not only achieves late-of-thheart performance across multiple benchmarks, but also provides transparent explanations at both local and global levels. Beyond detection, the in- terpretability of RAGLens enables actionable feedback mitigate hallucinations, improving the reliability of RAG systems in practice. These findings highlight the broader potential of sparse rep- resentation probing for enhancing model faithfulness and open up future directions for integrating lightweight, interpretable SAE- based detectors into real- world applications where trust and accuracy critical.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported in part by the US National Science Foundation (NSF) and the National In- stitute of Health (NIH) under grants ISS2106913,, I1S-2538206, IIS-2529378, CCF-2217071, CNS- 2213700, and R01LM014012-011 the authors and do not necessarily reflect the views of NIH or NSF.
Any recommendations expressed in this material are those of
REFERENCES
Samir Abdaljalil, Filippo Pallucchini, Andrea Seveso, Hasan Kurban, Fabio Mercorio, and Erchin Serpedin. SAFE: A sparse autoencoder- based framework for robust query enrichment and hal- lucination mitigation in LLMs. In Christos Christodoollpoolls,, Tanmoy Chakraborty, Car- olyn Rose, and Violet Peng (eds.), Findings of the Association for Computational Linguis- fics: EMNLP 2025, pp. 9335-9346 Suzhou, China, November 2025. Association for Computa- URL
tional Linguistics. ISBN 979-8-89) 19188997763387 -33-77. doi: 10. 18653v 1/2022 wew.dinnsneenllll.. org/ 2025. windinggigg.... emn 1p. 496/
https://aclaathoolie
Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it's lying. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Lin- guistics: EMNLP 2023, pp. 967-976, Singapore, December 2023. Association for Computational Linguistics. doi: 10.1 116113/1/2003..... wtn.222...iinngeeenl....m org http:/222...n...... ings emnlp. 68%
URL
Forrsst Bao, Miaoran Li, Rogger Luo, and Ofer Mendelevitch. HHEM-2211-Opn. 2024. URL httpa///aaaaaaaaaaaaaaoaaaaaaaaa 1
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Turner, Cem Anil. Carson Denison, Amanda Askell, Robert Lasenby. Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https:/transformer- hww.://20mmmosseaaaaieeaaaeeeeee
Anh Thu Maria Bui, Saskia Felizitas Brech, Natalie Hußfeldr, Tobias Jennert, Melanie Ullrich, Timo Breuer, Narjes Nikzad-Khasmaal Khasmakhi, and Philipp Schaer. The two sides the coin: Hallucination generation and detection with 1lms as evaluators for 1lms. In Gugliclmo Faggioli, Nicola Ferro, Petra Galuscáková, and Alba García Seco de Herrera (eds.), Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2024), Grenoble, France, 9-12 September, 2024, volume 3740 of CEUR Workshop Proceedings, pp. 727-758. CEUR-WS.org 2024. URL https:// ceur wS. /Vol 3740/pape::.. 71.
Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Mare Sturm, and Noemie Elhadad. Intel- ligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1721-1730. 1730, 2015.
Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and lieping Ye. INSIDE: LLMs' internal states retain the power of hallucination detection. In The Twelfth International Conference on Learning Representations, 2024. URL http//ppnnevewweeeeeeeee httpeeviieeeeeeffrrreeeeee forum?
id=2 Qbz
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference knowledge discovery and data mining, pp. 785-794, 2016.
Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. Lm vs Im: Detecting factual errors via cross examination. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 12621-12644 2023.
Jukka Corander, Ulpu Remes, and Timo Koski. On the distance for categorical probability distributions. Kybernetika, 57(6):879-99) 2021.
divergence and the variation
11

---

Published as a conference paper at ICLR 2026
Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders find interpretable IIm feature circuits. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Pa- quet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Sys- tems, volume 37, pp. 24375-24410. Curran Associates, Inc., 2024. doi: 10.52202/ 079017-0768. URL htttps://pooceeddiiiiiiee hewtiinnnnnnneus..eeueeu...uu www.ripp.ccppppprrr.r.... cccppapprr_lllle/pppeeeeeee
Paper Conference P pdf
Passant Elchafei and Mervat Abu-Elkheir. Hallucination detectives at semeval-2025 task 3: Span- level hallucination detection for Ilm-generated answers. In Proceedings of the 19th International Workshop on Semantic Evaluation (Semfival-20255, pp. 601-606, 2025.
Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. Ragas: Automated evalua- tion of retrieval augmented generation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguisties: System Demonstrations, pp. 150-158, 2024.
Javier Ferrando, Oscar Balcells Obeso, Senthooran Rajamanoharan, and Neel Nanda. Do i know this entity? knowledge awareness and hallucinations in language models. In The Thirteenth International Conference on Learning Representations, 2025. httpss/:::// /tt. het//orrumiid=wRRgo ]12q
Robert Friel and Atindriyo Sanyal. Chainpoll: A high efficacy method for IIm hallucination detec- tion. arXiv preprint arXiv:2310.18344 2023.
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. augmentel generation for large language models: A survey, arXiv preprint 2rXiv:2312.10997, 2(1), 2023.
Onkar Gujral, Mihir Bafna, Eric Alm, and Bonnie Berger. Sparse autoencoders uncover biologically interpretable features in protein language model representations. Proceedings of the National Academy of Sciences, 122(34):e2506316622 2025.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. Retrieval augmented language model pre-training. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume of Proceedings of Machine Learning Research, pp. 3929-3938. PMLR, 13-18 Jul 2020. URL https: : http:::://ppcceednngg./ .mlr.
pxess/v119/quu20a.
Jiatong Han, Jannik Kossen, Muhammed Razzak, Lisa Schut, Shreshth A Malik, and Yarin Gal. Semantic entropy probes: Robust and cheap hallucination detection in llms. In ICML 2024 Work- shop on Foundation Models the Wild, 2024.
Trevor Hastie. Generalized additive models. Statistical models S. pp. 249-307, 2017.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL
id=nZevKeeeFf
Xiangkun Hu, Dongyu Ru, Lin Qiu, Qipeng Guo, Tianhang Zhang, Yang Xu, Yun Luo, Pengfei Liu, Yue Zhang, and Zheng Zhang. Refchecker: Reference--asee fine-grained hallucination checker and benchmark for large language models. arXiv preprint arXiv:2405. 14486, 2024.
Lei Huang, Weijiang Yu. Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang. Qianglong Chen, Weihua Peng. Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Truns. Inf. Syst., 43(2), January 2025. ISSN 1046-8188. doi: 10.1145/3703155. URL https: :/da::///o..o /rg/
10. 3703155
Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelftth International Conference on Learning Representations, 2024, URL http///pppeeeiiieeeeee/ee net forum? id=F7 6bwwS1.ek
12

---

Published as a conference paper at ICLR 2026
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hlafield-Dodds, Nova DasSarma, Eli Tran-Johnson. Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bow- man, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Ka- plan. Language models (mostly) know what they know. arXiv preprint arXiv:207.052211 2022.
Wojciech Kryšciński, Bryan McCann, Caiming Xiong, and Richard Socher. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp. 9332-9346, 2020.
Patrick Lewis Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sehastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nip tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 3459-94724 Curran Associates, Inc., httplpppprrrlesseeeaeeeepe filess/ppeer//20// F122:a44520000110022222222111110
2020. URL https://prooeeednnnnnerri...s.s.
Haitao Li, Qian Dong. Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. Lims-as-judges: comprehensive survey on llm-based evaluation methods. arXiv preprint arXiv:2412.00559, 2024.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distrribtioo detection. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 21464-21475. Curran Associates, Inc., 2020. URL httpss//pproceecinnggg neurips. cc/paper files/
Yin Lou, Rich Caruana, and Johannes Gehrke. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 150-158, 2012.
Zheheng Luo, Qianqian Xie, and Sophia Ananiadou, Chatgpt as a factual inconsistency evaluator for text summarization. arXiv preprint arXiv:2303.15621 2023.
Varun Magesh, Faiz Surani, Matthew Dahl, Mirac Suzgun, Christopher D Manning, and Daniel Ho. Hallucination-free free? assessing the reliability of leading ai legal research tools. Journal of Empirical Legal Studies, 22(2):216-242 2025.
Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction. In International Conference Learning Representations, 2021. URL /top:ttt////oeereeieee
net/ feet//ooum?iie ficrrum?idd-nn JNo zb5@7m 7m
Potsawee Manakul, Adian Liusie, and Mark Gales. Selfcheckgpt: Zero-resource black-box halluci- nation detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9004-9017, 2023.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1906-1919, 2020.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine rine-grained atomic evaluation of factual pre- cision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12076- 12100, 2023.
Cheng Niu, Yuanhao WU, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. Ragtruth: A hallucination corpus for developing trustworthy attreval-augmente language models. In Proceedings of the 62nd Annual Meeting of the Association for Computa- fional Linguistics (Volume 1: Long Papers), pp. 10862-10878. 2024.
13

---

Published as a conference paper at ICLR 2026
Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. Interpretml: A unified framework for 2019.
machine learning interpretability. arXiv preprint
Agada Joseph Oche, Ademola Glory Folashade, Tirthankar Ghosal, and Arpan Biswas. A systematic review of key reurieval-augmented generation (rag) systems: Progress, gaps, and future directions. arXiv preprint arXiv:2507.1891 2025.
Consts-Cnnttnntin Popescu, Valentina E Balas, Liliana Perescu- Popseu,, and Nikos Mastorakis. Multilayer perceptron and neural networks. WSEAS Transactions on Circuits and Systems, 8(7): 579-588, 2009.
Subhey Sadi Rahman, Md Adnanul Islam, Md Mahbub Alam, Musarrat Zeba, Md Abdur Rahman, Sadia Sultana Chowa, Mohaimenul Azam Khan Raiaan, and Sami Azam. Hallucination to truth: a review of fact-checking and factuality evaluation in large language models. Artificial Intelligence Review, 2026.
Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter Liu. cuttof-diittrbbution detection and selective generation for conditional language models. In The Eleventh International Conference on Learning Representations, 2023. URI https: Mttts:://peerrvili
net
Wei Shi, Sihang Li, Tao Liang, Mingyang Wan, Guojun Ma, Xiang Wang, and Xiangnan He. Route sparse autoencoder to interpret large language models. In Christos Christodoulopou- los, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 6801-6815, Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-891769332.
6. doi: 20.18653/v1/202.. http:///11222..mmmmmmmaa mainlp-main.340 URL btttpss//aclantholory.oook emm 1p-1pppp-
main. 346/
Dong Shu, Xuansheng Wu, Haiyan Zhao, Daking Rai, Ziyu Yao, Ninghao Liu, and Mengnan Du A survey sparse autoencoders: Interpreting the internal mechanisms of large language models. In Christos Christodoulopoulls, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (eds.), Find- ings of the Association for Computational Linguistics: EMNLP 2025, pp. 1690-1712. Suzhou, China, November 2025. Association for Computational Linguistics. ISBN 979-8-89176-335 7. doi: 10.18653/w1/2025 wtt.://......nssn--em fndings--mmnl..8. URL https///aaaaaaaaa//a///aaa www.tthooooo..rr/2222...... Eirdings-emnip..e
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguis-
tics: EMNLP 202/, pp. 3784-3803, 2021.
Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. Improving the domain adaptation of retrieval augmented generation (rag) models for open domain question answering. Transactions of the Association for Computational Linguistics, 11:1-17, 2023.
Juntong Song, Xingguang Wang, Juno Zhu, Yuanhao Wu, Xuxin Cheng, Randy Zhong, and Cheng Niu. Rag-hat: A hallucination-awaar tuning pipeline for 11 in retrieval-augmented generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 1548-15 1158,158,,00 2024.
Gaurang Sriramanan, Siddhant Bharti, Vinu Sankar Sadasivan, Shoumik Saha, Priyatham Kattakinda, and Soheil Feizi. Llm-check: Investigating detection of hallucinations in large language models. quet, J. Tomezak, and C. Zhang (eds.), Advances in Neural Information Processing Sys- tems, volume 37, pp. 34188-3422 Curran Associates, Inc., 2024. doi: 10.52202/ 079017-1077. URL https::/proccediii
A. Globerson, L. Mackey, D. Belgrave, A. Fan, Pa-
ZhongXiang Sun, Xiaoxue Zang, Kai Zheng, Jun Xu, Xiao Zhang, Weijie Yu, Yang Song, and Han Li. RedeEP: Detecting hallucination in retreneel-augmmented generation via mechanistie inter- pretability. In The Thirteenth International Conference on Learning Representationss 2025. URL
https:://ppnnee net wtt.:e/eee/ooommm////z/ooooaaaa
14

---

Published as a conference paper at ICLR 2026
Praneet Suresh, Jack Stanley, Sonia Joseph, Luca Scimeca, and Danilo Bzdok. From noise to narra- tive: Tracing the origins of hallucinations transformers. In The Thirty-ninth Anmual Conference on Neural Information Processing Systems, 2025.
Manveer Singh Tamber, Forrest Sheng Bao, Chenyu Xu, Ge Luo, Suleman Kazi, Minseok Bae, Miaoran Li, Ofer Mendelevitch, Renyi Qu, and Jimmy Lin. Benchmarking LLM faithfulness in RAG with evolving leaderboards, In Saloni Potdar, Lina Nojas-Barahona, and Sebastien Montella (eds.), Proceedings of the 2025 Conference on Empirical Methods in Natural Language Process- ing: Industry Track, pp. 799 811, Suzhou (China), November 2025. Association for Computa- tional Linguistics. ISBN 879-8889176-3333 333-3. doi: 10.186533 wew.pp-emllpppppp 1aanlppnduutry5444 URL
https://accanthhn
org 2025. emnlp- wnw.iiin.utssr.. 54/
Liyan Tang, Tanya Goyal, Alex Fabbri, Philippe Laban, Jiacheng Xu, Semih Yavuz, Wojciech Kryšciński, Justin Rousseau, and Greg Durrett. Understanding factual errors in summarization: Errors, summarizers, datasets, error detectors. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11626-11644, 2023.
Liyan Tang, Philippe Laban, and Greg Durrett. Minicheck: Efficient fact-checking of Ilms on grounding documents. In Proceedings of the 2024 Conference on Empirical Methods in Natu- ral Language Processing, pp. 18187884, 2024a
Liyan Tang, Igor Shalyminov, Amy Wong, Jon Burnsky, Jake Vincent, Yu'an an Yang, Siffi Singh, Song Feng, Hwanjun Song, Hang Su, Lijia Sun, Yi Zhang, Saab Mansour, and Kathleen McKe- own. TofuEval: Evaluating hallucinations of LLMs on topic-focused dialogue summarization. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Confer- ence of the North American Chapter of the Association for Computational Linguistics; Human Language Technologies (Volume 1: Long Papers), pp. 4455-4480, Mexico City, Mexico, June URL
2024b. Association for Computational Linguistics. doi: 10.18653/v https::/acclathoot
org. 2024. nww.l 2517
Henk Tillman and Dan Mossing, Investigating task-specific prompts and sparse autoencoders for activation monitoring. arXiv preprint 2rXIv=2504.2027,,202 2025.
Truera. Trulens: Evaluation and tracking for 1lm experiments and ai agents. https: /
com//ruera//tull lens 2025. Version 2.3.1 (latest release as of September 2025).
Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. Language models don't al- ways say what they think: Unfaithful explanations in chain-of- thotuugh prompting. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neu- ral Information Processing Systems, volume 36, pp. 74952-74965. Curran Associates, Inc., neurg. cc https//eepppeeeeeese///////////e
2023. URL https://proceed.ns...eerr File #ile:/eccFaaSS33330ee11117621133 Pappeaaaaaaapaaapppaaaeeeee Paper Conference. pdd
files/paper/2023/
Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Anmual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 9440-9450, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 18653/1v 1024.ac-loong551. URL htthtts::// http::::://aaaaattoooooo -twp://.ooo00/022221111ooooooo
Jason Wei. Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 24824-24837. Curran Associates, Inc., 2022. URL http::::/pprccceeiigg.....t www.::iiiiin......s.se.........m http::////ppppeetllssee/p///e//p File/ http://666911332eeeefee1a1a1.aa1
www.perrroafeeeeneeeeeo .ddf
Zhepei Wei, Wei-Lin Chen, and Yu Meng. InstructRAG: Instructing aegrenee gener- ation via sytheszzdeee rationales. In The Thirteenth International Conference on Learning Representations, 2025. URL TtI.httts:: httpppareei.i..eeeeeorruaaeaaaaa
15

---

Published as a conference paper at ICLR 2026
Chunlei Xin, Shuheng Zhou, Huijia Zhu, Weiqiang Wang, Xuanang Chen, Xinyan Guan, Yaojie Lu, Hongyu Lin, Xianpei Han, and Le Sun. Sparse latents steer etrieval-auggeentd generation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 4547-4562, 2025.
Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang. Benchmarking retrieval-augmenter generation for medicine. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics; ACL 2024, pp. 0233-6251, Bangkok, Thailand, August 2024, Association for Computational Linguistics. doi: 10. 10.18653/v122044 findings-acl. 372. URL httppps:///a.aatthtt/ www.antw..ooooooooooooooooooooom org/ 2024. f winiiinnnss....... acl 1tt.gsssa..............
Cyril Zakka, Rohan Shad, Akash Chaurasia, Alex R. Dalal, Jennifer L. Kim, Michael Moor, Robyn Fong. Curran Phillips, Kevin Alexander, Euan Ashley. Jack Boyd, Kathleen Boyd, Karen Hirsch, Curt Langlotz, Rita Lee,, Joanna Melia, Joanna Nelson, Karim Sallam, Stacey Tullis, Melissa Ann Vogelsong, John Patrick Cunningham, and William Hiesinger. Almanac rerrieval-augmented language models for clinical medicine. NEJM AI, 10))AAoo2200066.. 2024. doi: 10. 10.1056/ Aloa2300068. URL httpt::tt/saat. jm org httg://o//Eull//e0 1056 /
Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, and Luoyi Fu. Enhancing uncertainty-basee -base hallucination detection with stronger focus. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro- cessing, pp. 915-932, 2023.
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren's s song in the ai ocean: A survey on hallucination in large language models. Computa- tional Linguistics, 51(4):1373-1418, 12 2025. ISSN 0891-2017. doi: 10.1162// URL
https://661.or2/10..111. 11tp:///....111/.111111 a.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph Gonzalez, and Ion Sto- ica. Judging llm-as-a-judge with mt-bench and chatbot arena. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and Levine (eds.), Advances Neural Informa- tion Processing Systems, volume 36. pp. 46695 Curran Associates, Inc., 2023. URL https://boceeddigg...uuuu.....ap http:///pppppppoooooeaasaaa///// FiLLss////D/////222//111 Bea.nd_Beehhaass.....
fl 8a 142113/0988877622222555533222223
pdf
Xiaoling Zhou, Mingjie Zhang, Zhemg Lee, Wei Ye, and Shikun Zhang. Hademif: Hallucina- tion detection and mitigation in large language models. In The Thirteenth International Confer- ence on Learning Representations, 2025. URL VwOXxPScXR
net mettfformmid
16

---

Published as a conference paper at ICLR 2026
A DATASET AND BASELINE DETAILS
A.1 DATASETS
Here are the detailed deseriptions of the datasets used our experiments.
RAGTruth. RAGTruth (Niu et al. 2024) is a large-scale dataset featuring nearly 18000 naturally generated responses from range of open- and closed-source LLMs under retrieval-augmentted gen- eration settings. The benchmark includes three subtasks: summarization (RAGTruth-Sumnary) data-to-text generation (RAGTruth-Data2Nn) and question answering (RAGTruth-QA). We the training split of RAGTrruth for feature selection and model fitting, and report results on the held-out test set.
Dolly. For the Dolly dataset in the "Accurate Context" setting (Hu al. 2024p, each example presents the model with context document verified to be relevant and accurate. The model is tasked with generating responses faithful to this context. Following (un et al. (2025), we perform two-fold valis-vali for model evaluation, alternating between halves of the test set for feature selection and assessment.
AggreFact. AggreFact (Tang et al. 2023) compiles outputs and annotations from various summa- rization and factual consistency datasets. Our experiments focus the "SOTA" subset, consisting of summaries produced by models such as T5, BART, and PEGASUS. Each summary is paired with its source document, and factual consistency is annotated by humans. We the validation set for feature selection and model training, evaluating final performance on the designated test split.
TofuEval. TofuFval [Tag et al. 2024b is a benchmark designed for topic-focused dialogue sum- marization. We utilize its MeetingBant portion, which consists of multi-turn meeting transcripts annotated with topic boundaries. LLMs are tasked with generating topic-specific summaries from the full dialogue. Annotations include binary sentence-level factuality well as free-form explana- tions for inconsistent content. For this dataset, feature selection and model fitting are performed on the development set, and evaluation is conducted on the test set.
A.2 BASELINES
We benchmark RAGLens against a diverse set of representative hallucination detection methods. For clarity, group all baselines into three main categories, detailed below.
Prompting-basse Detectors. This category captures hallucination signals by leveraging the gen- erative and reasoning capabilities of LLMs to produce token-level decisions. It encompasses various prompting strategies, such prompt engineering (Friel & Sanyal 2023). multi-agent collaboration (Cohen et al. 2023). as well as supervised fine-tuning. Following Sun et al. (2025), we include a
fine-tuned -lamm22-13BB (LB) baseline, where the detector is trained on RAGTruth using LoRA (Hu)
ct al. 2022). The full list of baseline detectors in this category includes: Prompt (Niu et al. 2024),
Llama2-13B(L) (Sun al. 2025). LMvLLM (Cohen et al. 2023), FActScore (Min et al. 2023
FactCC (Kryšcinski et al. 2020). ChainPoll (Friel & Sanyal 2023), RAGAS (Es et al. 2024).
Tru-
Lens [Truera] 2025), RefCheck (Hu et al. 2024), and P(True) (Kadavath al. 2022).
Uncertainty-based Detectors. These methods assess hallucination likelihood based on the uncer- tainty of LLM outputs, typically measured via sampled tokens or the distribution of output logits prior to token generation. The complete list of detectors in this category includes: SelfCheckGPT
MMankuul et al. 2023), LN-Entropy (Malinin & Gales 202T). Energy (Liu et al. 2020), Focus Zhang et al. 2023). and Perplexity (Ren et al. 2023).
Internal Representation-basee Detectors. These approaches probe the internal representations of the LLM. such as hidden states, attention patterns, or other intermediate representations, to iden- tify unfaithful generations. By analyzing these internal model dynamics, these detectors aim to capture subtle signals associated with hallucination that may not be reflected in output tokens or
17

---

Published as a conference paper at ICLR 2026
logits. Methods in this category include: EigenScore (Chen al. 2024). SEP (Han et al. 2024).
SAPLMA (Azaria & Mitchell| 2023), ITI (Li Lietaa al. 2024 and ReDeEP (Sun et al. (025).
B IMPLEMENTATION DETAILS
While trained SAE is required for detection in RAGLens, the rapid development of the SAE community (Shu al. 2 has led the public release of many pre-trained SAEs for widely used LLMs. Moreover, SAES are typically trained on ceneral-purposs corpora rather than task- datasets, allowing for their reuse in analyses beyond the scope of this work. To demonstrate the efficiency and practicality of our pipeline, we utilize publicly available SAEs whenever possible, showing that our method does not require resource-intensive, specially tuned SAEs for effective performance. Below, we list the specific SAEs used in our experiments:
co/ yuzzaauoe/LLL - 7b SAE, which includes SAEs for multiple layers, Specifically, we select the SAE trained on "layers.15", with an expansion factor of 32 and Top-K acti- vation (K = 192).
Llama2-7B: We use the pretrained SAE from
As no public SAE is available for Llama2-13B, we train our own using the sparsify package with default settings. The SAE trained on "layers. with an expansion factor of 16 and Top-K activation (K = 16).
Llama2-13B:
Llama3.2-1B: We use the pretrained SAE from https://huggingface.. 2----33.21111111 which covers multiple layers. For results in Section 4.3 we select the SAE trained on "layers.6.mlp" and for the layer-wise analysis in Section 5 we use all available SAEs. The SAEs have an expansion factor of 32 and
Top-K activation (K 32).
co/ SleutherAI/saee 11ama-aaa---m--m 3- 8---bb35 32x utilizing all available SAEs for the layer-wise analysis in Section 5 The SAEs have expansion factor of 32 and Top-K activation (K = 192).
Ilama3-8B: We use the pretrained SAE from hittps://huggineen
L1ama3.1-8B: We use the pretrained SAE from https://huggingfaet co/ Goodfire watee/lamaa-aa 1e.-1-B--nnstt Inn-----nstttuttSSA------
which contains the SAE trained on "Iayers.19". The SAE has an expansion factor 16 and ReLU activation.
Qwen3-0.6B: As there is no public SAE for Qwen3-0.6B, we train our own using the sparsify package with default settings. For results in Section 4.3] we select the SAE trained on "layers. 17", and for layer-wise analysis, we use all trained SAEs. The SAES have expansion factor of 32 and Top-K activation (K 16).
Qwen3-4B: Similarly, we train our own SABs for Qwen3-4B. For Section 4.3 select the SAE trained on "layers.22"; ; for layer-wise analysis, we use all available SAEs. The SAEs have an expansion factor of 32 and Top-K activation (K- 16).
To compute mutual information (MJ) in RAGLens, we estimate the MI value of each continuous SAE feature by discretizing feature values into bins using quantile thresholds. Specifically, we partition the value range into 50 bins per feature and compute MI values chunks for GPU acceler- ation. After ranking features based estimated MI, select the top 1000 features for subsequent Generalized Additive Model (GAM) fitting in RAGLens.
For GAM fitting, deploy the Explainable Boosting Machine (EBM) Nori et al. 2019), high- performance, tree-based GAM implementation that flexibly models nonlinear effects of features on the target output. Across all experiments, shape function to 32, a validation size of 10%, and a maximum of 1000 boosting rounds.
the maximum number of bins in each feature's
To generate semantic explanations for selected SAE features, collect representative activation cases by sampling 12 examples with the highest activations and 12 examples distributed across quantiles from the RAGTruth training data. These cases are then provided GPT-5, which summa- rizes the underlying semantic concept captured by each feature using the template in Figure
httpp::/ http://thttt...oo/o//maaaaaaa//a
18

---

Published as a conference paper at ICLR 2026
Prompt templates for all LLM text- sett-eenrration calls are shown in Appendix
All experiments are conducted on a server equipped with an AMD EPYC 7313 CPU and four NVIDIA A100 GPUs.
C CAUSAL INTERVENTION OF SAE FEATURES
For SAE features that consistently activate prior to hallucinated content, we investigate whether direct intervention on these features can causally influence the model's generation. Specifically, we manipulate the activation values of selected SAE features key tokens preceding hallucinated spans and observe the resulting changes in model outputs. This analysis assesses whether these features not only correlate with hallucination but also play a causal role in driving unfaithful generations.
Table 6 presents a case study on Feature 22790 from 1ama3.1-8B, which reliably activates before hallucinated numeric temporal details (e.g., firing the token "of" in the prefix "[-..] at the age of", which often leads to unsupported ages). When we suppress this feature (e.g., set its value to 0 -20), the model continues the problematic prefix with hallucinated, ungrounded numbers. In contrast, manually setting the feature to large positive value (e.g.. 20) steers the model to remain faithful the context, producing follow-up tokens that avoid hallucinated specifics (e.g.. using an unspecified age time). This suggests that Feature 22790 reflects the model's awareness of potentially hallucinated numeric or temporal details, and that overactivating can encourage more faithful behavior in such scenarios.
Table Examples of causal interventions Feature 22790 in Llama3.1-8B. This feature is consis- tently activated prior to hallucinated numeric/time specifics. Tokens in red indicate the hallucinated content.
Context
Prefix
Value
Output
-20.0
[...] 30.
No mention of the age
[...] at the age of
0.0
[...] 25.
20.0
[...] of an unspecified age.
-20.0
[...] in 2016.
No mention of the release date
[...] scheduled be released in
0.0
[...] in the future.
20.0 [...] in an unspecified time frame.
An additional case study on Feature 71784 in Llama2-13B is shown in Table 7 This feature is asso- ciated with hallucinations about opening hours (day/time) and ratings, and typically activates only when the hallucinated time or rating is already being produced (e.g.. "Monday" in an inconsistent hours-of-operation statement). For these cases, we perturb the feature value on the token immedi- ately preceding the hallucinated word (e.g.. "on" in "on Mondays"). The results also confirm that manipulating SAE features identified by RAGLens can steer the model's behavior and demonstrate causal relationship between these features and RAG hallucinations.
However, for features like the one in Table that only activate concurrently with or after hallucinated tokens, direct intervention is impractical for preventing hallucinations, as the problematic content has already been generated by the time these features fire. Furthermore, the distance between the hallucinated tokens and the token with high activation is not always consistent across features and examples. Thus, while causal intervention on SAE features is feasible in certain scenarios, is not universal solution for hallucination mitigation. This limitation motivates our focus post-hoc text-based feedback in the main mitigation pipeline.
D DISCUSSION ON SAE FEATURE VERSUS HIDDEN STATE
To further investigate the contribution of SAE-derived features to RAGLens performance, we con- duct ablation study by replacing SAE features with the raw hidden states of Llama2-7B, while
19

---

Published as a conference paper at ICLR 2026
Table 7: Examples of causal interventions Feature 71784 in Llama2-13B. This feature is consis- tently activated on ballucinations about opening hours (day/time) and ratings. Tokens in red indicate the hallucinated content.
Context
Prefix
Value
Output
[...]: the restaurant is open from
-20.0
11:00 AM to 21:00 PM. Monday to Sunday.
[...] "Monday": "0:0-0:0" [...]
[...] Restaurants
[..|] the restaurunt is open from
0.0
Hours:
11:00 AM to 21:00 PM. Monday Sunday.
20.0
[...]: Monday: 0:00 0:00 [...]
[...] no information available for the
-20.0
[..]] on hours of operation on Mondays.
[...] "Monday": "0:0-0:0" [...]
0.0
[...] on hours of operation on Mondays.
business's hours on
20.0
[...] on holidays.
retaining the MI-based feature selection and GAM classifier. Figure O compares hallucination de- tection performance using hidden states versus SAE features across varying numbers of selected dimensions (K'). When K' is large, hidden states achieve performance comparable to SAE fea- tures, which is expected since SAE features are derived from hidden states.
RAGTruth Acc
RAGTruth F1
TofuEval Acc
TofuEval F1
AggreFact Acc
AggreFact F1
0.80
0.80
0.70
Score
0.75
0.75
0.65
0.70
0.70
0.5
0.60
0.6
Hidden
SAE
0.4
30
8
6
4
2
10
8
4
10
10
10
10
4
log2
(K")
logz
(K))
log2
(K))
loga (K))
log2
(K')
logz (K")
Figure 6: Effect of varying the number of selected features (K') hallucination detection perfor- mance, comparing SAE features (SAB) and hidden states (Hidden).
However, as K' decreases, the performance of hidden states degrades more rapidly, especially single-task datasets such as AggreFact and TofuEval. This indicates that SAE features more effee- tively disentangle hallucinatiion-reltte a compact set of dimensions, particularly when the predictor is applied to narrow domains. Such compact representations improve interpretability and facilitate downstream applications like hal- lucination mitigation, as only a small number of salient features need to be monitored to achieve effective control.
signals from other information and concentrate them into
E DISCUSSION ON SAE FEATURE INTERPRETABILITY
To validate the semantic consistency of SAE features, we extend the analysis in Ta examining top-activated examples from the SAE's pretraining corpus. Specifically, we compute activations for Feature 22790 in Llama3. 8B over the first 10,000 samples from the Imsys/Imsys chat Im corpus) and inspect the highest activation cases, as shown in Table g Although the pretraining corpus con- tains diverse structures and languages, including clinical dialogue transcripts and telecom statements in German, we find that Feature 22790 is consistently activated in scenarios where the response is about to produce specific numbers or dates that are likely hallucinated. This pattern aligns with feature's summarized semantic meaning from RAGTruth and the representative examples shown in Table demonstrating that the feature robustly captures hallucination-relaan signals across both task-specific and pretraining data.
To further assess the robustness of the distilled feature explanation across diverse scenarios, we prompt GPT-5 to predict the activation level of Feature 22790 on 24 held-out RAGTruth test cases
chat-
20

---

Published as a conference paper at ICLR 2026
Table 8: Examples from the pre-training corpus with high activations of Feature 22790 in Llama3. 8B. Tokens highlighted in red indicate locations of strong feature activation.
Sample Index
Input
Output
TRANSCRIPT= "Hello doctor have fever and cough. Okay take paracetamol and go home and rest.' [...]
[..]] Patient advises they have
9384
been experiencing symptoms for the past two days [...]
[..]] "summary": "Mobiffink- Rechnung für den Monat März 2023" [..]]
3298
User: Telekom Deutschland GmbH [...]
(comprising top-activated and 16 guantile-distributed examples), using only the natural-language summary. For each case, the three most highly activated tokens are highlighted, and GPT-5 is asked to rate the expected feature activation on a scale from (feature not present) to 5 (very strong match). Comparing these predicted scores to the actual SAE activations yields Pearson correla- tion of 0.6731 (p 0.05), suggesting that the explanation reliably captures when the feature should activate across broad range of examples.
However, while SAEs are designed to disentangle distinct semantic concepts from hidden states, some SAE features may remain generic polysemantie, limiting their interpretability, which is challenge widely recognized in current SAE research (Bricken et al., 2023 Huben et al. 2024). Although RAGLens already achieves accurate and interpretable RAG hallucination detection using existing SAE foundations, its architecture-agnesse in SAE methods, which may enable even more transparent and effective detectors for RAG halluci- nations.
design allows it to benefit from future advances
F ABLATION STUDY OF SAE FEATURE EXTRACTION METHODS IN RAGLENS
In this section, investigate whether our SAE-based design, which combines max pooling over tokens with mutua-infformation-basss feature selection, offers advantages over alternative ways of using SAE features (Ferrando al. 2025 Tillman Mossing 2025 Xin et al. 2025 Suresh al., 2025 Concretely, we compare four variants on the RAGTruth subtasks:
1. Last Token Selection GAM: SAE features taken only from the last generated token, followed by MI-based feature selection and GAM classifier.
2. Max Pooled + No Selection + LR: max-pooled SAE features across tokens, using all dimensions as input to logistic regression (LR) classifier (no feature selection).
3. Max Pooled + Selection LR: max-pooled SAE features with MI-based feature selection, using LR as the classifier.
4. Max Pooled Selection GAM (RAGLens): our full RAGLens variant, which applies MI-based feature selection to max-pooled SAE features and then fits GAM.
Table 9 reports accuracy and AUC for these settings. Using max pooling with feature selection and GAM (Max Pooled + Selection + GAM) consistently outperforms both: (i) the last-token baseline (Last Token Selection GAM), which discards earlier activations that may precede hallucinated content, and (ii) the Max Pooled No Selection + LR bascline, which uses all SAE dimensions without selection and is thus less efficient.
Additionally, Max Pooled + Selection + LR performs comparably to using all features without lection, indicating that MI-based selection preserves the most informative SAE dimensions for de- tecting unfaithful model outputs. Overall, these results demonstrate that approach captures most hallucimation-relewwa signals in a compact, efficient feature set and achieves superior detection per- formance compared to alternative SAE usage strategies.
21

---

Published as a conference paper at ICLR 2026
Table 9: Ablation study of SAE feature extraction and classifier choices RAGTruth subtasks using Llama2-7B. We compare last-token versus max-pooled SAE features, with and without mutual information-based feature selection. Due the high computational cost fitting GAMs on all features, we use logistic regression (LR) as the classifier when no feature selection is applied.
RAGTruth Summaryy
RAGTruth-Q2
RAGTruth RAGTruth-Data2tx Data2txt
Source
Selection
Classifier
Acc
AUC
Ace
AUC
Acc
AUC
Last Token
Yes
GAM
0,6293
0.7507
0.6908
0.8101
0.7454
0.8296
Max Pooled
No
LR
0.6734
0.7305
0.7356
0.8344
0.7499
0.8397
Max Pooled
Yes
LR
0.6718
0.7663
0.7572
0.8472
0.7085
0,8014
Max Pooled
Yes
GAM
0.6973
0.8191
0.7717
0.8835
0.7668
0,8454
G PROOF OF THEOREM
Theorem 2 (Restatement of Theorem tokens t I.....T..
D Fix feature index l and suppress l in notation. For
0 with probability 1 1 1-p:, independently over t,
pe; with probability Pt,
21
(10)
V,
where Vt are i...d. from a Sabel-independent distribution on (0, oc). Lei = max Eb
Pr(f= = 1), = (p: + -Po), and Ap = P1 Po- IfT < 1, then (in bits)
I(2:6) T(1 -))
T) (Ap)) ((TT)))
(11)
2ln P
In particular, [(2; () > 0 iff p1 * Pot the leading dependence is linear in T and quadratic in Ap.
Proof. Write 1 ( > 0}. N= = Σ 1( 0}. and h(u) -u
-u logg - u) logg (1 u).
Step 1: Exact MI for the activation indicator.
Independence across tokens implies
qe == Pr(A-1 Pr(P 1 1- -1-
(12)
Hence ~ Bernoulli(g:) and
I(A;E) = M(Tq) + +(1 - T)00) [TT((a)) (1 T.) h((0))]
(13)
Step 2: Small-gap expansion of I(A; Let тh + T) ] + 90), and Aq
90- A second-order Taylor expansion of h about gives
h(g:) = hg)) + (1) + A h"( + 0((Ag))),
(14)
h(gu) = (g) Aa (p) + Aوg h(()) + O((Ag)))
(15)
2
M(r) = h(()) + (2x 1) 2) ('
( (((()) + (({)))))
(16)
8
Plugging into Equation 13 the linear terms cancel, and since h² h"(u) - -lu(1 u) In
((A:e)- = 7(1 (11-)) TT) (Aq)²
+ O((Ag))).
(17)
2 2ln ( - ②)
Step 3: Relating T(2; and [(A:),, with a JS-TV bound. Because A = 1f2 > 0} a deter- ministic function of ž, the chain rule gives
T(2;e) T(A;E) T(E;e) A).
(18)
22

---

Published as a conference paper at ICLR 2026
When A 0, almost surely, (2; | A 0) O.
When A = 1, write
A=IA=1,E
Σ (x) F(n),
(19)
n21
where (n)n) = Pr(N = n = 1, and F(") is the distribution of the maximum of n label- independent draws from F. Thus given A 1, the only dependence é is via the mixture weights [wa (n) (n))).
By the definition of total variation (TV) distance,
1 / P=|A==11(=)] dz
TV (P Ps(A=100)
(20)
< Pr(N 2 2 A = 1,1 2 2 A = 1,0),,
up to constant factors.
Since mutual information conditioned on is the Jensen-Shanme Shannon (JS) divergence between these two conditional distributions (with weight Pr(e = A = 1) over f), one can invoke standard bound:
JS( JS(paA==111 P3|A=1,0) S C TV (V A=
(21)
for some constant C depending on the mixing weight (Corander et al. 2021)
Thus,
[(z 1) A(Pr(N 2 2 A= 1,1) Pr(N 2 2 A = 1,0))²)
(22)
Multiplying by Pr(A = 1) < gives
[(z:t) I(A:e) OPr(N 2| 2 + Pr(N > 2 (0)))
(23)
which, under rarity (T' < 1), is o(TT))"
Step 4: Specializing to independence and the sparse regime. token rate pe,
Under independence with per-
Pr(N2210) 1 pe) -_ Tpe(1 pe) = ( pt OT
(n)
(24)
Thus
[(z;() I(A:0) + O(TTp9).
(25)
uniformly for pe with ğ 1 (p1 + po).
Step 5: Substitute sparse approximations.
For T' & I,
T
gr (1- = =Tp:
Pi
O(T
(26)
2
q Tp + 7(T
(27)
Ag = Tāp + pppp).
(28)
Plug these into Equation
17 Since 9 1 O(Tp).
(Ag)²
T (Ap))
+ (Ap) Tp) T(Ap)²
+ (Ap))
(29)
q(1 4) Tp
p
Moreover, (Dq)3= O(T31App o pT2pp) under Tp < and bounded App. Hence
T(1 T) T(Ap)?
I(A:E)
+ o(T²p?).
(30)
2 2
P
Combining Equation 30 with Equation 25 via Equation 18 yields
I (2; 2) = T - π) T(Ap)2
+ O(T2p?)
(31)
2ln D
which is the claimed statement since T2p² (Tp)2.
n
23

---

Published as a conference paper at ICLR 2026
H INTERPRETATION OF ADDITIONAL CALLUCINATIOO-RELAAAIE FEATURES
Table T0 highlights additional representative features that RAGLens activates when detecting hal- lucinations across LLMs of varying scales. These features capture wide range of hallucination patterns, including overconfident numeric spans, incorrect temporal assertions, and ungrounded en- tity mentions. Notably, smaller L.Mss tend to exhibit more generic hallucination signals (e-g.. "over- stated concrete details"), whereas larger L.MSs reveal more specific and nuanced patterns (e.g.. "pre- cise numeric spans not grounded retrieved passages"). This observation suggests that as L.MSs increase in size, they develop more specialized internal features for identifying complex hallucina- tion phenomena. This may contribute to the improved hallucination detection performance of larger models with RAGLens, shown in Figure
Table 10: Explanation and examples of representative SAE features in Llama2-7B, Llama2-1 Llama3.2-1B, and Llama3.1-81 that enable interpretable hallucination detection. Spans highlighted in red indicate tokens where the feature highly activated. The table illustrates how these features capture diverse hallucination patterns across models.
Feature ID)
Feature Explanation
Example Output
Example Explanation
Llama2-7B
120059
hallucination involving entity swaps and in- vented details.
[...] his castmates from "Central Intelligence" took a knee he [...]
There is mention of Kevin Hart's castmates from "Central Intelli- gence".
127083
unsupported concrete ad- ditions such as names, pairings, numbers, legal evidentiary claims
[...] such as Walker- Rubio or Clinton-Kaine pairing [...]
Clinton-Kart Kaine mentioned in the source content
Llama2-13B
26530
"amenity assertion" de- tector that spikes on the outdoor seating phrase hallucinations on day/- time (hours) and ratings
The restaurant has ca- sual ambiance and offers outdoor seating
Original text shows no outdoor seating
71784
[...] no information avail- able for the business's hours on Mondays [...]
Original text: closed on Monday
Llama3.2-1B
unsupported or swapped 78162
The team now fac- ing their in-state rivals, the Los Angeles Dodgers [...]
The Texas Rangers and Los Angeles Dodgers are not in-state rivals
named entities and pre- cise facts
121247 invented or overstated concrete details
[...] He was also a pro- fessor of film criticism at
It is not mentioned the original source.
NYU [...]
Llama3.1-8B
37877 precise numeric spans that aren't grounded in the retrieved passages
[...] soft ball stage occurs at a tempera- around 245- 250 [...]
The firm ball stage a temperature of about 245 to 250 degrees Fahren- heit
ture
40779
overconfident claims about hours, open/closed days, and amenities
[...] the exception of Friday when it closes at 20:00 PM
Friday opens from 11am and closes by 10pm
24

---

Published as a conference paper at ICLR 2026
I FURTHER ANALYSIS OF IDENTIFIED FEATURES VIA COUNTERFACTUAL PERTURBATION
To further validate that the features identified by RAGLens capture meaningful hallucination patterns in RAG-specific contexts, conduct case studies using counterfactual perturbation SAE feature 37877 from Llama3. 1-8B, which detects "precise numeric spans not grounded in retrieved passages" (see Table [OP. We select representative samples from three RAGTruth subtasks, summarization, data-to-text, and question answering, where this feature is highly activated and the generation is hallucinated. For each, we manually edit the context to construct counterfactual scenarios: (1) the output becomes consistent with the perturbed context, or (2) the output remains inconsistent, but in different way. For question answering. we also consider a version where the context entirely removed to examine feature activation in the absence of grounding.
Tables I present the results. For summarization (Table [T) and data-to-text (Table 13 when the context edited to make the output consistent, the feature value on previously highlighted tokens drops significantly; the context remains inconsistent, the feature stays highly activated. In question answering (Table (2). feature activation drops when the context is either edited to be consistent or removed, indicating that the feature specialized for detecting ungrounded numeric spans in context, rather than general hallucination. Overall, these studies demonstrate that RAGLens identifies features that robustly capture RAG-specific hallucination patterns.
J PROMPT TEMPLATES FOR LLM CALLING
This section presents the prompt templates we use for LLM calling in our experiments. Specifically, for the summarized SAE explanations in Table 3 we use the template shown in Figure 7 The hallucination mitigation approaches discussed Figures and for the instance- and token-level feedback, respectively. The LLM evaluation shown in is implemented with the template in Figure 10 following Luo et al. (023).
Section [.66 are implemented with templates in
Prompt template for summarizing SAE feature explanations
am trying to explain the semantic meaning of hallucination-relatee feature in retrieval- augmented generation settings. Please first examine if the highly activated tokens in each example are related to specific cases of hallucinations. Then, try to summarize the semantic meaning the feature based on these observations. Finally, give me a concise description of the feature meaning in one sentence, specifying what kind of hallucination (if applicable) is detecting.
### Example
#UD Here is the input: ( (input])
#### Here is the output: ((output))
#UUH Here are the feature activation associated with each output token: { [ (tokenl, valuel), (token2, value2), ...]l
### Example
Figure Prompt template for summarizing SAE feature explanations.
25

---

Published as a conference paper at ICLR 2026
Table 11: Counterfactual analysis of SAE feature 37877 from Llama3. 1-8B RAGTruthh Summary example. The table illustrates how feature activation changes in response to context perturbations, highlighting its specificity for detecting ungrounded numeric spans in summariza- tion. Key information in the original and updated contexts is highlighted in yellow, For the output, tokens are highlighted in red, with the intensity of the red background indicating the relative activa- tion value on each token.
Model Input
Summarize the following news within 78 words:
Iraq Service Campaign Medal Reinstated [...] According to memorandum signed by then- Under Secretary of Defense for Personnel and Readiness Gilbert Cisneros in August, the Pen- tagon once again awarded the Inherent Resolve Campaign Medal to eligible service members serving in Iraq. According to document received by Military Times, the medal's s eligibility has been renewed retroactive to January and will continue through December 31, 2024. [...]
Model Output (annotated with feature activation)
The US Department of Defense has reinstated the Inherent Resolve Campaign Medal for service members in Iraq due to renewed threats against US personnel in the Middle East. The medal's 's eligibility has been renewed retroactively from January 1. 202 and will continue through De- cember 31, 2024. The move was requested by the US Central Command, following Iraq's re- classification from combat mission. The medal was initially created in 2016 to acknowledge service in Iraq or Syria, but was limited to Syria 2017.
Context Perturbation (context inconsistent)
Original Context:
[...] the medal's eligibility has been renewed retroactive to January 1 [...]
Updated Context:
[..]] the medal's eligibility has been renewed retroactive to January 2023 [...]
Updated Activation:
[...] The medal's eligibility has been renewed retroactively from January 1, 202 and will
continue through December 31, 2024 [...]
Context Perturbation 2 (context consistent)
Original Context:
[...] the medal's eligibility has been renewed retroactive January 1 [...]
Updated Context:
[...] the medal's eligibility has been renewed retroactive to January 1, 2021 [...]
Updated Activation:
[...] The medal's eligibility has been renewed retroactively from January 1, 2021, and will
continue through December 31, 2024 [...]
K USE OF LARGE LANGUAGE MODELS
In this project, large language models (LLMs) used for multiple purposes:
We use checkpoints of open-source LLMs to extract hidden states, which are then analyzed to identify interpretable SAE features that help detect RAG hallucinations.
We use LLMs as judges to evaluate whether RAGLens feedback helps mitigate hallucina- tions in the original model outputs.
We use LLMs summarizers to describe the semantic roles of different SAE features based on their activations across multiple samples.
We use L.LMs proofread the paper.
26

---

Published as a conference paper at ICLR 2026
Prompt template for hallucination mitigation with instance-level feedback
User:
([input])
Assistant:
[[original_oupput)]
User:
There are hallucinations in your output. Please revise it.
Figure 8: Prompt template for hallucination mitigation with instance-level feedback.
Prompt template for hallucination mitigation with token-level feedback
User:
( ((input))
Assistant:
((original_output))
User:
There are hallucinations in your output, especially on the following spans: ([[spanl, span2, ...]}}
Please revise it.
Figure 9: Prompt template for hallucination mitigation with token-level feedback.
Prompt template for LLM-as-a-Judge mitigation results
Decide if the following summary/answer is consistent with the corresponding article.
Note that consistency means all information in the output is supported by the article.
Article: (ccontett)
Suumary//Answr: {
(reviseddouutpe
Explain your reasoning step by step then answer (yes no) the question:
Figure 10: Prompt template for LLM-as-a-Judge on mitigation results.
27

---

Published as a conference paper at ICLR 2026
Table 12: Counterfactual analysis of SAE feature 37877 from Llama3.1-81
RAGTruth-QA
example. The table illustrates how feature activation changes response to context removal and consistency edits, highlighting its sensitivity question answering. Key information in the original and updated contexts is highlighted in yellow, For the output, tokens are highlighted in red, with the intensity of the red background indicating the relative activation value on each token.
ungrounded numeric spans in retrieval-augmented
Model Input
Briefly answer the following question: how to tell the temperature water with a candy thermometer
Bear in mind that your response should be strictly based on the following three passages:
passage 1: [...] the candy forms a soft, flexible ball is a temperature 235-240 degrees Fahrenheit at the soft ball stage. passage 2: [...] If it forms into firm ball, is in the firm ball stage. This stage is at temperature
of about 245 to 250 degrees Fahrenheit [...]
passage 3: [...]
In case the passages not contain the necessary information to answer the question, please reply with: "Unable to answer based given passages."
Model Output (annotated with feature activation)
Based on the provided passages, here how tell the temperature of water using candy thermometer:
Passage I states that when the candy is in the thread stage, the temperature around 230- Fahrenhttt.
Passage 2 indicates that the soft ball stage occurs at a temperature of 245 Fahrenheit for candies such fudge and fondant. Therefore, to determine the temperature of water using a candy thermometer, you can follow these steps:
1. Cool some of the candy in the thread stage by dropping it into cold water.
2. Observe the candy forms a soft, flexible ball. If it does, the temperature is between
245 Fahrenheit,,, indicating that the water is the soft ball stage.
If the water is at the soft ball stage, continue cooling it until it reaches that temperature range, and then proceed the candy thermometer measure its temperature.
Context Perturbation (context removed)
Original Context:
[...] Bear in mind that [...]
Updated Context:
None
Updated Activation:
[...] indicates that the soft ball stage occurs at temperature around 245-250 [...]
Context Perturbation 2 (context consistent)
Original Context:
[...] This stage is at a temperature of about 245 to 250 degrees Fahrenheit [...]
Updated Context:
[..]] The soft ball stage is at temperature of about 245 to 250 degrees Fahrenheit [...]
Updated Activation:
[..]] indicates that the soft ball stage occurs at temperature of around 245-250 [...]
28

---

Published as a conference paper at ICLR 2026
Table 13: Counterfactual analysis of SAE feature 37877 from Llamm3.1-8BB RAGTruth--ataate example. The table demonstrates how feature activation responds to context perturbations, high- lighting its specificity for identifying ungrounded numeric spans in data-to-text generation. Key information in the original and updated contexts is highlighted in yellow, For the output, tokens are highlighted in red, with the intensity of the red background indicating the relative activation value on each token.
Model Input
Write objective overview about the following local business based only the provided struc- tured data in the JSON format. You should include details and cover the information mentioned in the customers' review. The overview should be 100 200 words. Don't 't make up information. Structured data:
{ ('name": "Mony's Mexican Food", 'address': '217 E Anacapa St', "city': 'Santa Barbara', 'state": 'CA', 'categories': 'Restaurants, Mexican', 'hours': {'Monday': '10:30-15:30; "Tues- day': 10:30015:300, "Wednesday': "10:30-15:00,, "Thursday': '10:30-15:30,, 'Friday': '10:30- 15:30', 'Saturday': "10:30-15:30) [...]]
Model Output (annotated with feature activation)
Sure! Here's an objective overview of Mony's Mexican Food based on the provided structured data:
[...] The restaurant has cozy and casual atmosphere, with WiFi outdoor seating available. However, customers can enjoy takeout or dine in during its hours of operation, which are from 10:30 AM to 3:30 PM from Monday Friday, and 10:30 AM to 4 20 PM Saturday. Despite the limited hours and lack of WiFi, Mony's 's Mexican Food remains popular choice among locals and visitors alike, with many return customers and high praise from reviewers.
Context Perturbation (context inconsistent)
Original Context:
[...] 'Saturday': 110:30-15:30
Updated Context:
[...] 'Saturday": 19:30-15:30' [...]
Updated Activation:
[...] and 10:30 AM to 4:00 PM on Saturday [...]
Context Perturbation 2 (context consistent)
Original Context:
[...] 'Saturday': : '10:30-15:30' [...]
Updated Context:
[...] 'Saturday': 110:30-16:00
[...]
Updated Activation:
[...] and 10:30 AM to 4:00 PM on Saturday [...]
29


Reviews:
[{"review_id": "HkAWhzwYA8", "reviewer": "Reviewer_rg9H", "summary": "The paper proposes a way to detect hallucinations in the context of RAG by training a generalized additive model on SAE-encoded feature vectors; token-level activations are channel-wise max pooled to produce instance level features via selection on mutual information. The authors compare against many baselines (though, slightly narrowly selected- they don't compare against pretty area-standard hallucination/factuality detectors like factscore/factcc/summac/stronger model internal state probe baselines) and show that their method performs better, across datasets like RAGTruth, Dolly, Aggrefact, and tofueval. Nonetheless, the approach is shown to reduce hallucination rates in a small mitigation study via token-level and instance-level feedback, by prompting the model to revise its original output with RAGLens' feedback. Ablations show that middle layers perform best in detecting hallucinations (though this isn't a new finding).", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Novel approach that provides both token-level hallucination detections through SAE features. Empirical results show that this approach outperforms all tested contemporary baselines. The approach is generally very lightweight if you have a trained SAE for the model. The method represents a contribution towards utilizing SAE features for hallucination detection in RAG settings.", "weaknesses": "The subsection \"4.3 GENERALIZATION ACROSS LLMS\" is not greatly named; the authors here only test if each model's own trained GAM outperforms the baseline of the model itself's chain-of-thought explanation of if it hallucinated or not. The method itself, due to needing to be trained on model-specific SAE features, doesn't translate across LLMs- that is, for every LLM, you will need to train your own model. The authors also don't really test generalization *across domains*, even for the same model- that is, how does a GAM trained on one domain (i.e. summarization) translate to detecting hallucinations in another (i.e. dialog).\n\nThe comparison models chosen miss out on some pretty standard area approaches for hallucination detection- factscore, FactCC, etc. Furthermore, novelty is limited- the paper largely combines existing ideas (SAEs, MI-based feature selection + additive modeling) rather than introducing a fundamentally new detection mechanism.  \n\nThe hallucination mitigation approach is not well evaluated. The evaluation is to see if the LLM can use the RAGLens feedback, when prompted, to mitigate the hallucination it caused. To evaluate whether the revisions are better, the paper mostly relies on LLM judges; only 45 examples are evaluated by humans, though the results are convincing and could use greater robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RyeJ7ILBTH", "reviewer": "Reviewer_AJ7o", "summary": "This paper proposes a framework leveraging Sparse Autoencoders (SAEs) to analyze and mitigate hallucinations in Retrieval-Augmented Generation (RAG) systems. The authors argue that hallucinations arise from misaligned interactions between parametric knowledge and external context, which can be disentangled through feature-level representation learning. By training SAEs on internal activations of RAG models, the paper identifies sparse and interpretable features correlated with faithful or hallucinatory outputs. The authors further construct a faithfulness detection model based on these features and explore case studies of interpretable neurons linked to reasoning or retrieval grounding. Experiments on RAGTruth and HalluRAG show improvements in detecting hallucinations and understanding model behaviors, demonstrating the potential of feature-level interpretability for trustworthy RAG.", "questions": "1.\tCan the authors test whether directly intervening on selected SAE features (e.g., amplifying or suppressing feature 22790) reduces hallucinations, to demonstrate causal interpretability?\n2.\tWhere do the 24 activation cases used for GPT-5 explanation distillation come from? Are they drawn from RAGTruth, or from broader pretraining corpora?\n3.\tHave the authors examined whether high activations in the pretraining corpus correspond to similar semantics as in RAGTruth? If so, do the feature functions remain consistent?\n4.\tCould the authors quantify explanation reliability—for example, by testing correlation between explanation-based simulated activations and real activations across large samples?\n5.\tHow does the feature-based faithfulness detector perform when external knowledge is incomplete or partially conflicting with internal knowledge?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tInnovative use of sparse autoencoders to dissect RAG internal activations and link specific features to hallucination phenomena.\n2.\tInterpretability focus — provides human-understandable explanations and visualizations for neuron activations contributing to hallucination or faithful grounding.\n3.\tComprehensive empirical setup, covering both quantitative metrics and qualitative feature studies on multiple benchmarks.\n4.\tClear motivation and relevance to improving trustworthy and explainable RAG systems.", "weaknesses": "1.\tLack of causal intervention — The paper provides interpretive insights but does not test whether manipulating identified SAE features can reduce hallucinations. Given that SAE features allow for activation-level control, it would be valuable to explore feature interventions to demonstrate causality.\n2.\tUnclear explanation data source — In Section 4.4, the authors show two representative features and mention that the semantic explanations were distilled from 24 activation cases. However, it is not clear where these cases originate. If they are from the RAGTruth dataset, they may not generalize beyond it; if they are from the pretraining corpus, examples should be shown in the paper to ensure explanation reliability.\n3.\tPotential bias in interpretation — Since the 24 examples are used to generate explanations, the feature semantics may reflect dataset-level biases. The authors could validate their explanations quantitatively, e.g., by correlating simulated activations (based on explanation-derived templates) with real activations to show consistency.\n4.\tAmbiguity in faithfulness detection — The method relies on features such as ID 22790, which are activated when models refer to external context. Yet, if the model’s internal and external knowledge are aligned, the same feature might be activated even without external grounding, potentially causing false positives. Similarly, when external information is incomplete or noisy, the model may rely partly on internal reasoning—raising questions about whether the proposed detection scheme remains reliable.\n5.\tLimited generalization discussion — The current framework is primarily designed for RAG tasks with correct retrievals. The paper should further discuss or test its applicability under retrieval noise, domain shifts, or reasoning-heavy tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7yOVXGD5rU", "reviewer": "Reviewer_uJwo", "summary": "This paper addresses the problem of hallucination in RAG systems, where large language models may produce content that contradicts retrieved evidence. The authors propose **RAGLens**, a lightweight and interpretable hallucination detector that leverages sparse autoencoders (SAEs) to extract semantically meaningful internal features predictive of unfaithfulness. Through information-based feature selection and additive feature modeling, RAGLens achieves superior detection accuracy compared to existing methods on the same LLM. \n\nThe contributions are: 1. It demonstrates that SAEs can capture nuanced RAG-specific hallucination features; 2. It introduces an interpretable and efficient detector that enables both accurate detection and explainable mitigation; 3. It provides empirical insights into where and how hallucination-related signals emerge within LLMs.", "questions": "1. In the $\\textbf{Analysis of Feature Count}$ section, why does the number of $K^{'}$ have such different effects across datasets? It has a significant impact on TofuEval, but its effect on AggreFact is much less pronounced.\n\n2. I would appreciate it if you could provide a more detailed explanation of how $\\textbf{Global Explanations}$ are realized, as I find this part unclear.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Unlike previous approaches, this paper takes an innovative direction by incorporating SAE-derived features for hallucination detection. This approach not only enables accurate identification of hallucinations but also offers interpretable explanations for their generation, making the proposed framework both elegant and effective. \n\nEmpirical results show that RAGLens achieves superior performance on two hallucination benchmarks compared with multiple baselines. The ablation studies further validate the soundness of key design choices, such as using a GAM as the predictor. \n\nMoreover, the theoretical analysis, including the proof that max pooling can distinguish hallucination-related activations, provides additional confidence in the reliability of the proposed method.", "weaknesses": "A key concern is that the paper does not clearly establish the causal connection between SAE features and hallucination behavior. Although the theoretical analysis argues that, under sparse activation, max pooling can amplify hallucination-related signals and suppress noise, it remains unclear whether the improved detection performance primarily stems from the SAE-derived features or from the predictive capacity of the GAM. If the authors could provide stronger evidence that the learned SAE features themselves are systematically associated with hallucination modes, the technical contribution would be more convincing. \n\nIn addition, while the main experiments include a reasonable set of baselines, the evaluation is limited to two LLaMA-based models (LLaMA2-7B and LLaMA2-13B). Although Section 4.3 reports results on other models, the comparisons are restricted to the CoT-based evaluation setting. This narrow range of model types raises concerns about the generalizability of the proposed method across a wide range of LLMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bWwmoPgn27", "reviewer": "Reviewer_sTa7", "summary": "This paper proposes a pipeline dubbed “RAGLens” for detecting hallucinations in retrieval augmented generation (RAG) outputs of LLMs. Specifically, the authors employ sparse autoencoders (SAEs) to extract interpretable and hallucination-relevant internal LLM features in RAG outputs. Key SAE features are selected based on the mutual information (MI) between the max-pooled token features and the hallucination label, and a generalized additive model (GAM) is fitted on these selected features to predict the hallucination label. Hallucination detection results from RAGLens are compared with existing hallucination detection baselines, and a hallucination mitigation strategy using the results of RAGLens is proposed.\n\n**Verdict**: This paper is clearly presented and empirically solid, but only a mildly novel application of known SAE-based hallucination detection techniques to RAG outputs. The problem space and proposed method do not seem particularly unique and well motivated, and therefore I do not recommend acceptance at this stage. However, the authors could convince me otherwise by addressing each of the major weaknesses below.", "questions": "- Q1: Why should RAG output hallucinations be considered separately from other types of hallucination? The authors should motivate their proposed method over existing SAE-based hallucination detectors.\n- Q2: What makes the proposed method particularly suitable for detecting hallucinations in RAG or grounded output generations if it does not incorporate information from the context?\n- Q3: Is it possible to mitigate hallucinations in outputs before generation? Can this be done with the proposed method?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "S1: The paper is mostly well-written and the methodology for the MI feature selection and GAM implementation is well laid out and explained.\nS2: A wide array of baseline methods of different varieties are used for comparison, as well as a good number of evaluation datasets. These results help to bolster the credibility and apparent utility of the method.", "weaknesses": "Major:\n- W1: While the paper is quite thorough in the method presented, the main idea of detecting hallucinations in RAG or grounded outputs using SAEs is of limited novelty. There are many papers showing that SAE features can be used for hallucination detection in generated outputs [1][2][3][4], and in fact there have been papers that show this for RAG outputs as well (including mitigation) [5]. While the proposed method is clearly effective, it is not properly motivated as to why RAG hallucinations should be treated differently from generic LLM hallucinations. Indeed the authors state “However, it remains unclear whether such features can reliably signal RAG-specific hallucination modes, such as evidence conflicts or unsupported novel content”. But they do not motivate why RAG outputs should present a different paradigm from generic hallucinated outputs. Some of the aforementioned works should be cited and contrasted with the problem setting in this paper.\n\n- W2: Similar to the previous point, the authors do not explain why their **method** should be particularly effective for RAG outputs or outputs with a provided context as opposed to generic LLM outputs. The method does not seem to make use of the provided context in the input prompt for more principled hallucination detection. Instead, the method is similar to existing hallucination detectors from SAE features, albeit with a slightly improved feature selection approach and linear model implementation for prediction. \n\nMinor:\n- W3: The hallucination “mitigation” approach appears somewhat misrepresented: the entire (potentially hallucinated) output needs to be generated before the “mitigation” can begin. Since the authors have access to the internals of the models, some intervention or steering prior to generation in order to mitigate hallucinations might be more useful.\nW4: It appears that the Figure 4 x-axis labels are incorrect for K’...they show 2e7-2e3, contradicting the text which indicates that K’ is compared from 1024 to 1.\n- W5: The SAE architecture and objective should be explained in more detail, as it is essential to the proposed method. This would help with readability.\n\n[1] Ferrando, Javier, et al. \"Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models.\" arXiv preprint arXiv:2411.14257 (2024).\n\n[2] Suresh, Praneet, et al. \"From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers.\" arXiv preprint arXiv:2509.06938 (2025).\n\n[3] Abdaljalil, Samir, et al. \"Safe: A sparse autoencoder-based framework for robust query enrichment and hallucination mitigation in llms.\" arXiv preprint arXiv:2503.03032 (2025).\n\n[4] Tillman, Henk, and Dan Mossing. \"Investigating task-specific prompts and sparse autoencoders for activation monitoring.\" arXiv preprint arXiv:2504.20271 (2025).\n\n[5] Xin, Chunlei, et al. \"Sparse latents steer retrieval-augmented generation.\" Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)