HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs
Authors: Xinyue Zeng
Junhong Lin
Yujun Yan
Feng Guo
Liang Shi
Jun Wu
Dawei Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Hallucination Detection
Primary area: foundation or frontier models, including LLMs

Abstract: The reliability of Large Language Models (LLMs) in high-stakes domains such as healthcare, law, and scientific discovery is often compromised by hallucinations. These failures typically stem from two sources: *data-driven hallucinations* and *reasoning-driven hallucinations*. However, existing detection methods usually address only one source and rely on task-specific heuristics, limiting their generalization to complex scenarios. To overcome these limitations, we introduce the *Hallucination Risk Bound*, a unified theoretical framework that formally decomposes hallucination risk into data-driven and reasoning-driven components, linked respectively to training-time mismatches and inference-time instabilities. This provides a principled foundation for analyzing how hallucinations emerge and evolve. Building on this foundation, we introduce **HalluGuard**, an NTK-based score that leverages the induced geometry and captured representations of the NTK to jointly identify data-driven and reasoning-driven hallucinations. We evaluate **HalluGuard** on 10 diverse benchmarks, 11 competitive baselines, and 9 popular LLM backbones, consistently achieving state-of-the-art performance in detecting diverse forms of LLM hallucinations. We open-source our proposed \model{} model at https://github.com/Susan571/HalluGuard-ICLR2026.

Published as a conference paper at ICLR 2026
HALLUGUARD: DEMYSTIFYING DATA-DRIVER AND REASONING-DRIVER HALLUCINATIONS IN LLMs
Xinyue Zeng* CS Department Virginia Tech
Junhong Lin' EECS Department MIT
Yujun Yan CS Department Dartmouth College
Feng Guo
Statistics Department
Virginia Tech
Liang Shi
Jun Wu
Dawei Zhou
Statistics Department
CS Department
CS Department
Virginia Tech
Michigan State University
Virginia Tech
ABSTRACT
The reliability of Large Language Models (LLMs) in high-stakes domains such healthcare, law, and scientific discovery is often compromised by hallucina- tions. These failures typically stem from two sources; data-driven hallucinations and reasoning- driven hallucinations. However, existing detection methods usu- ally address only one source and rely on task-specific heuristics, limiting their generalization to complex scenarios. To overcome these limitations, we introduce the Hallucination Risk Bound, unified theoretical framework that formally de- composes hallucination risk into data-driven and reasoning-driven components, linked respectively to training-time mismatches and inference-time instabilities. This provides a principled foundation for analyzing how hallucinations emerge and evolve. Building this foundation, introduce HALLUGUARD, NTK- based score that leverages the induced geometry and captured representations the NTK to jointly identify data-driven and reasoning-driven hallucinations. We evaluate HALLUGUARD on 10 diverse benchmarks, 1l competitive baselines, and popular LLM backbones, consistently achieving state-of-the-an performance in detecting diverse forms of LLM hallucinations. We open-source our proposed HALLUGUARD model at HalluGuard
INTRODUCTION
Large language models (LLMMs) are increasingly deployed in high-stakes domains such as health- care, law, and scientific discovery (Bommasani et al. [222] Thirunavukarasu et al. [2023] Ke al.
2025). However, adoption in these settings remains cautious, as such domains are highly regulated and demand strict compliance, interpretability, and safety guarantees (Dennstädt et al., 2025 Kat-
inig et al., 2024). A major barrier is the risk of hallucinations, generated content appears unfaithful or nonsensical. Such errors can have severe consequences (Dennstädt et al., 2025), as the example in Figure [ generated incorrect medical diagnosis may delay treatment or lead to harmful interven- tions. Therefore, detecting hallucinations is not merely a technical challenge but a prerequisite for trustworthy deployment, undetected enrors undermine reliability, accountability, and user safety.
Generally, hallucinations LLMs arise from two primary sources (Ji et al., 2023 Huang et al.) 2025): data-driven hallucinations, which stem from flawed, biased, incomplete knowledge en- coded during ge-ttriniig or fine-tuning, and reasoning trasonnnn-drrven driven hallucinations, which originate from inference-time failures such as logical inconsistencies or breakdowns in multi-step reasoning (Zhang et al. 2023 Zhong et al. (2024). Detection methods broadly split along these two dimensions. Ap- proaches for data-driven hallucinations often compare outputs against retrieved documents refer- ences (Shuster et al. 2021 | Min et al. 2023 Ji et al. (002). or exploit sampling consistency as in SelfCheckGPT (Manakul et al. 2023). In contrast, methods for reasoning-driven hallucinations rely on signals of inference-time instability, including probabilistic measures such as perplexity (Ren
"gqual contribution.

---

Published as a conference paper at ICLR 2026
Hallucination Emerginn
Hallucination Evolving
Hallucination Evolving (Snowiballing)
Inpul
35-jsar-endd has a J-moonh
Step "night weight dusphome Gallaono), so this is most Lady lyaghocan.
Step lymphonss,
Slep
cough, & soditentional
Recsuse patient systamic R-syuptoos, PEI able scort is warnarzed ofter buopsy:
The likely
wetphh loer, and kooht
the patical neal take lyugh sode boopsy.
lymphhome: ned slaps inchale lymph mode biopsy follewed PET
What's dlagnosis and next slap"
AT Doctor
Humann Doctor
Diagnaste
Halluzination
Iligh suspicion for tunx
Data-driven Hallucination
Reasoning- drrivin
MiSleading distonted dhagtosis lead delayed crootment and presible Guality Grality
cwiczy. with chest X-ray and spatum testing de nixtisisp
Hallucination
Figure 1: An illustration of hallucination emerging and evolving in the context of disease diagnosis.
et al. 2023), length- normalized entropy (Malinin Gales, 2021], semantic entropy (Kuhn et al, 2023). , emergy-based scoring (Liu et al. 2020). and RACE (Wang al. 2025). Others probe internal representations, for example. Inside (Chen et al. 2024), which applies eigenvalue-baid -basd covariance metrics and feature clipping, ICR Probe (Zhang et al. 2025), which tracks residual-stream updates, and Shadows in the Attention (Wei et al.] perturbations. While these methods shed light the mechanisms underlying hallucinations, most remain tailored to single hallucination type and fail to capture their evolution. Yet growing evi- dence indicates that data-driven and reasoning-driven hallucinations often evolve during multi-step generation (Liu et al., 2025 Sun et al., 2025). As shown in Figure I it emerges from an initial dis- ease misclassification and evolves into a distorted diagnosis, delaying treatments and risking fatality. This gap brings two central questions: (1) How can we develop a unified theoretical understand- ing of how hallucinations evolve? (2) How can we detect them effectively and efficiently without relying on external references or task-specific heuristics?
al.] (025). which analyzes representation drift under contextual
To address these challenges, we propose a unified theoretical framework-Hallicinatton Risk Bound, which decomposes the overall hallucination risk into two components: a data-driven term, cap- turing semantic deviations rooted in inaccurate, imbalanced, or noisy supervision acquired during model training; and reasoning -driven term, reflecting instability introduced by inference-tinee time dy- namics, such as logical missteps or temporal inconsistency. This decomposition not only elucidates the mechanism behind hallucinations but also reveals how they emerge and evolve. Specifically, our analysis shows that hallucinations originate from semantic approximation gaps, captured by representational limits of the model, and are subsequently amplified by unstable rollout dynamics, evolving across decoding steps. As such, our framework offers a unified theoretical lens for charac- terizing the emergence and evolution of these hallucinations.
Building on the theoretical foundation, we propose HALLUGUARD. a Neural Tangent Kernel(NTK)- based score that leverages the induced geometry and captured representations of the NTK to jointly identify data-driven and reasoning-drive driven hallucinations. We evaluate HALLUGUARD comprehen- sively across 10 diverse benchmarks, 11 competitive baselines, and 9 popular L.LM backbones. HALLUUGARR consistently achieves state-of-the-ar hallucination detection performance, demon- strating its efficacy. We open-source our proposed HALLUGUARD model at HalluGuard
PRELIMINARIES
Hallucination Detection There are two primary sources of hallucinations in LLMs (j et al. Huang et al. 2025); data-driven hallucination, which stems from incomplete or biased knowl- edge encoded during pre-training or fine-tuning, and reasoning-driver hallucination, which arises from unstable or inconsistent inference dynamics at decoding time. This distinction has implicitly guided broad range of detection strategies, which we examine through these two lenses.
2023
For data-driven causes, a recurring signal is elevated predictive uncertainty. A common formulation adopts the sequence- level negative log- likeeiiohooo::
u(y x. 0)
log po x),,
T
which quantifies the average uncertainty of generating a sequence = .... yrl from input x and 0 denotes model parameters. This directly recovers Perplexity (Ren et al. 2023), where low scores
2

---

Published as a conference paper at ICLR 2026
imply confident predictions, while high scores indicate implausible generations due weak priors. To capture more nuanced uncertainty, later methods extend this formulation to sample settings. The Length-Normalized Entropy (Malinin & Gales| 2021) penalizes dispersion across stochastic where K denotes the number of independent stochastic rollouts sampled from the model for given input, offering finer-grained view of model indecision. This perspective is further enriched by Semantic Entropy (Kuhn et al. (023)), which projects sampled responses into semantic space, and by energy-based scoring (Liu et al. (020), which replaces log- probability with a learned confidence function. Collectively, these methods reflect progression from token-level likelihoods to semantically grounded multi-sample uncertainty estimators.
generations y = [y' .......y' y
In contrast, reasoning-dre -riven hallucinations arise from brittle inference trajectories, where identical contexts may yield inconsistent or incoherent outputs. A commonly used measure of such instability is the cross-sample consistency score:
K
C(' x. 0) = C
Σ
sim(y's ,y),
(2)
where C= K (K-1) and sim(..)) is similarity function such as ROUGE-L (Lin 2004), co- 2
sine similarity, BLEU (Chen et al. 2023). Low scores reflect diverging generations and un- stable reasoning. Several reasoning-driven detection methods can be interpreted through this lens. Early approaches used surface-level lexical overlap metries (Lin al.) (022b), while SelfCheck- GPT (Manakul et al. (023) advanced this by evaluating factual entailment across responses, and FActScore (Min et al. 2023) extended this further by comparing outputs retrieved reference doc-
uments. More recent efforts probe internal signals directly: Inside (Chen et al. (0224) analyzes
the covariance spectrum of embedding representations, and RACE (Wang et al. 2025 diagnoses
instability in multi-step reasoning.
NTK in LLMs. NTK provides a principled framework for analyzing the training dynamics in the overparameterizee regime characteristic of modern LLMs (Jacot et (018). Formally, for network output (x, 9) with input x and parameters 6, the NTK is defined as:
@ +' oof 0) Vof(z'
This kernel ((x, 00) quantifies the similarity of training dynamics between inputs æ and x'. In the infinite-width limit, it converges deterministic value at initialization and remains nearly constant throughout training (Lee et al.] (2020b). This stability reduces the highly nonlinear optimization of deep networks to a tractable kernel regression problem. By examining the eigenspectrum of the NTK, one can probe how internal representations are shaped during training: which features are prioritized (c.g., syntax versus semantics), how quickly different tasks converge, and why overpa- rameterized networks generalize effectively to unseen data (Ju et al., [2022). In this way, the NTK transforms the apparent complexity of LLM optimization into clear lens on how these models capture, process, and generalize information (Zeng et al. 2025).
3 METHODOLOGGY
3.1 PROBLEM SETTING
Our analysis reveals that hallucination not unified failure mode but rather shifts with the task structure. On the instruction-followine Natural benchmark (Wang et al.] (022), 88.9% of the overall 3499 errors are from logical missteps (reasoning-driven, while 11.1 are factual inaceu- racies (data-driven). By contrast, the math-focused MATH-500 (Hendrycks et al., (021), the 1985 wrong generations are dominated by 1946 reasoning errors (98.1%). with only 19 factual flaws (1.9%). This contrast highlights that, in practice, hallucinations rarely pure but often mix- tures of data-driven bias and reasoning-drin -diven instability mustaility-motivaaing our formal decomposition of hallucination sources.
Problem Definition.
Let D' denote the discrete space of all possible finite-length textual token sequences. We define a continuous semantic embedding space Un S Rd equipped with a norm || ||- Each vector Un represents the semantic representation of a reasoning chain composed of step-wise logical
3

---

Published as a conference paper at ICLR 2026
statements. We define task-specific encoder T 3 Un that maps discrete textual sequence into this continuous hypothesis space. In this framework, for an imput x with ground-truth output sequence y" E V, define the target semantic representation as u* with parameters emits a random sequence Y (Yi...., Yr) E Y via the autoregressive decoding distribution po (gve y<et, x). yielding predicted semantic representation un == o(Y) Un. Thus, the model's expected sematic output is defined as [E[un] := B
(vy") Un. An LLM
To analyze inference dynamics, we consider perturbations in local neighborhood of the decoding process. Let IK" denote the r-dimensional continuous space of the model's ' internal states (e.g., prefix embeddings or hidden activations). We parameterize a small perturbation by 8 E R', restricted to a local bz-ball Bр == {ō E R' Ilól|2 I p}. Let Po(. x, 5) denote the perturbed decoding distribution induced by We define the mean semantic response map Gy : IR. + Up. Gy (6) = By-PP(-lxx (-xx))) (o(Y) with its corresponding inference Jacobian J == DGy(0) E Rd Thus, we formally define the problem as follows:
Problem 1 (Hallucination Dynamics Characterization).
Given: (1) The target semantic representation u* := 4 E for a ground-truth output y E Y; (2) the random sequence Y E y emütted via the autoregressive decoding distribution pa Vee, x), yielding a predicted representation un == (YY) with expected value TE. TR(un): and (3) the inference constraints defined by local perturbation ó E R' restricted to the f--ball B,
Find: formal geometric mechanism to characterize how hallucinations emerge and evolve by analyzing the Mean Semantic Response Map Gv (8) and the Inference Jacobian J, which captures the sensitivity of the model's 's reasoning trajectory to internal instabilities.
3.2 HALLUCINATION RISK BOUND
To bridge the formal setup with the phenomenon of hallucination, we first disentangle the sources of hallucinations. Intuitively, hallucinations may arise either from systematic biases in the knowledge encoded by the model driven). The following proposition formalizes this idea by decomposing the total hallucination risk into two components.
or from instabilities during autoregressive decoding (reasoning-
We first impose the following assumptions:
A1. (Un, || II) is a finite-dimensional Hilbert space. The encoder + : ) - U, is measurable, and the random variable ((Y) has finite second moment under the model's unperturbed decoding distribution: Ey [+] < so. This ensures that the mean semantic representation [E)( is well-defined in Un.
A2. Let (J), dy) be the discrete metric space equipped with edit distance. The encoder + : (V, (yy + (Un, Il) is Li-Lisschitz continuous: [ - #(y))l s
La dy v') "g.v'E
A3. For any perturbation 8 in the closed ball B, = {5 RT |8112 S p}. the mean se- mantic response map Gr) = Ey-Px(-1x.6) [x{()] is twice Fréchet differentiable in neighborhood of 8 O and admits the expansion Gy = Gy (0) + J8 + R(0). where 2 H. 11 45 E B,. for
J = DGy (0) dRd xr and the remainder satisfies || |[()] S
some constant H, > 0.
Proposition 3.1 (Hallucination Risk Decomposition). Under Al-A3, applying the triangle inequality yields natural split of the risk:
| ual |l
< ||u - E| Emll]]
+ || - E(u,]]
data
reasoning
This decomposition distinguishes errors caused by systematic bias in the learned representation from those introduced during stochastic rollout.
Characterizing Data-Driven Hallucination. To quantify the data-driven term, we take inspira- tion from the NTK, which has proven effective in analyzing training dynamics of overparameterrzzed models. Here, NTK geometry provides a way to measure how well the model's 's representation space aligns with task generation under small perturbations.
4

---

Published as a conference paper at ICLR 2026
Let Up, denote the hypothesis subspace accessible to the model under perturbations. By Céa's be bounded
lemma Céa 1964) with curvature penalty, the data-driven
A inf I|u" ull. 7 WEU
||u - E|ua]l
(4)
where Amin (Ka)) is the smallest eigenvalue of the NTK Gram matrix embedded pertur- bations Ksp, and A S ||T]], where Un, - Un denotes the operator mapping. Intuitively, the ratio measures the conditioning of the feature map: well-conditioned NTK spectra allow a closer approximation to the true generation.
Thus, the ratio can be further controlled in terms of pretraining-fieet mismatch:
A
Apt logO L) + k Emismatch
7
Signal,
where logO(P,,) L) complexity term from parameter count P and prompt length L, Emiamatch denotes the Wasserstein distance between prompt and query distributions, Signal, measures task- aligned energy in the top-k eigenspace. kpp and & are task and model-dependent constants. Thus, data-driven hallucinations grow when the mismatch is large or when the task signal is weak.
captures reasoning-diie instability that accumulates during autoregressive decoding. Here, we model gen- cration as a martingale process, where deviation from the expectation is controlled by concentration inequalities. Specifically, Freedman's 's inequality (Geman et al., 1992) gives
Characterizing Reasoning-Driven Hallucination. The reasoning-driven
luA Elua]ll S K expp
Kr²
a(e 1),
(6)
where K is the number of rollouts averaged, 3 summarizes per-step growth in local Jacobians, scales the cumulative effect and C is a task and model-dependent constant. This bound shows that -orivenng-rivenn
reasoning- driven hallucinations grow exponentially with sequence length T.
We now synthesize the two components into a unified result that characterizes the overall risk of hallucination. By combining the NTK-conditioned approximation bound for data-driven deviation with the Freedman-style concentration bound for reasoning-drive instability, we obtain the follow- ing unified bound of data-driven and reasoning- driven hallucinations (detailed proof is provided in Appendix A):
Theorem 3.2 (Hallucination Risk Bound). Let u² := (2y") denote the semantic embed- ding of the ground-truth output and ut o(Y) that of the model-generated output. Under
Assumptions A1-A3, suppose there exists 3 such that IT Jell, eT Then the total hallucination risk satisfies
||u--wu -u <
[ + kpt log O(P,L) + k
"mismaachh Signaly
inf ||u" ull + [CI expp ke? a(e uEU,
data-driven
reasoning drives term
Here, |.]| denotes the total sampled trajectories.
3.3 HALLUCINATION QUANTIFICATION VIA HALLUGUARD
While Theorem 3.2 makes explicit how data-driven and reasoning-driven hallucinations emerge and evolve, applying it directly at inference is impractical since direct step-wise Jacobians for billion- parameter LLMs are intractable, our decomposition.
seek proxy score that computable, stable, and faithful to
Let K. denote the NTK Gram matrix with eigenvalues X ... 0 and condition number re(C)) = /Amin- Let Je be the step-t input-output Jacobian of the decoder, and define Fmax = sup, |J,ll2 the uniform spectral bound(note that max is independent of the spectrum of K.).
Under Assumptions A1-A3, a standard NTK approximation argument yields infaet, I|u" u/ S
Ca -et ||u*]| that (et(KC) capture the representations in systematic bias.
5

---

Published as a conference paper at ICLR 2026
| J 111
For autoregressive rollout, based on the property of Jacobian, we have
Ih ||Jell2 - exp E log [ellz) so that we have [ Jell
ex Since
3 S log mmax with o max = sup, Jelle thus we have the upper bound as || I Jella I o T = ellog emax)l Thus, log max serves as a stable and tractable proxy for the per-step amplification rate.
Perturbation analysis of K. together with classical eigenvalue sensitivity results (Trefethen & Bau 2022). yields Var[us, I Ge m((C)² ||6]|, showing that instability grows quadratically with the condition number x(CC). To temper this effect and ensure additivity, we penalize ill-conditioned representations via - log x², where log compression brings well-behaved dynamic range.
In summary, det(K) quantifies representational ad- equacy, log max captures rollout amplification, and -lg x² penalizes spectral instability, together forming compact and tractable proxy consis- tent with the Hallucination Risk Bound. The lightweight projection layers are self-supervised spectral calibration modules, optimized offline (via AdamW) to align NTK spectral properties across heterogeneous backbones into a stable, comparable geometric without hallucination labels or task-specific supervision, with the backbone fully frozen and zero runtime overhead during infer- ence. Detailed proofs are provided in Appendix
Table 1: Correlation between NTK proxies and task families.
SQuAD Math-500 TruthfulQA
del(IC)
0.84
0.42
0.61
log Fmax log x²
0.39
0.88
0.67
Empirical validation. We empirically validate how those proxies correlate with different task families. In Table D (eC correlates most strongly with the data-centric task SQUAD (0.84), in- dicating its role in capturing factual fidelity. In contrast, for the reasoning-oriented MATH-500, the highest correlation is observed with log max log n² (0.88), reflecting the importance of amplifi- cation and stability in multi-step reasoning.
Motivated by the above, we formally define HALLUGUARD follows, which provides principled and unified lens for hallucination detection:
HALLUGUARD((u))
dct((C) + log Fmax
log x²
4 EXPERIMENTS
We comprehensively evaluate HALLUGUARD across 10 diverse benchmarks, 11 competitive base- lines, and popular LLM backbones. We aim evaluate its efficacy from the following five ques- tions: QI: How does HALLUGUARD perform across different task families? Q2: How does HAL- LUGUARD perform across LLMs of different scales? Q3: How does each term capture trends across task families? Q4: Can HALLUGUARD guide test-time inference to improve downstream reason- ing? Q5: How well does HALLUGUARD generalize to detecting fine- -raingd hallucinations beyond benchmarks?
evaluates HAL.LUGUARD as a detection m1-h)d)Q1-Q3), Section applies HALLUGGARD in score-guided inference (Q4) and Section 4.4 analyzes HAL- LUGUARD on fine-grained hallucination via a case study on semantic data(Q5).
Section 4.1 details the setup; Section 4.2
4.1 EVALUATION SETUP
Benchmarks. We evaluate across 10 widely used benchmarks spanning three distinct categories. For data-grounded QA, we include RAGTruth (Niu et al.] 2024). NO-Open (Kwiatkowski et al.) 2019), Hotpot (Yang et al. 2018) and SQuAD (Rajpurkar et al. 2016), which emphasize factual correctness through external evidence. For reasoning-oriented tasks, we use GSM8K (Cobbe et al.) 2021], MATH-500 (Hendrycks et al. 2021). and BBH (Suzgun et al. 2023), which require multi-step derivations prone compounding errors. Finally, for instruction follwwiinnn.ooowingg settings, consider Truthful0: (Lin et al. 2022a). HaluEval (Li et al.] (202a) and Natural (Wang et al. 2022). which probe hallucinations under open-ended adversarial prompts.

---

Published as a conference paper at ICLR 2026
Baselines. We compare HALLUGUARD with 11 competitive detectors spanning diverse strategies. Uncertainty-bot based methods include Perplexity (Ren et al.
2023). Length-Normaliza Predictive
Entropy(LN Entropy(LN-EEnropy (Malinin & Gales, 2021). Semantic Entropy (Kuhn al. 2023 Energy
Score (Liu et al. 2020) and P(true) (Kadavath et al. 2022). Consistency-basee approaches cover
SelPCheckGPT Manakuul et al. 2023). Lexical Similarity (Lin al. 2022b), FActScore (Min al.,
2023
and RACE Wang et al. (2025) Internal-state methods are represented by Inside (Chen et al.
2024
and MIND (Su et al.) 2024).
LLM Backbone Models. We evaluate publicly available LLMs spanning different scales and architectures. These include five models from the Llama family (Llama2-75 Llama2- -3Bn Llama2- 70B, Llama3-8B, and Lamma,22-3B) (Touvron et al., 2003 Grattatiori et al., (2024), along with OPT- 6.7B (Zhang et al., 2022). Mistral-7B- 7 (Jiang et al.) 2023), QwQ-32B (Yang et al. (024). and GPT-2 (117M) (Radford et al.) 2019). All models are used in their off-the-shelf form with pre-trained weights and tokenizers provided by Hugging Face, without further fine-tuning.
Evaluation Metrics. We evaluate hallucination detection ability under two regimes following Ja- niak et al. (2025): ROUGE-based reference evaluation (x,,) and LLM-AS-A-JUDGE (+1m). For performance measures, we report the area under the receiver operating characteristic curve (AU- ROC) and the area under the precision-recall curve (AUPRC), AUROC is widely used to assess the quality of binary classifiers and uncertainty estimators, while AUPRC highlights performance under class imbalance. In both cases, higher values indicate better detection.
4.2
MAIN RESULTS
Q1: How does HALLUGUARD perform across different task families? To evaluate how HAL- LUGUARD performs across different task types, conduct experiments all benchmarks. For clarity, Table 2 presents representative results from three task families: data-centric (RAGTzuth), reasoning-oriented (Math-500), and mnstruction-followee
(TruthfulQA). As shown, HAL- LUGUARD consistently outperforms all baselines across backbones. On Math-500, it reaches 81.76% AUROC and 79.76% AUPRC, improving On RAGTruth, it attains 84.59% AUROC and 81.15% AUPRC, with gains of up to 7.7%. On TruthfulQh, it achieves 77.05% AUROC and 73.79% AUPRC, exceeding the next strongest base- line by as much as 6.2%. Overall, HALLUGUARD establishes new state-of the-art results across di- verse task families, with particularly pronounced improvements on reasoning
the second-best method by up 8.3%.
eassoningooreented oriented benchmarks.
Q2: How does HALLUGUARD perform across LLMs of different scales? We fur- ther investigate whether the effectiveness of HALLUGUARD depends on model scale, as smaller backbones are typically more prone hallucination. Table [ reports representa- tive results small Llama3-8B), mid-sized((Llama-133B) and large-scale(llamaz 70B) models using SQuAD, GSM8K, and HaluEval. Across all settings, HALLUGUARD consistently surpasses baselines, with the largest margins smaller models-for instance, 72.89% AUPRC, on HaluEval with Llama2-7B.
more than 10% above the second best. Mid- sized models also exhibit clear gains (e.g.. 79.01 % AUROC, on GSM8K), while even large-scale mod- els like Llama2-70B see steady improvements (e.g.,
SQuAD
AUROC(S)
83.8% AUROC, SQuAD). Overall, HALLU- GUARD benefits most on small backbones while maintaining consistent advantages across scales.
Math-500 AUROC(S)
Q3: How does each term capture frends across task families? As shown in Figure each term faithfully tracks the ground-truth trend within its re- spective task family. On data-centric SQuAD, the data-driven term closely follows the dashed gold curve across the variant hallucination rate, capturing the smooth AUROC decline. On reasoning oriented orrieted MATH-500, the reasoning- driven term mirrors the monotonic AUROC drop reasoning drift in-
20
30
50
Halliciaaiion
Label
Data-driven Only
Reasocing drives Tera Only
Figure 2: Ablation results comparing in- dividual terms with ground-trutt trends on SQuAD (top) and Mat Math-500 (bottom).

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison on representative benchmarks: cassoning-orienter (Math and instruction-following (TruthfulQA), We highlight the first and second best results.
data-centric (RAGTruth)
GPT2
OPT-6,7B
Mistral-7B
QwQQ-2B
AUROC,
AUPRC,
AUROCaa
AUTRO
AUROC,
AUPRC,
AUROC
AUTRO Cas
AUROC
AUPRC-
AUROC
AUROC+
AUPRC,
AUROCae
AU Cin
HALLUGUARD
75.51
73.40
62.40
56.60
80.13
76.77
71.01
63.58
82.31
80.79
64.89
67.25
84.59
81.15
71.82
66.68
Inside
73.42
73.08
61.99
56.39
79.49
7182
66.1
62.46
75.32
19
64.58
61.05
77.72
73.47
66.05
64.73
MIND
58.54
54.79
43.47
41.85
63.82
62.58
51.03
44.78
73.13
71.53
58.25
58.6
64.23
63.06
47.37
51.47
Perplexity
58.07
56.68
43.84
41.53
64.47
61.57
47.12
52.98
65.42
63.63
53.28
51.36
73.91
72.92
59.77
RAGTruth
LN-Entropy
64,42
60.79
49,41
45.04
60.81
57.91
48.76
42.27
64.22
60.92
52.24
48.41
63.81
62.26
47,52
52.17
Energy
65.53
62.42
51.8
47.22
66.54
63.28
54.21
49.19
64.36
62.26
48.64
53.93
73.26
71.21 65.43
62.32
Semantic Ent.
60.72.
59.41
50.55
45.86
70,2
68.34
54.54
56.74
66.01
64.49
53.01
55.5
66.48
64.41
51.54
50.11
Lexical Sim.
64.72 63.1
55.04
48.04
67.28
64.62
52.55
54.86
64.96
61.17
52.34
45.11
70.87
67.41
61.25
51.01
SelfCheckGPT
65.4
62.79
52.85
52.43
66.64
64.89
$2.69
51.17
71.19
68.45
63.13
60.23
65.79
62.45
54.76
51.29
RACE
64.83 62.84
51.8
48.44
64.26
61.03
52.74
46.22
66.34
64.54
51.88
53.86
71.13
57.58
55.54
P(true)
66.19
64.04
48.2
56.27
68.44
65.48
57.53
53.08
72.54
71.8
57.25
59.42
65.32
63.01
53.01
52.12
FActScore
65.72
64.39
51.94
47.51
61.53
58.2
51.86
45.57
63.98
60.71
53.54
49.34
66.72
64.03
58.21
49.17
HALLUGHUARD
67..44
62.05
59.05
73.1
70.88
63.67
61.88
79.85
76.5
67.13
60..7
81.76
79.76
68.77
65.46
Inside
66.18
66.81
56.15
58.62
20.64
65.22
63.28
59.28
67.2
65.49
51.3
53.46
80.8
21.49
64.05
63.42
MIND
55.41
51.77
39.01
41.59
55.48
53.46
38.59
40.88
65.71
63.7
49.61
52.54
61.75
60.18 53.46
50.04
Perplexity
53.28
50.22
43.86
38.98
64.89
62.
48.65
$1.99
61.97
60.05
$1.15
42.87
60.28
57.75 51.62
43.38
LN-Entropy
60.84
58.76
42.76
47.48
58.71
55.01
43.55
42.02
68.96
69.44
58.79
57.49
63.96
62.18 46.01
49.5
Energy
$5.09
51.99
46.2
39.5
53.96
50.98
42.56
34.12
66.27
62.72
49.48
50.06
69.61
68.66 54.35
57.36
Semantic Ent.
58.16
54.81
49.61
40.39
62.63
59.52
50.14
45.02
64.99
61.33
50.11
45.53
62.76
60.95 45.77
45.75
Lexical Sim.
51.37
47.18
38,37
39.06
61.27
58.06
44.13
42.96
58.25
55.92
46.31
46.01
69.46
67.59 55,93
52.6
SelCCheckGPP
54.51
51
44.62
44.01
57.36
53.21
42.55
38
63.68
62.5
51.7
53.03
64.56
62.49 55.85
45.8
RACE
55.99
54.66
41.39
38.32
64.23
62.03
56.03
53.44
66.88
64.33
49.57
48.5
59.5
55.83
46.13
41.07
P(true)
54.57
52.88
45.45
44.74
$7.02
55.49
48.81
37.84
$7.11
55.21
43.93
47.05
61.49
59.03 44.37
44.69
FActScore
56.76
53.85
40.25
40.01
54.51
53.2
38.45
36.49
62.11
58.64
53.52
47.27
58.82
57.47
49.48
42.74
HALLUGUAARD
72.1
52.01
69.59
58,52
$2.65
77.05
73.79
63.62
62.26
74.26
72.76
64.07
lnside
70.42
68.76
60.09
52.01
62.1
59.78
51.07
$1.38
62.53
60,99
$2.3
49.35
70.89
64.44
56.61
56.01
MIND
59.45
56.79
45.22
43.71
60.56
58.55
47.49
49.63
59.2
57.98
47.23
41.79
62.81
61.5
52.56
46.37
Perplexity
50.57
47.87
40.64
35.63
55.07
52.26
44.43
42.79
60.8
59.69
47.33
41.62
25.29
43.95
43.92
TruthfuQQA
LN-Eattrpy
58.04
56.99
41.94
47.21
12
54.01
47.06
38.4
59.67
56.25
41.99
41.25
60.76
58.21
46.24
42.64
Energy
55.02
53.31
38.78
45.16
54.42
5188
36,21
42.57
58.93
55.25
50.76
41.72
64.15
61.32
51.78
50.02
Semantic Ent.
57.08
43.35
51.48
47.81
54.44
36.62
40.35
66.75
63.85
46.71
Lexical Sim.
52.54
50.56
39.94
33.42
59.74
55.72
49,89
46.81
66.16
64.05
54.08
51.65
55.24
51.36
46.39
39.57
SelCCheskGiP
56.04
54.48
43.78
44.38
58.93
56.47
47.65
39.02
61.14
58.91
42.97
47.01
55.86
54.95
41.08
37.35
RACE
53.02
50.33
33.81
62.95
67.89
54.61
51.93
71.06
60.4
57.44
55.75
43.19
P(true)
55.52
53.41
38.33
38.38
54,88
53.1
38.22
40.96
55.8
52.01
40.88
38.72
57.18
55.16
46.19
38.21
FActScore
53.82
51.42
41.33
35.2
54,57
51.26
42.51
35.52
53.97
50.2
42.97
36.16
62.31
60.23
45.06
49.9
creases. These results show that each term is well matched to its task family and faithfully tracks performance trends as hallucination rates rise.
4.3 TEST-TIME INFERENCE
Test-time reasoning remains challenging, as models need to generate coherent multi-step solu- tions without drifting into errors. To assess whether hallucination detection can mitigate this dif- ficulty, we integrate detectors into beam search and evaluate Qwen2.5-Math-71 on MATH-500 and Llama3.1-8B on Natural. As shown in Table 4 HALLUGUARD achieves the strongest gains: on MATH-500, it reaches 81.00% accuracy, around 10% higher than I0 Prompt; Natural. it at- tains 70.96%, exceeding IO Prompt by 15.72%. These results demonstrate that HALLUGUARD not only detects hallucinations but also strengthens test-time reasoning by guiding models toward more reliable solutions.
4.4 CASE STUDY
Fine-grained hallucinations-lexicall similar yet semantically incorrect outputs-pose a particular challenge for detection. To evaluate whether HALLUGUARD can comprehensively capture such subtle emrors, we use the PAWS dataset (Zhang et al.) 2019), which contrasts paraphrases with high surface overlap but divergent meanings. Following Li et al. (2025), we adopt ROUGE- refer- ence signals for evaluation (Table 5). Across model scales, HAL.LUGUARD consistently surpasses
8

---

Published as a conference paper at ICLR 2026
Table 3: Performance comparison across backbone scales (small, mid-sized, and large) on three benchmarks: SQuAD, GSM8K, HaluEval. We highlight the first and second best results.
Laama27BB
Llama-3-8B
Llama2-138
Llama2-70B
AUROCY
AUPRC-
AUROC
AUPRIC tee
AUROCY
AUPRCC
AURDC Cro
AUPRiCa
AUROCT
AUPIC-
AUROC
AUPRCit
AUROC,
AUTFCC
ADROC
AUPRCite
HALLUGUARD
77.16
71.18
64.38
79.56
78.29
67.97
63.27
81.45
78.39
64.39
65.07
83.8
81.77
70.46
73.24
Inside
73.63
75.74
22
59.11
76.
72.44
65.62
62.9
74.68
74.81
61.01
59.51
81.24
75.09
69.48
62.4
MIND
64.57
61.11
52.39
53.13
62.29
59.58
44.49
48.61
68.64
66.95
54.92
52.49
73.46
71.71 57.76
56.77
Perplexity
63.93
61.77
46.97
48.2
70.51
67.51
55.71
$2,68
70.
69.22
60.33
54.82
74.23
70.88
62.24
58.05
SQuAD
L.N Entropy
65.96
53.43
52.84
63.7
60.4
42.85
61.66
59.16
49.05
46.27
72.44
68.91
52.63
Energy
59.83 56.11 46.19
43.18
64.41
61.02
56.17
46.21
61.02
59.73
48.26
42.08
69.01
66.19 58.44
49.82
Somantic Ent.
60.29 57.73
43.63
48.83
66.52
62.62
52.37
52.7
70.58
67.22
53.31
52.94
72.01
68.51 56.49
50.9
Lexical Sim.
70.31
69.08
53,97
53.31
66.43
63.56
53.19
50.96
68.53
67.42
50.73
54.12
68.95
67.91
60,52
56.56
SeltCheckGPPT
68.26 67.09 60.06
57.31
73.99
72.15
65.26
54.02
65.47
61.65
53.12
49.89
73.07
70.49
56.59
54.65
RACE
71.35 69.23
59.18
54.73
68.17
66.02
54.65
53.06
64, 19 60.45
47.53
45.66
64.05
62.39
54.38
50.07
P(true)
62.55
46.84
52.32
67.42
63.94
47.52
71.56
57.51
45.66
62.71
46.85
FActScore
70.32
68.63
58.13
53.01
71.2
69:45
61.92
54.91
66.65
63.2
56.41
53.42
68.33
65,26
56,93
48.46
HALLUGUARD
75.89
72.83
62.29
63.46
75.2
72.9
63.62
61.79
79.01
76.73
64.38
64.97
77.33
73.97
60.48
61.26
Inside
74.61
68.35
58.57
62.58
73.73
67.51
56.02
57.28
75.79
76
60.91
59.77
72.3
72.26
54.49
58.39
MIND
65.88
63.4
48.28
48.17
66.57
65.55
48.84
53.4
61.49
59.55
51.63
$1.45
66.41
63.44
52.05
53.57
Perplexity
66.23
53.52
52.31
57.61
53.63
41.59
60.96
46.27
47.44
64.32
62.81
GSM8K
LN-Entropy
59.45
55.95
43.04
44.08
68.22
66.05
53.03
53.21
61.31
58.90
45.83
40.86
61.81
60.46
44.5
44.76
Energy
58.15
54.71
43.65
36.71
59.79
56.52
50.31
42.23
$7.58
56.07
43.39
38.94
65.27
62.94 $2.8
46.6
Somantic Ent.
57.95
54.68
42.78
41.95
66.9
64.81
50,47
55.36
62.72
59.09
49.33
44.35
60.63
57.01 46.22
40.24
Lexical Sim.
65.8
63.7
52.12 54.07
63.29
59.87
53.17
50.02
63.83
60.20
54.43
44.82
63.27
59.41
47.42
47.38
SelfCheckGPT
60.99
57.54
49.28
44.43
65.72
62.01
54.49
50.34
57.98
54.58
46.72
39.86
68.06
65.09 52.99
50.89
RACE
63.37
62.33
53.53
49.94
64.49
61.47
53.28
47.55
64.20
50.15
45.35
68.35
66.66
50.41
51.16
P(true)
65.95
63.63
54.95
48.25
62.59
58.88
47.21
42.2
67.08
65.60
53.66
55.12
60.16
58,
47.73
49.49
FActScore
56.69
53.71
45.78
39.52
65.69
61.95
53.69
46.06
55.76
54.17
44.91
43.18
59.84
55.85
44.05
39.49
HALLIIGUARD
75.72
72.89
66.65
63.15
73.43
71.19
64.95
54.8
78.15
74.15
65.39
61.14
80179
79.54 67.68
68.51
Inside
21.33
67.63
59.73
53.15
67.95
64.93
60.31
52.21
72.01
71.97
$6.51
60.64
74.62
68.33
62.22
64.4
MIND
54.8
51.43
44.15
43.34
64.54
60.89
49,09
43.13
55.05
53.28
39.16
45.17
57.98
56.01 45.82
41.69
Perplexity
54.02
52.53
36.76
40.51
61.31
59.36
50.62
46.01
$4.99
51.39
42.64
35.64
62.85
60.59
48.29
43.85
HaluEval
LN-Entropy
59.47
58.33
50.2
46.91
64.89
60.72
51.78
46.39
68.18
63.53
49.70
48.09
60.16
58.89
50.29
48.42
Energy
62.29
59.6
50.68
42.24
62.74
61.61
50.17
52.01
60.54
59.04
43.53
50.37
63.13
58.44
48.79
48.01
Semantic Ent.
59.39
55.94
48.53
46.35
$5.25
53.05
44.5
44.35
59.44
57.72
45.38
40.77
61.57
57.99
49.07
45.39
Lexical Sim.
63.61
61.16
55.01
44.75
56.59
55.39
44.45
45.57
53.46
52.06
41.34
40.57
64.37
60.92 54.29
50.86
SelCCheckGiP
64.29
61.83
48.4
45.49
65.44
63.13
57.02
48.23
65.24
63.52
53.71
54.33
57.12
55.26
40.5
43.06
RACE
59.78
59.14
48.1
40.47
61.98
60.32
48.08
46.29
60.65
59.11
49.92
44.51
62.11
58.24
40.5
43.06
P(true)
57.46
40.47
56.32
42.55
43.75
65.77
63.01
49.98
45.47
55.75
54,94
43.97
FActScore
63.93
61.33
46.9
51.87
61.73
57.85
49.92
42.15
65.15
63.71
55.98
54.61
62.66
60.3
53.13
46.42
Table 4: Performance of hallucination score-guides test-time inference across reasoning tasks. We
highlight the first and second best results.
Dataset
ID Prompe
Ours
Inoide
MIND
Poplesity
I.N- Entroyy
Fnergy
Samantic Bar.
SeliChook-
RACF
P(trae)
FActScone
MATH-500
72.70
81.00
74.90
77.10
77.10
76.20
78.00
72.50
67.10
60
Natural
55.24
70.96
67.42
68.32
67.51
68.04
68.59
68.10
65.68
66.90
68.16
67.74
baselines: it achieves 90.18% AUROC and 87.64% AUPRC on Llama2-70B, and 91.24% AUROC and 88.53% AUPRC on QwQ-32B-Q ewQ-32B-exceeding the next-best method by nearly five points. Even on GPT-2, it leads with 83.27% AUROC and 80.46% AUPRC. These results confirm HALLUGUARD's effectiveness in capturing fine- reeergrinedd semantic inconsistencies beyond benchmark settings.
Table 5: Results PAWS measuring semantic hallucination detection with Llama-3.2-3B. Llama2- 70B, and QwQ-32B. We highlight the first and second best results.
Method
Ours
Inside
MIND
Perplesity
LN-- Entropy
Energy
Semantic Fat.
Letical Sim.
SefCCeek- GPT
RACE
P(tnie)
FActScare
1.ama3.2
AUROC 85.63
80.46
78.93
71.27
72.19
73.05
75.11
64.58
77.82
79.47
73.56
68.44
AUPRC
$2.14
77.28
75.41
67.55
68.34
70.22
72.41
59.67
73.41
76.28
70.43
63.58
IJama2
AUROC
90.18
85.47
83,92
76.23
77.14
79.06
68.35
82.71
84.26
77.39
72.62
AUPRC
87.64
82.38
81.06
71.42
72.59
74.28
76.32
63.44
78.89
81.73
74.18
67.58
QwQ
AUROC
91.24
85.41
84,56
76.72
77.43
78.29
80.42
69.54
83.59
86,38
78.53
73.46
AUPRC
88.53
82.27
81.37
72.63
73.29
75.44
78.18
64.27
79.42
83.41
75.21
68.32

---

Published as a conference paper at ICLR 2026
5 RELATED WORK
In this section, we review prior hallucination-derrcction methods by their detection target-Data- driven hallucinations and reasoning-driven hallucinations.
Detecting Data-Driven Hallucinations. Recent work has shown that internal activations encode rich indicators of such flaws. Chen al. (2024) proposed EIGENSCORE, which computes statis- tics of hidden representations from the eigen matrix to estimate hallucination risk. [S et al. (2024) introduced MIND, an unsupervised detector that models temporal dynamics of hidden states with- out requiring labels, along with HELM benchmark to enable standardized evaluation. AAaria & Mitchell (023) demonstrated using linear probes on intermediate states to predict truthfuiness.
Detecting Deasoning-Driven Hallucinations. There are other works targeting inference-time in- consistencies during generation-such as logical errors, instability across decoding steps, or tempo- ral drift in extended outputs. Manakul et al. (023) proposed SELFCHECKGPT, which assesses self-consistency by sampling multiple candidate generations and measuring their alignment using entailment and lexical overlap. Kalai & Vempala (2024) introduced suite of calibration based un- certainty scores designed to capture hallucination risk directly from output distributions. Ding et al. (2025) proposed REACTSCORE, which integrates entropy with intermediate reasoning traces to de- tect failures in multi-step decision-making. FACTSCORE (Mín et al. (023) decomposes outputs into atomic factual units and verifies each against retrieved passages using entailment- scoring.
6 CONCLUSION
The reliability of LLMs is often undermined by hallucinations, which arise from two main sources: datu-driven, caused by flawed knowledge acquired during training, and reasoning-driven stemming from inference-time instabilities in multi-step generation. Although these hallucinations frequently evolve in practice, existing detectors usually target only one source and lack a solid theoretical foundation. To address this gap, propose a unified theoretical framework-a Hallucination Risk Bound, which formally decomposes hallucination risk into data-driven and reasoning ponents, offering principled view of how hallucinations emerge and evolve during generation. Building on this foundation, we introduce HALLUG UARD, a NTK-based score that measures sensi- tivity to semantic perturbations and captures internal instabilities, thereby enabling holistic detection of both data-driver and reasoning driven ballucinations. We evaluate HALLUGUARD across 10 di- verse benchmarks, 11 competitive baselines, and popular LLM backbones, where it consistently achieves sfte-of--thearr performance, demonstrating robustness and practical efficacy. Looking forward, leveraging HalluGuard's sensitivity to error propagation offers promising pathway for developing prognostic indicators in interactive multi-turn dialogues, enabling systems to predict and preempt hallucinations before they fully manifest.
caasoning-driven driven com-
REPRODUCIBILITY STATEMENT
We have taken several measures ensure the reproducibilityy of work. A complete description of the theoretical framework, including the formal assumptions and proofs of the Hallucination Risk Bound, is provided in Section 3 and Appendix A Detailed experimental settings and evaluation protocols are documented in Section 4 and Appendix C.T covering all 10 benchmarks, 11 base- lines, and LLM backbones. Together, these resources ensure that both our theoretical claims and empirical results can be independently validated and extended by the community.
ETHICS STATEMENT
This study is based exclusively on publicly available datasets and open-source large language mod- els, and does not involve human subjects or the use of private data. All scientific concepts, method- ological designs, experimental implementations, and resulting conclusions remain entirely the re- sponsibility of the authors.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
We thank the anonymous reviewers for their constructive comments. This work is supported by the National Science Foundation under Award No. IIS-2339989 and No. 2406439, DARPA under contract No. HR001 12490370 and No. HR001 124S0013, U.S. Department of Homeland Security Amazon- -iizzonniiigiiaa Tech Initiative for Efficient and
under Grant Award No. 17STCIN00001-08- Robust Machine Learning, Amazon AWS, Google, Cisco, 4-VA, Commonwealth Cyber Initiative, National Surface Transportation Safety Center for Excellence, and Virginia Tech. The views and conclusions are those of the authors and should not be interpreted as representing the official policies of the funding agencies or the government.
REFERENCES
Amos Azaria and Tom Mitchell. The internal state of an I1m knows when it's lying. In Findings of the Association for Computational Linguistics: Conference on Empirical Methods in Natural Language Processing(EMNLP) 2023.
Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Ste- fano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Pe- ter Henderson, John Hewitt, Daniel Ho. Jenny Hong. Kyle Hsu, Jing Huang. Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Ece Kamar, Michal Kosinski, Ryan Chi-Ying Hsich, Drew A. Linsley, Long O. Mai, Nikolay Manchev, Christopher D. Manning. Yian Yin, Christopher J. N. de M. L. Matthews, Lu- cia Mondragon, Ognjen Oreskovic, Mark Sabini, Yusuf Sahin, Clark Barrett, Christopher Potts, James Y. Zou, Jiajun Wu, and Percy Liang. On the opportunities and risks of foundation models. ArXiv, 2021.
fm. stan edu/ wwtp://sses/s//ss rww.rt.pprrpppp.
Jean Céa. Approximation variationnelle des problèmes aux limites. In Annales de I l'institut Fourier, volume 14, pp. 345-44- 444, 1964.
Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Lims' internal states retain the power of hallucination detection. International Conference on Learning Representations//CLR) 2024.
Yuyan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, and Yanghua Xiao. Hallucination detection: Robustly discerning reliable answers large language models. Conference Information and Knowledge Management( CIKM), 2023.
Lenaic Chizat, Edouard Oyallon, and Francis Bach. On lazy training in differentiable programming. Conference on Neural Information Processing Systems(Neur/PS), 2019.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rejichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers solve math word problems. 2021. URL https://arxiv.
at///bb//2110 14168
Fabio Dennstädt, Janna Hastings, Paul Martin Putora, Max Schmerder, and Nikola Cihoric. Im- plementing large language models in healtheare while balancing control, collaboration, costs and security. NPJ digital medicine, 8(1):143, 2025.
Yue Ding, Xiaofang Zhu, Tianze Xia, Junfei Wu, Xinlong Chen, Qiang Liu, and Liang Wang- Drhscore: Reasoning-awarc hallucination detection via semantic breadth and depth analysis in 1lms, 2025. UtL URL /ttpppps:///axvv.... org.
abs abs 2509. 11569
Stuart Geman, Elie Bienenstock, and René Doursat. Neural networks and the bias/variance dilemma. Neural computation, 4(1):1-58, 1992.
11

---

Published as a conference paper at ICLR 2026
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar. Artem Ko- renev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes. Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang. Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind That- tai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Kore- vaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arricta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang. Jiecao Yu, Joanna Bitton, Joe Spisak, Jong- soo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeand Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Ku- mar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoy- chev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ra- mon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Ro- hit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Scohyun Sonia Kim, Sergey Edunov, Shaoliang Nic, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mi- haylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do. Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang. Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang. Yaclle Gold- schlag, Yashesh Gaur, Yasmine Babaci, Yi Wen, Yiwen Song. Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, An- drew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, An- nie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leon- hardi, Bernie Huang. Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Mon- talvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery. Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smoth- ers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harri- son Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj,
12

---

Published as a conference paper at ICLR 2026
Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Saan-Bptistte Gaya, Jeff Marcus, Jeff Tang, Jen- nifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Jun- jie Wang. Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang. Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz. Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Ro- driguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu. Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov. Satadru Pan. Saurabh Mahajan, Saurabh Verma. Seiji Yamamoto, Sharadh Ra- maswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang. Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satter- field, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk. Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Ku- mar, Vishal Mangla, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiao- jian Wu, Xiaolan Wang. Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary Vito,, Zef Rosnbrick. Zhao- duo Wen, Zhenyu Yang. Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024. URL https://arxins https/////xxiv.orr/aab.s.. wwt. 2407. 217733
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. Confer- ence on Neural Information Processing Systems(Neur(PS), 2021.
Lei Huang, Weijiang Yu, Weitao Wang, Yujia Wang, Shi-Qi Chen, and Ju-Hua Wang. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 2025.
Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. Efficient attentions for long document summarization. In The North American Chapter of the Association for Computa- tional Linguistics: Human Language (echm)logiess/AACCL), pp. 1419-1436, Online, June 2021. Association for Computational Linguistiic(AACL. doi: 10.18653/v 1/202 wtw.://..an..nnannnniiaaaa .naacl-main. 112. URL https: ac http://aaaaaaacooo.orr////////// nacci 112
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. Conference on Neural Information Processing Nystem)(NearIPSS) 2018.
Denis Janiak, Jakub Binkowski, Albert Sawezyn, Bogdan Gabrys, Ravid Shwartz-Ziv, and Tomasz Kajdanowicz. The illusion of progress: Re-evaluating hallucination detection llms. Conference on Empirical Methods in Natural Language Processing(EMNLA 2025.
Ziwei Ji, Naycon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM
13

---

Published as a conference paper at ICLR 2026
Computing Surveys, 55(12):1-38, March 2023. ISSN 1557-7341. doi: 10.1145/3571730. URL
http://dx: wttpp//.w.........ooo........... 3577173
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https: wawxiv.orggabs////.....
Peizhong Ju, Xiaojun Lin, and Ness Shroff. On the generalization power of the overfitted three-layer neural tangent kernel model. Conference on Neural Information Processing Sys- tems(NeurIPS), 2022.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Haffield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bow- man, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kermion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann. Sam McCandlish, Chris Olah, and Jared Ka- plan. Language models (mostly) know what they know, 2022.
Adam Tauman Kalai and Santosh S. Vempala. Calibrated language models must hallucinate. ACM Symposium on Theory of Computing (STOC), 2024.
Markus Kattnig, Alessa Angerschmid, Thomas Reichel, and Roman Kern. Assessing trustworthy ai: Technical and legal perspectives of fairness in ai. Computer Law & Security Review, 55:106053, 2024.
Zong Ke, Yuqing Cao, Zhenrui Chen, Yuchen Yin, Shouchao He, and Yu Cheng. Early warning of cryptocurrency reversal risks via multi-source data. Finance Research Letters, pp. 107890, 2025.
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. International Conference on Learning Representations(//LLR)) 2023.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wee Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguisticx(/ACLL 7:452-466, 2019. doi: 10.1 a_00277
URL https: http:://aaaaanttooooorooroe
Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, and Jascha Sohl-Dickstein. Finite versus infinite neural networks: an empirical study. Conference on Neural Information Processing Systems(NeurlPS), 2020a.
Jaehoon Lee, Lechao Xiao, Samuel S Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl- Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent & Journal of Statistical Mechanics: Theory and Experiment, 2020(12): 124002, December 2020b. ISSN 1742-5468.
doi: (0.1008//7742-5461bbo2222 URL http:
7/dx. doi org/ 1088 1108/1174-22 5468/4468868000 abbbbbb
Jiawei Li, Akshayaa Magesh, and Venugopal V. Veeravalli. Principled detection of hallucinations in large language models via multiple testing, 2025. URL http/ttst///aiiv....aa www.g/a////aa..
18473
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian- Nie, and Ji-Rong Wen. Halueval: A large- scale hallucination evaluation benchmark for large language models. Conference on Empirical Methods in Natural Language Processing(EMNNP) 2023a.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating ob- ject ballucination in large vision-language models. In Conference on Empirical Methods in Natu- ral Language AmocesssingEMNNLL. 2023b. URL httts:://ppnzeviiiieee net/ forummmi ummiich
xo7IJw0k7XF
14

---

Published as a conference paper at ICLR 2026
Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-8 74-81,22 2004.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. Association for Computational Linguistics(ACL), 2022a.
Zi Lin, Jeremiah Zhe Liu, and Jingbo Shang- Towards collaborative neural-symbolic graph se- mantic parsing via uncertainty. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics(ACL), pp. 4160-4173, Dublin, Ireland, May 2022b. Association for Computational Linguistiis(ACCL). doi: 10. 1/202223/11/202.. findings-acle acl.328. URL htt.tpppss/aacaatthooo.....
wtwp://.iniiinnnnninnnngn 3288
Chengzhi Liu, Zhongxing Xu, Qingyue Wei, Juncheng Wu, James Zou, Xin Eric Wang, Yuyin Zhou, and Sheng Liu. More thinking, less seeing? assessing amplified hallucination in multimodal reasoning models. Conference on Neural Information Processing Systems(Neur/PS), 2025.
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based out-of-distribure detection. Conference on Neural Information Processing Systemss(Nurl/PS) 2020.
Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction. International Conference on Learning Representations/CCRR, 2021.
Potsawee Manakul, Adian Liusie, and Mark J. Gales. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. Conference on Empirical Methods in Natural Language Processing(EMNIPP 2023.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision long form text generation. Conference on Empirical Methods in Natural Language Processing(EMNLP 2023.
Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. Association for Computational Linguistics(AAL)) 2024.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. Conference on Empirical Methods in Natural Language Processing(EMNNiP)) 2016.
Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Salch, Balaji Lakshminarayanan, and Peter J. Liu. Out-of-distribution detection and selective generation for conditional language models. International Conference on Learning Representatiion((/II..)) 2023.
Tomáš Ko ciský, Jonathan Schwar7, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. The NarrativeQA reading comprehension challenge. Transactions of the Association for Computational Linguistics(TACL) 2018.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation reduces hallucination in conversation. In Conference Empirical Methods in Natural Language (TucessingEEMNNP) 2021.
Weihang Su, Changyue Wang, Qingyao Ai, Yiran HU, Zhijing Wu, Yujia Zhou, and Yiqun Liu. Un- supervised real-time hallucination detection based on the internal states of large language models. Association for Computational Linguisticx(ACL), 2024.
Zhongxiang Sun, Qipeng Wang. Haoyu Wang, Xiao Zhang, and Jun Xu. Detection and mitigation of hallucination in large reasoning models: A mechanistic perspective. Conference on Neural Information Processing System)(Neur(PS) Workshop, 2025.
15

---

Published as a conference paper at ICLR 2026
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. Challenging big- bench tasks and whether chain-of-thought can solve them. Findings of the Association for Com- putational Linguisties(ACI, 2023.
Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kavya Elangovan, Lio Gutierrez, Teng Fong Tan, and Daniel Shu Wei Ting. Large language models in medicine. Nature Medicine, 29(8): 1930-1940, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang. Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, llyyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URt/ttps:: :
www.vvv..ggaabbaaaaeeeeee 2200 09288
Lloyd N Trefethen and David Bau. Numerical linear algebra. Society for Industrial and Applied Mathematicx(SIAM), 2022.
Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
Changyue Wang, Weihang Su, Qingyao Ai, and Yiqun Liu. Joint evaluation of answer and reasoning consistency for hallucination detection in large reasoning models, 2025.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, An- jana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, Mehrad Moradshahi, Mihir Par- mar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddhartha Mishra, Sujan Reddy. Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi, and Daniel Khashabi. Superr-nauralinstrrutions Generalization via declarative instructions on 1600+ nlp tasks. Conference on Empirical Methods in Natural Language P ProcessingEEMNNPPP 2022.
Zeyu Wei, Shuo Wang, Xiaohui Rong. Xuemin Liu, and He Li. Shadows in the attention: Contextual perturbation and representation drift in the dynamics of hallucination in llms, 2025. URL httpss
16894
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang. Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jin- gren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang. Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng. Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wen- bin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report, 2024. URL htttps://arat
www.ooggaaaa... abs / 1407.10677
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio. William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse. explainable multi-hop question answering. Conference on Empirical Methods in Natural Language Processing(EMNNI.)) 2018.
16

---

Published as a conference paper at ICLR 2026
Xinyue Zeng, Haohui Wang, Junhong Lin, Jun Wu, Tyler Cody. and Dawei Zhou. Lensllm: Un- veiling fine-tuning dynamics for Ilm selection. International Conference on Machine Learn- ing(ICML), 2025.
Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. How language model hallucinations can snowball. International Conference on Machine Learning(ICML) 2023.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christo- pher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettle- moyer. Opt: Open pre-trained transformer language models, 2022. URL https://arxiv.
erg/abs/ 2205. 01068
Yuan Zhang, Jason Baldridge, and Luheng He. Paws: Paraphrase adversaries from word scrambling. The North American Chapter of the Association for Computational LinguisticsNNAACL), 2019.
Zhenliang Zhang, Xinyu Hu, Huixuan Zhang, Junzhe Zhang, and Xiaojun Wan. Icr probe Tracking hidden state dynamics for reliable hallucination detection in llms. Association for Computational Ling 2025.
Weihong Zhong, Xiaocheng Feng, Liang Zhao, Qiming Li, Lei Huang, Yuxuan Gu, Weitao Ma, Yuan Xu, and Bing Qin. Investigating and mitigating the multimodal hallucination snowballing in large Dision-languag models. Association for Computational LinguistiesAACL), 2024.
17

---

Published as a conference paper at ICLR 2026
A PROOF OF HALLUCINATION RISK BOUND
A.1 ASSUMPTIONS VALIDATION
We provide theoretical and practical justification for the assumptions adopted in Section 3.2, which serve to ensure the well-posedness and interpretability of the proposed Hallucination Risk Bound. These assumptions follow standard practice NTK-based analyses and stability theory, and are consistent with the empirical behavior observed in modern large language models.
Setup For completeness, we briefty recall the main notation used in Section 3.2 Let y denote the discrete metric space of finite-length token sequences. Let Un S Rd be di-dimensional Hilbert space equipped with inner product (--)) and induced norm [|- The task-specific encoder ; Un is assumed to be Læ-Lipschitz with respect to dy-
Given input x, the model defines a decoding distribution (. x) over V, and we denote the embedded random variable by u, o(Y) where Y ~ ( x). For perturbations 5 E R' restricted to the local ball B,, the perturbed decoding distribution is denoted Po(. x, x, and the mean semantic response map is defined by (ō) By Po(--o(-) (x) (o(Y),
())))))))()((() with Jacobian J DGy(0) E R4 The NTK Gram matrix on embedded perturbations denoted K. E R with eigenvalues λ 2 ... 2 X, > O and condition number r((C)) Amax/ Amin- All expectations are taken with respect to the specified decoding distribution, and all norms are Euclidean unless otherwise stated.
Assumption A1 (Integrability and well-defined expectation). Assumption A1 ensures that the semantic embedding [ [Y)) is well-defined as Bochner expectation in the finite- dimensional Hilbert space Un. The bounded second-moment condition guarantees that the expec- tation exists and is finite, which is standard minimal requirement in stochastic analyses of neural network outputs. Such integrability assumptions are commonly adopted in NTK based analyses (Jacot et al. 2018: Lee al. 2020b), where control of second moments ensures stability of kernel spectra and well -oseedosssssss of linearized approximations
Assumption A2 (Lipschitz continuity of the encoder T). Assumption A2 imposes controlled relationship between the discrete sequence space (),, dy) and the continuous embedding space Un. The Læ-Lipschitz condition ensures that bounded perturbations in edit distance induce proportion- ally bounded deviations in semantic representation. Such Lipschitz regularity is standard in high- dimensional learning theory (Vershynin 2018) and is frequently invoked to establish stability under structured perturbations in representation learning. Importantly, this assumption is imposed only on the encoder map Φ, not on the full autoregressive model.
Assumption A3 (Local Fréchet smoothness of the mean semantic response). Assumption A3 formalizes the local linearization principle underlying NTK theory. By requiring twice Fréchet dif- ferentiability of the mean response map Gy = Ey=Ps()x.,) (Y()) within the perturbation ball B,, we ensure that Gy admits a controlled second-order expansion with uniform curvature constant H.. This local quadratic remainder bound is consistent with classical finite-width NTK lineariza- tion results (Lee et al. 2020a] Chizat et al. 2019), while avoiding unrealistic global smoothness requirements. Crucially, the assumption is imposed only the expected semantic response, not on the discrete decoding distribution itself.
Remark. Collectively, these assumptions provide a bridge between discrete autoregressive gener- ation and continuous functional analysis. By restricting smoothness and curvature requirements to the localized perturbation neighborhood B, and to the expectation-level map Gy, we avoid impos- ing global regularity conditions over the infinite token space Y This localization ensures that the Hallucination Risk Bound is derived under mathematically controlled conditions while remaining aligned with the practical inference dynamics of large-scale language models.
A.2 PROOF OF SECTION 3.2
We restate the main inequality from Section 3.2 Note that due to the stochastic nature of autore- gressive decoding, the bound holds with high probability. With probability least å over the
18

---

Published as a conference paper at ICLR 2026
generation process, the total hallucination risk satisfies:
Ke² C
||u" uull S 1 kpp log O(P, L) + Emismatch
inf NEUn
||u - ull +
[C]
exp
a (e - 1)
Signalg
Duta-driven term
Ressoning-driven term
Step 1:
Triangle inequality split Variance Decomposition). Let ū E[ua] be the ex- pected semantic representation under the decoding distribution. By the triangle inequality in Un. we decompose the hallucination risk into approximation error (bias) and stochastic residual (variance):
Iu"
unl
||u²
uill ||
x|| + I|an al|.
Step 2: Approximation term via Céa's lemma. To bound the deterministic approximation error, we cast the model's expected representation u as the solution to a variational problem in the Hilbert space Un. Let Let a(u, v) (u, Ko)u, denote the coercive bilinear form induced by the Neural Tangent Kernel (NTK) Gram operator K, and let f(v) := (u" Ko)u, be the bounded linear functional defining the target projection. Assuming ü acts as the Galerkin projection of the target u* onto the trainable hypothesis space, it satisfies the weak formulation a(ū, v) = f(v) for all E Un. By Céa's lemma, the projection error bounded by:
ū||
inf wEU
ul/, ||u" -o--ull
y
where and are the continuity and coercivity constants of the NTK-induced bilinear form a(.s)), respectively.
Step 3: Variance term via Bernstein concentration. We now bound the stochastic residual ||ll u||. Let C denote the set of independent sampled reasoning trajectories used during decoding. Under our local perturbation assumption (Assumption A3), the deviations of the hidden states bounded by the local neighborhood radius p. Applying Bernstein's inequality for bounded random vectors in Hilbert space (Vershynin) 2018), the tail probabilities decay exponentially. For error tolerance and an absolute constant > 0, we have with probability at least 1 - 8::
|| a|l
[ exp (-c
Ke² C
ale -1),
where is scaling constant, T is the sequence length, and S log maax bounds the per-step Jacobian spectral norm.
Step 4: Substitution. Combining both terms yields the high-probability bound:
A inf ||u" - u|| + (C]
Ke² 1).
upll We now bound the condition ratio A/y via NTK decomposition.
||u"
exp
(-c)
alesT
7 wEU,
Step 5: Decompusition of NTK Continuity Constant We decompose the bilinear form a(:,))
into three components:
ao Ept 8mmm:
where ag is the infinite-width baseline kernel, 6pr is the perturbation due to pre-training noise, and 5mm is the domain mismatch from fine-tuning. Consequently, the continuity constant satisfies A Ag + Apt + Amm-
Bounding Aps: Following standard matrix concentration bounds for finite-width NTKs (Jacot et al. 2018), the pre-training deviation scales logarithmically with the network parameters. Let be the number of parameters, L the prompt length, and kpt pre-training scaling constant; we have:
pt 7kip log ((P, T.).
19

---

Published as a conference paper at ICLR 2026
Bounding Amm Using spectral generalization bounds under data distribution shift (Lee et al. 2020b), the mismatch penalty is governed by the task-specific signal strength Signal the empirical mismatch eπror Emismatch: and scaling constant k:
Emismatch
mm
S 7k
Signalg.
Substituting both inequalities into the ratio for A/7, and normalizing An/y Ra we obtain:
S 1 + kppt log ((P, L) +
Emismatch
Signal
This completes the proof.
B HALLUGUARD DERIVATION AND INTERPRETATION
B.1 PRELIMINARIES AND NOTATION
Let K. E Rrxx be the NTK Gram matrix formed on r light semantic perturbations (see Assumptions A1-A3 in the main theory section). Denote its eigen decomposition K. VAV with
A diag( Ap),
2 0.
Let Amax = A1. Amin Ar, (<C)) Amax/ Amins and det(KC) IT A Let denote the NTK
feature matrix whose columns span the hypothesis subspace Un, so that K. t T Φ, | 2|2 = v Amax and minn
( Vmmin. For the autoregressive decoder, let be the step-t onput-output Jacobian,
and write mmax :- sup; |Jzll2-
We will use the following two standard inequalities repeatedly:
Maclaurin/AM -Monnigennaalss::
II ==1
A
1/r
A
tr(K)
Submallinhicativvityy
||AB]| 2
I
||l|2
Bl1
(10)
B.2 REPRESENTATIONAL ADEQUACY VIA det(KC) WITH EXPLICIT CONSTANTS
Assumptions for this subsection. Beyond A1-A3, assume mild source condition and spectral envelope:
S1 (Source condition) Let T denote the infinitediimensional NTK integral operator. We assume there exists regularity exponent 0 and constant R2 0 such that
u² E Range(T*) Equivalently, the spectral coefficients satisfy: Σ (u" ...
x < R2
This is standard in kernel approximation and encodes RKHS regularity.
S2 (Spectral envelope) Let π and À denote uniform upper and lower bounds on the kernel spectrum. We assume there exist constants 0 < A < Ī < oo and a decay rate a 1 such that X; < À for all i, and the tail eigenvalue satisfies X, \r (Polynomial decay is common stylization; other envelopes can be treated similarly.)
Lemma B.1 aprooaaproo error under source condition). Let Un span mi.... u,) Un- der SI,
inf ||"" - ull 1Iu- TT, S R, Ao41 NEU,
where Xr+1 denotes the next-eigenvalue of the seffnite-dimensinne kernel operator (or, equivalently, the empirical tail eigenvalue if more perturbations are added).
Proof. Write "x Een Citi with & = (u" , vz). By the source condition, ||u* - Ill, u²11
Eize& Eisr 2 x <
2x
Σ Er>
v 2g R2.
X
20

---

Published as a conference paper at ICLR 2026
To connect the representation enror the empirical NTK Gram matrix K., we leverage the algebraic relationship between the smallest eigenvalue X, and the determinant.
Lemma B.2 (Lover-bounding Ar by del((C)). Suppose del(KC) X-
S ffor all and X > O. Then
det(
and X 2 xa(r-1)
Proof. Since (et((C) yields the second inequality.
-T A S X
Ap.
obtain X det(K)) Raising to power
F
Theorem B.3 (Determinant- baasd adequacy bound with explicit constants). Under Al-A3 and S1- S2, Under AI-A3 and SI-S2, the approximation error is bounded by:
inf
I|u -ul - C. dct(K.) "
NEUp
with explicit constants independent of the target sequence:
and Ca = Rg.
Moreover, if the empirical spectrum satisfies A, 2 Ar one may choose
1
X min r-1 a log( ()ee(())) cd
S
which improves with slower decay (smaller o:).
Proof.
and the fact that eigenvalues monotonically decreasing (A,41 Ap), By Lemma B.I and
have:
inf ||u" ull R. X
NEUx
Recall that the determinant of the empirical Gram matrix is the product of its eigenvalues, del(KC) II A, Since X is the minimum eigenvalue of the rank-r matrix, it follows strictly that XG S det(KC), which implies X, S 1/t Raising both sides to the power of s yields Xē I dct(K))s/r, Substituting this upper bound into the approximation error gives:
inf ||u - ul s Rg det(())*/ s/r. NEU,
Setting Ca == R& and ca := s/r completes the proof.
In practice, tracking the direct determinant cause numerical underflow in high-dimensional spaces. We use log det(AC) via the Cholesky decomposition as our empirical score, aggregating with z-normalization across components avoid scale domination by any single dimension.
B.3 ROLLOUT AMPLIFICATION JACOBIAN PRODUCTS (EXACT CONSTANTS)
Theorem B.4 (Amplification bound with exact constant). Let Js be the step-t Jacobian and Fmax sup, |ell. Then
T
T
S II [
S
max
Defining = log Fmax gives BT TT Taxe hence
@T
< T
with equality if and only if [|Je||2 Omax for all and the top singular directions align across factors.
Proof. The first inequality is equation 10 applied iteratively. The second is by definition of mmax- Setting = log maax yields equality in the worst case. Alignment of top singular vectors is the tightness condition for submultiplicativity.
21

---

Published as a conference paper at ICLR 2026
Token-dependent refinement. If one defines d, || and Pavg + E logor, o then I Jell2 I exp E. log oo) = e which tighter but requires per-step measurements.
B.4
CONDITIONING- INDUCED VARIANCE WITH x(K)² SCALING
We now give explicit projector-pertuubation derivation showing the quadratic dependence on the condition number.
Setup. Let P ( + be the orthogonal projector onto Uni then the linearized output is Ub = Pu". Consider a feature perturbation 2 induced by prefix perturbation satisfying
|| _ La ||ô]| (A2/A3).
( (Ø AT) and define Let the perturbed projector be P ( ( ( AT))
AP = P P.
Lemma B.5 (Projector perturbation bound). There exists an absolute constant Cn > 0 such that
|
Amax
A ll2
| s Cn Omin (+)2 |
On
|A0122
On k(K)
Amin
Amiin
Proof idea. Use standard bounds for the perturbation of orthogonal projectors onto column spaces (e.g., Wedin's sinO theorem and Stewart-Sun, Matrix Perturbation Theory, Thm 3.6). One shows
| I 2 2 4) +
A + 0(1A+11)) 12).
Since 1) = Amin and | T A l2 z || |
Amax A 2, the result follows
for sufficiently small || absorbing lower-order terms into CnI
Theorem B.6 (Variance amplification with explicit constant). Let un (Φ) = Pu" and un ( (2+A4) Pu". Then
A 2|2
Mu ④ + A (.)())) < On x(K)
||0'||
Amiin
If A is induced by random prefix perturbation 8 with | < L0l|o| and B|181|2 oz. then
Var[us] S Elluy ( + AΦ) un (D]l I c. x(K)² 18812
with
L 1u0
Amiin
Proof. By Lemma B.5
[u + AΦ)
un
A < ||P||||| S
|a41l2 On x(K.) Amiin ||=" Square both sides and take expectation over 8, using || S to obtain the stated variance bound with the explicit constant Cy..
Lo ||ō||
Interpretation. The x(K)²² factor arises from two sources: (i) x(K) from the projector sensitivity (Lemma B.5). and (ii) 1/ 1/Amin from converting || a mean-squared bound after squaring and averaging, yielding overall k²²scaling in the variance constant.
B.5 CONSOLIDATION: COMPACT SURROGATE CONSISTENT WITH THE RISK DECOMPOSITION
Combining Theorem B.3 Theorem B.4 and Theorem B.6 obtain computable surrogate aligned
with the Hallucination Risk Bound:
Adequacy: (At(
Amplification: log Fmax
Conditioning penalty: log x(C))²
This motivates the score
HALLUGUARD det( + log Fmax
log x(X)²
with the following explicit, implementatiion-radyy notes:
22

---

Published as a conference paper at ICLR 2026
Use log det( via Cholesky for stability; replace det in the score with log det if desired (monotone equivalent).
Estimate max either as sup, [|Jell or its tighter average form Savg 1 T Σ, log || (then use Pavg in place of log max))
z-normalize each component across a validation set before summation to avoid scale dom- inance; optionally fit task-specific weights if permitted.
C EXPERIMENT
C.I SETUP
Framework. All experiments use PyTorch and HuggingFace Trans formers with fixed random seed for reproducibility. Unless otherwise noted, computations run in mixed precision (fp16). Hardware details (A100/H200) are reported once in the main setup section.
Implementation
Generation Configuration. For default evaluation of detectors, we use nucleus sampling with temperature = 0.5, top-p- = 0.95, and top-k 10, decoding K=10 candidate responses per input (unless otherwise specified). These decoding trajectories also operationalize semantic per- turbations as natural variations within the model's local predictive distribution, thereby instantiating semantically proximate neighborhood around the primary response and capturing the local geom- etry of the reasoning manifold required for NTK construction. For score guided test-time inference (Section 4.3, we beam search (beam size 10) and score candidate trajectories at each step with the chosen detector. For stability analysis, HALLUGUARD extracts sentence representations from the final token the middle transformer layer (L/2), which empirically preserves semantics relevant to truthfulness.
NTK- Score Computation. For each set of generations, we form task-specific NTK fea- ture matrix and compute the semantic stability score from its eigenspectrum. We add small ridge 10 for numerical stability and compute singular values via SVD.
Perturbation Regularization. To prevent pathological activations that amplify instability, HAL- LUGUARD clips hidden features using an adaptive scheme. We maintain memory bank 330000 token embeddings and set thresholds at the top and bottom 0.2% percentiles of neuron activations; out-of-range values are truncated to attenuate overconfident hallucinations.
Optimization. Backhone language models are fine-tuned. We train only HALLUGUARD'S lightweight projection layers using AdamW with learning rate selected from {1 x 10-5, 5 x 10 1 x 10-4 and weight decay from {0.0, 0.013. The best setting is chosen on held-out validation split.
Implementation Details. For score-guided inference we apply beam search with beam size 10, rescoring candidates stepwise with different hallucination detectors.
Ablation Setup. All ablations reuse the main paper's splits, prompts, and decoding; we vary only HALLUGUARD internals and explicitly control the hallucination base rate. On the generation side, we modulate prevalence by adjusting temperature/top-2 and beam size; to stress the two families, increase the prefix perturbation budget p and rollout horizon T amplify reasoning drift, and (when applicable) toggle retrieval masking to induce data-driven errors. On the detection side, AU- ROC/AUPRC are threshold-free when fixed operating point needed, we decision threshold on the validation set by (i) matching target predicted-positee rate Wrarget via score quantiles or π is
CFN
(ii) fixing a desired FPR (e.g.. 1%, 5%, 10%): cost-sensitive Bayes rule
CFP + CFN optional when misclassification costs are specified. Unless noted, we toggle one factor at a time and sweep {0.75, 1.0, 1.5), T E {12, 16, 24). and the number of semantic probes {2, 4, 8); no additional training is performed beyond optional temperature/z-score calibration on the training split. We report mean std over seeds.
π
23

---

Published as a conference paper at ICLR 2026
C.2 ABLATION STUDY ON log x²
To empirically validate the necessity of the stability term - -log we performed controlled ab- lation on MATH-500. We systematized the reasoning drift (d) by progressively increasing the per- turbation budget and rollout horizon T. As shown in Figure 3 the absence of this term leads to severe instability. While the ablated model (orange dashed line) performs competitively in low-drift regimes (d < 0.15), it exhibits significant performance volatility the reasoning task becomes more complex. contrast, the full HALLUGUARD score (green solid line) effectively penalizes these ill- conditioned regimes, maintaining smooth and robust detection profile. This confirms that log x² functions as an essential spectral regularizer, preventing the score from becoming unreliable under high-entropy inference states.
0.9
Empirical envelope
HALLUGUARD (full) w/o -log K² term
0.8
0.7
AUROC
0.6
0.5
0.4
0.0
0.2
0.6 Reasoning drift level d
0.4
0.8
1.0
Figure 3: Ablation study of the stability term (- log x²2 on MATH500.
Table Ablation on stability term - log x²2 (MATH500).
Method
Pearson
MSE
HalluGuard
0.985
0.0192
w/o log x
0.8904
0.0381
The enror in table 6 nearly doubles without the stability term, confirming that spectral conditioning essential for stable reasoning-risk quantification.
C.3 ABLATION STUDY SEMANTIC ENCODER
To examine sensitivity the semantic encoder ④ we replace the default representation with widely adopted alternatives, including BERT, SimCSE, and E5. We evaluate across multiple backbone models and benchmarks.
repporss AUROC and AUPRC on RAGTruth, GSM8K, and TruthfulQA. Across all settings, variants. For example, QwQ-32B
HALLUGUARD consistently outperforms (RAGTruth), replacing the default encoder with BERT reduces AUROC from 84.59 to 81.44.
These results indicate that the performance gain does not stem from surface semantic similarity of final outputs. Instead, the method captures geometric structure reasoning trajectories, which external encoders cannot fully preserve.
24

---

Published as a conference paper at ICLR 2026
Table Encoder ablation across backbones and benchmarks (AUROC/ AUPRC).
Backbone
Method
RAGTruth
GSM8K
TruthfulQA
AUROC
AUPRC
AUROC
AUPRC
AUROC
AUPRC
HalluGuard
75.51
73.40
72.04
69.88
72.10
68.76
+BERT
72.48
70.12
67.31
64.90
68.02
65.01
GPT-2
+SimCSE
73.21
71.05
68.44
66.02
69.14
66.27
+E5
74.02
71.66
69.12
66.80
70.03
67.10
HalluGuard
80.13
76.77
72.57
70.31
69.59
68.36
+BERT
77.44
74.20
67.95
65.48
66.12
64.80
OPT-6.7B
+SimCSE
78.11
74.83
69.01
66.40
67.08
65.72
+E5
78.66
75.31
70.04
67.25
67.80
66.41
HalluGuard
82.31
80.79
80.62
77.30
77.05
73.79
+BERT
79.02
76.91
75.51
72.08
73.14
69.52
Mistral-7B
+SimCSE
79.88
77.66
76.40
73.01
74.08
70.40
+E5
80.41
78.20
77.12
73.74
74.66
71.05
HalluGuard
84.59
81.15
75.81
74.68
74.26
72.76
+BERT
81.44
78.03
70.92
68.90
70.35
68.01
QwQ-32B
+SimCS
82.10
78.66
72.10
69.82
71.20
68.70
+E5
82.66
79.12
73.05
70.44
72.02
69.31
HalluGuard
77.51
75.30
79.01
76.73
78.50
77.56
+BERT
74.26
72.04
73.12
70.60
74.41
72.88
LLaMA2-13B
+SimCSE
75.11
72.83
74.20
71.51
75.36
73.54
+E5
75.78
73.44
75.14
72.32
76.10
74.22
C.4 COMPUTATIONAL EFFICIENCY ANALYSIS
To assess practical deployment feasibility, measured inference latency NVIDIA A A100/H200 GPU. Our setup utilizes batched parallel sampling to generate = 10 trajectories, en- suring sub-lineaa scaling of the computational cost. The core HALLUGUARD aperations-speciffcally feature clipping and computing the NTK score via the Gram matrix-add minimal latency, requiring less than ms of post-processing time per query.
Perpleuty
Inidde
Energy
Reckirr
DutSicoee
Moma6-18
Bacchoen Mori
Figure 4: Per--uuetiio Inference Time (Seconds) on BBH Across Hallucination Detection Methods.
C.5 DETECTION PERFORMANCE ANALYSIS
Across all five model families and three benchmark regimes, HALLUGUARD consistently achieves state-of-thed of-tth-artt detection performance, particularly in the safety-critical low-FPR regions as shown in Table g
25

---

Published as a conference paper at ICLR 2026
Perpleity
HIND
Semontt: Fnz.
RACE Kurticore
Rchlogeer
Make
Qutb
Eecdora Hood
Figure 5: Per- Questiesio Inference Time (Seconds) on HaluEval Across Hallucination Detection Methods.
Conpenational Cust MBT- 3C0
Perploitty halluGaard Inaide
HIMD L4E2troyy Enargy
Scmantic End. Jookel Sims CCheeeGFP
RACE Pitratt FAcSsorr
OF12
378
Uoma0-7
Jami
Figure 6: Per-Question Inference Time (Seconds) on Math500 Across Hallucination Detection Methods.
Computational Coel RAGTrut
Metalll
andd
144rmne
rokcal Rceclie
Pitnee Marticore
1nskie
Energy
Eecdone Hood
Figure 7: Per-Question Inference Time (Seconds) on RAGTruth Across Hallucination Detection Methods.
We additionally expanded our evaluation to include SAPLMA, LLM-Check, and ITI. As shown in Table 9 HALLUGUARD delivers the strongest performance not only on AUROC/AUPRC but also on deployment-crittcaal, low-FPR operating points, including Fl and TPR at 5% and 10% FPR. Across all three benchmarks (RAGTruth, GSM8K, HaluEval) and all backbones (GPT-2 through QwQ-32B and LLaMA2-13B), HALLUGUARD consistently achieves the highest Fl and the highest or near- higheti TPR under fixed low-FPR constraints. In contrast, SAPLMA and LLM-Check ex- hibit noticeably lower recall in the stringent 5% FPR regime. These results demonstrate that HAL- LUGUARD is better aligned with maintaining high detection sensitivity under tight false-positive budgets, requirement that is central to reliable hallucination detection in real-world systems.
26

---

Published as a conference paper at ICLR 2026
Perpleity
HEND
Semontt: Fnz.
Rchlogerr
Kurticore
Miki
Eacdore Hood
Figure 8: Per-Question Inference Time (Seconds) on SQuaD Across Hallucination Detection Meth- ods.
Comp wwrponntieeeiioom Coet on Tuuhrags
Perpleitt
Semondiic eeiical Sin
Halurkard
Petroct
Irokde
Energy
Sc Pcheclgg
Murtscore
Eacdora
Figure 9: Per-Question Inference Time (Seconds) on TruthfulQA Across Hallucination Detection
Methods.
C.6 TIGHTNESS OF BOUND
Evaluation of bound tightness. To rigorously stress-test the Hallucination Risk Bound of The- orem 3.2, we conducted a controlled synthetic study grounded in the empirical reasoning-depth distribution of the Snowhalling dataset (Zhang et al. 2023). We instantiated empirical hallucination trajectories by injecting low-variance Gaussian noise into the base components D(T) and R(T), comparing them against the closed-form theoretical prediction. As illustrated in Figure IO] while the theoretical curve acts as a conservative upper envelope, it exhibits a nearly parallel growth trajec- tory to the empirical risk. Crucially, it faithfully captures the exponential curvature and compound- ing dynamics of the Snowballing Effect. This confirms that the bound possesses high structural fidelity: correctly models the scaling law of error propagation across depth ranges, validating its effectiveness as a ranking proxy despite the absolute numerical offset.
Evaluation of NTK proxy tightness. To quantitatively validate that our NTK- proxy faith- fully captures the amplification behavior of stepwise Jacobians, we conduct diagnostic experiment on GPT-2-smal 11 (117M), where per-step Jacobian norms are fully tractable. For held-out set of GSM8K prompts and decoding steps < 18, we compute:
the empirical stepwise Jacobian magnitude Jell,, obtained via automatic differentiation on the next-token logits, and
reasoning driven NTK proxy, log Fmax log x2, defined in Eq. (7), which upper- bounds the per-step amplification rate and penalizes spectral ill-conditioning of the NTK Gram matrix.
27

---

Published as a conference paper at ICLR 2026
Table 8: Performance comparison on representative benchmarks: ceasoning-oriented (BBH), and instruction-followinn (TruthfulQA).
data-centric (RAGTruth),
GPT2
OPT-6.7B
Mistral-7B
QwQ-12B
LLaMA2-13B
TPRS 10re
TPR ItP
TRS 10s
TRS35
TPR 10%
TPRe3
TPR
Fl
F7
F1
F1
HALLUGUARD
71.22
64.86
51.41
77.03
73.52
59.12
75.19
69.44
59.21
81.91
74.13
63.52
74.66
68.91
57.42
Inside
66.12
59.72
48.31
72.91
70.25
60.37
70.45
68.1
12
52.41
79.03
74.66
61.09
73.08
70.11
56
MIND
58.33
54.11
38.72
62.55
87
47.65
71.91
66.74
54.39
64.02
59 12
45.63
68.55
63.50
48.78
Perplexity
55.42
40.51
69.74
66
52.18
70.42
55.32
57.01
44.75
RAGIruth
LN-Entropy
62.17 57.52
46.44
58.33
52.99
43.28
65.30
61.27
49.92
67.15
62.42
51.33
63.28
59.07
46.14
Energy
59.71
56.23
44.81
60.44
57.18
45.03
63.54
59.42
48.62
72.09
68. 15
58.42
66.10
61.33
49.41
Semantic
57.28
53.42
41.92
64.81
52.01
67.10
50.66
66.12
49.31
64.55
60.18
47.75
Lexical Sim.
61.41 57.09
45.03
65.81
61.44
49.51
62.50
59.12
50.92
70.91
67.53 55.21
66.29
59.88
51.03
SeliChecGGPT
56.22 52.84
40.63
60.79
55.68
45.72
63.12
59.47
48.33
66.54
62.92
51.41
68.21
65.12 53.60
RACE
60.12
44.90
59.77
65.44
52.73
69.61
62.55
59.42 45.66
Pitrue)
58.91 55.47
42.13
67.44
63.20
51.43
71.22
66.91
10
63.44
60.33
49,27
70.18
65.77 52.78
FActScore
62,10 58.21
46.33
59.22
54.14
44.32
63.87
60.77
47.98
68.33
64.02
53,41
65.92
61.37
49.84
HALLUGUARD
68.33
64,11
56.42
74.91
69.14
62.10
73.22
69.88
57.21
78.55
69.91
61.45
71.10
68.25
59.92
Inside
65.41
61.22
52.83
71.02
67.10
60.21
68.17
64.75
53.92
79.17
72.33
64.22
67.10
63,52
55.91
MIND
54.12
50,22
40.11
53.44
63.92
59.88
47.01
48.83
65.11
60.22
49.52
Perplexity
52.91
49,33
40.44
61.88
58.12
49.22
62.91
59.42
50.11
59,91
55.72
49.03
60,88
57.41
48.62
L. Entropy
59.12
55.44
44.92
54.61
51.75
43.18
66.44
63.21
54.09
62.75
59.12
47.52
68 20 64.88
55.41
BBH
Energy
53.94
45.03
64.55
60.11
49.99
52.84
62.77
50,22
Semantic Ent.
57.41
54.32
47.21
61.22
58.42
49.74
63.21
59.10
48.62
63.55
60.24
48.88
64.91
61.44
50.72
Lexical Sim.
50.41
46.77
38.92
60.71
57.11
45.55
59.42
56.88
48.91
70.33
67.10
55.32
58.33
55.42
47.41
SellChekGPT
55.21 52.14 43.92
58.10
55.78
46.22
62.82
50.44
65.22
62.44
54.21
63.44
60.77
52.33
RACE
56.14 53,72
43.88
63.11
59.71
52.81
65.77
62.55
50.72
58.88
55. 14
46.18
66.10
62.41
49.81
P(true)
54.31 52.22
44.10
58.22
56.10
48.52
56.91
53.55
43.92
61 61
58.21
46.77
57.33
54.88
45.91
FActScore
56.20 52.42
41.77
55.44
52.12
61.62
58.22
51.33
59.33
56.42
49.14
63.44
60.22
52.44
HALLL GUARD
75.11 71.20
63.21
67.44
64.55
58.12
78.92
74.22
65.33
76.44
72.01
59,92
75.33
69.11 63.08
Inside
71.10
68.55
60.77
61.77
54
50,10
63.88
61.33
53.41
69.
55.14
62.14
59.94
52,80
MIND
57.44
54,91
45.33
59.92
56.88
48.33
58.72
56. 14
47.21
61.21
58.88
52.02
60.44
58.20
49.03
Pempexity
49.52 46.71
38.82
54.12
51.74
43,90
59.72
57.55
46.88
54.44
51.72
42.58
60.33
57.21
47.41
TruthfuQQA
LN Entropy
57.11 54.88
42.98
55.33
52.41
45.91
59.66
56.22
40.10
60.44
58.02
46.22
61.41
57.17
43.88
Energy
54.11
52.17
38.91
53.44
51.14
36.88
58.21
$4.77
49.92
63.02
60.44
51.33
58.41
55.33
50.42
Semantic Ent,
60.08
56.44
44.15
50.14
47.33
35.92
53.74
52.11
37.02
65.33
63
50.77
55.02
53.11
38,44
Lexical Sim.
51.22
49.20
39.03
58.72
54.71
48.77
65.71
63.50
53.10
54.77
51.44
45.88
66.41
64.14
54.88
SeliCheckGPT
55.72 53.44
42.78
58.33
55.72
47.14
60,88
$7.44
43.91
55.42
54.44
40,77
61.72
59.51
44.10
RACE
52.22
49.88
41.44
63.14
66.88
54.05
70.55
67.11
59.77
55,44
52.11
45.33
71.33
68.22
60.02
P(true)
55.54
52.11
38.82
55.72
52.33
39.22
57.41
53.10
41.22
56.88
54.77
45.55
57.12
53.33
41.88
FActScore
52.91
50.14
40.44
54.11
50,22
41.33
52.88
49.91
42.55
61.55
59.22
44.72
53.41
50.71
43.10
Table 9: Comparison with SAPLMA, LLM-Check and ITI across benchmarks and backbones.
196
121
11.0
11.0
3640
M432
13.00
40:0
9.5
0.5
0.6
USA
2-12
153
22
2.6
121
115
11
118
1101
3111
100
Figure 11 reports the scatter plot comparing the NTK proxy against empirical ||Ja||2 across all prompts and steps.
Validation of Term Decomposition To validate the architectural premise of our Hallucination
Risk Bound Section 3.2] we visualize the evolution of the decomposed risk components across rea- soning depth I on the Snowballing dataset (Zhang et al.) 2023). As shown in Figure Figure [2] the total risk is driven by two distinct dynamic behaviors. The data-driven term (green dotted line) exhibits linear or near-constant progression, reflecting static retrieval or snowledge-encoding that persist regardless of depth. In contrast, the reasoning-driven term (purple dotted line) demon- strates exponential amplification consistent with the Snowballing Effect, remaining negligible at shallow depths but rapidly dominating the total risk as T increases. Crucially, this reveals a phase transition in hallucination dynamics: at lower depths (T 15), errors are primarily data-driven, whereas at higher depths, reasoning instability becomes the governing factor. This dichotomy em- pirically justifies our hybrid scoring mechanism, confirming that unified detector must account
errors
28

---

Published as a conference paper at ICLR 2026
4.5
Empirical hallucination risk
Theoretical bound
4.0
3.5
3.0
Risk
2.5
2.0
1.5
1.0
0.5
5
10
15
20
25
30
Reasoning depth
Figure 10: Empirical hallucination risk versus our theoretical bound
NTK Proxy vs. True Jacobian Norm
(Rollout Instability Analysis)
500
0.991 0.00e+00)
500
Empirical Jacobian Norm: 1|Al Cirax()
400
Empir Jacobian Norm
200
100
Upper
Proxy: 1ogomax
log
Figure 11: The NTK proxy closely tracks empirical Jacobian amplification on GPT-2-small, showing near-perfect monotonic alignment and a consistent conservative envelope across decoding depth.
for both the static semantic bias and the dynamic rollout instability to be effective across varying generation lengths.
C.7 CORRELATION OF REASONING-DRIVER AND DATA-DRIVEN TERMS WITH DIFFERENT TYPES OF DATASETS
To empirically verify the independence of the proposed risk components, analyzed their relation with detection performance across distinct task families. As illustrated Figure [4 and Figure [3] observe sharp geometric decoupling: the data-driven term aligns strongly with data-
29

---

Published as a conference paper at ICLR 2026
0.5
Our Bound
Data-driven Term Reasoning- drrivnnng-drieen Term
Ground Truth Risk
Reas-nning Hallucinatio the
0.4
Data hallucination
0.3
Risk
Gap between our bound and ground truth
0.2
0.1
V.
2.5
5.0
7.5
10.0
12.5
15.0
17.5
20.0
Reasoning depth 7
Figure 12: Risk decomposition across reasoning depth T on Snowballing dataset.
centric benchmarks (c.g.. RAGTruth) while showing negligible correlation with reasoning tasks. Conversely, the reasoning-driven term dominates on reasoning-oriente datasets (e.g.. MATH-500). This double dissociation reinforces the structural validity and orthogonality of our decomposition, confirming that each term captures distinct, non-redundant failure mode.
reasoning term AUROC
0.85
data term vs AUROC
0.80
AUROC (reasoning-centis
0.75
0.70
0.65
0.60
0.55
reasoning term p = 0.82
0.50
data term = 0.03
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Proxy score
Figure 13: Correlation Between data-driven and reasoning-driven terms and AUROC on Reasoning- Centric MATH500.
C.8 CASE STUDY
Case Study 1 GSM8K (Multi-step Arithmetic): Bias saves $3/day for four weeks and buys a S12 toy. How much money does he have left?" Ground truth: $72.
Drift Snowballing. Task: "John
30

---

Published as a conference paper at ICLR 2026
X
data vs AUROC
0.90
X
reasoning term vs AUROC
X
X
0.85
AUROC (data-centric)
XX
0.80
x
0.75
0.70
data term p = 0.85
reasoning term 0.05
0.0
0.2
0.4
0.6
0.8
Proxy score
Figure 14: Correlation Between data-driven and reasoning-driven terms and AUROC Data-
Centric RAGTruth.
Length (T)
Model Behavior
HalluGuard Response
T=1-8 Stable setup
Correct restatement and arithmetic planning weeks" - "40 days"
Data-driven term dominant: risk flat Slight rise data-driven signal Reasoning-driver share begins to rise Reasoning-dri driven dominates (snowballing)
T=9-14 Seed error
T=15-22 Propagation
"3 x 40 120"
T=23-40 Amplification
Final answer: $108
Table 10: Evolution of hallucination in GSM8K arithmetic reasoning.
Case Study 2 DonemDe Summarization: Misalignment Overreach - Fabrication. Task: Summarize 5,000-token policy document
Ground truth: Security audit exception applies only specific log types.
Length (T)
Model Behavior
HalluGuard Response
T=1-20 Accurate extraction
Correct recovery of retention rules Incorrect merge of distant sections
Low risk; strong alignment Data-driven signal increases
T=21-40 Misbinding
T=41-95 Drift
Overgeneralized suspension claim New false rule introduced
Reasoning- driven share rises
T=96-170 Fabrication
Reasoning- driven dominates
Table 11: Eyolution of hallucination long-document summarization.
C.9 COMPARISON WITH INSIDE AND MIND
Inside and MIND serve as empirical uncertainty diagnostics. Inside analyzes covariance spectra of static representations, while MIND measures temporal variations in hidden states. Both methods extract hho signals and produce a single uncertainty score.
In contrast, HALLUGUARD derives structured risk decomposition from generative dynamics, sep- arating data-driven and reasoning-drin driven sources via NTK spectral geometry and instability ampli- fication. This formulation explicitly models compounded reasoning errors.
31

---

Published as a conference paper at ICLR 2026
We evaluate all methods the Snowballing benchmark Zhang et al. (2023), which emphasizes progressive reasoning instability. As shown in Table 12 HALLUGUARD consistently outperforms Inside and MIND across all backbone models.
Table 12:
Comparison with Inside and MIND on the Snowballing benchmark across different back- bone models (AUROC/ AUPRC).
Method
GPT-2
OPT-6.7B
Mistral-7B
QwQ-32B
LLaMA2-7B
LLaMA2-70B
HalluGuard
88.52/82.14
92.63/87.42
94.87/89.66
97.41/95.08
93.28/88.03
97.96/95.37
Inside
74.11/66.39
78.24/70.51
83.32/75.80
87.55/80.47
81.72/73.11
89.03/82.77
MIND
69.42/58.73
74.56/64.37
78.67/68.52
84.03/73.68
77.91/65.89
86.28/78.41
C.10
ADDITIONAL EVALUATION ON MULTIMODAL AND LONG-CONTEXT REGIMES
To evaluate generalization beyond short-form reasoning tasks, we extend experiments to (i) multi- modal hallucination detection POPE Li et al. 2023b). and (ii) long-context generation Gov- Report Huang et al. (2021) and NarrativeQAls Ko cisky et al. (2018).
Across all backbone models, HALLUGUARD consistently achieves the strongest AUROC and AUPRC, demonstrating robustness under multimodal noise and long-range dependency drift.
Table 13: Comparison of methods across different backbone models on POPELAUROC/AUPRCC,
Method
GPT-2
OPT-6.7B
Mistral-7B
QwQ-32B
LLaMA2-7B
LLaMA2-70B
Perplexity
61.12/53.04
68.27/60.18
72.41/64.09
79.36/73.22
70. 15/62332.31
83.48/76. 19
HalluGuard
74.33/68.27
81.22/75.36
86.47/80.51
91.58/86.42
85,39/78.44
94.63/89.27
Inside
70.08/64.12 12
77.19/70.33
83.44/75.28
89.27/82.36
81.22/74.41
92.51/87.39
MIND
66.17/58.22
73.31/66.14
79.28/71.39
86.44/79.33
77.18/69.27
89.36/83.48
LN-Entropy
63.09/55.11
71.24/62.18
76.37/67.06
84.33/75.29
14.12/65.18
87.42/80.35
Energy
62.14/54.22
69.17/61.26
75.29/66.31
83.41/74.18
73.21/64.33
86.39/79.41
Semantic Ent.
64.18/56.04
72.29/63.14
77.41/68.22
85.48/76.39
75.17/66.41
88 88.46/81.27
Lexical Sim.
65.24/57.19
73.33/64.21
78.46/69.37
85.52/77.44
76.31/67.29
88. 88.59/82.31
SeIPCheckGPT
58.11/50.28
63.22/55.31
67.38/58.24
74.41/66.33
64.27/56.21
78.46/70.39
RACE
69.14/63.17
76.28/69.41
82.33/74.29
88.47/80.36
80.36/73.22
91 91.44/85.33
P(true)
67.22/59.26
74.31/66.18
80.41/71.33
87.44/79.28
78.29/69.33
90.38/83.41
FActScore
18.19/61.33
75.39/68.22
81.47/73.88
88.52/81.41
79.34/71.48
91.46/85.37
Multimodal Hallucination (POPE).
Table 14: Comparison of methods across por(AUUROC/AURRC,
different
backbone
models
GovRe-
Method
GPT-2
OPT-6.7B
Mistral-7B
QwQ-32B
LLaMA2-7B
LLaMA2-70B
Perplexity
18.13/49.22
64.41/55.37
67.29/58.46
75.34/66.18
63,28/54.33
78.57/69.4 .41
HalluGuard
72.38/66.41
79.27/72.39
84.46/78.31
90.58/84.42
82.44/76.33
93.62/88.51
Inside
69.17/62.24
76.33/68.41
81.44/73.36
88.42/80.31
79.36/71.29
91.47/85.39
MIND
68.21/56.18
72.41/63.29
77.38/68.33
86.33/77.41
75.27/66.38
88.46/82.2
LN-Entrppy
63.12/54.27
70.33/61.22
75.41/65.34
83.39/74.21
72.18/62.33
86.38/79.33
Energy
61.09/52.14
69.18/60.41
74.37/64.28
82.34/73.19
71.26/61.44
85.41/78.36
Semantic Ent.
64.17/55.11
71.22/6..11
76.39/67.42
84.46/75.38
73.24/64.28
87 87.49/80,36
Lexical Sim.
65.26/56.17
72.38/63.38
76.44/68.41
85.43/76.37
74.22/65.19
87.53/81 .44
SellCheckGPT
55.14/46.29
60.31/51.22
63.44/54.19
70.26/60.41
59,33/49.24
73.41/63.38
RACE
68.28/60.33
75.41/66.29
80.36/72.41
87.42/79.33
78.32/70.24
90.38/84.41
P(true)
66.34/57.22
73.39/64.31
78.48/69.44
86.38/77.41
76.33/67.25
89.44/83.36
FActScore
67.41/59.36
74.42/66.41
79.39/71.46
87.41/78.47
77.47/69.44
90.41/84.38
Long-Context Generation (GovReport).
32

---

Published as a conference paper at ICLR 2026
Table 15:
Comparison of methods
across
different
backbone
models
on Narra-
(ieQAAAUROC/AURRC)
Method
GPT-2
OPT-6.7B
Mistral-7B
QwQ-32B
LLaMA2-7B
LLaMA2-70B
Perplexity
56.14/47.22
62.33/53.18
65.41/55.39
72.26/63.41
61.27/51.33
76.38/67.2 .9
HalluGuard
70.36/64.41
77.22/70.37
83.48/76.29
89.53/83.47
81.33/74.36
92.57/87.41
Inside
67.18/60.27
74.39/66.41
80.46/72.31
87.44/79.36
78.41/69.38
90,43/84.32
MIND
63.27/54.18
70.31/61.29
76.33/67.24
84.39/75.41
74.36/64.47
87.41/80.32
LN-Entropy
61.19/52.11
68.27/59.33
73.42/63.21
82.41/73.29
72. 42114/1141
85,36/78.4 44
Energy
60.08/51.14
67.18/58.34
72.37162.47
81.33/72.41
70.27/60.33
84.44/77.44
Semantic Ent.
63.22/55.09
69.31/61.46
75.44/66.33
83,47/74.41
73.26/63.44 44
86.47/79.39 .99
Lexical Sim.
64.17/56.22
70.37/62.34
76.41/67.41
84.33/75.44
74.41/65.27
87.46/80.41
SelfCheckGPT
52.14/43.29
57.33/48.31
61.48/51.66
68.41/58.477
56.39/46.31
71.36/61.4
RACE
66,29/58.31
73.42/65.38
79.33/71.28
86.41/78.44
77.28/68.39
89.43/83.38
P(true)
64.31/56.24
71.39/63.38
77.77/68.6
85.38/77.41
75.29/66.33
88.38/82.44
FActScore
65.44/57.36
72.41/64.41
78.52/70.38
86,44/78.33
76.41/68.44
89.44/83.39
Long-Context Generation (NarrativeQA).
D USAGE OF LLM
Large language models (LLMs) were employed in a limited and transparent manner during the preparation this manuscript. Specifically, LLMs were used to assist with linguistic refinement, style adjustments, and minor text editing to improve clarity and readability. They were not involved formulating the research questions, designing the theoretical framework, conducting experiments, interpreting results. All scientific contributions-inclut analyses, and conclusions-are the sole responsibility of the authors.
conceptual development, methodology,
33


Reviews:
[{"review_id": "9vHZg6WuDN", "reviewer": "Reviewer_dMiw", "summary": "In this paper, the authors investigate the phenomenon of hallucinations in Autoregressive LLMs, and propose two specific components to analyze in detail: those that originate due to poor data from the training stage, and the second being inference-time decoding issues that arise in generation. Toward this, the paper introduces HalluGuard, a scoring method that decomposes the hallucination risk into distinct terms by bounds obtained from eigenspectrum of the Neural Tangent Kernel under semantic perturbations. In practice, the paper proposes three scoring components, $det(\\mathcal{K})$ i.e the determinant of the NTK gram matrix for the data-driven term, $log~\\sigma_{max}$ the supremum Jacobian norm and $-log \\kappa(\\mathcal{K})^{2}$ utilizing the condition number to account for autoregressive reasoning-driven term. The proposed HalluGuard Score is shown to be effective in practice on standard datasets and LLM models as well, achieving notable improvements over several baseline approaches.", "questions": "Kindly refer to the questions mentioned in the weaknesses section above. I would be happy to raise my score further if these could be adequately addressed.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1) The primary strength of the paper is to cast the hallucination detection problem in a formal theoretical framework, to decompose the hallucination risk into data-driven and reasoning-driven sources. This lays appropriate groundwork to then analyse the source of hallucinations themselves, which otherwise is often heuristic at best.\n\n2) The mathematical framework is introduced in a clear and lucid manner, though some terms are introduced with limited motivation (condition number term). Furthermore, the authors empirically demonstrate that the scores proposed for the data-driven and reasoning-driven components vary as expected in a practical setting over standard datasets such as SQuAD, Math-500 and TruthfulQA.\n\n3) The proposed HalluGuard method is shown to be effective on several datasets such as RagTruth, BBH, TruthfulQA, SQuAD, GMS8K and HaluEval. The method is also shown to perform better than numerous baselines, though the choice of comparison points for prior works could be improved (kindly refer to Weaknesses section below).\n\n4) The utilization of HalluGuard toward improving beam search in test-time inference to boost performance is another strong and notable contribution. The paper further presents a detailed case-study in fine-grained hallucination detection, wherein lexically similar yet semantically incorrect outputs are analyzed, and helps demonstrate the efficacy of the proposed scoring method.", "weaknesses": "1) While the core theory as presented in Theorem 3.2  decomposes risk into two terms (data-driven and reasoning-driven), the final score is an additive combination of three terms. The inclusion of the third term ($-log~\\kappa^2$), while explored in the appendix as a penalty, appears to be arbitrary and appended without adequate motivation overall. Furthermore, the paper relies on a Freedman inequality to show that the reasoning-driven error term grows exponentially with sequence length T. However, this is extremely loose in practice, and an overly simplistic justification for the reasoning-driven term in the final score.\n\n\n2) Unexplained Jacobian Proxy: While the paper states that computing the direct step-wise Jacobians for billion parameter LLMs are intractable, the final score still utilizes the eigenspectrum of the NTK Gram Matrix. It is quite unclear from the paper about how this translates to the computational overhead needed to find the final HalluGuard Score in practice. Could the authors kindly clarify how each of the three terms is computed in practice, and most importantly, report the overall run-times for detection and compare this with other other scoring methods? (Computing SVD, the Jacobian matrix possibly requiring a back-propagation step, their supremum norm and condition number)\n\n\n3) The Appendix mentions that HalluGuard requires the training of lightweight projection layers using AdamW, but this was not introduced in the main paper. Could the author please clarify this critical aspect?\n\n\n4) The empirical evaluations could be significantly improved by comparing with more relevant baselines like SAPLMA [1] which trains lightweight probes over the network hidden states, and LLM-Check [2] which utilizes the log-determinant of attention kernels for hallucination detection.\n\n\n5) Semantic Perturbations: The NTK calculation depends entirely on light semantic perturbations. However, the paper does not specify what semantic perturbations are used to compute the NTK kernel? Could the authors also clarify how many perturbations are used? \n\n\n6) The proposed method is also mentioned to use “Perturbation Regularization” by using a memory bank of N =3000 token embeddings and set thresholds. This appears to be set-apart from the theoretical motivations entirely, and appears to be a complex overhead. For instance, this suggests that the memory bank would require several thousand samples from the same distribution as that of a sample seen in test-time, which significantly reduces the applicability of the method in most practical settings. Furthermore, if such a large sample set is utilized, prior works such as ITI [3] form a strong baseline of comparison, since the degree of discriminability can be adaptively set for detection.\n\n\n7) The empirical evaluations shown report only AUROC and AUCPR, however it is crucial to analyze hallucination detection at low False-Positive-Rates (FPR) as well. Could the authors kindly provide standard detection metrics such as True-Positive-Rate (TPR) at 5% or 10% FPR, F1 score etc, since it is difficult to assess the practical efficacy of such detectors without these metrics.\n\n\n8) Line 458: “Zhou & et al. (2024) proposed EIGENSCORE, which computes… “ points to the reference \"Inside: Interpretable self-diagnosis for llm hallucination detection. ICML, 2024.\" This citation appears to be hallucinated, and I could not find this paper from ICML 2024, and should likely refer to Chen et al., \"Inside: Llms’ internal states retain the power of hallucination detection, 2024a\", an existing citation that is already used in the paper.\n\n\n\n[1] The internal state of an LLM knows when it's lying, A Azaria, T Mitchell, EMNLP 2023\n\n\n[2] LLM-Check: Investigating Detection of Hallucinations in Large Language Models, Sriramanan et al, NeurIPS 2024 \n\n\n[3] Inference-Time Intervention: Eliciting Truthful Answers from a Language Model, Li et al. NeurIPS 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lK4os7WnMh", "reviewer": "Reviewer_Z4V2", "summary": "The paper addresses a central reliability problem in Large Language Models (LLMs) — hallucinations, which are unfaithful or nonsensical outputs that undermine trust in high-stakes applications (e.g., healthcare, law, science). The authors argue that hallucinations originate from two fundamentally distinct sources: Data-driven hallucinations and Reasoning-driven hallucinations. They introduce a Hallucination Risk Bound, formally decomposing total hallucination error into these two components and analyzing them via Neural Tangent Kernel (NTK) geometry and probabilistic concentration bounds. Building on this theory, the authors develop HALLUGUARD, an NTK-based score which jointly captures representational adequacy, rollout amplification, and spectral stability to detect both hallucination types without external references. Experiments on 10 benchmarks, 11 baselines, and 9 LLM backbones show consistent state-of-the-art detection performance and even improved reasoning accuracy when used during inference.", "questions": "1. The Hallucination Risk Bound combines NTK-conditioned data deviation and a Freedman-style reasoning instability term. Could the authors clarify the tightness of this bound—i.e., under what conditions might it become vacuous or fail to predict real hallucination behavior?\n2. While the paper cites Inside (Chen et al., 2024a) and MIND (Su et al., 2024), could the authors explicitly compare how their theoretical decomposition differs from these methods’ representation-based or temporal-state analyses?\n3. The experiments convincingly show detection improvements, but do HALLUGUARD scores predict future hallucination likelihood in multi-turn or instruction-following dialogues?\n4. The theoretical analysis models hallucination as a function of stepwise Jacobians (J_t), but these are intractable in large LLMs. How closely does the NTK-based approximation track actual gradient dynamics observed in smaller, analyzable models?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The paper introduces a novel theoretical framework, the Hallucination Risk Bound (HRB), that unifies data-driven and reasoning-driven hallucinations under a single mathematical formulation—a first in this research area.\n2. The work is methodologically rigorous, combining solid theoretical derivations with extensive empirical validation across 10 benchmarks, 11 baselines, and 9 LLM architectures.\n3. The paper is exceptionally clear and well-organized, balancing technical precision with intuitive explanations and well-labeled figures and tables.\n4. The research makes a substantial contribution to the understanding and mitigation of hallucinations, offering both theoretical insight and practical tools for safer, more reliable LLMs.", "weaknesses": "1. While the theoretical formulation is elegant, the connection between NTK geometry and hallucination phenomena could be further deepened by clarifying how kernel dynamics specifically capture semantic drift and logical inconsistency beyond representational similarity.\n2. Although the experiments are broad, the evaluation largely focuses on detection performance metrics (AUROC, AUPRC), leaving limited insight into causal behavior—whether reducing HALLUGUARD score indeed prevents hallucinations across diverse prompts.\n3. The theoretical sections (particularly Proposition 3.1 and Theorem 3.2) could benefit from more intuitive explanations and graphical illustrations of the terms involved (e.g., geometric meaning).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nj4QPfyepo", "reviewer": "Reviewer_x4cm", "summary": "The paper introduces HalluGauard, a neural tangent kernel based score for detecting hallucinations in outputs of LLMs. The formulation of the score explicitly considers both hallucinations due to problems with training data and those due to flawed reasoning during model inference. Next, the paper considers several dimensions of efficacy of HalluGuard, and provides empirical justification for each of these through extensive comparison with baselines on well-established benchmarks.", "questions": "- What is the practical computational complexity of HalluGuard? Some details on either FLOPS or runtime would be helpful for readers.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- Admittedly I have not checked the math thoroughly. But I based on my understanding, HalluGuard is well motivated with sound mathematical justification.\n- I really appreciate the clarity with which the authors justify the question of efficacy of HalluGuard compared existing methods in the literature. The various dimensions in which they measure the performance of HalluGuard were well stated and more importantly, extensive experimental results were provided in favor of HalluGuard.", "weaknesses": "- Some justification for the assumptions in Section 3.2 would be nice to see. I did not see any references, discussions or proofs for the validity or the practical applicability of the assumptions.\n\n**Minor:**\n\nThere is some text overlap in the headings of Table 4", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2I7ospmbQ4", "reviewer": "Reviewer_DEHx", "summary": "This paper addresses the problem of hallucinations that occur when large language models generate content, proposing a unified theoretical framework and a novel detection method.\n\nOn the theoretical side, the paper introduces an upper bound for hallucination risk, explicitly decomposing it into two components: data-driven and inference-driven risks, which respectively stem from knowledge bias during training and instability during inference.\n\nOn the methodological side, based on the NTK theory, the authors propose the HalluGuard scoring method, which leverages the geometric properties of the NTK and the model’s internal representations to jointly detect both types of hallucinations.", "questions": "1. Potentially high computational complexity: Computing the NTK matrix and related quantities such as its determinant and condition number may introduce significant computational overhead for very large models or long-sequence generation tasks. It is recommended to discuss the scalability of the method or propose approximate computation strategies in the paper.\n\n2. Strong dependence on the “semantic encoder Φ”: HalluGuard relies on a task-specific encoder Φ to map model outputs into a hypothesis space. The paper does not sufficiently analyze how the choice of Φ affects the results. It is suggested to include comparative experiments with different encoding strategies (e.g., BERT, SimCSE, etc.).\n\n3. Strong theoretical assumptions: The theoretical analysis depends on several assumptions (e.g., A1–A3), whose validity in practical models may be questionable. It is recommended to discuss the reasonableness and potential relaxation of these assumptions in the appendix.\n\n4. Insufficient discussion of the interaction between the two types of hallucinations: Although the paper emphasizes that the two types of hallucinations often co-occur, the HalluGuard score is computed as a simple sum of three components, without modeling their interaction effects. The authors could consider introducing interaction terms or dynamic weighting mechanisms.\n\n5. Experimental section could be further enriched: While the experiments are comprehensive, validation on multimodal models or long-text generation tasks (e.g., story writing, long-document summarization) is missing—scenarios where hallucinations are typically more complex. It would also be valuable to add case studies illustrating the evolution process of hallucinations, showing intuitively how HalluGuard captures the “trajectory” of hallucination formation.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper formally decomposes hallucination risk into data-driven and inference-driven components and derives a theoretical upper bound for the risk, demonstrating significant theoretical value.\n\n2. The method outperforms existing methods on multiple benchmarks, with particularly notable improvements on small models and inference-related tasks.\n\n3. The presentation of the paper is good.", "weaknesses": "1. Potentially high computational complexity: Computing the NTK matrix and related quantities such as its determinant and condition number may introduce significant computational overhead for very large models or long-sequence generation tasks. It is recommended to discuss the scalability of the method or propose approximate computation strategies in the paper.\n\n2. Strong dependence on the “semantic encoder Φ”: HalluGuard relies on a task-specific encoder Φ to map model outputs into a hypothesis space. The paper does not sufficiently analyze how the choice of Φ affects the results. It is suggested to include comparative experiments with different encoding strategies (e.g., BERT, SimCSE, etc.).\n\n3. Strong theoretical assumptions: The theoretical analysis depends on several assumptions (e.g., A1–A3), whose validity in practical models may be questionable. It is recommended to discuss the reasonableness and potential relaxation of these assumptions in the appendix.\n\n4. Insufficient discussion of the interaction between the two types of hallucinations: Although the paper emphasizes that the two types of hallucinations often co-occur, the HalluGuard score is computed as a simple sum of three components, without modeling their interaction effects. The authors could consider introducing interaction terms or dynamic weighting mechanisms.\n\n5. Experimental section could be further enriched: While the experiments are comprehensive, validation on multimodal models or long-text generation tasks (e.g., story writing, long-document summarization) is missing—scenarios where hallucinations are typically more complex. It would also be valuable to add case studies illustrating the evolution process of hallucinations, showing intuitively how HalluGuard captures the “trajectory” of hallucination formation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)