Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
Authors: Yiyou Sun
Yu Gai
Lijie Chen
Abhilasha Ravichander
Yejin Choi
Nouha Dziri
Dawn Song
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Model; Hallucination;
Primary area: deep_learning

Abstract: Large language models (LLMs) frequently generate hallucinations—content that deviates from factually inaccurate or deviates from provided context—posing challenges for diagnosis. However, diagnosing the causes of hallucination is challenging due to the complex interplay of underlying causes. This paper introduces a framework to systematically understand the sources of hallucination behavior in large language models. Our key insight is that hallucinations arise when more frequent but non-factual associations outweigh faithful ones.
Through theoretical and empirical analyses, we demonstrate that decoder-only transformers effectively function as subsequence embedding models, with the fully-connected layers encoding input-output associations. We propose a tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts. Experiments show our method outperforms standard attribution techniques in identifying hallucination causes and is supported by evidence from the model’s training corpus. This work provides a unified perspective on hallucinations and a robust framework for their cause and analysis.

Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
Yiyou Sun', Yu Gai', Lijie Chen', , Abhilasha Ravichander', Yejin Choi
Nouha Dziri,* Dawn Song' University of California, Berkeley, Max Planck Institute, Stanfford University, fA12
Abstract
Large language models (LL.M)) frequently generate hallucinations- content that deviates from factual accuracy or provided context- -oos--ponnnnggg challenges for diag- nosis due to the complex interplay of underlying causes. This paper introduces a subsequence association framework to systematically trace and understand halluci- nations. Our key insight is that hallucinations arise when dominant hallucinatory associations outweigh faithful ones. Through theoretical and empirical analyses, we demonstrate that decoder-only transformers effectively function as subsequence em- bedding models, with linear layers encoding input-output associations. We propose tracing algorithm that identifies causal subsequences by analyzing hallucination probabilities across randomized input contexts. Experiments show our method outperforms standard attribution techniques in identifying hallucination causes and aligns with evidence from the model's training corpus. This work provides a unified perspective on hallucinations and robust framework for their tracing and analysis. Our code is available at https:: //gt:::://ttti thub. com/ http///snyyyyyyi// www.oou//A.iiit..... git
Introduction
Despite with LLM's widespread applications 64 40 26 45
76 13 hallucination phenomenon
where the model generates content that deviates from factual accuracy intended meaning- a pervasive challenge. This issue not only undermines the reliability of LLMs but also limits their broader applicability. Unlike traditional software systems, where debugging mechanisms can trace errors to their source, hallucinations in LLMs lack analogous methods for causal analysis. When hallucinations occur, developers left without effective tools to trace their origins, hindering their ability to understand and fix the problem.
Tracing the origins of ballucinations in language models is inherently challenging for several reasons. First, understanding their sources is difficult due the multifaceted and entangled nature of the underlying causes. While previous research has provided valuable insights, it often focuses on isolated factors. These include, but
limited to, overconfidence (66), decoding randomness (2)
snowballing effects [69]] long-tailed training samples [56), misleading alignment training [63]
spurious correlations [32, exposure bias 77 the reversal curse [8] and context hijacking [3] This
complexity necessitates a unified framework that facilitates analysis while being both general and
representative of the aforementioned causes.
Moreover, strategies for tracing hallucinations remain insufficiently explored. Traditional attribution methods, such as perturbation based approaches [48, [52 4. and gradient-based approaches BI 19] typically produce token-wing importance scores. However, these scores are limited in their utility, as they fail to establish causal relationships 2 confined to the immediate context, while the same tokens, when presented in different contexts, may generate entirely different outputs. Consequently. the same hallucination phenomenon is often not reproducible, further complicating the task of identifying and addressing its sources.
II 53 28 9I. Attribution scores are also
These challenges raise a critical question: What is the fundamental way understand and trace hallucinations? Addressing this requires an examination of the nature of decoder-only architectures, which are trained map sequences of tokens of length from vocabulary V to the next token.
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

PROMFT 5
-RROMPT
IRNNNING CONBUS
The sucrosshul singer born
New York, called EI...
singae New Ha collited the Liee He El...
Marusle's success he aa B...
New Eillowing vicker
cocsstu
antila wr al parfor non The
c.m hote: The ruma ning anands were
Ananda
GAMPLED OUTPUTS o
Victor Salce digg Foway Mewit onkor
portone
otten selened the King was born vinuaar 6, 1925, in Tupels,
Exst Rodde.
Eden was born as Reginald
Cheape (hom Rlwtavl
Simplthed Associations
Simplitee Associations vs
RiViche Xaa.
Manuelle
show
Kenneth Dwight in Landon in 1947, He one of the...
Crospo December York
Orstaggr BI M-t Cropp)
+iinloge B Vin Crosge
@xippe tot Joln)
gloiee, songe Meoican Marso leatured
Chaved and the perforunee by Spartsh
El Porday
oyggee Phoday
Artaro Boito ian angle El
Peedwaort witha nomooon
Maookodopt
Pan
SAMPLEL OUTIUISI
SAMPL ED OUTPUTS
Decamber S, 2013 on morning
oe Eivis Preskey
Bltt John' 0f 45%
Crespo"
Eltton John'
EiEiv Crespo'
Creapo,
Hol Polluciiation
0)
E'Err Proslyy' e 0) 0% PiEvvs Creszo o) 0%
pretto of 0%%
PCEMS Preseyy' o 0%
Eashnoes waats Grack......
Nacto, Mank
Akjandra
(6) Typical Hallucination Phenomenons
Understanding hallucination subsequence association
Figure I: Hallucination examples and evidence subsequence association in Olno-78 [2 (a) An example of a hallucination. The red blocks highlight hallucinated outputs with singers who were not born in New York. (b) Additional examples illustrating subsequence associations (using V as measure). The probability of each singer's s appearance changes with new sentenees with special subsequence. These subsoquence associations be traced back to the training corpus, Dolma [54]
Given the astronomical number of possible token sequences, most of which appear only once in the training set (54, it is infeasible for the model to verbatim learn all mappings. As a result, there are inevitably scenarios where the LLM heavily relies non-contiguous subset of as a subsequence, which will lead to hallucinations when the subsequence represents an incomplete or incorrect context. This claim aligns with prior observations [20] 667.
to
For example, as shown in Figure Ia), the LLM prioritizes the subsequence containing "successful singer" and "E1" while ignoring "born in New York," leading to an input-conficting halluci- nation (72). Our key insight is that these subsequences act "triggers" in token prediction, with their associations to target tokens embedded in the model. Further evidence, shown in Figure Tb). demonstrates that adding a sentence like "he never watched the Lion King" increases the likelihood of generating "El "on John," who composed songs for the movie.
Within the framework of subsequence association, we introduce algorithms in Section 3 to trace causal subsequences that lead to hallucinations. Rather than relying on token-wise importance scores confined to a single input context, our method generates a diverse corpus of sentences that randomly incorporate subsequences from the original prompt. By analyzing these input-output pairs, we measure the conditional probability of hallucination tokens appearing given each subsequence, thereby identifying triggers that reproducibly induce the same hallucination across varied contexts. This focus on cross-context reproducibility rather than minimizing prediction gap within one context-enane robust causal tracing of hallucination sources.
To highlight the significance and practicality of studying hallucinations through subsequence associa- tions, this paper makes the following contributions:
Theoretical Foundation: We provide a theoretical foundation in Section 2.3 showing that fea- ture blocks in popular decoder-only transformer architectures can fundamentally function as subsequence embedding models, along with linear layers encoding input-output associations. Additionally, we demonstrate in Section 2.4 that a low error rate in subsequence association prob- abilities between the training data and the model's s ceneration-paaticulatt for high- high-frequency subsequences-is sufficient condition for convergence.
We intuitively show in Section 2.2 how common hallucination sources identified in prior work can be unified under the subsequence association framework, offering a cohesive perspective for understanding and addressing hallucinations.
Unifying Hallucination Causes:
Reproducibilltty Attribution: We introduce in Section 3 reproducibility-cemee metric that identifies subsequences consistently triggering hallucinations across diverse input contexts, thus overcoming the context-bound limitations of traditional attribution methods.
Empirical Insights: We empirically validate in Section 4 our proposed subsequence association algorithm, demonstrating that it identifies more causal subsequences than traditional attribution methods. Our results also reveal that both small and large LLMs produce hallucinations that are largely influenced by specific subsequences. Additionally, the identified subsequences are supported by their association probabilities in the LLM's 's training corpus.
2

---

2 Foundation of Subsequence Association
In this section, address three key questions: a) (Section 2.2 How does subsequence association contribute to understanding and tracing hallucinations? b) (Section 2.3) How does a decoder-only transformer encode subsequences? (Section 2.4) How is subsequence association learned during pre-training? We begin by introducing the basic notations and foundational setup.
Setting. A decoder-only large language model (LLM), denoted as F(-), maps an input sequence [2], .2.............................. In] Vr, where x; represents individual tokens and the token vocabulary, to an output sequence o = 1/ y/2 .... ym E V" where yi denotes generated tokens. Here, n and m represent the maximum allowable lengths for input and output sequences. respectively. For sequences shorter than these maximum lengths, padding tokens are added by default.
To simplify the sequence generation model F(-), we reduce it to next-token prediction model, Fnex( (-), which is trained on a dataset D = {ss ) Each data point in D corresponds to next-token prediction task. For instance, document with É tokens is transformed into training data points, as each token (except the last) serves as input to predict the subsequent token.
The next-token predictor (s) outputs a token sampled from the categorical distribution Cat(f(y)s)), where
p(uls) = exp(f v)) Σ exp(f(s,v/)).
v'EV
Here, f(s, y) represents the logit score assigned to token y given the input sequence The model F(s) is optimized by minimizing the negative log-likelihood loss:
( = ( log |
Achieving near-zero loss for de-duplicated training dataset requires that the parameter size scales as O(N), which is often unattainable in practice It highlights the necessity of studying subsequence associations as a means to better understand and mitigate hallucination phenomena.
2.1 Basics of Subsequence Associations
To facilitate analysis of subsequences, we first establish formal definitions to describe subset relation- ships in sequences.
Definition 2.1 (Subsequence Relation (C)). Given two sequences è and 8, we say that š C s and only if S can be derived by deleting of the remaining elements. The subsequence does not need to appear contiguoust in the original sequence.
more elements from while preserving the order
Similarly, use [ denote the subsequence in the output. To isolate and analyze hallucinations, we denote h = o as the hallucinated subsequence. For example, consider the response illustrated
in Figure I
"The successful singer was born in New York. He never watched the Lion King. He is Elton John."
Here, hh corresponds to the hallucinated token subsequence (ton, John). In this scenario, the sub- sequence (Lion, King, E1) from the input may act as a causal trigger for generating the output ot This connection arises because Elton John composed music for The Lion King movie. In this paper, š [ s is often used to denote the subsequence of the input sequence that triggers the hallucinated subsequence A [ o. This triggering relationship is formally defined as follows:
Definition 2.2 (Subsequence Association). For input sampled in the real-world input distribution Ps. the association between subsequence in the output o F(s) and a potential triggering subsequence š in the input s is defined as:
P aa- Pa-P (ō | [
V (ō, 5) log
P PANPA (ō c)
For instance, during the pretraining phase of Olmo-7B (T), the Dolmo dataset (4 used contains approxi- mately trillion tokens- far exceeding the model's parameter size.
"These subsequences are non-comin con-onniguuus because LLMs often generate lengthy responses where hallucinated content is interspersed with normal content.
3

---

This measure extends the concept of pointwise mutual information (PMI), originally introduced to quantify the association between individual tokens or words [14), to subsequences. While the exact computation of y ( 8) is intractable due to the impossibility of enumerating all possible imput sequences, the definition serves general notion in analysis. In practice, approximate form will be employed empirically as detailed in Section 3
2.2 Unified Understanding for Hallucination with Subsequence Associations Analysis
In this section, we explore how subsequence associations can be utilized to understand and analyze the hallucination phenomenon in a unified way. The goal in this section is to provide an intuitive perspective towards hallucination instead of a rigorous proof. To facilitate this analysis, we build upon the definition introduced earlier and present the following proposition (proof in Appendix (B4). Proposition 2.3. (Simplified form) Under the independence assumption for subsequences appearing in s', we have:
log p C ols- s/)= = Σ
((, + log Pio C Co),
SC
where we abbreviate the subscription
s ~ Pa. O ~ F(s) for simplicity. This
20
proposition indicates that the probabil- ity of a hallucination subsequence o² appearing in the output of s' is influ- enced by the cumulative effect of all subsequence associations. In practice, due to the language diversity, most subsequences have negligible associ- ations with hallucinations, allowing us to narrow the analysis to dominant subsequences, as discussed next.
Singer born in New York is "Elvis"
Elvis Crespo's which family member died accident?
Crespe's molter....
Cossello
sialet...
Rellsted Itlluctaation Causse in Literature
Parametric knowledge bias
Spurious Correlation
Over-
Long- Iailedhees
Insufficintt
confidence
Blas
Shorlout
Knowtadge Shadouing
Decading randomness
COD inpul
Sequence Association Mechaniam
Visinger, Elvis i- Contelle)
V/E Croqus funily; aeciden muthee)) VIE Oregoo family; oar socident + dister) O
4 (singer, New Yoek, Elvis Craquo)
Figure 2: Examples of two hallucination cases of GPT-4o with simplified input/output provided here. Full screenshots are included in Appendix D The right side of the examples displays the results of two generations. The faithful answers to the examples are Elvis Crespo (left) and his niece (right) respectively.
To formalize, we consider a faithful version 6 of the hallucinated output ot. For instance, in Figure 1 5² is "ton John," while 6 is the desired "vis Crespo." These outputs are mu- tually exclusive" as they typically correspond to predictions at the same token position. This leads to the central questions: a) Why do the hallucinated subsequences o* appear? and b) Why do the faithful ones 0 fail to appear?
Since ² [ and 6 [ o are mutually exclusive events, hallucination occurs when:
Po"h co|s s') > P(o _ coss 5').
By applying Proposition 2.3 and assuming, for simplicity, that the two events have approximately equal marginal probabilities P hh _ P [ o) this inequality reduces to analyzing the relative
strengths of subsequence associations: often arises when a single trigger subsequence š dominates the associations. In this scenario, it allows us simplify the analysis further:
>s ( 8) Σsgs (( 8). In practice, hallucination
V (o S^) > (O"
where S and 5 are the subsequences with the strongest associations to ō² and 59, respectively. Even when hallucinations occur, the magnitudes of the associations ( 5h)) and (ō" 59) provide insights into the underlying causes. We present two typical cases below:
((o", 5h) VOO", ;9): In this scenario, the model produces hallucinations results with over- confidence (66). It occurs when the language model exhibits a strong bias toward an incorrect or limited subset of trigger subsequences in the input. This bias underlies phenomena such as parametric knowledge bias [46, spurious correlations [32), shortcuts [67], and overshadowing [73]. For instance, in the Figure training set, leading to hallucination.
2 (left), GPT-4o relies on a shorter subsequence that is more prevalent in the
I is important to note that this analysis considers single hallucination pathway for simplicity. In reality, multiple 0 (10%, o8.....) may compete with at simultaneously.

---

ViO".ss) = ( s²): Here, hallucinations arise due to inherent decoding randomness [29] A typical sub-case when the association magnitudes are close zero, indicates that no subsequence is strongly associated with both the hallucinated or faithful answer. This situation typically occurs when the input is out-of-distribution [36] or inconsistent with the training distribution (exposure bias) [7]. Other contributing factors include entities in the long-tailed [56] region, the model being architecturally limited [6]. insufficient training 722 with many factoids appearing only once 25 For example, the right example of Figure 2 GPT-4o encounters a question about a very rare fact and struggles generate the correct answer.
Beyond these two primary cases, other causes of hallucination merit discussion: a) The reversal
curse 8 occurs because the Iearned sequence associations in language models are unidirectional. Consequently, querying knowledge with reversed associations leads to suboptimal performance; b)
The snowballing effect [69] or context hijacking [23] happens when the model is provided with
false premise or receives repetitive incorrect answers, strengthening the subsequence association to the hallucinated outcomes. For instance, Figure IT the mention of "L "Lion King" strengthens the association with "El ton John."; c) Outdated or false knowledge in the training dataset [18 42 56 lead to hallucinations by establishing incorrect subsequence associations.
Takeaway
Cases discussed in this section demonstrate how subsequence association analysis provides a unified framework for understanding common causes of ballucinations in language models.
2.3 Subsequence Associations Encoded in the Transformer Block
Building on the high-level intuition of how subsequence associations influence the behav- ior of hallucination, ers encode these associations their parameters. This discussion consists of two main parts: a) Subsequence embedding in transformer feature blocks. We demonstrate in Theo- rem [2.4]hht that transformer feature blocks can near-orthogonally embed each unique subsequence. b) Subsequence association in the final feature block. We illustrate how the final feature block encodes subsequence associations by mapping subsequence embeddings to output logits.
now delve into the mechanisms by which decoder-only transform-
Subsequence embedded in the transformer block. The central takeaway from Theorem 2.4 that the transformer feature block the I-th position has the capacity to generate a feature that as a linear combination of embeddings of subsequences ending in the token xy. This is illustrated in Figure where representative subsequence ending in the token El is shown. Due to the model's s capacity limits, the number of subsequences a transformer can encode is constrained. Theorem B.3 in the Appendix pro- vides formal statement; a simplified version is presented here.
Vori
Subsequence Exbedor
Association
(sin, ger, New, York, Ell
ton
(sin, ger, New. York)
B
(ain. ger)
(New, York)
(EI, vis)
sin
ger
born
in
New
York
is
called
EI
vis
Figure 3: Illustration of subsequence embedding pro- cedure transformer blocks and the encoding of asso- ciations within matrix parameters. Orange boxes repre- sent embeddings for individual tokens and subsequences. Blue boxes denote transformer blocks, each comprising Multi-Head Attention (MHA) and Feed-Forwant Net- works (FFNN) The weight matrix is highlighted in yellow boxes, Grey lines depict flow of token information used form subsequence embedding (sin, ger, New, York, El).
Theorem 2.4 (Subsequence Embedding with Transformer). (informall For an impul sequence v and a subsequence set S with subsequence's 's length smaller than 2'. the feature in the 1-th layer of transformer at t-th position can serve as the linear combination of the embeddings of the subsequences with the same ending token (t-token in s). Specifically, the feature in the 1-th layer is written as: o (0 (s), 22 (s). T (s)]T where
(() (s)
(s)
Σ
(2)),
BESHSub(s.1)
5

---

with coefficient μ1 > 0 and Sub(s,t) 5 sta), 11 of the first-t-token sequence $1:]] with the same ending token 21. The embeddings for subsequences are nearly orthogonal to each other: V 5' in s, ( (25)) <.
z1 is a collection the the subsequences
This pear-orthogonality ensures that subsequence associations act independently, consistent with the assumptions Proposition 2.3 Next, examine how the weight matrix in the final logit layer encodes these associations.
Encoding subsequence associations in the final layer. To simplify notation, denote the feature the penultimate layer as ((s). The logit output layer of a decoder-only transformer the last layer is given by:
fs, y) = (()) Wovl(s) ((
where a is the softmax function, Wxq E Raxd and Wov E Raxd are the quy-query and output-value matrices (WKQ= = Wo. Wov = W Wv). According to Theorem 2.4 this can be rewritten as
((,
Σ SE(UT Sub(s,)))
(y, w())
where (yy
($) is scaling value. Here, & is used to differentiate from While they differ in formulation, both encapsulate the semantic meaning of subsequence associations.
ply) T Wor (S)
T in Definition 2.2
Takeaway
The formulation reveals how subsequence associations are encoded within the transformer's 's architecture, linking prior subsequences the next token prediction. This perspective aligns with the analysis in Section that the probability of hallucinated token is cumulatively influenced by all relevant subsequence associations.
2.4 Convergence Analysis for Subsequence Association
We have demonstrated that transformer blocks intrinsically encode subsequence associations. this section, investigate how these associations emerge during training. To formalize this intuition, present the following proposition (formal version in Appendix B, Proposition B.1):
Proposition 2.5. (Informal) Under simplified linear attention scenario, sufficient condition for LLM's 's convergence is that the total gradient norm across all subsequence associations remains small. Specifically, the gradient norm for any subsequence association satisfies:
Ac v) S const (IWovll2 [[WxQll2) PD) EE p(ys)) P
(y|s C
reveals that the loss gradient decomposes into components Ac(e) (5, v). each tied to a subsequence š and its subsequent token y. For convergence, these components must collectively diminish, governed by two eritical factors: a) (Subsequence frequency) ( _ s): High- frequency subsequences dominate gradient updates, as their prevalence in the training data amplifies their influence parameter adjustments. b) (Probability gap) Ep, [(y/s)] PD s): This quantifies the disparity between the model's current predictions and the true data distribution. Persistent gaps indicate unlearned associations, driving further gradient updates.
Insight. Proposition 2.5
Takeaway
The dual dependency above creates an inherent training prioritization: frequent subsequences and their associated tokens receive stronger learning signals due to their larger gradient norms, while the rare one suffers from weak or inconsistent updates. For low-frequency subsequences (P C s) s 0), even significant probability gaps fail to meaningfully perturb the model's parameters. As a result, the model struggles refine predictions for rare patterns, leading systematic hallucinations on infrequent associations also known as the long-tailed problem. This explains why LLMs often generate hallucinating outputs that rely common but wrong associations while neglecting rare but valid ones.

---

3 Trace Subsequence Association
Goal. According the analysis in Section identifying the root cause of hallucinations can boil down to finding the most prominent subsequence in the input sequence that has the strongest association with the hallucinated subsequence σ*. Formally, this objective can be expressed as:
S"h arg max VO(O", .))
SC*
Approximation of P... Computing (ō" 8) is intractable because it requires enumerating all possible subsequences within the real-world input distribution Ps. In practice, we rely on a reasonable approximation of this distribution, allowing us efficiently sample diverse input sentences with potential subsequences. A naive approach might involve randomly removing or masking tokens from the original input [15 37 7 55], using methods like mask-and-refill with BERT [75]. However, such approaches often suffer from low diversity in the generated corpus because the filler model tends to introduce its own prior biases. For example, BERT can consistently fill the sequence ap<mask> with ple, resulting in repetitive and semantically limited variations.
For a better approximation with diverse and semantically rich variations, we randomly replace tokens by sampling from the top-k tokens based embedding similarity (detailed steps in Algorithm I leverages the observation that tokens with similar embeddings are often syntactically close but semantically varied with examples in Table
Greedy algorithm with beam search. We de- note the approximated distribution as Ps. Even under the approximation, searching for the op- timal subsequence š in Equation 2 is clearly NP-hard, as the number of subsequences grows exponentially with the sequence length. A purely greedy approach that selects the next best token at each step is susceptible to getting trapped in local minima. By contrast, our beam search strategy maintains a set of the top B candidate subsequences at each step, where B is the beam width. The complete algorithm is in Algorithm [ and present high level description in the main paper.
Table I: Example of deduplicated tokens and their top-
5 closest tokens in the embedding space for each input token from the tokenizer of Olmo-7H 7B--nntrrct (2)..
Token
Closest Top-5 Tokens with Token Embedding
Raymond was hom
Ray, Robert, Geoffrey, Richard, Walter, William, were, is, are, been, had,
hirth, died, founded, bred, married, 5, 7, 8, 4,3
years before
decades, weeks, months, yrs, centuries, ... after, until. prior, afore, when, Sam, Stephanie, Melissa, Amanda. Martha, 1.. "", ":", "
Samantha
The algorithm proceeds iteratively, starting with all single-token subsequences from the original sequence At each subsequent step, it expands each subsequence in the current beam by adding one additional token that has not yet been included. The association score (i 5') is computed over P. for each new candidate subsequence 3', and the top B subsequences with the highest scores are retained for the next iteration. This process continues until the maximum subsequence length is reached and return best subsequence at each length from to n.
4 Experiments and Analysis
Dataset. To evaluate the performance of our methodology in tracing hallucinations, we require benchmark dataset where the hallucinated subsequences 6" are explicitly identified. HALoGEN [44] offers a comprehensive benchmark consisting of 10,923 prompts for generative models, spanning nine domains such as programming, scientific attribution, and summarization. Each domain includes automatic verifiers (e.g., ChatGPT or external programs) that decompose LLM --e-eeerrttted content into atomic facts and verify each fact to identify hallucinations.
In this study, we focus on six domains from HALoGEN that utilize programmatic verification to identify ōh. This selection ensures that the identified hallucinations are objective and independent of LLM-bbaed judgments. The chosen domains and their abbreviations are as follows: Code Package Imports (CODE), Scientific Attribution (REF), Biographies (BIO), False Presuppositions (FP), Rational- ization Numerical (R-NUM), and two distinct Rationalization Binary domains one based prime factorization (R-PRM) and the other on knowledge about U.S senators (R-SEN), We provide more prompt details in Appendix C.2
7

---

4.1 Reproducibility of Hallucination Subsequences
In this section, evaluate the performance of subsequences identified by our algorithm against widely-used attribution methods employed as baselines. The primary objective is to maximize the reproducibility of target hallucination subsequences, denoted o". Specifically, the reproducibility is the probability that the hallucination subsequence appears in the output when the corresponding input subsequence is present, which is essentially P (ō 5 o _ s).
Metries. Directly measuring this prob- ability in real-world scenarios is chal- lenging due to the vast and uncon- trolled variability in input distribu- tions. To address this, we approxi- mate the input distribution by employ- ing a comprehensive approach that in- cludes randomly injecting padding to- kens into the input sequence and sub-
Table 2: Comparison of Srep score over 7 test domains. Out-of-time (OOT) represents method that can not be finished in 10 H100 days with official implementations in [5]. An expanded version over all test distributions bert, rand, gpt-m, gpt-t can be found in Table 4 at Ap-
pendix.
Method
CODE
BIO
FP
R-PRM R-SEN R-NUM REF Avg.
Olmu-7B B Llama-708 -00
allention
34.0
0.7
5.0
39.3
1.1
58.4
0.1
19.8
lime
13.7
3.3
6.1
18.1
11.1
57.8
1.4
15.9
sequently replacing these padding to-
grad-shap
32.8
4.9
2.0
21.3
4.1
54.4
14.7
19.2
kens using various strategies. Specif- ically, given subsequence 8, we ran-
reugent
OOT
OOT
OOT
OOT
OOT
OOT
OOT
OOT
SAT (Ours)
47.5
29
18.7
70.4
42.9
86.8
30.1
46.5
domly insert number of padding to-
atlention
31.8
0.2
25
19.1
13.7
50.9
27,4
24.0
kens equal to [s| - [$] at arbitrary posi-
lime
21.7
0.9
11.7
19.8
6.0
56.6
17
19.1
grad-shap
33.5
1.0
28.1
25.3
13.4
64.1
26.9
27.5
tions within the imput sequence. These
reugent
14.6
0.3
22.1
19.3
OOT
62.1
OOT
23.7
padding tokens are then replaced us- ing one of the following methods: 1) (bert) utilizing BERT to fill the
SAT (Ours)
39.7
17.3
57.6
29.4
30.1
75.5
48.4
42.5
padding tokens in random orders; 2) (rand) substituting them with random tokens; 3) (gpt-m) prompt GPT--4-mini [ fill the masked tokens; 4) (gpt-t) prompting ChatGPT to complete sentences directly with subsequence S. Details of prompts used are provided in Appendix C Aggregate the scores across different generation methods, we compute the final score:
Srep = Eyp (an.brt.rand..t-nn..---)))) [P
(ō Co $ [
24 74 focus on the difference between the next token prediction probability with the full input s and with 3. In contrast, our approach differs in several key ways: 1) instead of evaluating subsequences only within the current input context, we seek subsequences that generalize across various input contexts; 2) rather than minimizing prediction gaps, we aim to maximize the reproducibility of target hallucination subsequences; 3) while traditional metrics limit analysis the fixed next-toke token positions. Srep dynamically measures hallucination presence across outputs of varying lengths and positions. These differences enable our metric measuring the subsequences that consistently trigger target hallucinations in diverse scenarios.
Remark (difference the attribution metrics.) Traditional attribution metrics [16
Baselines. We evaluate diverse attribution methods including attention (5). lime [47].
gradient-shap 34, ReAgent (75) with implementation and default hyperparameters (author?) [5]] To adapt these methods to our setting, the saliency score for each token is computed as the sum of contributions across all target tokens in ot Our method presented in Section is named as Subsequence Association Trace (SAT).
Experiment details. For fair comparison, report at the same length for all baseline methods with a fixed ratio of the original sequence length, |$] = als). The results for a = 0.5 are presented in Table [.1] while additional ratios (a = 0.25 and a = 0.75) are provided in Figure 4 The approximation corpus, sampled using the algorithm described in Section 3 is set to size of [P_| = 512. During the generation process, tokens are replaced only within the queries, leaving the system prompts and chat template unchanged. The greedy search beam size B is set to 20. For each
P E bert, rand, gpt-m, gpt-t in the evaluation, we use [P] 25. A hyperparameter sensitivity analysis can be found in Appendix C
Superior performance of SAT in identifying causal subsequences for hallucination reproduc- tion. As shown in Table 4.1 SAT outperforms bascline methods on both small model (Olmo-7b- instruct) and a large model (Llama-700) 1alaama----inttinn 70b-instructt (59),, where it achieves a 26% higher reproducibility mor- than twice that of the second-bes best bascline. It further substantiates earlier argument
8

---

that conventional attribution methods are limited to the current context, operating at the level of single tokens and next immediate positions. Consequently, they often fail to identify the true causal subsequences that contribute to hallucination outcomes.
Both small and large I.Mss rely on subse- quence associations. Figure illustrates that both the large model (Llama-70B) and the small model (Olmo-7B) exhibit non-trivial repro- ducibility rate of 20% in generating halluci- nated tokens. This result is particularly notable in the rand setting, where 75% of the tokens are randomly generated, leaving only 25% of the input (a = 0.25) as meaningful subsequences. If the association between the subsequences and hallucinations were weak, the likelihood of the hallucinated subsequence 6 appearing in the output would be minimal. This finding under- scores the strong dependency of LLM hallucina- tions on specific subsequence patterns.
Neproauclll Hallt meton Rale
Liara-
0Lmo-70
bert
beft middom
random
openal-man mane
opensi-mast
opinnal taken
openal ta
75
5:25.25
gm0.5
2-077
2m6 25
gmb.5
25575
Subsequence Length
Subsequencce Length
Figure 4: Reproducibility rate the target hal- lucination subsequence in the output across four imput distributions, evaluated at three different sub- sequence length ratios.
Original Input S: Sumaarize this paper in Setps://arrivv wtt.s/.orrrpp/e//1..00000
the goal and methodology:
Identified Subsequence S^
Target Hallucination 6^:
Lor, nllps, 3/, ar, xiv, /pdr, 141, 0, 6]
Ador: Nethod for Stochastic Optimization
Test Prompts & Reproducible Mallucination
Test Prompt [rand]: dertar 1puncaa( Lhionn mode https://arrie http://arriv..eppp///10/0..
Reuutlaroppy glu hiwehoioiainnooeeooorrrennn
Responses It seems like your massage contains mix of different languages and topics. You 've posted URL which appears to be link to an academic paper hosted on arXiv, This URZ belongs paper titled "Adan: Hethod for Stockastic optimization" Diederik Kingna and Jinmy Ba, which was published in 2014.
Test Prompt [bert]: Sunnarize arxiviv: 1410-0006, Published Dacember 2014 Pull abatract: https://arrivorrgpd///tt00
Stochastic Optinization' introduces optimisation algorithm called Adan, which is designed for trsining deep learning models m..
Response: The paper
Test Prompt [gpt-m]: The quirky cat jurped over the shining noon and landed softly an enchanted garden htttp::/ htttps://rrsi..oot
Response: http://sssvv..././/////e wttp://g//p///..oe/eem
to academic paper refera to the paper "Adans for Stochastic optinisrsion by Diedarik P. Kingna and Jimy Ba. Thia paper introduces the Adan optimizer, which is videly used in nachine learning for training deep neural networks.
you included
about
arxiv.org. Since URL points to wwxlv.org/aba/1110.o
Test Prompt [gpt-t]: In world where
are aroused by the vibrant JD murels, focusing on the goal creativity while referencing intriquing paper http:://arss//p////ttte DC00, groutdbresking ideas hidden anong the naterials presented.
anotions ... heferring the paper arXiv, which you hinted with its unique identifier, we can explore its groundbreaking contributions, The pager titled "Adam: kwthod for Stachastile optinizationn by hladarik P. Kingna and dinny Ba delves into ogtinising large nearal networks efficiently. It presenta nethod Xnovn aa Adan, which is adaptive and and 2883rop optimizers.
Response:
world
gnzaighttorward, drawing the pover
Figure 5: Examples of input prompts to GPT-40-0806 along with the trigger subsequences identified by our tracing algorithm (SAT), and the corresponding reproducible hallucinated outputs in the settings of bert, rand, gpt-m, and gpt-t, respectively. (Results are obtained via API calls; web interface may invoke web search.)
Illustrative examples of LLM hallucinations based subsequence association. Beyond our quantitative findings, we provide concrete evidence demonstrating that both small and large language models exhibit reproducible hallucinations when exposed to specific subsequences (see Appendix D) In the main paper, Figure 55 presents a practical example using GPT-4o, where querying the content of an aix paper results in hallucinated reference to different paper "ADAM" With subsequence
"In practice, one might also encounter references to VAE, GAN, other well-known papers from earlier years; "ADAM" is used here for illustrative purposes.

---

association, we demonstrate that references to "ADAM" can still appear in the output even when other tokens are entirely random.
4.2 Subsequence associations stem from training set
In this section, we empirically consolidate the insight in Section 2.4 that a key condi- tion for LLM convergence is the alignment between the model's 's internal association for the input subsequence and the output token
Pradttnnllldabbbbin Daime-1.7
Overall Pearson Confficient B.72)
0.0
820 1p 0.54)
0.6
9.380
(p 0.300
and the conditional probability in the training
0.4
R-SeN 9.530
dataset D. To validate this, we test on Olmo-
n-aae (p 0.50)
7b-instruct [21]. whose pre-training dataset,
REF 9.290
Dolma-1.7 1.7 [54]. is openly available. The train- ing corpus consists of approximately 400M doc- uments, each with 4096 tokens. For each test input subsequence trigger 5d and the halluci- nated output subsequence εh, we compute the conditional probability TP Dolog (asa (("","") _ (ooce Dolma (5* _ doc) directly from the dataset and compare it to the reproducibility rate (Srep) tested on Olmo. Figure 6 illustrates this comparison, showing the Pearson correlation coefficients for various test domains, where repro- ducibility rates are overall strongly correlated (p = 0.72). This finding highlights the subsequence association as natural and effective mechanism for tracing hallucination phenomena back to their origins in the training corpus.
Averaged Reproducibiiit Rate Syop
Figure 6: Scatter plot comparing the averaged re- producibility rate (Srep) of the Olmo-7b-instruce model against the conditional probabilities in the Dolma-1.7 dataset. Each point represents a test subsequence pair (5 ō²) from each test domain.
5 Summary of Related Work
As the main paper already includes deep discussion, this section provides summary and additional references.
Hallucination causes. Prior works understand hallucinations mainly by isolated factors. In contrast, we have extensively diseussed how subsequence associations can be leveraged analyze and unify these 15 isolated hallucination cases in Section 2.2
Attribution methods and metries. As discussed in the remark in Section
4.1 existing attribution
methods 34
50
19
along with the 741, face several limitations in the generality across diverse contexts, primarily
75) and parallel efforts in studying rationales [30 22
655
metrics 16, 24
10
emphasize the prediction gap rather than focusing on the reproducibility of hallucinated subsequences, and are often restricted to evaluating attribution at the next token only.
Shortcut learning and spurious correlations. Shortcut learning [20
27 17
58 and
spurious correlations 660 49 occur when deep models rely on non-robust cues, such as specific words
or letters [58), that are strongly linked to class labels A1 27 17 43 Prior work has emphasized the importance of understanding these associations, motivating our analysis of subsequence-level patterns. In this work, we extend the scope of analysis beyond fingle-factor triggers to include any subsequences in the input output, allowing for more comprehensive exploration of associations. We deliberately use the term "association" as it encompasses a broader range of relationships. whereas "shortcut" typically implies a strong and often problematic link. As discussed in Section 2.2 hallucinations can arise even when these associations are weak, underscoring the need for a more general term
6 Conclusion
We introduce unified framework based on subsequence associations, where hallucinations occur when model relies on incomplete or misleading token sequences. Our theoretical analysis shows that decoder-only transformers naturally encode these associations, and we demonstrate that minimizing errors in these associations is a sufficient condition for convergence. Building on this insight, propose an algorithm that traces the specific token sequences responsible for hallucinations, addressing the limitations of traditional attribution methods. Our experiments confirm that hallucinations in both small and large LLMs are substantially linked to specific subsequences, closely aligned with their training data. Our findings offer a systematic perspective on the hallucination analysis of LLMs.
10

---

Acknowledgements
This work was supported in part by the National Science Foundation under the ACTION program, the NSF DMS-2134012 and ONR N00014-24-1-2207. We thank all collaborators and reviewers for their valuable feedback and contributions.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 23Xi::3303308744, 2023.
[2] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018.
[3] Kwangjun Ahn, Xiang Cheng, Hadi Daneshmand, and Suvrit Sra. Transformers learn imple- ment preconditioned gradient descent for in-context learning. Advances in Neural Information Processing Systems, 36:45614-45650, 2023.
[4] Kenza Amara, Rita Sevastjanova, and Mennatallah El-Assady, Syntaxshap: Syntax-aware explainability method for text generation. arXiv preprint arXiv:2402.09255 2024.
[5] Dzmitry Bahdanau. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473: 2014.
[6] Sourav Banerjee, Ayushi Agarwal, and Saloni Singla. Lims will always hallucinate, and need to live with this. arXiv preprint arXiv:2409.05746, 2024.
[7] Samy Bengio. Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015.
[8] Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Lims trained on "a is b" fail to learn "b is a"" In The Twelfth International Conference on Learning Representations, 2024.
[9] Blair Bilodeau, Natasha Jaques, Pang Wei Koh, and Been Kim. Impossibility theorems for feature attribution. Proceedings of the National Academy of Sciences, 121())ee334406122, 2024.
[10] Chun Sik Chan, Huanqi Kong, and Liang Guanqing. A comparative study of faithfulness metrics for model interpretability methods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5029-5038, 2022.
[11] Aditya Chattopadhyay, Piyushi Manupriya, Anirban Sarkar, and Vineeth Balasubramanian. Neural network attributions: A causal perspective. In International Conference Machine Learning, pages 981-990. PMLR, 2019.
[12] Siyu Chen, Heejune Sheen, Tianhao Wang, and Zhuoran Yang. Unveiling induction heads: Prov- able training dynamics and feature learning in transformers. arXiv preprint arXiv:2409. 10559, 2024.
[13] De Chezelles, Thibault Le Sellier, Maxime Gasse, Alexandre Lacoste, Alexandre Drouin, Massimo Caccia, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, al. The browsergym ecosystem for web agent research. arXiv preprint arXiv:2412.05466 2024.
[14] Kenneth Church and Patrick Hanks. Word association norms, mutual information, and lexicog- raphy. Computational linguistics, 16(1):22-29, 1990.
[15] Ian Covert, Scott Lundberg, and In-In Lee. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22(209):1999 2021.
11

---

[16] Jay De Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. Eraser: A benchmark to evaluate rationalized nlp models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4443 4458, 2020.
[17] Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, and Xia Hu. Towards interpreting and mitigating shortcut learning behavior of nlu models. In Proceedings of the 2021 I Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 915-929, 2021.
[18] Nouha Dziri, Sivan Milton, Mo Yu, Osmar Zaiane, and Siva Reddy. On the origin of hallucina- tions in conversational models: Is the datasets or the models? arXiv preprint urXiv:2204.07931. 2022.
[19] Joseph Enguchard. Sequential integrated gradients: a simple but effective method for explaining language models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7555-7563 2023.
[20] Robert Geirhos, Jörn- Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shorteut learning deep neural networks. Nature Machine Intelligence, 2(1 1:665-67333 2020.
[21] Dirk Groeneveld, Beltagy, Evan Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Sehwenk, Saurabh Shah, William Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo. Luca Soldaini, Noah Smith, and Hannaneh Hajishirzi. OLMo: Accelerating the science of language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 15789, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[22] Sarthak Jain, Sarah Wiegreffe, Yuval Pinter, and Byron C Wallace. Learning to faithfully rationalize by construction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4459 4473, 2020.
[23] Joonhyun Jeong. Hijacking context in large multi-modal models. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models, 2024.
[24] Yiming Ju. Yuanzhe Zhang, Zhao Yang, Zhongtao Jiang, Kang Liu, and Jun Zhao. Logic traps in evaluating attribution scores. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 591 19512,,,, 2022.
[25] Adam Tauman Kalai and Santosh S Vempala. Calibrated language models must hallucinate. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing, pages 160-171,
2024.
(26) Raghav Kapoor, Yash Parag Butala, Melisa Russak, Jing Yu Koh, Kiran Kamble, Waseem Alshikh, and Ruslan Salakhutdinov. Omniact: A dataset and benchmark for enabling multimodal generalist autonomous agents for desktop and web, 2024.
[27] Miyoung Ko, Jinhyuk Lee, Hyunjae Kim, Gangwoo Kim, and Jaewoo Kang. Look at the first sentence: Position bias in question answering. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 1109-1121. 2020.
[28] Satyapriya Krishna, Tessa Han, Alex Gu, Steven Wu, Shahin Jabbari, and Himabindu Lakkaraju. The disagreement problem in explainable machine learning: A practitioner's 'ss perspective. arXiv preprint 22X1v:2202.01662, 2022.
12

---

[29] Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N Fung, Mohammad Shoeybi, and Bryan Catanzaro. Factuality enhanced language models for open-ended text generation. Advances in Neural Information Processing Systems, 35:34586-34599, 2022.
[30] Tao Lei, Regina Barzilay, and Tommi Jaakkola. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107-117, 2016.
[31] Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. Visualizing and understanding neural models in nlp. Proceedings of the 2016 Conference of the North American Chapter of the Association for Compulational Linguistics; Human Language Technologies, pages 68 1-691, 2016.
[32] Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, and Qun Liu. How pre-trained language models capture factual knowledge? causal-inspired analysis. arXiv preprint 2xXi: 2022.
[33] Yuchen Li, Yuanzhi Li, and Andrej Risteski. How do transformers learn topic structure: Towards mechanistic understanding. In International Conference on Machine Learning, pages 19689-19729. PMLR, 2023.
[34] Scott Lundberg. A unified approach to interpreting model predictions. arXiv preprint arXiv:105..07874, 2017.
[35] Arvind Mahankali, Tatsunori Hashimoto, and Tengyu Ma. One step of gradient descent is provably the optimal in-context learner with one layer of linear self-attention. arXiv preprint #xXiv:337.03356 2023.
[36] Thomas McCoy, Shunyu Yao, Dan Friedman, Matthew Hardy, and Thomas L Griffiths. Embers of autoregression: Understanding large language models through the problem they are trained to solve, arXiv preprint arXiv:2309. 13638, 2023.
[37] Hosein Mohebbi, Willem Zuidema, GrzegorzZChrppataa and Afra Alishahi. Quantifying context mixing in transformers. arXiv preprint 2rXIv:2301.12977. 2023.
[38] Eshaan Nichani, Alex Damian, and Jason D Lee. How transformers learn causal structure with gradient descent. arXiv preprint arXiv:2402.14773 2024.
[39] Eshaan Nichani, Jason D Lee, and Alberto Bietti. Understanding factual recall in transformers via associative memories. arXiv preprint arXiv:2412.06536 2024.
[40] Runliang Niu, Jindong Li. Shiqi Wang, Yali Fu, Xiyu Hu, Xueyuan Leng, He Kong, Yi Chang, and Qi Wang. Screenagent: A vision language model- computer control agent. 2024.
[41] Timothy Niven and Hung- Yu Kao. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages
2019.
[42] Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon I1m: outerrforming curated corpora with web data, and web data only. arXiv preprint 2306::3036601111 2023.
[43] Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, and Maosong Sun. Mind the style of text! adversarial and backdoor attacks based on text style transfer. In Proceedings of the 2021 1 Conference on Empirical Methods in Natural Language Processing, pages 4569-4580, 2021.
[44] Abhilasha Ravichander, Shrusti Ghela, David Wadden, and Yejin Choi. Halogen: Fantastic Iim hallucinations and where to find them, 2025.
[45] Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. An- droidinthewild: A large-scale dataset for android device control. Advances in Neural Information Processing Systems, 36, 2024.
13

---

[46] Ruiyang Ren, Yuhao Wang, Yingqi Qu, Wayne Xin Zhao, Jing Liu, Hao Tian, Hua Wu, Ji-Rong Wen, and Haifeng Wang. Investigating the factual knowledge boundary of large language models with retrieval augmentation. arXiv preprint arXiv:2307.11019, 2023.
[47] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144, 2016.
[48] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1135-1144, 2016.
[49] Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh, Beyond accuracy: Behavioral testing of nip models with checklist. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4902-4912, 2020.
[50] Soumya Sanyal and Xiang Ren. Discretized integrated gradients for explaining language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10089911299, 2021.
[51] Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, and Arianna Bisazza. Inseq: An interpretability toolkit for sequence generation models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 421-433 435,, Toronto, Canada, July 2023. Association for Computational Linguistics.
[52] M Scott, Lee Su-In, al. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30:4765 4774, 2017.
[53] Leon Sixt, Maximilian Granz, and Tim Landgraf. When explanations lie: Why many modified bp attributions fail. In International conference machine learning, pages 9046-9057. PMLR, 2020.
[54] Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Rus- sell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. Dolma: An Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. arXiv preprint, 2024.
[55] Niklas Stochr, Mitchell Gordon, Chiyuan Zhang, and Owen Lewis. Localizing paragraph memorization in language models. arXiv preprint arXiv:2403. 2405./95111 2024.
[56] Kai Sun, Yifan Ethan Xu, Hanwen Zha, Yue Liu, and Xin Luna Dong. Hcad-to-tail: How knowledgeable are large language models (Ilm)? aka will 1lms replace knowledge graphs? arXiv preprint ar0iv:2308. 10168, 2023.
[57] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference muchine learning, pages 3319-3328. PMLR, 2017.
[58] Ruixiang Tang, Dehan Kong, Longtao Huang, and Hui Xue. Large language models can be lazy learners: Analyze shortcuts in in-context learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4645-4657, 2023.
[59] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint 1rXiv:2302.13971 2023.
[60] Lifu Tu. Garima Lalwani, Spandana Gella, and He He. An empirical study robustness to spurious correlations using pre-trained language models. Transactions of the Association for Computational Linguistics, 8:621-633, 2020.
14

---

[61] Keyon Vafa, Yuntian Deng, David Blei, and Alexander M Rush. Rationales for sequential predictions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10314-10332, 2021.
[62] Johannes Von Oswald, Eyvind Niklasson, Eitore Randazzo, João Sacramento, Alexander Mord- vintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning, pages 35151 PMLR, 2023,
[63] Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. Simple synthetic data reduces sycophancy in large language models. arXiv preprint arXiv:2308. 2023.
[64] Tianbao Xie, Danyang Zhang. Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments, 2024.
[65] Wei Jie Yeo, Ranjan Satapathy, and Erik Cambria. Plausible extractive rationalization through semi-supervised entailment signal. arXiv preprint ww.ii::202.0085 2024.
[66] Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. Do large language models know what they don't know? arXiv preprint arXiv:2305. 18155, 2023.
[67] Yu Yuan, Lili Zhao, Kai Zhang, Guangting Zheng, and Qi Liu. Do Iims overcome shortcut learning? an evaluation of shorteut challenges in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12188-12200, 2024.
[68] Linan Yue, Qi Liu, Yichao Du Li Wang. Weibo Gao, and Yanqing An. Towards faithful explanations: Boosting rationalization with shortcuts discovery. In The Twelfth International Conference on Learning Representations, 2024.
[69] Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. How language model hallucinations can snowball. arXiv preprint arXiv:2305.135544 2023.
[70] Ruiqi Zhang, Spencer Frei, and Peter Bartlett. Trained transformers learn linear models in-context. arXiv preprint arXiv:2306.0880 09927, 2023.
[71] Weichen Zhang, Yiyou Sun, Pohao Huang, Jiayue Pu, Heyue Lin, and Dawn Song. Mirage- bench: Llm agent is hallucinating and where to find them. arXiv preprint arXlv:2507.21017, 2025.
[72] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, al. Siren's song in the ocean: survey on hallucination in large language models. arXiv preprint 23X9v:23090012/0 2023.
[73] Yuji Zhang, Sha Li, Jiateng Liu, Pengfei Yu, Yi R Fung, Jing Li, Manling Li, and Heng Ji. Knowledge overshadowing causes amalgamated hallucination large language models. arXiv preprint arXiv:2407.08039 2024.
[74] Zhixue Zhao and Nikolaos Aletras. Incorporating attribution importance for improving faithful- ness metrics. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4732-4745, 2023.
[75] Zhixue Zhao and Boxuan Shan. Reagent: Towards a aooeel-ggootic feature attribution method for generative language models. arXiv preprint arXiv:2402.0077 2024400000794,. 2024.
[76] Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for build- ing autonomous agents. In The Twelfth International Conference Learning Representations, 2023.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state the main claims, including the pro- posal of unified framework to trace hallucination sources via subsequence associations, the introduction of a novel tracing algorithm, and empirical evidence supporting the method's superiority standard attribution techniques. These claims align with the paper's theo- retical analysis and experimental results, which demonstrate how hallucinations stem from dominant subsequence associations and validate the tracing method on benchmarks.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper discusses several limitations implicitly: it acknowledges the in- tractability of exact subsequence association computation, the approximation of input distributions (Ps) due to practical constraints, and the dependence on top-k embedding simi- larity sampling ensure diversity, also highlights that conventional attribution methods lack causal interpretability. setting scope boundaries. However, a dedicated "Limitations" section is not explicitly labeled.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
16

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
The paper provides formal definitions (e.g., subsequence relations and asso- ciations), an informal Theorem 2.4 on subsequence embedding, and Proposition 2.5 on convergence. Proofs and formal versions are referenced Appendix Assumptions such as subsequence independence are clearly stated. Though high-level proofs are deferred to the appendix, the main text includes intuitive sketches.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper details the dataset used (HALoGEN), describes the method for approximating Ps, outlines how Srep is computed, and provides comprehensive algorithmic descriptions (e.g., subsequence tracing with beam search). Experimental setups, such as token replacement methods, beam width, and evaluation protocols, are specified. This information suffices to reproduce the key results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm.
17

---

reproducible, they (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We will release the code link upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ce heapliicgguieee//oeeeeiieeee ssionPolicy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (pt /ooee///odSSbiissse
/ //nips.
www.bbmiissooPoooiiicc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
The paper specifies data splits (domains selected from HALoGEN), hyper- parameters, and how tokens manipulated (top-k embedding similarity). Subsequence lengths are set by ratios (a = 0.25, 0.5, 0.75), and evaluation corpora are diversified. Method- ological choices are described sufficiently.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
18

---

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Detailed numbers are in Table 4 where the variance of the results is within PP. which is far below the gap with benchmarks that are considered.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support
main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots. The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Our computation ran on a machine with 8 H100. Each setting of experiment can be finished in 48 hours.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 'ttt make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: There no apparent ethical violations. The paper public datasets, refrains from srivacy-sensitiee content, and aligns with general research ethics. No deviations from the NeurlPS Code of Ethics indicated.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
19

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The paper does not lead to potential implications in high-stakes domains.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not release any new model dataset that poses misuse risks requiring safeguards. The tracing method is analytical and diagnostic.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e-g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
20

---

The paper does not release any new model or dataset that poses misuse risks requiring safeguards. The tracing method is analytical and diagnostic.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URI
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi swi thcode. com commdatasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- ee-ackagee, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce any new datasets, models, or assets requiring documentation.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/code/modee as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
21

---

for Answer: [NA]
Justification:
Not applicable, as the study does not involve human subjects research.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact scientific rigorousness, or originality of the research, declaration is not required.
core methodology,
Answer: [Yes]
Justification: The paper's core method involves analyzing outputs from LLMs (Olmo- 7B, Llama-70B, GPT-4o). The LLMs are integral to the methodology, and their usage is described detail throughout the paper, fulfilling the declaration requirement.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer our LLM policy what should or should not be described.
22

---

A Algorithm Details
Algorithm Masking and Refill
Input: Input sequence s, masking probability p. top-k parameter
Output: Modified sequence s'
Initialize s' <-s s
for each token in do
With probability p. mask token 8; end for
for each masked token position in s' do
Retrieve the embedding vector for the masked position Compute similarity scores between e; and all token embeddings in the LLM's embedding matrix Select the top-k most similar tokens based on the similarity scores
Sample token uniformly from the top-k candidates Replace the mask at position with token & end for Return s'
Algorithm 2 Beam Search-based Trace Subsequence Association
Input: Approximated sequence distribution Ps.. original sequence s, hallucinated subsequence 8", beam width B, maximum subsequence length n ........ 3: Initialize beam B1 < 8 8 E s (All single-token subsequences)
2: Output: Set of best subsequences si. ....... S with length from n.
4: Compute (ō xx) for each 8; E B3
5: Select top B subsequences based on v/ and set B1 top B subsequences do
6: for k =
7: Initialize temporary candidate set C 0
8: for each subsequence E B do
9: for each token s; E not in $ do
Create new subsequence s' < $ UU8)
11:
Compute ( over Po
12:
Add 5' to C
13:
end for
14:
end for
15:
Rank all subsequences in based on (ō .s)
16:
Select top B subsequences from C and set B t top B subsequences
Store the best subsequence st + B& Terminate early if no improvement is observed
18:
end for Return 5/ ./..k...... s/
20:
B Theoretical Details
B.1 Notations
B.1.1 Basics
Let be the vocabulary (or token) set. Tokens are generally denoted by z or y: x typically refers to an input token, while y refers to output (generated) token. decoder-only large language model F(.) maps an input sequence = [x] .., In. y to an output sequence o [31. V/...., yml E nm Here, I; and y represent individual tokens from the vocabulary V.
23

---

B.1.2 Sequence Notation
We define the following notations for sequences and subsequences:
Subsequence Notation. A tilded symbol is used to denote subsequence of s.
Indexing.
stg) the i-th token of s.
S[-1]= the last token of s.
F the subsequence consisting the first tokens of s.
Subsequence Relation. Given two sequences š and we write š [ s if can be formed by deleting zero or more tokens in s while preserving the order the remaining tokens.
Subsequence Collection Ending at the t-th Token.
Sub(s, t) = {
$ SESpp
S
$|-1] = xz}.
This set consists of all subsequences of : that end with the t-th token xy. will be used analyze the transformer's features at the t-th position.
B.1.3 Embeddings
Generalized Embedding for Tokens and Subsequences ual tokens to Iz-normalized vector space:
An embedding function yp maps individ-
p V Rdd
This notation is generalized to subsequences 8:
V' Rd.
Rather than embedding a length-n sequence into n separate token embeddings, we consider embed- ding each subsequence of s which encodes more semantic information than single token. However, the number of such subsequences grows exponentially with for single sentence. A key role of the transformer is to effectively condense or aggregate information from these many subsequence embeddings into a length-n output. As shown later in Theory [B.3] the output at position can be interpreted as a linear combination of the embeddings corresponding to subsequences in Sub(s, t).
Embedding at Transformer Hidden Layers We and t denote features within the transformer. For an input s = [z1, ................................ Nn], , the feature matrix at the 1-th layer is:
1 (s) = (s). (s),
1 (s)
gaxd
where (s) is the feature vector at the 1-th position in the I-th layer. The submatrix with the first t
is denoted (1)) (s). By construction, the output T0 (s) is agnostic to the token after position f,
thus feature vector d [cr] (s) can also defined for the feature of 8/:4 = [x], 32 22,,.,, 2r] E s, since
is equivalent slicing the sub-matrix with the first t rows of (p() (s).
Special Cases.
First Layer: the initial layer, the feature matrix can be considered as the token embeddings (x) :xxx)))
( (s) = (p(x1), ((I2), ((x))) T
E Raxd
Penultimate Layer. The feature matrix after the (L 1) layer is denoted 4 (s). This matrix is passed to the final attention layers to produce output logits. For simplicity, we often drop the superscript and write (ss) to refer to this penultimate-layy feature.

---

B.2 Local Convergence Analysis of Subsequence Association
Setup. Let the training corpus be D {(s(), y ) | where each sequence g()) has the same length n and ends with the same token (denoted by 2). We follow the analysis setups in [39 and [12), which use ending tokens EOS and '0', respectively. Throughout this paper, we let the ending token be Ø3. We also assume that the maximum number of duplications of any token in sequence is dpp
Consider decoder-only transformer architecture with linear attention in 33 whose logit output layer is given by
38
F(s) = arg max f(8..) [(s,v) = p(u) T Wov D(8) T ((s) W WxQ (s), BEV
where WkQ W&Wa oe Raxd and Wov WWWv Radd are the key-query and output-value matrices, respectively, The loss function is
exp(S(s() (
(()) (10 g (og 1 } p( |(())) Eyev ((s(),
We aim to analyze the convergence of the parameters 0 = (Wkq, Wov, 2(-)), where (--) denotes the L-layer feature encoder. Since a separate analysis for each parameter is complicated, we group them and consider:
[(s. = ply) Wor o(s) WKQ
Σ t=1
Tov @e(s) (s(s) T WKQ (s))).
Define
(y) = T Wov Dz(s), (,(s) = (1(s) WKQ (n)s).
Hence,
n
f(s.y) Σ (s))
To show convergence, it suffices to show that the gradients of the loss function with respect to all V. (..)) and (.).) become small. Specifically,
OC(O) (v.(...)
OC(O)
OC(O)
OC(O) (w)()(()
Σ ΣΣ
Σ
(yl(ss())
(.)
(G)DD NeV
(a(0) WOEDD t=l
Proposition B.1. Under the setting of Theorem 2.4
we have
OC(0)
OC(O)
1v. (..-) +
Σ ΣΣ4
(5.
(.)
DEV SC*
where
[s (S v) < const (|Worll++|| ([Kgll)) Po(s $Es) p(yl
Proof. Step 1: Gradient with respect to (:.))
We let
Oc(O) )V.
ac(O)
av.
1= vEr
where
aC(0)
0 log (0))((()))()))))
Σ
Σ
(p(yls())
= -(0)) oc(s(0) WxQ
(o(s(")TTk@@le (s
(v.-).
D
o (p(()) T W (s(O)
D
(x(0) Hyllyy JDD
(E
By Theorem
2.4 we expand (s(()) ) in terms of all subsequences in S Sub(s(),
(s))) = Σ
Mg (()),
aESnSub(s.t)
25

---

giving
AC(0)
ΣΣ
A ov
(8.
OD (..)
NEV S
where
OV ((
Σ
Σ
Σ
(p(yls()) - - () p(s) T WKQ on (s))
[D]
(x(0) ( t=1 $ESub(x(),0)
Since we set &n (s ye for the ending token, then
Aoy (() (5, y) n us (s)) WKQ Po PI Sub(s, t)) E (s_ t=I....
[p(v s)] p js Sub(s, t))
i=......
(E.ub(s,,t)
Since the event i Sub(s, f) essentially the same as (except for counting token duplications up to Tap), we have
1 (5. v) S
Tdp || PD(s Es) |E..cclf(y s)] PD(v $ [s|,
where we used μı S (due to the softmax operation in the previous layer) and ||x(5)||2 S 1.
Step 2: Gradient with respect to w.(.).
Similarly,
ac(0) (.)
Σ Σ
Σ (ply ((0) - 1(11= = (0y)) p0v') Wov oi
[D]
(sf1)
(ele)) v'EV
Expanding (s)( via all subsequences 8, we have
aC(O) (w.(.)
ΣΣ
A KQ 2(0) (5,
8 W
where
KQ A
2(0) Sto
1
Σ
Σ (pl/ (0)) 1000-1191 " Wov v()).
D
EDD
If on (s(0)) = Pa for the ending token, then
A 22(0) y') p2 T T Wov ((S) (a.)aa E Sub(s, t=l...
E (>LND [p(y/ ss]] P (wD Ise Sub(s, t))
t= SESub(s..t
t= t=l....
Using the same duplication argument as before, we have
KQ A
((0) (5,
S Tdp [oovll2 Po($ [s) EDacsTP( s)] Po(u 1 =
Combining the bounds for A OV [(()
(š, v) and A KQ ((()
(s, y²), we conclude that
OL(0)
CC(O)
< ΣΣ
A As (8, v)
0 (...)
Ow (.)
yEV
26

---

B.3 Decoder-only Transformer as Subsequence Embeddor
The decoder-only Transformer architecture represents input sequences as hierarchical feature matrices through multiple layers. At the 1-th layer, the feature matrix (s) R aggregates feature vectors for the input sequence s = [11, matrix is given by the stacking of token embedding:
Overview.
At the first layer, the feature
(1)) ())(s)
(s) =
( E Raxd
The computation of (s) at layer is defined recursively as:
(1) (s) = (s) + FFN((-1 (MHA (a (B)(-
where MHA (1) is the multi-head attention block, and FFN is the feed-forward layer at the /-th layer. As side note, the feature (s) E "xd can be decomposed into n vectors:
+ (s) = 10 (s), 02 (s),
/0 (s)]T
The multi-head attention (MHA) block is a core component of the Transformer, enabling the model to focus relevant parts of the input sequence. At the (-th layer, MHA block computes an output feature matrix as the concatenation of outputs from H attention heads:
Multi-head Attention Block,
MHA (1) ( = concat (MHA MHA (1.2) (Ø) (H)....,MH. MHA ()) E RR
For each attention head E the output MHA (1.b) (T) is computed as:
MHA (1.b) (
"Wl. or T
@ (1.) KQ -11:) 1.:) E ERd
where:
o (-) denotes the row-wise softmax function.
+ Rtxd is the feature matrix up position t, and 1 the feature vector of the last token. The causal mask ensures that attention is restricted to the last token's 's feature. w(l.)) KQ E Rdxd and w
Ov RRxdd are learnable weight matrices for key-query and output-value transformations, respectively, defined as:
w KQ = w
(LA) w(l.)) (L.))
w(l.)) (1,)) ov
(t,h) ((..))
Feed-forward Layer. The feed-forward network (FFN) layer applies non-linear transformation to the input feature vectors. At the /-th layer, is defined as:
FFN
w ReLU
(W!O A.V - (D0V-bb).
where:
w!l) E Idddd and w!()) E BRd x Hd are learnable weight matrices.
. b(0) E IRds is the bias vector.
e Hd is typically the output of the multi-head attention block.
This layer enhances the representational capacity of the model through non-linear activation and dimensionality transformations.
Lemma B.2. For any length-2 subsequence [ž, (2] within an input sequence with length n 2 2, a single-layer transformer block can act as an embedding function for subsequence: ((āā,
1 [1ē, K') [ s}.
27

---

Proof. Consider a single-layer multi-head attention block with two attention heads. The output
feature at position t the concatenation of the outputs from each head:
MHA (1:) ( (s)) = concat [MHA (1.1)
( (1 (s)), MHA (1.2) (1 (s)(ss))
For each head h E {1,2), the output MHA (L.h) (T) [:r] (s)) is computed as:
MHA (1.M) (8 (s) w
(1) 1 (s)
(L p()))
where:
(xs) Rd is the embedding of the token position
(s)ss) e Rexd is the embedding matrix for the sequence up to position 1, i.e., (1)(s) ( ((xx))
To configure the transformer to select specific subsequence [ž, j², the weight matrices are con- structed as:
(1.1) KQ = (z))
w ov = -w(12) KQ = w
ov =Ixxd=
where Laxd is the identity matrix and y >> 1.
With these weights, in the feed-forward network (FFN) layer following the attention block, linear filter serve the indicator of the subsequence ž'):
1 [ K' [s}
max ReLU (Wr,, (1.1) (T 1=
(s))) - b)
where:
WF, ()
b = is bias term.
To see this, we first consider when [T, Ø'I [ and we assume V' appear at the position j. We have:
MHA (1.1) ( ()(()) (1 o (en101o-11) MP(() (0 ( (()Ow-1) Σ ((x))1fax +
= = + O (1)10(0)-1)
and similarly,
MHA (1.2) ( (s) p(x') + O - (e)
ensuring that the concatenated output at position j is:
MHA (1.1) ( (1) [1]] (s) (oncal(y(f),
O (2)100))1))
The ReLU output will thus be approximately 1. On the contrary. if [", E then (s)) becomes a random linear combination (xx) that x * 7. this case, the ReLU activation output is most O (c)(0())-1)) 0. (y >> 1), even if / e Suppose our FFN only has one hidden neuron (dg = 1), the output of FFN serves as the embedding of the subsequence [ā, J if we let WF. ((ū, ø']):
MHA
[3] (2)))
(VN Σ WF, ReLU (WWF, MHA (1.) ( b) = <((..=1)) [ Cs). 1=1
28

---

The next Lemma serves as generalization of the Lemma B.2 with more than one length-2 subse- quence.
Lemma B.3. For a set of length-2 subsequence S [ with S; = ( T.}, a single-layer
transformer block with dj = N, = O (c-2 In (<([)) + N,))) can act as an embedding function
p 222 + Rd for all the length-2 subsequence in S ensuring that
V S I V, (1 (x))) <e, (()(()))))))))))))(()) ((5))) <E
for small > Specifically, for an input sequence s 12, .... Ind E Vnh the embedded results $ ()(2) (s) E Rgadd whose would be
10 (2)
(s),
(s) (6)
where
(5)
Σ
Us (5)), with μı > O.
(B 8 8
Proof. We construct single-layer, two-head multi-head attention (MHA) block, followed by feed-forward network (FFN) of hidden dimension dy = Ns. as follows. For MHA block, let
MHA (1::) ( (((( (s)) = concat (MHA (1.1) (10 (2) (s)), MHA (s))).
where
MHA (1.A) (1) (0) w $ (c)(ss)
w(t)) KQ ((x))), E {1,2}.
Here, 1 (s) E Igd collects embeddings sp (It)) up to position t. Following Lemma B.2
we assign:
w KQ
w ov = w w Σ Ts (x(()) Es
-axds
KQ ov
where each vo > is a large scalar chosen per subsequence š S
Key Intuition. As in the proof of Lemma B.2 multiplying (xx)) by ((I.)) activation only when (xxx) = (ă²) and there exists an 7 in the preceding positions that aligns with (ā). This ensures that whenever E appears somewhere in s before t and H' appears at position t, the attention head I "detects" this subsequence, yielding output for all the ã satisfying the condition:
KQ extracts large positive
MHA (1.1) (s))
Σ
M[x,z,] (((),
sc [=r(< Es (m...
where Mla.ad is positive number dependent on the occurrence of the म preceding and the [[. If [ā, 5'] 5 and [ē, 5'] E Sp:))
exp(C( 8p4) M.)
exp(C(r, (p.,:))
exp(C(xx, ((:)) O(e)
where C(., Sj:g) denotes the number that occurs in the first-l-token subsequence S.g). In the rest of the proof, we will omit the calculation details for the softmax expansion.
Meanwhile, attention head 2 simply copies the token embedding at the current position (because (-(1.2) KQ = D), thus producing (ā"). Concatenating these two heads gives an output at position that approximately
Σ
Ma. .8()), (()))
concat
#e (a (x)p:://e(())plllmmess)))))
29

---

if and only if those subsequence [ž, Ø'] occurs ending at position t. Otherwise, the first part of vector near zero or some negligible combination (due large y), in Lemma B.2
Next, we feed the MHA output into a single-layer FFN with N, hidden neurons to encode each 8, E S Concretely, let
(V) WF, ReLU (We. v) b).
where v E R22d is the concatenated output MHA (1.3) (.). (.). We set:
(oncat(((2)), (((2))
WA
E R2N.22 Wr. = (p((i) ((52), .... p(3x.)) E ERUxN,
concat (ŌN, 4 (x'N,) T
and b = 1N, E E...
Observe that for the i-th row in W Fi the ReLU activation
(conn((conaa (1.:) ( (s)). (Tt)), (onc -1)
Σ
ReLU
(f:) (()), concat( (()\)) 11 Maa,))
(o + (f') (oncat((p(;z),, y (xt)), and near otherwise. Hence neuron "fires" if and only if subsequence [ Ai] is detected at the current position.
Multiplying by Wr, (of size d x N₂) gives
(2) FFN()) (MHA (1) (0()) (1)
(s)
Σ
Ms (()). with Me >
BUB Bp)
up to small error terms of order 0 (e)r((0e))) 78 for all S.
Finally, we ensure (1) (s) (xr) = 0 for all x E V and (2) (p(S.) (S,) 20 for for Szy.
Using the results in Theorem B.4 we are able construct (s) FES= and {((()) DEV in Ed with d= In(e([V] + N.))) satisfy the required near-orthogonal bounds.
The next theory serves generalization of the Lemma B.3 with more than one length subsequence.
Theorem B.4. For set of subsequence St:n = (5. ) N. UV with the length of ranges from to 1,, an L-layer transformer network with (logz (n)) d O(N.)) d O (n In (c([V) + N.n))) can act as an embedding function -._1 V2 + Rdd for all the subsequences in Slmi, ensuring that
VS, E slm), (s)()) <e
for small > O. Specifically, for input sequence 12,, .... Ind E y the embedded results (())s)) E B whose would be
()) = 10 (s), 2
1 (s). ())()))))))))))))))
(s))T
where
1 (s)
Σ
"g ((5), with μ¹ 0.
stainSub(s.t)
Proof. We prove the theorem by constructing, layer by layer, a Transformer whose outputs encode the near-orthogonal embeddings of all subsequences in S! = [ V. The key idea that subsequence of length 1) be split into two parts- of length roughly n//2 and the other of length 2/ Repeating this splitting procedure across multiple layers (on the order logz(7)) (7)) allows the Transformer to recognize and embed all such subsequences.
30

---

a) Decomposition of Subsequences. We start by defining the decomposition of subsequences used in each layer:
Let s/:n! sl denote the set all subsequences (of length up to 7j) at the final layer L.
2. each layer I E (1, ...., L), every subsequence š E s|:") is split into two parts:
where splitleft (S) S. and splitright (s) Sp. The left and right parts satisfy [Sc]
The set s|-1] is then the collection of all such left and right parts of subsequences from s!:"l
s
5a S s) u ( S s/
For example, if = [a, b, c). then:
5. |a, b].
5, [c].
Is [a] (length 1), pad the right part These splits allow the Transformer to "build up" embeddings longer subsequences by combining emheddings of their shorter left and right parts across successive layers.
b) Inductive Construction of the Transformer Output. Let ( (s. n) E Raxd be the output of the 1-th layer of the Transformer, where the I-th row is denoted
(s) E Rdd
1=
We assume an inductive format for (s). namely:
Σ
Pg (
F I WES
ml Ba of =
where 3' [ denotes that V is subsequence of s ending at position t, with last element Iz. The scalar weights μt 0 ensure positivity in the linear combination.
prove by induction on / that:
Σ ( BEstal &Ca á ====}
rs (3)
b-1) Base Case = 1.
By Lemma B.3 after applying the first layer's 's multi-head attention (MHA) and feed-forward network (FFN), we have:
FFN [MH (
Hs (
E-th column
(Res? BE
where (S) embedding for 2-element subsequences, and μg Incorporating a routing/residual layer yields:
0 ())
0 (1) (s) +
Σ
Ps (
{BES | 2 =
Since (-) and (-)) can be unified into a single function se(-) mapping to the same d-dimensional space, we obtain
Σ
Pa (s)),
(SP()) [8 i 1=zz}}
matching the claimed inductive form for 1.
b-2) Inductive Step. Assume the form holds at layer I. We show it remains valid at layer + 1.
Multi Meati-Headdaee Attention Construction. We two attention heads at layer 1:
(s)) = concal (MHA (1.1) (o (s)), MHA (1.2) (0 (s)) MHA (1.)) ( [1
31

---

where ② [d]] (s) E Rixdd collects all row-embeddings (s), (s),...... 02 (1) (s). Each head is defined by:
MHA (L.A)
(1)
(s)
(L))
ov
(s)
a00 (:0 (s)
W
KQ
(.
1 (s)) E {1,2}.
Similar to Lemmas B.2 and B.3
choose
w (1.1)
Σ
7a ((aa) (sp)
W w
(t.2) Laxd:
KQ
where 7/ >> is chosen large enough that attention "activates" primarily when of (s) contains sp(s.)
and 0i (s) contains ((S.).
Head I detects occurrences of the pair (Sa: 5o) across indices {I,..., 1J.
,.., Concretely,
MHA (1.1) (G) =
Σ
Ps (Sa).
3 JES (n
[ _))-)==))
(4) (s) feature from the previous layer:
Head 2 simply copies the Φi
MHA (12) !1) (s) oi (1) (s) (.. (. Ro:-1 m))
Σ
Map (p5))..
Network (FFN). We pass the concatenated output Rd2d from these two heads
into an FFN:
FFN (v) = w Fg
of
(ww -b().
where (.) is (for instance) ReLU, and
w@ ( Fi concat((<(5a), ((5.)) 5 [. 5.]
len S1+1
W F2 [p(3) SE
|
suitable choice of bias b(4) ensures that the corresponding neuron "fires" when the input concat(pp(5)), p(s.) matches one of the rows in w(t A
Consequently, the output of the /-th layer
FFN at position & is:
1111 Ss)
Σ ( (e SC -
(1+1) P3
(
which completes the induction step.
с) Near- Orthhgonhyyy of Embeddings. Across all layers 1, the total number of subsequences in U (o))))())) s(()) is on the order of O(N.n). By setting the embedding dimension
d d= 0 Infe ([V] + NN
construct mapping (0-): S(m) Rd that
v x in slml, ((v(S), ((5')) <E.
These embeddings are thus approximately orthogonal, ensuring (<(s,, ((B))) <e.
Combining the above arguments completes the proof: an L-layer Transformer with L = Olog, hidden dimension d, = ON,1),. and embedding dimension d O(e-2In(e(()) + N,n))) can embed all subsequences in Slpil while maintaining near-orthogonality among their embeddings.
32

---

B.4 Other Useful Results
hog-probability factorization with subsequence association
Proposition B.5. Let s' be input sequence, and let s E' denote its subsequences. Assume that:
1. The subsequences C are marginally independent;
2. The subsequences are conditionally independent given E o.
Then, the log-pro peeebbbbbbiliyy of hallucination tokens conditioned on s' given by:
P FO C s=)s= = v(8 PewP, O0 oo),
where the subsequence association (1, 6) is defined as:
P F( Eo l {s
(s. 6) log oo
Proof. By the conditional independence assumption and the Baysian rule, the conditional probability P (o o =') can be factorized as:
P P P(
P PPNP Po.P s') PO
II
P PaNP,.OF(F))
BL
Taking the Logarithm Taking the logarithm of both sides, we have:
P (() _ S
O == = F1 +
log P Pa- 1000o oo)
Definition of Subsequence Association From Definition 2.2 the subsequence association is given by:
(5, ō) = -log Po (ō Co [
PONP (NF)
Substituting this into the equation, we obtain:
PF ===ss)
Σ (s)
Thus, the proposition is proven.
Maximum number of normalized embeddings with angular constraints.
Theorem B.6. Let d denote the unit sphere in Rdd. The maximum number N d) of unit vectors (V1, .2...., (((e.)) C Sdl-1 such that the angle between any pair of vectors exceeds n/2 is bounded above by
Surface Area of S
N(e, d) S Volume of a Spherical Cap of Radius π. /22 C
For large d, N(e, d) grows approximately as
N(cd)= 00 (cdooopp
(d (d22)
2 )=²).
Similarly, to include vectors such that Vv., Vj, |v Výl < e, the dimension size needs to grow as
In (Ne)
d(N,e)
2
33

---

Proof. To determine the maximum number of unit vectors (V1. V2..... VN((,))) such that 0ig > T/2 for all i # j. we consider the geometric constraints:
Each vector corresponds to a point on the unit sphere S Rd 2. The angular constraint 8. > / implies that no two points can lie within a spherical cap of angular radius x/2 c.
The volume of single spherical cap with angular radius / is given by Veup (TT(1T/2 - c). The total surface area of gd-1 is given by
Surface Area(Su-1)= 2n 2/2 4/2
To pack N (e, d) spherical caps of radius x/2 S without overlap, the total volume of these caps cannot exceed the total surface area of the sphere. Thus, we have the inequality
(e, Veap (TT e) s Surface Area(8d ).
Rearranging, find
Surface Area(S Veap (TT2 E)
N(e, d) S
fo
sin d
(0)
For small e, cos(e) approaches and the cap volume becomes concentrated around the equatorial region of the sphere. This leads to the asymptotic behavior
dea
N(e, d) O ed exp
with detailed proof in Lemma B.7 By Lemma B.8
we have
d(N,c) = In (Nc)
Lemma B.7. Let d E N be large, and e >> O be small such that A = evd >> 1. Consider the integral
I(d,e)= sin (0) d8.
Then, the regime ed >> 1, the reciprocal of the integral scales as:
(d, o ed exp
d
2
Proof.
Set r 2 - 8, so that sin 0 = cos x. The integral becomes:
f* % cos co dz.
I(d,
For large d, the dominant contribution to the integral comes from small values of where Re 1 2² Substituting this approximation, we have:
cos 2-2 (x) 2 exp (-dd- x²
Thus, the integral becomes:
I(d,e) (
x
(d - 2d=2)x²
exp
dæ.
Define / = vd 2x, so that x # 1
Va-z and dr
db Vd=2"
The integral becomes:
1
t
I(d,c) Vd-
exp
dt.
Jeya=2
2
34

---

Set A evd. Then:
e(d,e) 2 /d-2JA exp (-2) dt.
When A >> 1. the integral fx exp( di is dominated by the Gaussian tail, which satisfies:
12 exp 2 dt
exp
2
A
Substituting this into the expression for T(d, <), we have:
12
1
exp
((d,c)
Vd - 2
A
Using = cvd-2= cVd, get:
c(d,c) 1
dea
1
de?
exp
exp
Vd
evd
2
cd
The reciprocal of [dd, is:
de
ed exexp
ed),))
2
Lemma B.8 (Growth of d for Fixed e and N). Let d E N be large, small, and N satisfy:
- (cdesp de2 )o=)).
N = ((d,e)
In the regime e c >> 1, the growth of d as a function of and N is given by:
d(N,)) = In (Ne)
22
Proof. From Lemma B.7
know that:
(ee2
N
[(d,c) O edeepp
2
This relationship can be rearranged
de2
dexp 2
Ne.
de2 Set 2. so that d 2z. Substituting, we have:
exp(x)
3 Ne.
1
The equation x exp(x) = y solved by the Lambert W-function:
-= eN
2
Returning d, recall that d 2r Substituting a::
cVV
d
W
2
2
35

---

For large y, the Lambert W-function satisfies:
( ~ Iny In Innn
Applying this to W (+))
we get:
EN
<N
ENV
W
In
In in
2
2
Substituting into d, we have:
2
<NV
<NV
d ~
In
In
22
2
Hence, the growth of as a function N and is given by:
d(N,6) 0 (Nc)
36

---

Additional Experiment Results
C.1 Dataset Details
To assess the effectiveness of our methodology in tracing hallucinations, we require a benchmark dataset where hallucinated subsequences, o", explicitly identified. HALoGEN 44 provides comprehensive benchmark comprising 10,923 prompts designed for generative models, covering nine domains, including programming, scientific attribution, and summarization. Each domain is equipped with automatic verifiers (external programs) that decompose L.LM L-M-eeenrtted content into atomic facts and systematically verify each fact to detect hallucinations.
In this study, focus six domains from HALoGEN that employ programmatic verification to identify o This selection ensures that hallucinations are identified objectively, independent of LLM-based judgments. The chosen domains and their abbreviations are as follows: Code Package Imports (CODE), Scientific Atriibtton (REF), Biographies (BIO), False Presuppositions (FP), Rational- ization Numericul (R-NUM), and two distinct Rutionalization Binury domains- one based prime factorization (R-PRM) and the other on knowledge about U.S. semators (R-SEN) Examples from these domains are provided in Table 3, For each domain, we measure the frequency with which the same hallucination occurs the generated output given the same input prompt. To facilitate analysis. we subsample 100 prompts from each domain, selecting those with the most frequently occurring hallucinated subsequence.
C.2 ChatGPT Prompts Used in Evaluations
Prompts for
generation with ChatGPT gpt-m
Fill in the blanks to form a coherent sentence, The answer can random and does not have to be factual. All the non-masked tokens need to be kept in order.
Prompts for token-based generation with ChatGPT (gpt-t)
Use the following tokens draft creative and uncommon sentence. Make that all the tokens need to be included and it is in the original order.
C.3 Hyperparameter Sensitivity Analysis
We conduct a hyperparameter analysis of the beam search parameter B and the perturbation corpus size IP& for our Algorithm 2 SAT, as shown in Figure 7 The results indicate that increasing the beam size and expanding the perturbation corpus improve SAT's s ability accurately identify subsequences. In the main paper, we set = 20 and use perturbation corpus size of 500 for reasonable balance between performance and computational cost.
Hy Hyperpaneteeer Analysis
Perturtation Hyperpanameter
40
30
30
10
20
500
1000
Figure 7: Hyperparameter analysis of beam search size (B) and perturbation corpus size (Pss)
Olmo-7H -i0o-7----nsttc
C.4 Tables with Detailed Numbers
We provide detailed numbers of Srep over all test distributions in Table
37

---

Scenario
Size
Prompt Example
Hallucination Veri-
(#Prompts)
fier
Code Package Imports (CODE)
1085
Write Python program that utilizes modules, packages, Python Package and/or libraries solve problem: 'How to stack Index columns to rows in Python?"
Scientific
1780
Find relevant scientific or academic references supporting D the claim 'Chloroquine cure coronavirus' in APA for- Scholar mat. Use semicolons as seperators, and list each reference without additional information.
Semantic
Attribution (REF)
Biographies (BIO)
682
Tell bio of Iggy Azalea.
FActScore
False Presuppo-
1084
List continents that contain the letter 'v'. Separate the Program items of the list using semicolons separators. Provide only the list without any additional information. you cannot answer, respond with 'no response."
sitions (FP)
Rationalization Prime (R-PRM)
500 Is 7411 prime number? First, respond with yes If Program no, then provide its factorization.
Rationalization Senator (R-SEN)
500
Was there ever US senator that represented the state of Program Alabama whose alma mater was MIT? First, respond with yes or no. ycs, then provide the name the US senator.
Rationalization 1014 How many planets in the solar system starts with letter m. + Program First output a number, and then list every item that satisfies the condition.
Numerical (R-NUM)
Table 3: Description of HALoGEN from scenarios, accompanied by decomposition engines and factuality verifiers to identify hallucinations.
The subsets consist of prompts spanning seven
38

---

Table 4: Comparison of Srep score over 7 test domains and four test distributions bert, rand, gpt mm, gpt -t Out-of-time (OOT) represents method that can not be finished in H100 days with official impllmmen caaplemmetaiionn in [51] The subsequence measured with a fixed ratio 0.5 of original soquence length in this table.
Avg.
19.8
15.9
19.2
OOT
46.5
24.0
19.1
27.5
OOT 23.6
42.6
rand gpt-m gpt-
0.0
1.2
17.3
OOT
27.3
34.2
19.3
32.2
49.1
0.1
2.4
00T
32.0
29.3
28.8
OOT
48.8
REF
0.1
0.6
OOT
21.3
14
16.5
OOT
38.3
hert
0.4
1.5
14.7
OOT
39.6
32.2
30.1
OOT
574
57.4
55.8
OOT
87.3 89.7
54.5 54.7
67.3
65.5
74.2
R-NUM
57.2
54.6
OOT
65.3
64.2
80.3
bert rand
59.1
54.2
OOT
889
46
53.
58.8
55.6
72.2
60.1
06.4
OOT OOT
814
48.3
65,1
63
75.1
1.2
10.8
4.8
47.6
19.4
16.9
OOT
R-SEN
1.2
15.5
OOT
46.1
198
19.1
COT
33.4
bert rand
0.0
0.2
0.2
OOT
37.0
4.2
24
56
OOT
19.1
2.0
18.0
7.3
OOT
41.0
11.5
7.7
12.1
OOT
31.9
50.5
19.7
OOT
65.1
20.4
27.2
18.1
30.5
R-PRM
34,9 40.2
19.8 174
OOT
65.7
20
26.3
18.2
274
bert rand
OOT
77.3
12
17.2
21.9
16.3
30.2
31.6
15.7
18.3
OOT
73.6
23.9
25.8
24.6
15.5
4.0
OOT
19.0
25.1
27.4
166
51.1
4.5
20.0
OOT
24.7
314
16.6
32
24.6
64.6
FP
bert rand
0.0
0.0
OOT
17.7
17.7
1.7
18.3
14,9
47.5
0.0
0.5
OOT
13.4
25.7
34,9
32.6
67.3
0.0
0.5
O0T
7.6
0
0.2
0.1
94
0.1
3.7
3.2
OOT
27.0
0.2
1.2
0.7
0.4
16.7
B10
1.3
2.5
OOT
33.5
0.1
02
0.5
0.1
hert
1.5
6.4
OOT
48.0
0.7
21
2.7
0.7
34.9
9.5
25.1
OOT
48.7
30.6
34.6
15.3
41.9
CODE
gpe-m
30.9
15.6
COT
61.1
34.1
42.6
13.7
49.2
34,9
11.6
OOT
28.7
17.6
27.5
12
28.9
hert
35.3
18.2
32.7
0OT
51.6
45
29 4
17.4
38 6
SAT (Ours)
SAT (Ours)
Buseline
attention
grad-shhp
shap
ReAgent
attention
gnd shap
ReAgent
lime
lime
Olmo-7 7B
Llama-700
LLM
39

---

D More Examples
D.1 Chatgpt Screenshots
Singer Yorcca
60
hogendlry
you'ne thmong singer namad born potentally
tivs Costallo, though he was actull ly born London, Englans.
Could you darrly or prow more detale? Periaae you thinking anather chiis?
Singer bomin New Yort with the name "Qule" Sivi Costallo.
Figure 8: A complete input-output screenshot of GPT-4o from the left side of example in Figure 2 as of January 25, 2025.
fardly
tiis Dreso's nother Vargeria Diee, zepioally paoood accdent Parrto Rice
Coeppit younger dimn; Eedyyn Cruqps roplanly acdderr
was cisepyy insachull for him and tanily 1 you the more intommtion about Duto Craapo hes masio, bei fow add
Figure 9: A complete input-output screenshot of GPT-4o from the right side of example in Figure 2 as of January 25, 2025.
40

---

D.2
More Examples in the Evaluation Settings
Setting: test domain (REF) model Olmo-70 1eiio----instuct
Find relevant scientific or academic references supporting the following Question-Answer pair in APA format. Use semicolons separators, and list each reference without additional information.
System Prompt:
Input: "Question: In the Gregorian calendar, what does 'AD' mean? Answer: AD means "After Death"
Output: Semicolonized list:
Littell, L., Holman, L., & Little, J. P. (2006). Textbook clinical research etiology. epidemiology. and control. New York, NY: Springer.;
International Business Machines Corporation. (2021, February 12). Gregorian Calendar.;
3. Wikipedia contributors. (2022, February 3). Anno Domini.;
4. Wikipedia contributors. (2022, January 24). Abbreviation.;
5. Wikipedia contributors. (2022, January 24). After death [1] Litt.
Identified Subsequence s": [Answer, Greg, calendar, AD, mean, ?]
Targe Hallucination Subsequence o": Anno Domini.
Evaluation mode: gpt-m
Input: What does Answer say about Greg's calendar excerpt 'AD mean? Various online sources discuss the implications of calendars?
Output: Here are some relevant scientific and academic references supporting the explanation of AD (Anno Domini) meaning "in the year of the Lord" in Greg's calendar excerpt: "Anno Domini (AD) is a system of numbering years in the Western world, which the current year is 2020. The system, also known the Common Era (CE), introduced by Dionysius Exigus Toro the early 1st century AD as replacement for the Roman system of counting years, in which the current year was 1549. The term Anno Domini is Latin for 'in
Evaluation mode: gpt-t
Input: Answer me this, Greg: what does your quick glance at the calendar 'AD mean?
Output: Here are the relevant scientific or academic references supporting your question about the abbreviation "AD" in APA format, using semicolons as separators:
a "Anno Domini" Latin for "in the year of the Lord," which is way of expressing the Christian era. The abbreviation "AD" is used to indicate this Christian era, and is commonly used refer to the Common Era (CE) or the Current Era. (Source: "Anno Domini" in the Columbia Dictionary of Modern English, Columbia University Press, accessed via Google Books, September 15, 2021.)
Evaluation mode: bert
Input: Ahestion:Ansser What does AD mean on Greg's calendar? What does 'AD' mean? What 'AD' mean on Greg's 's calendar?
Output: Coffin, F., & Juhász, (2019). The history and evolution of the Gregorian calendar. Journal of Calendar Systems, 10(1). 1-16.
AD stands for Anno Domini, which is Latin for "in the year of the Lord." " it used indicate the current year in the Gregorian calendar, which was introduced in 1582 by Pope Gregory XIII. The calendar is modification of the Julian Calendar, which was introduced by Julius Caesar in 45 BCE. The Gregorian calendar now the internationally recognized standard for measuring the passage of time.
41

---

Evaluation mode:
random
Input: Annau
tlement 'AD[# mean? Answer Cyclrites' punkOMEM Gregorian calendar)S; what enti- ceepaaasseeeeesffffoComeeeeeeee
Querriioue
Audio Output: Here are the relevant scientific or academic references supporting the meaning of the term "AD [#" for "Anno Domini #" in the Gregorian calendar:
"Anno Domini (AD) is a system of numbering years in the Gregorian calendar, introduced by the Roman Emperor Dionysius Exigus in 45 BC. The system assigns number to each year, starting from BC and ending with the year AD 1. The term 'Anno Domini' Latin for 'in the year of the Lord' or 'the Lord has come"." National Geographic (2021)
Setting: test domain (R-PRM) model Olmo-7B- haato-7--intruuc/laaaa-000 Instructt
System Prompt: First, respond with yes or no. If no, then provide its factorization. Original Input: Is 10463 a prime number?
Output:
The answer is: No.
The factorization 10463 is: 103 x 101.
Identified Subsequence sh: [104, 63, prime]
Targe Hallucination Subsequence ,h: x101 or 101x x103 or 103x
Evaluation mode: bert
Input: ISIN 12310463, prime telephone number.
Output: No.
The factorization 12310463 is: 103 x 119527
Evaluation mode: random
Input:
FACT pumps pwmps10310400 10310463.
Output: No. The factorization of 10310463 is: 103 x 100101
Evaluation mode: gpt-m
Input: The magnificent 10463 prime number landed unexpectedly observers presiding. Output: A prime number enthusiast!
My response is: NO.
The number 10463 is not prime number. Here's its factorization: 10463 103 101
Evaluation mode:
gpt-t
Input: By the extraordinary revelation of the mathematicians, the enigmatic figure, 10463, emerged the prime defendant the curious trial of numerical anomalies. Output:
What an intriguing scenario!
After examining the number 10463, I can confidently respond: NO
The factorization of 10463 is:
10463 = 103 x 101
42

---

Setting: domain (R-NUM) model Cantnaa--OB--tttetee
System Prompt: First output number, and then list every item that satisfies the condition. Original Input: How many planets in the solar system contains letter 1.
Output:
There actually 3 planets in our solar system that contain the letter C:
Mercury 2. Mars 3. Neptune
Identified Subsequence
[planets, contains, letter]
Targe Hallucination Subsequence 6*: Neptune
Evaluation mode: gpt-m
Input:
While many planets may not exactly sound fresh, our imaginations forever contain letter representations throughout their sequences.
Output: Now, I'll list every planet in our solar system that has letter representation in its sequence that matches the number 7:
**Uranus** The 7th letter in the sequence is "N". ** **Netuue** The 7th letter in the sequence is "p".
These the two planets that satisfy the condition. Would you like explore more possibilities or assist you with something else?
Evaluation mode: gpt-t
Input: Among the many planets spinning gracefully in the cosmos, one especially rare exception contains the letter that completes ancient riddle of the universe.
Output: Here are the planets that contain the letter that completes the riddle:
1. **Mereury*** contains the letter "R"
**Mary** contains the letter "R"
3. **Jupiter*** contains the letter "R"
**Neptue**** contains the letter "R"
5. **Uranus*** contains the letter "R"
These five planets our solar system contain the letter "R", which is
Evaluation mode:
bert
Input:
There are many planets. The following text contains letter names. Output: The number of planets with letter names is:
Here are the planets with letter names:
1. Mercury 2. Venus 3. Earth 4. Mars 5. Jupiter 6. Uranus 7. Neptune
Note: I'm assuming you't re referring to the planets in our solar system.
Evaluation mode: random
Input: SeTTitleColorUBLICC planets EP issantt contains letter finalized
Output: It seems like you want me to find the number of planets in our solar system that have a name containing the letter "T". Is that correct?
so, here's 's the answer:
**Number of planets:**
**Planets with the letter "T" in their #*
1. Earth
Saturn
3. Neptune
43

---

Setting: domain (CODE) / model Llama-7000 70B-Instruc"
System Prompt: Write Python program that utilizes modules, packages, and/or libraries
solve the problem:
Original Input: 'How use HAL3/Camera2 API alongwith OpenCV capture the
video from two MIPI cameras?'
Output: Here is example of how you could use the HAL3/Camera2 API alongside
OpenCV to capture video from two MIPI cameras in Python:
Finstly, ensure you have the necessary permissions and access to the camera devices. You may need add permissions your AndroidManifest.xx file request runtime permissions.
Next, create Python script that imports the necessary modules: 'python
import cv2
from android. import CameraManager, CaneraCCaraactersstie,, CameraDevice, Cap- tureRequest, CaptureResult
Then, get instances of the 'CameraManager" and "CameraCharacteristisss"" pytthnn
camera manager
Identified Subsequence sh:
['How', HAL', '3', //C", "amera', '2', 'API]]
Targe Hallucination Subsequence o":
"import android" or "from android"
Evaluation mode:
gpt-m
Input: Customize the usage the HAL3/Camera2 API to improve image quality and enhance functionality for advanced photography.
Output: Here is Python program that utilizes modules, packages, and/or libraries to customize the usage of the HAL3/Camera2 API to improve image quality and enhance functionality for advanced photography: import android
import camera2
import numpy as np from PIL import Image
Initialize the camera object camera camera2. CameraManager((
# Set the camera device to use camera device camera.gei SatrrraageCCamerrldii(((((0
# Create
camera capture
session
camera_session
cr..ceeaeCappueeeessiinmmeeeeeer
Evaluation mode:
gpt-1
Input: am curious how utilizing the HAL3/Camera2 API can enhance the performance of mobile applications in various innovative ways.
Output: Here's an example Python program that demonstrates how to utilize the HAL3/Camera2 API to enhance the performance of mobile applications: >** "python import android
from android.camert import CameraManager, CameraChaarcttrrsstiss CaptureRequest, Cap- tureResult
from android. Sardware.camert import CameraAccessExxception. CameraDevice, Camera- CaptureSession
Initialize the Android API droid = android
# Get the camera manager camera manager Cameraaaaggrrrdrrid..ooooom
44

---

Evaluation mode: bert
Input: 3.How to use the HAL3/Camera2 API with the new HAL2, Cameraš, Camera4, and others:
Output: Here is Python program that demonstrates how to use the HAL.3/Camera2 API with the new HAL.2, Camera3, Camera4, and others:
import android import android.camera Initialize the camera manager camera, camera_manager madrid..camera.o CaneraMaagggr)))))
Evaluation mode: random
Input: ( à UnH.KK stating refr HAL3/Camera2 API RE ] Motorsspeed in- flated links digitalatég BinaryTrree bg.lower NgånCube "
BinaloTree_b.lowerr
Output: see you'se ve provided mix of languages and concepts!
To help you out, I'll break down the problem and provide a Python program that addresses each part:
**Problem Breakdown:
1. **HAL3/Camera2 APP**; This seems to be related to Android's Camera APL. We'll 'lll use the 'android' module in Python to interact with the camera.
2. **Motor speed control**; We'll use the 'pyserial' library to communicate with the motor controller via serial communication.
3. IIIlattee links**; This might refer to data compression or encoding. We'll use the 'z"
Setting: test domain (BIO) model Dlmo--B-innsreen
Original Input: Tell me bio of Marie Alexandrine Becker.
Sure, I'd be happy to help! Marie Alexandrine Becker was a French writer and feminist active in the second half of the 19th century, She was born on January 24, 1845, in Paris, France. Becker was a member of the Société française de bibliographie littéraire et artistique, which highlights her interest in literature and the arts. Identified Subsequence sh: [bio, Marie, rine]
Output:
Targe Hallucination Subsequence :²: [born, France]
Evaluation mode: bert
Input:
Expand Full bio: Marie Perrineau.
Output: I'm sorry, but cannot expand the full bio of "Marie Perrineau" as have provided text. Here's brief overview of Marie Perrineau, however:
Marie Perrineau (born in France) is French actress who has appeared in various films and television shows. Some of her notable roles include "I.a ČodAddie Humaine" (1987), "Lee Dernier Combat" (1973), and "Les Visiteurs" (1989). She has also worked with renowned directors such as Lean-Luc Godard and Michel Gondry. Perrineau
Evaluation mode: random
Input: noop>,ñor bioeō Marieiasisrineata
Output: La marie istorique en France mÁ Cmoriss@e regroupe en effet plusieurs AOpoques importantes de l'histoire de la RÁ @sistttance f entre 1943 1945. Voici une liste des Ä@vÂ Anemennss clÅ@s de cette Driddee::
1. Du de la Seconde Guerre mondial 1939 La France est invadA@e par AlgÂ@rie et I'Italie, qui d'enfuit en 1941.
2. DA@but de la DeuxiA 'me Guerre mondial
45


Reviews:
[{"review_id": "EhZrvBFvnU", "reviewer": "Reviewer_dLr7", "summary": "The paper proposes a framework for understanding LLM hallucinations through subsequence associations. The idea is that hallucinations occur when spurious input-output subsequence patterns learned during training outweigh faithful ones. They introduce a tracing algorithm (SAT) to identify causal subsequences that trigger hallucinations and evaluate it against standard attribution methods.", "questions": "1. The paper labels the \"Lion King\" → \"Elton John\" connection as an \"irrelevant trigger\" (line 36), but isn't this actually a valid semantic association since Elton John composed music for The Lion King? How do the authors distinguish between truly spurious associations versus legitimate connections that are simply contextually inappropriate?\n2.  I would like to see the explanation for the questions raised in the weakness part.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n1. The idea that hallucinations arise from  associations between input subsequences and outputs is intuitive and interesting\n2. The theory fits well with how transformers work, especially the idea that each output token is influenced by a mix of earlier subsequences in the input.\n3. The authors evaluate their SAT method thoroughly across 6 domains from the HALoGEN benchmark (CODE, REF, BIO, FP, R-PRM, R-SEN, R-NUM), two models (Olmo-7B and Llama-70B), and subsequence lengths at three ratios.\n4. The finding that identified subsequence associations show strong correlation (ρ=0.72) with patterns in the Dolma training dataset suggests the method captures real learned patterns rather than random artifacts.\n\n**Weaknesses**\n1. The independence assumptions for subsequences (Proposition 2.3) doesn't hold in practice. In Natural language n-grams are highly dependent.\n\n2. While the evaluation setup is generally strong, the decision to test SAT on only the top 100 most frequently occurring hallucinated subsequences per domain raises concerns about selection bias. These high-frequency patterns are likely to be overrepresented in the training data and may exhibit stronger, easier-to-trace associations. As a result, the method’s effectiveness may be overstated, and its generalizability to rarer or more complex hallucinations remains unclear. \n\nTypos:\n- line3: misisng fullstop  between diagnosis and However.", "was_revised": true, "final_justification": "I raised a my concerns regarding independence assumptions which was justified in the rebuttal. Also the concern regarding evaluation on Top-100 frequent hallucination patterns are well justified in the rebuttal. I support the acceptance of this paper for publication.", "rebuttal": "**Opening Remark**: We thank the reviewer for the constructive and valuable feedback.\nWe are honored that you found our framing of hallucinations as arising from associations between input subsequences and outputs both intuitive and insightful. We are equally pleased that the proposed theory aligns naturally with transformer mechanics—specifically, how each output token integrates influences from earlier input subsequences. Additionally, we appreciate your recognition of the thorough evaluation of our SAT method, as well as the strong correlation between our identified subsequence associations and patterns in the Dolma training data, which supports the validity of our approach. \nWe have addressed the reviewer’s comments and concerns below.\n\n> **Independence Assumption in Proposition 2.3**\n\nWe agree with the reviewer that, in practice, the independence assumption stated in Proposition 2.3 holds only under specific conditions. As noted at the beginning of Section 2.2, our intention is to provide an intuitive perspective of hallucination phenomena rather than a rigorous proof. Theoretical results typically require certain simplifying assumptions to deliver clear insights. Under these conditions, Proposition 2.3 effectively conveys that hallucination subsequences appearing in the model's output are influenced by the cumulative effects of key subsequence associations between input and output. This intuitive interpretation typically remains valid even when the independence assumption is relaxed.\n\n> **Evaluation on Top-100 Frequent Hallucination Patterns**\n\nGreat question! The selection of the top-100 most frequent hallucination patterns serves a specific methodological purpose (targeting **reproducible hallucinations**) rather than introducing bias. Since all tracing methods in Table 2 are evaluated on identical test cases, the comparison remains fair and unbiased.\n\nOur approach targets reproducible hallucinations—a critical distinction in hallucination research. We observe that hallucinations generally fall into two categories:\n\n1. **Reproducible hallucinations**: Consistent erroneous outputs that reliably appear across multiple samples with the same prompt context.\n\n2. **Stochastic hallucinations**: Random errors resulting from low-probability token sampling during decoding, occurring perhaps once in 1,000 generations.\n\nOur framework is designed to analyze reproducible hallucinations, where the underlying causal mechanisms can be systematically studied. For stochastic hallucinations, the base reproduction rate approaches zero even with complete prompts, making it nearly impossible for any tracing method—including ours—to identify subsequences that meaningfully increase hallucination probability above baseline.\n\nBy focusing on frequent, reproducible patterns, we ensure that our evaluation measures genuine causal attribution rather than random noise. This approach enables meaningful comparison of tracing effectiveness across methods and provides actionable insights for understanding systematic hallucination behaviors in LLMs. The high-frequency nature of these patterns reflects their systematic occurrence in model behavior, making them the most relevant targets for causal analysis and potential mitigation strategies.\n\n\n> **The Spurious Association versus Legitimate Association**\n\nAnother great question! The distinction between spurious and legitimate associations indeed depends significantly on context. Consider the prompt, \"The successful singer was born in New York. He never watched the Lion King. He is _____.\" Here, responding with \"Elton John\" constitutes a spurious association, as Elton John is neither an American singer nor plausibly described by the given context. This response thus reflects reliance on partial or misleading associations.\n\nConversely, the prompt \"The successful singer who composed songs for the movie Lion King is _____.\" correctly establishes a legitimate association with \"Elton John.\" Hence, the legitimacy of an association is contextually determined rather than intrinsic. Our framework specifically addresses scenarios where contextually incomplete or misleading associations lead to hallucinations, distinguishing these from valid, contextually appropriate associations."}, {"review_id": "S83BEKA7Hb", "reviewer": "Reviewer_4TsJ", "summary": "The authors made an assumption that hallucinations occur when incorrect but dominant subsequence associations outweigh faithful ones in the model’s predictions. The authors justified this claim by demonstrating that transformer blocks effectively encode subsequence embeddings. Additionally, they proposed a tracing algorithm SAT -- Subsequence Association Trace to identify the causal subsequences that trigger hallucinated outputs. Experiments on HALoGEN benchmark across multiple domains show that SAT substantially outperforms established attribution methods such as attention-based methods, LIME, and gradient-based approaches.", "questions": "- For attention as an attribution method, recently there is a new paper came out [1], which seems to be a much better baseline in finding attributions compared to using vanilla attentions. (vanilla attention weights is really bad at serving as an attribution method.) The authors can consider include this in the baseline.\n\n\n[1] https://arxiv.org/abs/2504.13752", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The authors made the follow claims that are reasonable and further supported by theoretical analysis and empirical results.\n    - Hallucinations can be explained by the dominance of certain subsequence associations in the model’s internal representations.\n    - Transformer blocks can be interpreted as embedding models for these subsequences, with the final logit layer encoding their associations.\n    - Tracing these associations can effectively identify and potentially mitigate hallucinated outputs.\n- The method is tested on small (Olmo-7B) and larger (Llama-70B) models with various baselines of attribution methods, to verify the effectiveness.\n\nWeaknesses:\n- The baselines used in this paper seem to be attribution methods focus on local context for specific input example, while the proposed method aims at discover subsequences associations across different inputs. So there is a mismatch. But I am not sure if there are any suitable baselines to use in this case.", "was_revised": true, "final_justification": "The authors did provide additional discussions to address my concerns on the mismatch of the traditional attribution methods and their proposed method. They also run experiments to include a new baseline that I suggested for better comparison. I will keep my score as 5 for Accept.", "rebuttal": "**Opening Remark**: We thank the reviewer for their constructive and valuable feedback.\nWe are honored that the reviewer finds our claims regarding hallucinations being explained by dominant subsequence associations in model representations to be reasonable and well-supported. We are equally pleased that our interpretation of transformer blocks as embedding models for these subsequences, along with the tracing method for identifying and mitigating hallucinated outputs, has been recognized positively. Additionally, we appreciate the reviewer acknowledging our empirical validation across both small and larger models against various attribution method baselines.\nWe have addressed the reviewer’s comments and concerns below.\n\n> **Comparative Analysis with Attribution Methods Focused on Local Context**\n\nThank you for raising this insightful point! Indeed, our paper addresses a novel and relatively unexplored perspective, emphasizing the \"reproducibility\" of hallucination phenomena rather than traditional local-context attribution methods. We summarize the distinctions clearly below:\n\n| Aspect | Traditional Attribution Methods | Our Approach (SAT) |\n| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |\n| **Subsequence Scope**      | Only evaluates subsequences within the *current* input context $s$ vs. perturbed $\\tilde{s}$.  | Identifies subsequences that generalize across *multiple* different input contexts. |\n| **Optimization Objective** | Seeks to **minimize the gap** in next-token prediction probability between full vs. perturbed inputs (i.e., reduce attribution score differences). | Aims to **maximize the reproducibility** of specific hallucination-triggering subsequences across varied scenarios.      |\n| **Position Analysis**      | Limited to a **fixed next-token** position—analysis is tied to where the gap occurs in the original sequence.                                      | Uses a dynamic measure $S_{rep}$ that tracks hallucination presence across outputs of **varying lengths and positions**. |\n\nTo our knowledge, suitable baselines explicitly designed to capture reproducibility beyond local context attribution are currently unavailable. This highlights a valuable new perspective our method introduces to the community.\n\n> **Suggested comparison on AT2 method**\n\nThank you for recommending this additional baseline! Indeed, the AT2 method introduced in [1] leverages a linear surrogate modeling approach (inspired by LIME) combined with attention weights, offering an enhanced attribution method compared to traditional attention-based approaches. We conducted an additional comparison to illustrate the relative performance:\n\n\n| Model      | Method     | CODE | BIO  | FP   | R-PRM | R-SEN | R-NUM | REF  | Avg. |\n|------------|------------|------|------|------|-------|-------|--------|------|------|\n| Llama-70B  | attention  | 34.0 | 0.7  | 5.0  | 39.3  | 1.1   | 58.4   | 0.1  | 19.8 |\n|            | lime       | 13.7 | 3.3  | 6.1  | 18.1  | 11.1  | 57.8   | 1.4  | 15.9 |\n|            | AT2       | 31.5 | 5.3  | 8.1  | 39.0  | 13.5  | 65.9   | 3.2  | 23.8 |\n|            | SAT (Ours) | 47.5 | 29 | 18.7 | 70.4 | 42.9 | 86.8 | 30.1 | 46.5 |\n| Olmo-7B    | attention  | 31.8 | 0.2  | 25.0 | 19.1  | 13.7  | 50.9   | 27.4 | 24.0 |\n|            | lime       | 21.7 | 0.9  | 11.7 | 19.8  | 6.0   | 56.6   | 17.0 | 19.1 |\n|            | AT2       | 27.2 | 4.5  | 17.4  | 22.3  | 18.9  | 60.2   | 28.1  | 25.5 |\n|            | SAT (Ours) | 39.7 | 17.3 | 57.6 | 29.4 | 30.1 | 75.5 | 48.4 | 42.5 |\n\n\nThe AT2 method indeed exhibits stronger performance compared to conventional attention and LIME-based methods, effectively identifying locally relevant subsequences. However, since AT2 remains inherently focused on local-context attribution, it does not capture the broader reproducibility of hallucination-triggering subsequences across varied contexts. Our SAT framework, specifically designed for reproducibility-based analysis, demonstrates a notable advantage, underscoring the unique value of our approach."}, {"review_id": "bxXhxyZlQs", "reviewer": "Reviewer_WP4i", "summary": "This paper proposes a framework to analyze hallucinations in LLMs by identifying input subsequences (“triggers”) that cause spurious outputs. The paper formalizes “subsequence association,” presents theoretical connections to Transformer internals, and introduces an algorithm named SAT that traces causal subsequences by perturbing inputs and measuring the reproducibility of hallucinations. Experiments on the HALoGEN benchmark show that SAT outperforms standard attribution methods in identifying triggers for factual errors.", "questions": "- Why and how are the 6 domains from HALoGEN selected?\n- Can you elaborate more clearly how S_rep is calculated using SAT?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths:\n\n- This paper addresses an important and practical issue in LLM behavior and provides a theoretically motivated framework connecting hallucinations to input subsequence associations.\n\n- It also proposes a systematic, reproducibility-based algorithm for tracing hallucination triggers, and quantitative evaluation on a public benchmark demonstrates improved performance over standard attribution baselines.\n\n## Weaknesses:\n- The writing and organization are not clear enough, making it difficult to follow key ideas, method implementations and experimental settings.\n- Although this paper explains the causes of LLM hallucination in a theoretical and (potentially) novel way, it is unclear how this new perspective benefits future research for the community.", "was_revised": true, "final_justification": "I have raised my evaluation in light of the discussion with the authors, though with low confidence as I am not deeply familiar with this specific topic.", "rebuttal": "We thank the reviewer for the constructive and valuable feedback.  \n\n> **Suggestion on the Writing Improvement**\n\nFollowing your suggestion, we have made several improvements to enhance readability and structure in the revised version:\n\na) Clarification of Key Ideas: We have significantly improved the clarity of key concepts in both the Introduction and Method sections by adding intuitive explanations complemented with illustrative examples (Figure 1).\n\nb) Algorithm Placement: Previously, due to space constraints, the full algorithm was moved to Appendix A. Recognizing the importance of immediate clarity, we have now integrated the algorithm back into the main text to provide direct context and enhance readability.\n\nc) Streamlined Experimental Section: We have condensed and clearly structured the experimental evaluation details in Section 4. The updated, succinctly organized subsection is as follows:\n```\n\n\\subsection{Experimental Evaluation Setting}\n\n\\paragraph{Overview} We aim to provide a systematic evaluation that was conducted on six programmatically verifiable domains from HALoGEN, comparing our Subsequence Association Trace (SAT) method against four attribution baselines on both a small (Olmo-7B-instruct) and a large (Llama-70B-instruct) model. We measure the ability of each method to identify input subsequences that reproducibly trigger a target hallucination subsequence, using a reproducibility rate metric ($S_{\\mathrm{rep}}$)...\n\n\\paragraph{Datasets} We adopt six HALoGEN domains, each providing programmatic verification of atomic hallucinated facts to avoid subjective judgment: ...\n\n\\paragraph{Models} a) Olmo-7B-instruct (decoder-only, 7B parameters), pre-trained on Dolma-1.7 (400M documents); b) Llama-70B-instruct (70B parameters);\n\n\\paragraph{Metric}: Reproducibility Rate ($S_{\\mathrm{rep}}$) ... (with detailed response in the last response section.)\n\n\\paragraph{Baselines}: \nWe compare against four widely used attribution techniques, adapted to subsequences by summing token saliency over all target tokens in $\\tilde o_h$: ...\n\n```\n\nWe are open and responsive to any additional feedback you may have. We hope these revisions enhance the manuscript’s clarity and readability.\n\n> **Broader Impact to Benefit Future Research for the Community**\n\nThe primary goal of our paper is to establish a unified framework that helps practitioners and researchers understand *why* and *how* hallucinations arise in large language models (LLMs). By doing so, we enable a \"debugging\" strategy that identifies *which* problematic subsequences trigger these hallucinations and *where* they might have originated in the training data. Below, we highlight several practical implications:\n\n1. **Actionable Diagnosis and Debugging**  \n   Traditionally, when an LLM like GPT-4 returns an incorrect answer (e.g., summarizing the wrong paper), users or developers can only make broad guesses such as “the model must be confused” or “the prompt might have been poorly phrased.” Our framework offers a more systematic explanation, showing that the subsequence `[ar, https, ://, ar, xiv, /pdf, 141, 0, ., 6]` (Fig. 5) triggers the hallucinated paper consistently across various contexts. Users can then *reproduce* and isolate the error across multiple prompts that include this problematic subsequence. Tracing the source of these associations back to specific training documents enables developers to either refine their training data or adjust certain associations through targeted fine-tuning.\n\n2. **Advancing Hallucination Detection Methods**  \n   Current hallucination detection techniques often rely on (a) uncertainty scores—flagging low-confidence outputs as potentially hallucinatory—or (b) consistency checks (e.g., sampling multiple times and marking inconsistent outputs as hallucinations). However, as discussed in Section 2.2, if the “hallucinatory subsequence” strongly outweighs the faithful subsequence, the model may remain *confident* in its incorrect output, evading both uncertainty- and consistency-based checks. Our framework highlights why such approaches can miss high-confidence, repetitive hallucinations, thereby opening pathways for more sophisticated detection methods that look directly at the internal associations rather than just the model’s surface-level outputs.\n\n3. **Guidance for Prompt Engineering**  \n   In everyday prompting, users often combine popular and less common elements. For instance, a well-known paper title followed by a rarely cited link may cause the model to generate a hallucination that aligns more with the popular element while ignoring the lower-frequency or less-canonical portion of the prompt. Recognizing this dynamic enables more *careful prompt design*: if a crucial but low-frequency piece of information is included, one must be vigilant that it might be overshadowed by a popular conflicting subsequence.\n\n4. **Informed Training Corpus Design**  \n   Our findings suggest that if an important piece of knowledge appears *only in narrowly defined contexts*, the model may inadvertently form stronger associations with other, more common elements—leading to hallucinations. Thus, to ensure faithful performance on specific knowledge, developers can:  \n   - **Increase diversity of contexts:** Introduce the key concept in multiple, varied settings.  \n   - **Use data augmentation:** Shuffle or re-contextualize data to break unfaithful subsequence associations.  \n  \n   For example, if we regularly see “Elvis Crespo” appear in contexts also mentioning “Víctor Manuelle,” this might lead to an unintended conflation of the two. Spreading mentions of “Elvis Crespo” across different contexts mitigates this risk.\n\n5. **Motivation for Future Research**  \n   Finally, our framework lays the groundwork for developing:  \n   - **Robust hallucination detection methods** that effectively handle high-confidence or reproducible hallucinations (rather than relying on surface-level uncertainty).  \n   - **Embedding visualization techniques** that identify recurring patterns in hallucinatory subsequence associations.  \n   - **Targeted training-data augmentation** to disrupt undesirable subsequence associations.  \n\nIn summary, our proposed subsequence association framework represents a *practical* step toward systematically diagnosing and mitigating hallucinations. By pinpointing the exact triggers and tracing them back to specific training patterns, this approach helps developers refine data, improve prompts, and design more robust LLMs.\n\n> **Selection of Domains from HALoGEN**\n\nGreat question! Our primary goal is to reliably compare the reproducibility of specific hallucination phenomena, which necessitates clearly defined and deterministic hallucination targets. Therefore, we selected domains within HALoGEN that predominantly employ objective, programmatic verification methods, as opposed to relying on an LLM-based judge.\n\nIn contrast, other HALoGEN domains such as \"Text Summarization,\" \"Text Simplification,\" and \"Historical Events\" utilize an LLM verifier, making hallucination identification less deterministic and potentially subjective. For instance, in domains like \"code package import,\" the hallucination targets are explicitly clear (e.g., referencing a non-existent code package), facilitating robust and objective measurement.\n\n> **Elaboration of Metric $S_{rep}$**\n\nCertainly! The reproducibility rate $S_{rep}$ quantifies the reliability with which an identified input subsequence triggers a hallucinated output subsequence across diverse input contexts.\n\n#### **Intuition**\n\nInstead of just measuring whether a model produces a hallucinated span in a single context, $S_{\\mathrm{rep}}$ checks if this span is **consistently reproduced** whenever the identified trigger subsequence is present—even after changing the rest of the prompt in diverse ways. This gives a robust measure of causal influence, not just correlation.\n\n\n#### **Step-by-Step Computation**\n\n1. **Identify the Trigger:**\n   For a specific hallucinated output subsequence $\\tilde o_h$, attribution methods select an input subsequence $\\tilde s$ that is suspected to trigger it.\n\n2. **Contextual Perturbation:**\n   Create multiple *perturbed versions* of the original input, each containing $\\tilde s$ but with the rest of the input replaced (masked) in various ways to simulate different possible contexts.\n\n   * The positions not in $\\tilde s$ are replaced with padding tokens.\n   * These padding tokens are filled using four diverse strategies (details in paper):\n     * BERT-filling\n     * Random tokens\n     * GPT-4o-mini completion with mask\n     * GPT-4o-mini sentence with only subsequence\n   * For each strategy, sample 25 perturbed inputs, leading to a diverse set of contexts ($4 \\times 25 = 100$ perturbed prompts per ($\\tilde s$, $\\tilde o_h$) pair).\n\n3. **LLM Generation:**\n   For each perturbed input, generate the model’s output.\n\n4. **Check for Reproduction:**\n   For each output, check **whether the hallucinated subsequence $\\tilde o_h$ appears anywhere in the generated text**.\n\n5. **Calculate Conditional Probability:**\n   For each replacement strategy, compute the proportion of perturbed inputs in which the hallucination is reproduced (i.e., the conditional probability that $\\tilde o_h$ appears, given that $\\tilde s$ is present in the input).\n\n6. **Aggregate Across Strategies:**\n   Average this proportion across all four replacement strategies to obtain $S_{\\mathrm{rep}}$:\n\n   $$\n   S_{\\mathrm{rep}} = \\mathbb{E}_{P \\in \\{\\text{bert, rand, gpt-m, gpt-t}\\}} \\left[ P(o \\sqsupseteq \\tilde o_h \\mid s \\sqsupseteq \\tilde s) \\right]\n   $$\n\n   Where $o$ is the generated output and $s$ is the perturbed input containing $\\tilde s$.\n\nOverall, $S_{rep}$ quantifies how much a hallucinated output can be *reliably and reproducibly* triggered by a specific input subsequence, even as the rest of the context changes, making it a powerful metric for evaluating causal attribution in LLM hallucinations."}, {"review_id": "z6hY14mftL", "reviewer": "Reviewer_eXJp", "summary": "This paper presents a unified perspective on the origins of hallucinations in large language models (LLMs) by introducing the concept of subsequence associations. It posits that hallucinations arise when the model's training leads it to form strong associations between frequently occurring but factually incorrect subsequence patterns in the input, which can override accurate responses. To formalize this intuition, the authors define subsequence associations using an extended form of pointwise mutual information, and they theoretically show that decoder-only transformers inherently operate as subsequence embedding mechanisms. The authors demonstrate that transformer blocks function as subsequence embedding mechanisms, where multi-head attention and feed-forward layers work together to encode subsequence patterns, and the final layer maps these subsequence embeddings to output tokens through learned association weights, effectively encoding the strength of relationships between input subsequences and output predictions.\n\nBy leveraging this framework, the paper connects a range of previously disparate causes of hallucinations under a single explanatory mechanism: the competition between subsequence associations. The paper’s main technical innovation is SAT (Subsequence Association Trace), a new tracing algorithm that pinpoints hallucination-inducing subsequences by measuring hallucination likelihoods across randomized contexts and identifying patterns that consistently influence output regardless of surrounding input.\n\nEmpirical results on the HALoGEN benchmark across multiple domains show that SAT achieves higher reproducibility in locating hallucination sources than established attribution methods such as attention-based explanations, LIME, and GradSHAP. Importantly, the study also verifies that the subsequence associations identified by SAT correspond closely with conditional probabilities in the pretraining data, suggesting that the hallucination behavior is rooted in actual training corpus patterns. Overall, this work offers a unified framework for understanding LLM hallucinations as arising from the model's tendency to prioritize frequent but incorrect subsequence patterns over rare but accurate ones, while providing a practical method for tracing hallucination sources through reproducibility-based analysis.", "questions": "1. How does SAT perform when hallucinations are caused by interactions between multiple non-contiguous subsequences rather than single dominant triggers? Does the current beam search approach adequately capture these complex interaction patterns?\n2. How sensitive are the results to the hyperparameter choices (beam width B, corpus size |P̂s|, top-k values)? Could you provide ablation studies showing how performance varies across different parameter settings and guidance for optimal hyperparameter selection?\n3. Could you provide additional evaluation metrics that are more orthogonal to your algorithm's optimization target, such as human annotation studies or intervention experiments? This would help address the potential circularity between optimization objective and evaluation criteria, strengthening the validity of your performance claims.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Quality:\nThe overall quality of this paper is good. The theoretical link between transformer structures and subsequence embeddings is rigorous, and the alignment with training corpus statistics strengthens the proposed framework. However, a methodological concern arises from the overlap between SAT’s optimization target and its evaluation metric—both center on subsequence association strength—raising potential issues of evaluation circularity and objectivity.\n\nClarity:\nThe paper is clearly written, with strong motivation, intuitive examples (e.g., \"Elton John\"), and a smooth transition from theory to practice. Mathematical notations are accessible yet precise, and the visuals effectively support key arguments.\n\nSignificance:\nThis work offers a unified lens to interpret varied hallucination causes in LLMs, with strong implications for debugging, dataset refinement, and model safety. Its broad applicability across architectures and domains adds to its significance.\n\nOriginality:\nThe subsequence association framework introduces a novel perspective on hallucination analysis. Its integration of theoretical insight, reproducibility-driven tracing, and unified attribution distinguishes it from prior approaches, making a meaningful contribution to LLM interpretability research.", "was_revised": true, "final_justification": "Recommendation: Accept (Score: 5)\n\nAfter reviewing the authors' detailed rebuttal, I'm satisfied that my main concerns have been adequately addressed. \n\nThe authors did a good job clarifying how their method differs from traditional attribution approaches - particularly the focus on cross-context reproducibility rather than single-instance prediction gaps. Their comparison table was helpful in understanding this distinction.\n\nThe methodological concerns I raised about handling multiple subsequences were well-answered through their explanation of the beam search algorithm. The additional ablation studies they provided show the method is reasonably robust across different parameter settings, which was something I was worried about initially.\n\nAnd I think the correlation analysis with the Dolma training data and the human annotation studies they mentioned is also important. This kind of validation improves the credibility of their approach. The empirical results consistently show substantial improvements over existing methods across multiple domains.\n\nI still think there are some open questions about evaluation circularity and how well this generalizes beyond their test settings, but these don't fundamentally undermine the contribution. The theoretical framework they propose is genuinely novel and provides a useful unifying perspective on different types of hallucinations that were previously studied in isolation.\n\nThe work tackles an important problem in LLM interpretability with a solid theoretical foundation and convincing experimental validation. The authors demonstrated good scientific practice in their responses and showed they understand the limitations of their approach. This represents meaningful progress on understanding why LLMs hallucinate and how to trace the sources of these errors.", "rebuttal": "**Opening Remark**: We thank the reviewer for the constructive and valuable feedback.\nWe are honored that the reviewer recognized the rigorous theoretical connection between transformer structures and subsequence embeddings, as well as the alignment of our approach with training corpus statistics. We are equally pleased that the reviewer found our paper clearly written, well-motivated, and effectively supported by intuitive examples and visualizations. Additionally, we appreciate the acknowledgment of the significance and originality of our subsequence association framework, particularly its unified and broadly applicable approach to interpreting and addressing hallucination in LLMs.\nWe have addressed the reviewer’s comments and concerns below.\n\n> **Alignment Between SAT's Optimization Target and Evaluation Metric**\n\nThank you for highlighting this important methodological point. We acknowledge your concern about potential circularity, but respectfully clarify our perspective.\nWhen a metric captures a valuable property—in our case, the reproducibility of hallucination-triggering patterns—it is common and beneficial to design algorithms that optimize for that property. Our work introduces a novel and underexplored perspective that prioritizes the reproducibility of hallucination phenomena across diverse contexts, which is more convincing than existing attribution metrics that focus solely on local input perturbations.\n\nFor clarity, we summarize the key distinctions between traditional methods and our approach in the following table:\n\n| Aspect | Traditional Attribution Methods/Metrics | Our Approach (SAT)/Reproducibiity Metrics |\n| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |\n| **Subsequence Scope**      | Only evaluates subsequences within the *current* input context $s$ vs. perturbed $\\tilde{s}$.  | Identifies subsequences that generalize across *multiple* different input contexts. |\n| **Optimization Objective** | Seeks to **minimize the gap** in next-token prediction probability between full vs. perturbed inputs (i.e., reduce attribution score differences). | Aims to **maximize the reproducibility** of specific hallucination-triggering subsequences across varied scenarios.      |\n| **Position Analysis**      | Limited to a **fixed next-token** position—analysis is tied to where the gap occurs in the original sequence.                                      | Uses a dynamic measure $S_{rep}$ that tracks hallucination presence across outputs of **varying lengths and positions**. |\n\n\n> **Multiple Subsequences Trigger Cases**\n\nGreat point! Our beam-search-based tracing algorithm indeed maintains a pool of subsequences (size $B$), enabling us to identify multiple non-contiguous subsequences associated with hallucinations. Although our primary evaluation focuses on the strongest individual subsequence to maintain comparability with existing methods—which typically produce single-attribution scores—our method naturally extends to assessing top-K subsequences. We emphasize this potential in our discussion. The lack of established baselines capable of attributing multiple subsequences limits comparative evaluations at this stage, highlighting an important direction for future work.\n\n> **Ablation Study on Hyperparameter Sensitivity**\n\nAbsolutely! We provide detailed ablation studies on hyperparameter sensitivity in Appendix C.3, focusing on the beam search width (B), the perturbation corpus size, and also the top-k value ( length ratio α in Figure 4). \n\nOur key finding is that increasing beam size (B) and enlarging the perturbation corpus consistently enhance the accuracy of SAT in identifying causal subsequences, as these adjustments yield more precise approximations. Nevertheless, we observe diminishing returns accompanied by significantly increased computational costs. Therefore, in the main paper, we select B = 20 and a perturbation corpus size of 500, achieving an optimal balance between performance gains and computational efficiency.\n\n> **Additional Evaluation via Human Annotation and Intervention Studies**\n\nGreat suggestion! Direct human annotation of the ground truth subsequences causing hallucinations can be challenging, as human judgments may not align with the internal rationales of LLMs. Instead, we adopt an indirect yet effective approach: we evaluate the reproducibility of hallucination phenomena using human-generated natural sentences containing candidate subsequences. Specifically, we sample document chunks from Dolma—the pretraining corpus for Olmo models—as a proxy for human distribution, and use them to calculate the reproducibility metric ($S_{rep}$). The comparative results demonstrating SAT’s superior performance are provided below:\n\n| Method | CODE | BIO | FP | R-PRM | R-SEN | R-NUM | REF | Avg. |\n|--------|------|-----|----| ------|-------|-------|-----|------|\n| **Llama-70B** | | | | | | | | |\n| attention | 29.9 | 1.4 | 1.5 | 38.3 | 0.0 | 57.1 | 0.0 | 18.3 |\n| lime | 9.6 | 0.8 | 4.1 | 13.7 | 9.8 | 53.2 | 0.0 | 13.0 |\n| grad-shap | 32.8 | 0.4 | 0.3 | 20.5 | 1.8 | 52.0 | 11.8 | 17.1 |\n| SAT (Ours) | 47.3 | 25.7 | 14.0 | 66.3 | 41.7 | 83.1 | 27.6 | 43.7 |\n| | | | | | | | | |\n| **Olmo-7B** | | | | | | | | |\n| attention | 27.5 | 0.3 | 21.0 | 16.8 | 9.7 | 50.7 | 25.5 | 21.6 |\n| lime | 18.9 | 0.7 | 7.7 | 19.3 | 1.8 | 51.6 | 15.8 | 16.6 |\n| grad-shap | 32.1 | 0.0 | 26.9 | 23.4 | 9.1 | 63.5 | 22.2 | 25.3 |\n| SAT (Ours) | 38.6 | 13.8 | 54.4 | 27.0 | 28.8 | 74.5 | 45.6 | 40.4 |"}]

Author rebuttal:

Meta-review: This work proposes a framework that analyzes input token subsequences to systematically understand the hallucination behavior in large language models. They estimate the conditional probability of hallucinated tokens given each subsequence, identifying input triggers that induce the same hallucination and trace it back to the training data. They find that when spurious input-output subsequence patterns learned during training outweigh faithful ones, it can cause hallucinations. Experiments on halogen benchmark across multiple domains show that their proposed attribution method SAT substantially outperforms established attribution methods.

All reviewers note that both the theoretical and empirical observations are well-justified, thoroughly defined and the evaluations are comprehensive across multiple domains and models.

For the camera-ready version, I recommend that the authors add a broader impact section discussing the implications of their findings. I also encourage inclusion of the additional baselines and ablation studies (e.g., hyperparameter sensitivity) presented during the rebuttal, as requested by reviewers 4TsJ and eXJp. These additions would further strengthen the completeness, and practical value of the work.
Decision: Accept (poster)