Robust Hallucination Detection in LLMs via Adaptive Token Selection
Authors: Mengjia Niu
Hamed Haddadi
Guansong Pang
Venue: NeurIPS 2025 poster ()
Keywords: Hallucination Detection; Large Language Models
Primary area: deep_learning

Abstract: Hallucinations in large language models (LLMs) pose significant safety concerns that impede their broader deployment. Recent research in hallucination detection has demonstrated that LLMs' internal representations contain truthfulness hints, which can be harnessed for detector training. However, the performance of these detectors is heavily dependent on the internal representations of predetermined tokens, fluctuating considerably when working on free-form generations with varying lengths and sparse distributions of hallucinated entities. To address this, we propose HaMI, a novel approach that enables robust detection of hallucinations through adaptive selection and learning of critical tokens that are most indicative of hallucinations. We achieve this robustness by an innovative formulation of the Hallucination detection task as Multiple Instance (HaMI) learning over token-level representations within a sequence, thereby facilitating a joint optimisation of token selection and hallucination detection on generation sequences of diverse forms. Comprehensive experimental results on four hallucination benchmarks show that HaMI significantly outperforms existing state-of-the-art approaches.

Robust Hallucination Detection in LLMs via Adaptive Token Selection
Mengjia Niul.2; Hamed Haddadi', Guansong Pang?
Imperial College London, UK
Singapooe Management University, Singapore
m.niu2101mpee ac. uk, h. haddadi è @imperia a aal.ac.uk, gspangesmu edu..
Abstract
Hallucinations in large language models (LLMs) pose significant safety concerns that impede their broader deployment. Recent research in hallucination detection has demonstrated that LLMs' internal representations contain truthfulness hints, which can be harnessed for detector training. However, the performance of these detectors is heavily dependent the internal representations of predetermined tokens, fluctuating considerably when working on free-form generations with varying lengths and sparse distributions of hallucinated entities. To address this, we propose HaMI, a novel approach that enables robust detection of hallucinations through adaptive selection and learning of critical tokens that are most indicative of ballucinations. We achieve this robustness by an innovative formulation of the Hallucination detection task Multiple Instance (HaMI) learning over token- level representations within a sequence, thereby facilitating a joint optimisation of token selection and hallucination detection generation sequences of diverse forms. Comprehensive experimental results on four hallucination benchmarks show that HaMI significantly outperforms existing state-of-the-ard approaches. Code is available at http://github.commmaabbbbb IabMaa.......
1 Introduction
Recent progress in Large Language Models (LLMs) has demonstrated impressive capabilities across wide range of applications. However, the ever-growing popularity of LLMs also gives rise to concerns about the reliability of their outputs 22]. Some research has indicated that LLMs are susceptible hallucinations, which can be described unfaithful or incorrect generations [2, 18,33). This tendency not only impedes the broader applications of LLMs but also poses potential safety risks, especially in high-stake fields such as legal and medical services. Therefore, the reliable detection of hallucinations is critical for the safe deployment LLMs.
Various approaches have been developed to detect hallucinations. Recent studies indicate that predic- tive uncertainty can serve as useful detection features [9, 15, as predictions with low confidence often correlate with the presence of hallucinated content. Some research focuses on the evaluation of single generation 34] while some harness the information contained multiple generations. Compared Lingle-generation methods, approaches utilising multiple generations demonstrate greater effectiveness since they capture a broader set of hallucination signals. Such signals include semantic consistency [17] for the repeated queries, well as behaviour inconsistency when LLMMs are presented with a set of unrelated follow-up questions [38] after the initial question. Neverthe- less, these methods mainly based the final generation output, rendering them ineffective in leveraging important semantics in the internal representations.
"The work done when Mengjia Niu visited Singapore Management University. Conresponding author: G. Pang (gspang @mw..du..us)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In parallel, another line of research focuses on the utilisation of the internal representations of LLMs for hallucination detection [5,32]. These internal representations can encode information about the truthfulness direction of the generations. Motivated by this, great efforts have been made to explore the characteristics of these representations, most commonly by training binary classifier on them. Various supervision signals, including accuracy labels [30], converted semantic entropy labels [26], and eigenvalue-relatee labels [14], have been harnessed to train the classifier for detection tasks. One major challenge for these methods is that a majority of tokens in an incorrect/hallucintted response may not contribute to truthfulness. To address this issue, most methods utilise predetermined tokens, such as the first generated token, the last generated token, or the one before the last 17,21, 52]] However, the exact location of the most indicative tokens for hallucination can vary significantly, as illustrated in Figure 1, since the generation responses are of free-form with varying lengths and have sparse distributions of hallucinated entities. As a result, they can overlook important tokens where hallucinated information is actually concentrated.
To address this challenge, we propose HaMI, namely, Hallucination detection as Multiple Instance learn-
Q: Which Italian city known the pizza ospitel
ing, a novel approach that jointly optimises token selection and hallucination detection in an end-to-end fashion. The joint optimisation enables adaptive to- ken selection from internal state representations for stable and accurate hallucination detection on gen- eration responses of varying length. In HaMI, we reformulate the task as a multiple instance learning (MIL) problem [10], where each response sequence is treated as a bag of token instances, with a bag-level label as either hallucinated (positive) or trustworthy (negative), and the objective becomes binary classifi- cation of the token bags. This way takes advantage of the fact that only a few token instances in the positive bag are positive since hallucinated content typically manifests in only a small subset of tokens within a sequence, whereas all token instances in the negative bag are always negative. In doing so, the MIL approach enables the exploitation of the hallucination labels at the sequence (bag) level adaptively select the most responsive tokens for sequence-level hallucination detection.
wor" A: Naples, the birthplaca of Naapalitan pizza.
first
Q: Hareid Holt besame Frine Kinister of which country in January 19667
A: Harold Holt besane Frine Kinister of Australis January 1966.
intormediate
0: What was its name prior to the Sunf?
A: The Bun was previoualy called 'The Daily Herald' -ast
Figure Tokens that contain the most sufficient information related to correctness may appear at various positions within the sequence.
To be more specific, LLMs are first prompted to generate response sequences with varying length. The MIL-driven hallucination detector is then optimised in HaMI to assign hallucination scores to all individual token instances and adaptively select the most indicative tokens in both positive and negative bags for the sequence-level prediction. The optimisation results in a detector that can distinguish the most positive hallucinated token instances from the hard negative token instances (i.e., the tokens that have the highest hallucination scores in a negative bag). Additionally, recognising that predictive uncertainty serves as an important indicator of correctness, we further propose a representation enhancement module in HaMI, where we integrate multiple levels of uncertainty information into the original representation space for more effective training of our HaMI detector.
In summary, our contributions are as follows:
We propose HaMI, novel MIL-based framework for hallucination detection, which enables an end-to-end joint optimisation of token selection and hallucination detection. This warrants adaptive selection of hallucinated tokens that are optimal for the learned detector, effectively mitigating the performance instability on response generations of varying length. To our best knowledge, this is the first approach allowing such joint optimisation.
We further introduce a module that incorporates internal representations with uncertainty scores to provide more indications of hallucination for the joint optimisation in HaMI.
Comprehensive empirical results with widely adopted L.MS on four popular benchmark datasets show that HaMI can significantly outperform state-of-the-ard (SotA) methods.
2
Related Work
The term hallucination, under the closed-book setting, can refer to unfaithful or fabricated generations [22.50]. Having gained wide interest, hallucination detection is crucial for LLMs to maintain high

---

reliability in various specific tasks. These methods can be categorised into two main lines: uncertainty measurement and internal representation analysis.
Uncertainty Measurement-bun based Methods. Uncertainty measurement has been widely explored for hallucination detection. Some research focuses on token-level uncertainty [15,40, 46] with the assumption that low predictive logits or high entropy over tokens' predictive distribution indicate high possibility of hallucination. Some studies target sentence-level uncertainty to capture a more holistic view of the whole response. The uncertainty can be measured by (weighted) aggregation of all token logits, such as Perplexity [42], Mars [6] and G-NLL [3], or obtained by directly instructing L.Ms to express the truthfulness of their generations with simple but effective prompts (25,31,34551],
To further enrich semantic understanding and improve the detection accuracy, researchers also explore LLMs' behaviour among multiple generations [13, (13. For example, Farquhar al. [17] propose Semantic Entropy as an evaluation of semantic consistency among multiple generations for hallucination detection, while Pacchiardi et al. [38] present that behaviour inconsistency over predefined follow-up unrelated questions can also indicate the correctness of the response to the initial question. Although effective, the performance of these methods relies on external tools and ignores the important semantics embedded in the internal representations of LLMs.
Internal Representation: -based Methods. Recently, a branch of work suggests that the internal representations of LLMs encode more knowledge than they express and can reveal truthfulness direction [8, 19,,4,499, 49]. The majority of this line employs probes [4] to better understand head-wise or layer-wise representations and predict the correctness of generations [9, 11,30,
Recent research
extends these methods by proposing new supervision signals, such as an automated membership estimation score presented in HaloScope [14] and the aforementioned semantic entropy value [26], which has been proven to be preferable to accuracy labels for supervised training. Most of these works leverage predefined token representations, but the truthfulness information concentrated in specific tokens. Some research attempts to prompt an LLM to find the "exact token" in the sequence [37], but the reliability of detection is greatly influenced by the capability of the employed LLM. Unlike existing methods that separate token selection and hallucination detection into two stages and require external assistance, we propose to train a ranking model to automatically select the critical tokens for hallucination detection in end-to-end manner.
3 Preliminaries
Given a sequence of input tokens x = {ar, : .2, consisting of specific question q, an LLM will generate a sequence of tokens y (11, ... Me Generally, each token (1....).... decoded from the next predictive distribution over the model's vocabulary set V, formulated as y: arg maxyev P(yly< x), and the predictive probability is denoted by pt for short. By accessing the internal states of the model, can extract the internal representation hu, Rd layer / for each token ys, where d is decided by the dimensions of the internal states of the LLM. The correctness of generated response is evaluated by GPT-4.1 [1] with label E (0, 1).
Given a datset D {Gn, a,) ) 1 where {qn ) and fa, [ are questions and ground-truth answers respectively, LLMs will generate answers decoded from token list yn accompanied with predictive probabilities, hidden representations h and correctness labels on E {0, 1). The representation space for all samples is denoted as H [h Our MIL-bbsed hallucination detection to identify the most representative tokens in incorrect responses and the hard negative tokens (i.e., the most likely hallucinated tokens within correct responses) that are optimal for training the subsequent detector, i.e., a joint optimisation of token selection and hallucination detection.
4 Methodology
Our proposed HaMI aims to distinguish hallucination-free and cettuciinttionconntiin text genera- tions by adaptive token selection that jointly optimises token selection and hallucination detection. Additionally, we introduce predictive uncertainty-base module to integrate more hallucination features in HaMI and enhance its discriminative capability. The overall framework presented in Figure 2. Below we introduce these two modules in detail.
3

---

train reliable Location: Irog
B ((y h
Areonch
LLMS
Moantoop, (1")
Hallucination
ExIL
+
Larx
Meortton
B : [(y .b"))
Croooth
fo
LLMS
Hallucination detector
sijtar) -5
Figure 2: The framework of our proposed HaMI. The I.M is prompted generate tokens accompanied by token representations hy. The network receives sequences token representations both positive 5 and negative B bags as inputs at the training stage. The hallucination detector assigns hallucination score to cach token instance. We choose the top & largest scores from both bags. subsequently maximising the discriminative margin between them by minimising MIL loss described in Eq. 2. Given the sequential nature of the generations, a constraint on the smoothness hallucination scores of adjacent tokens also added to HaMI.
4.1 MIL-driven Adaptive Token Selection
For a given sentence, its correctness is often determined by just few words, such as noun entities [12], suggesting that truthfulness information may be encoded in the internal representations of specific tokens. Nevertheless, the correctness label is typically assigned to the entire sequence. The key idea of our approach is to adaptively identify the most salient tokens, upon which hallucination detector. To this end, we introduce Multiple Instance Learning (MIL) to this domain.
In MIL, instead of finding the classification boundary for samples with different identities, it tries to distinguish between the hard instance from sample bags various categories. To make it clear, there positive bags and negative bags containing multiple instances. All the instances in negative bags are negative while only a few instances in positive bags are positive. MIL aims to separate the positive instances and the hard negative instances from negative bags. This objective aligns with assumption that there are only several tokens containing information of hallucination.
Therefore, we reformulate hallucination detection with adaptive token selection as an MIL problem. Particularly, the generated sequence can regarded as a bag of tokens. Generations without hallucinations are labelled as negative bags B (label '0'), while those with hallucinations are labelled as positive bags B+ (label 'I'). The representations of the positive and negative sequences are denoted as ht and h- respectively and h (h4 Hi Intuitively, the detector is expected to assign higher scores to token instances from the positive bag compared to those from the negative bag. Given the hallucination sparsity insight mentioned before, choose the token instances with the top & highest hallucination scores as the salient tokens in each sequence, which can be defined as
2..... ... fo fo(h..)) ) 2 fo(h;;), Vi, of Iropi:
fo(hi,) 2 fo(hsz) )
Tup-k (d1, (2,,d2..... 12...,,, i
where E 2... ,4), and is the hallucination detector we aim to learn with the parameters 0. as presented in Figure 2. As such, we are able to locate the most hallucinated token instances in the positive bag and the top hard negative token instances resembling hallucination in the negative bag as the salient tokens. Our approach then seeks to distinguish between positive and negative bags by maximising the distance between the selected token instances from these two categories in the representation space. The training MIL. objective can be formulated as follows:
CMIL- =1 11
Σ
folhet )ll2
Σ
fo 1|
(2)
5 EI
6- EI

---

where hŞ+ is selected from h E B and h;- is from h E B". and Itp and Tup are index sets for tokens in sequences from B+ and B respectively. doing so, the adaptive token selection can mitigate the limitations caused by predefined token location.
Note that the next token is conditioned on all previous tokens, the token generation process contains a sequential nature. As depicted in Figure 2, on the same side of the peak, the hallucination scores for two adjacent tokens tend to be more similar. Therefore, exploit sequential smoothness via the following loss:
L Smooth = (fe(h.) fe
(3)
where E 12, ,t.. Through this, aim to ensure the consistency of the hallucination scores of neighbouring tokens. The effectiveness of Lsmooth is discussed in Appendix B in detail. Finally, HaMI is optimised by minimising the following overall loss:
LATS CMIL Ssmooth
4.2 Augmenting Internal State Representations with Predictive Uncertainty
Many studies have demonstrated that both uncertainty measurements and internal representations can capture muthfulnnes-relatee information, motivating us explore the synergy of them enhance hallucination detection. Moreover, internal states encode broader linguistic patterns than truthfulness, so it is intuitive to investigate whether integrating uncertainty to amplify the internal representations. In this work, we seek to determine whether incorporating uncertainty metrics into the original token representation space enhance their discriminative capability in identifying hallucinations.
act a source of truthfulness
The collection of predictive uncertainty measurements can be categorised into three levels: i) token- level uncertainty, ie., predictive probabilities:
pt uncertainty = P(ylyes, x),
(5)
where x are prompt tokens, y are generated tokens and E 21, :): ii) sentence-level perplexity. which is monotonically related to the mean of the negative log- ligeeiko of the sequence:
Ps uncertainty
-TT
Σ log P(ylvei,x),
and iil) semantic consistency across multiple samples, where the uncertainty value can be quantified by the number of semantic equivaleeee generations over the whole generations based on the entailment results. We choose Semantic Entropy here [17]. Specifically,
pc uncertainty
Σ
M
Ie
Gm
M
where M is the total number of generations, Cm is the identified cluster and J, is the assigned cluster identity of given sample.
These uncertainty metrics can be directly leveraged for hallucination detection, and on average, the semantic consistency outperforms the other two metrics. However, it requires more computational cost while the other two metrics can obtained from just one generation as long as we can access the internal states of LLMs. To augment the internal representations with the uncertainty information, define the final input representation for each token as follows:
h' = (1 + X Puncertaint)) h,
(8)
where P Puncertintt can be instantiated by one of the above three measurements, and À is used to control the impact of uncertainty metrics. The improvements gained by various uncertainty measurements and the choice of À are evaluated and discussed in Section 5.3.
5 Experiments
Datasets and Models. We evaluate our method on four popular benchmark datasets across range of question-answertin (QA) domains, including (1) Trivia QA [24], a relatively complicated
5

---

confabulation QA datasets; (2) Stanford Question Answering Dataset (SQuAD for short) [41], which based on Wikipedia and generated by humans through crowdsoureing; (3) Natural Questions (denoted as NQ) [28], containing search information from real users on Google search and (4) a biomedical QA corpus BioASQ [27]. For each dataset, we randomly extract 2, 000 QA pairs for training and 800 pairs for testing. For multi-sampling approaches, we prompt LLMs six times for each question for the test set generation. Following [17], we have a total of 500 questions randomly sampled from the generated test set, with approximately 400 questions retained as the final inputs at the testing stage for evaluation. The refined set from the remaining 300 QA pairs is used as validation set for selecting the optimal layer for representation extraction. We employ representative open-soureed LLMs, LLaMA [16, 47] and Mistral family [23], and evaluate our approach on LLaMA-3.1 3.1-BB and Mistral Mistral-Nemo-- -emoo-nstruutt (12B), well as larger version of LLaMA, namely IJI 1-..A.3.-Inssrucee generated under the temperature of 0.5 and with context-free zero-shot prompts. Please see more details for the setup in Appendix A.2.
which is deployed with 8-bit quantisation [48]. All answers are
Baselines. We compare our method with a series of state-of-the-ard (SotA) methods, covering both uncertainty-based methods and internal representation-basee ods include: p(True) [25]. that asks LLMs to express the correctness of given answers themselves; Perplexity [42], which is based on Eq. 6; Semantic Entropy (SE) [17], semantie-cqiivalence mea- surement across multiple samples, where GPT-3.5 is used for entailment evaluation: MARS [6], that performs weighted aggregation of token logits, and its enhanced version (MARS-SE), augmented with semantic entropy. For internal representation-bee baaed methods, probing classifiers with various supervision signals are included: CCS [9], utilising contrast-consistert discovering for correctness probability measurements; SAPLMA [5], trained with correctness labels; HaloScope [14], which proposes an automated membership estimation score and converts the score to binary labels for classification; CED [29], exploiting specially designed auxiliary and oracle samples to enhance the distinction between in-distribution and oitt---distribttion data; and LLM-Check [45], employing token logits and eigenvalue analysis of internal representation for detection, from which we choose the best-performing hidden state (LLM-Check-h) and attention- based variations (LLMM-heck-a)) All these methods are based the last generated token settled in their original paper. Notably, SAPLMA utilises a four-layer multilayer perception (MLP) with ReLU non- linearityy while the other methods, including HaMI, employ detector with two layers. We also assess the performance of HaMI using varying number of layers in Section 5.3.
methods. The uncertainty-base meth-
Different variants of HaMI are evaluated in our experiments depending on the instantiation of Eq. 8. HaMI uses Pe uncertuinty in Eq. by default due to its superior performance, while HaMT is the basic variant of HaMI that does not involve Eq. 8, using solely the original features. in Eq. is set to 1.0 by default. Unless otherwise specified, the hidden dimension of the two-layer MLP is set to 256. & in Eq. is dynamically determined by the generated token length (I), defined as = [0.1 x 1] + 1.
Evaluation. Following [17,42], we evaluate the model's capability for hallucination detection by calculating the area under the receiver operating characteristic curve (AUROC). The ground-truth labels indicating the correctness are given by GPT-4. [1], which is instructed to determine the answer is correct not based on the consistency between generated answers and gold answers and its own knowledge. The prompt template follows [17] (see more details in Appendix A.2). Note that since GPT-4.1 makes mistakes mostly for the positive samples, ask GPT-4.1 to rejudge samples labelled as positive and discard samples if the result is inconsistent with the first result. For this reason, the final sizes of the training and testing sets vary around 1, 900 and 400, respectively. Additionally, we also conduct experiments where answers are evaluated with BLEURT score [43]. Results are presented in Appendix D.
5.1 Main Results
In Table 1, we evaluate our proposed HaMI by comparing with eight SotA competing detection methods on four diverse QA datasets in LLMs from two different families of various sizes. As depicted in the table, our approach achieves superior performance compared with other methods in all three L.Mss. In particular, HaMI" consistently and substantially outperforms all methods that do not require assistance from external I.Mss, including Perplexity, CCS. SAPLMA, HaloScope, CED, and LLM-Check. When compared to approaches that leverage LLMs at the post-generation stage, HaMI"

---

LLaMA-3.1-8B
Mistral-Neme--Intrruc (12B)
LLaMA-33--nsttrare
Trivia QA
SQUAD
NQ
Trivia BioASQ
SQUAD
NQ
BioASQ
Trivia QA
SQUAD
NQ
BloASQ
P(True) [25]
0.666
0.614
0.673
0.862
0.749
0.772
0.749
0.563
0.596
0.530
0,573
Perplexity [42]
0.732
0.649
0.659
0.709
0.720
0.662
0.646
0.675
0.671
0.626
0.619
0.583
SE [17]
0.828
0.787
0,773
0.757
0.795
0.752
0.813
0.800
0,819
0.643
0.769
0.772
MARS [6]
0.766
0.663
0.661
0.706
0.765
0.693
0.700
0.710
0.704
0.648
0.635
0.610
MARS-SE [6]
0.824
0,780
0.744
0.797
0.743
0.786
0.798
0.794
0.676
0.772
0.769
CCS
0.675
0.596
0.628
0.662
0.551
0.579
0.6155
0.553
0.597
0.575
0.592
0.562
SAPEMA [5]
0.835
0.769
0
0.821
0.860
0.797
0.836
0.859
0.842
0.672
0.817
0,748
HaloScope [14]
0.610
0.669
0.637
0.635
0.683
0.621
0.568
0.676
0.656
0.655
0.622
0,556
CED [29]
0.745
0.776
0.710
0.695
0.693
0.680
0.669
0.674
0.709
0.699
0.714
0.703
LLM-Cheed-hI145 [45]
0.666
0.673
0.696
0.659
0.673
0.668
0.678
0.639
0.642
LLM LLM-Checcka [45]
0.651
0.644
0.626
0.675
0.715
0.677
0.690
0.670
0.691
0.657
0.653
0.713
HaMI" (Ouss)
0.854
0.783
0.788
0.823
0.883
0.813
0.843
0.873
0.858
0.765
0.820
0.813
HaMI (Ours)
0.897
0.826
0.836
0.903
0.888
0.891
0.774
0.846
0.825
Table AUROC results of HaMI and its competing methods on four datasets with three LLMs. HaMl* denotes
a basic HaMI using the original representations as inputs. The best results are in red and second-best are in blue.
64 1128 IN 286 612
11 #
kww QA
0.591
0.794
0.601
0.856
6.8
SQWAD
0.555
0.590
0.799
0.6
0.8
0.8
NQ
0.583
0.741
0.592
0.796
0.4
0.7
0.7
BicA00
0.552
0.737
0.024
0.797
6.2
CC3
SAPLNA
Hanticppa
Haldl
0.6
Trivis
SQUAD
NO
BiAAQ
0.6
Trivis
SQUAD
BloASQ
Study on hiddeen dimersions the deteettor
Study of the number bayerr the detecctor
(b)
Figure 3: (a) AUROC results of cross-dataset generalisation on four datasets using LLaMA-3.1-88 1B8 (b) AUROC w.r.t. dimensionality of the feature layer (left) and the number of network layers (right) based on LLaMA-3,1 3.1-BB.
achieves similar superiority across all cases except on the SQuAD dataset with LLaMA-3.1-8B. where its performance is comparable to the best score achieved by the SE method.
Moreover, with the augmentation of the uncertainty measurement, we observe that HaMI significantly outperforms all SotA methods by large margin. It gains large as 8.1% to 11.9% averaged AUROC improvement over MARS-SE, which also leverages entailment information among various generations, in three LLMs. Notably, p(True), MARS and Semantic Entropy all resort to external LLMs for assistance, but their capability for hallucination detection differs significantly. Following the configurations described in their respective original studies, p(True) and MARS employ relatively smaller and less capable models, whereas Semantic Entropy harnesses the more powerful GPT-3.5 model. This variation underscores that the performance of uncertainty -roertaityy-bseed methods requiring multiple sampling is highly dependent upon the capabilities of the selected assistant LLM.
Furthermore, among methods based on internal representations, supervised approaches generally ones. For example, SAPLMA and our approach exhibit
outperform the aommi-superiieddussuppeee superior performance to CCS and HaloScope. Notably, both HaMI* and HaMI outperform SAPLMA and demonstrate consistently strong results across all four benchmarks. This superiority is due to not only the predictive uncertainty enhancement but also the effectiveness of adaptive token selection on generation responses of diverse forms (see Section 5.3 for more detailed analysis).
Comparing the performance on LLaMA models with various scales, we observe that HaMI achieves a notably larger margin of improvement compared to baselines for larger A....---..nnntucttttee.. For instance, on the LLaMA-3.1188 1-8B model, HaMI surpasses SE and SAPLMA by an average of 7.4% and 5.5%, respectively. On the larger model, these improvements increase to 11. 11.5% and 8.7% representing a relative improvement of over 50%.
7

---

1LaMA-3,1-8B Trivia QA
Mistral
LL.MA-3,1-88
Mistral
SQUAD
Trivia QA
SQUAD
Trivia QA
SQUAD
TTivia QA
SQUAD
First
0.849
0.774
0.844
0.799
Original
0.854
0.783
0.813
Before Last
0.878
0.778
0.849
0.807
P
0.856
0.762
0.884
0.825
Last 0.890
0.804
0.873
0.825
P
0.871
0.787
0.897
0.831
Ours 0.897
0.826
0.903
0.837
P uacarniasy
0.897
0.826
0.903
0.837
Table Ablation study results the ATS module (left) and uncertainty-enablld repressettttttttoo module (right).
5.2 Cross-dataset Generalisation Ability
The ability to generalise across datasets is essential to facilitating real-world applications of LLMs in diverse domains. We conduct experiments on the aforementioned four datasets to assess whether the proposed HaMI can effectively generalise among various datasets, with three strong competing methods in the line of internal-repressnnaaiioo analysis as baselines. For each dataset, we report the average AUROC scores of detectors trained on one of the other three datasets. The results are illustrated in Figure 3a. It is clear that our method HaMI consistently achieves the best generali- sation performance across all four datasets, outperforming the best competing method SAPLMA by 7% 9%. Compared to the within-dataset performance in Table 1, the maximum performance decline in HaMI no more than 4.5%, observed on the BioASQ dataset, which is significantly lower than that of the competing methods. We also compare the generalisation capability of HaMI with waining-free baselines. The results for HaMI trained on one dataset and evaluated on oth- separately are presented in Table 2, where the results for the training-free baselines are the best AUROC of training-fre free methods prrooodss-(rrrree Perplexity, SE, MARS, MARS-SE, CED, LLM- Check-h, and L.LM-Check-a- each dataset. Notably, across all settings, HaMI can consistently outperform
baselines unseen target datasets, regard- less of which dataset it is trained on. The observation is similar to the one with the results aggregated over multiple detectors. These findings affirm the effectiveness of HaMI as reliable hallucination detector in unseen datasets.
Trivia QA
SQUAD
NQ
BloASQ
Trained on Trivia QA Trained SQuAD Trained on NQ
0.802
0,818
0.796
0.794
0.870
0.804
0.791
Trained on BloASQ Training-Fre Free Baselines
0.853
0.790
0,781
0.828
0.787
0.777
0.757
Table 2: Cross-dataset generalization capability compared with training- taninng-rreeete baselines.
5.3 Ablation Study
Analysis of MIL-based Adaptive Token Selection. Unlike existing methods that predetermined critical tokens for capturing truthfulness information, HaMI proposes the MIL-based adaptive token selection (ATS) module. Table 3 Left compares the perfor- mance of our ATS module with commonly used meth- ods, including the First generated token, the Last gen- erated token, and the Before Last generated token. The results show that our ATS method outperforms the alternatives across both Trivia QA and SQuAD datasets on both LLMs, yielding an average improve- ment of 6% over the First token, 4% over the Before Last token and 2% over the Last token.
Poothen
Palday's abild
Loving bnd lcaae.
1.00
Intaretioa so the mallent concizent in ateay covectoy cpproxteately
hegasies bag
l1ich Dunicvs Sensiiee betier bnove Cartos Ni
Dechel.
de ln' colone wil taltop- hwlaccattinn sove
Figure 4: Adaptive token selection results showing tokens with the top-2 highest hallucination scores.
Furthermore, our ATS module exhibits substantially better robustness to the length of generations than the competing methods. In particular, the average number of generated tokens for the Trivia QA dataset with the LLaMA model is 10. which is smaller than the other three cases (Trivia QA with Mistral model - 13.5. SQuAD with LLaMA model 17.5, SQuAD with Mistral model 15). Despite this variation, the ATS module performs robustly on all four cases and even performs more effectively as the length increases. In contrast, the performance of the predefined token location methods is unstable and tends to degrade as the length of the generation grows. Moreover, the Last token achieves better performance across all scenarios, and the First token is just the opposite. It

---

Trivia, SMM-SUAD- NNQ -BioASQ
Original Feature
Enhanced Feature
0.9
0.9
AUROC
0.8
AUROO
0.8
0.7
0.7
0.6
0.6
8
14
20
0.00
0.25 0.50 0.75
1.00
11.25 1.50 1.75 2.00
Layer
2
(a)
(b)
Figure 5: (a) Performance of HaMI w.t.. internal representations extracted from different layers. (b) The impact of augmentation strength À in Eq. 8 on HaMI. All results are based on LLaMA-3.1-8B
suggests that the last token can capture full semantics under specific length conditions, while the first token may miss content in the subsequent predictions.
To further support the validity of the token selection process, conducted additional study, a manual evaluation using 100 randomly sampled examples from the Trivia QA dataset. In this analysis, manually annotated hallucinated tokens and measured the token detection recall rate of our method throughout training. Specifically, HaMI can achieve a recall rate at 0.84 as training progresses. This suggests that our approach enables the model identify salient tokens even without token-level supervision. Figure 4 presents an illustrative example of the scoring results over tokens in positive bag and negative bag, where we can observe that there is significant distinguishability between the positive and negative tokens since the maximum scores of the instances in the positive bag are substantially greater than those in the negative bag. Tokens denoted by blue characters have scores that are very close to the largest ones and we find that these tokens can be adaptively identified by our method as concentrated answers in comparison noted that while selected tokens are associated with the exact answer, they may appear at any location in their vicinity, which can be captured by our smoothness loss.
ground-truth answers. Additionally, is
Analysis of Different Predictive Uncertainty-enabler Features. Here systematically examine three uncertainty measuring methods presented in Section 4.2 for the internal representation enhance- ment as detailed in Eq. 5, 6 and 7 respectively. We use simple combination strategy defined in Eq. 8. In the right panel of Table 3, we can observe that both Ps unceatainty and Pe uncertainty contribute to improved detection capabilities over the baseline (i.e., the original feature representations) while the token-level uncertainty Pincertainty yields limited effectiivnesss Specifically, the enhancements attributed to semantic consistency Pé uunertainty are the most significant, exhibiting improvements up to 6.7%. These observations suggest that the effectiveness of uncertainty -based enhancement is tied to the inherent ability of the uncertainty metric to distinguish hallucinations from correct responses. Notably, although the improvements observed with ps performance of SotA multi-sampling approaches (such SE in Table 1), without incurring the costs associated with multiple generations and external LLM employments. This highlights the potential of HaMI for deployments in various practical environments that involve external tools or not.
uncertainty are less pronounced, it surpasses the
Performance of HaMI w.r.t. Representations from Different LLM Layers. We evaluate detec- tion performance using representations extracted from various layers LLM with LLaMA-3.1- across all benchmarks. We apply both original internal representations and semantic-equivalencee enhanced representations for investigation. As depicted by the dashed line in Figure 5a, the AUROC values for original representations exhibit a clear increase, peaking between layers 12 and 16, before declining relatively stable level. This observation suggests that the truthfulness content evolves across the initial to middle layers. The performance of the uncertainty enntancet-eehaneed representations (presented in the solid line) maintains a relatively consistent trend, with the highest AUROC scores concentrated in the middle layers as well. The comparison of these trends indicates that incorporating predictive uncertainty enhances the distinctiveness of the representations, particularly in the earlier layers where less semantic information is typically available.

---

Performance of HaMI w.r.t. Various Model Scales. We investigate the sensitivity of HaMI on the model scale by varying the number of layers and hidden dimensions in the two-layer MLP network. As illustrated in Figure 3b, HaMI exhibits low sensitivity to both hyperparameters across all four benchmarks. Specifically, for two-layer MI.P (employed in HaMI and most baselines), the performance remains steady as the size of the hidden dimension changes from 64 to 512. Similarly, increasing the number of layers from I to yields only marginal changes in AUROC, suggesting that shallow architecture is sufficient for effective hallucination detection. These findings highlight the robustness of our method w.E.I. variations in network architecture.
Performance of HaMI w.r.t. Uncertainty Augmentation Strength (A). Figure 5b presents results based on various choice of À used in Eq. 8. It is clear that the model's performance is steadily improved as x changed from 0.0 towards 1.0. Beyond this point, performance becomes stable.
6 Conclusion
In this paper, we introduce the very first approach that supports joint token selection and hallucina- tion detection, HaMI, enabling adaptive identification of the most hallucinated tokens for learning an optimal hallucination detector. This helps largely enhance the robustness of the detection on generation responses of varying lengths and hallucinated entities. Specifically, HaMI incorporates straightforward yet effective MIL formulation to automatically highlight salient tokens that are optimal for training the subsequent hallucination detector. Additionally, we also explore integrating uncertainty metrics into the original representations to enrich them with more information about truthfulness. Extensive empirical results demonstrate that HaMI substantially outperforms existing SotA methods across diverse popular datasets and LLMs. Our ablation studies offer important insights into how different designs in HaMI help improve the robustness of the detection. While our experiments primarily focus on the QA domain, the principles underlying our method are task-free, suggesting potential applicability to a broad spectrum of other tasks.
Acknowledgments
This research is supported by A*STAR under its MTC YIRG Grant (M24N8c0103), the Ministry of Education, Singapore under its Tier-1 Academic Research Fund Kong Chian Fellowship.
and the Lee
References
[1] Achiam, Adler, S. Agarwal, L.. Ahmad, Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint
wwXiv:2303.08774 2023. 3, 6, 21,22 22
[2] V. Adlakha, P. BehnamGhader, X. H. Lu, N. Meade, and S. Reddy. Evaluating correct- ness and faithfulness of instruction-followii MMKi:2337..18777 2023.
models for question answering. arXiv preprint
[3] L. Aichberger, K. Schweighofer, and S. Hochreiter. Rethinking uncertainty estimation in natural language generation. arXiv preprint arXiv:2412.15176 2024. 3
[4] G. Alain. Understanding intermediate layers using linear classifier probes. arXiv preprint 161401661.001644 2016.
[5] A. Azaria and T. Mitchell. The internal state of LLM knows when it's lying. In H. Bouamor,
J. Pino, and K. Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967-976, Singapore, Dec. 2023. Association for Computational Linguistics. 2, 6,
[6] Y. F. Bakman, D. N. Yaldiz, B. Buyukates, C. Tao, D. Dimitriadis, and S. Avestimehr. MARS: Meaning-aware response scoring for uncertainty estimation in generative LLMs. In L.-W. Ku, A. Martins, and V. Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pages 7752-7767, Bangkok, Thailand, Aug. 2024. Association for Computational Linguistics. 1, 3, 6,7
[7] F. Barbero, A. Arroyo, X. Gu, C. Perivolaropoulos, M. Bronstein, R. Pascanu, et al. Why do Imms attend to the first token? arXiv preprint 2rX1v:2504.02732, 2025. 2
10

---

[8] R. Bayat, A. Kaliiiiia M. Pezeshki, S. Chandar, and P. Vincent. Steering large language model activations sparse spaces. arXiv preprint arXiv:2503.001 000000000007..... 2025. 3
[9] C. Burns, H. Ye, D. Klein, and J. Steinhardt. Discovering latent knowledge in language models without supervision. arXiv preprint (tGi:222/2.03822) 2022. 1,3, 6,7
[10] M.-A. Carbonneau, V. Cheplygina, E. Granger, and G. Gagnon. Multiple instance learning: A survey of problem characteristics and applications. Puttern Recognition, 77:329-353, 2018.
[1]] C. Chen, K. Liu, Z. Chen, Y. Gu, Wu, M. Tao, Z. Fu, and Ye. Inside: Lims' internal states retain the power of hallucination detection. arXiv preprint arXiv:2402.0 033744 2024. 3
[12] Y.S. Chuang. Y. Xie, H. Luo, Y. Kim, J. Glass, and P. He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03383 2023. 1,4
[13] Dhuliawala, M. Komeili, J. Xu, R. Raileamu, X. Li, A. Celikyilmaz, and Weston, Chain-of- verification reduces hallucination in large language models. arXiv preprint arXiv:2309.11495, 2023.
[14] X. Du, C. Xiao, and Li. Haloscope: Harnessing unlabeled LLM generations for hallucination detection. In The Thirty-eighth Annual Conference on Neural Information Processing Systems,
2024. 2, 3, 6,
[15] J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkhura, and K. Xu. Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models. In L.-W. Ku, A. Martins, and V. Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5050-5063, Bangkok, Thailand, Aug. 2024. Association for Computational Linguistics. 1,3
[16] A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al. The llama herd of models. arXiv preprint drXiv:2407.2/788 2024.
[17] S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017) 630, 2024. 1, 1, 5, 6, 7, 21
[18] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):1-55, 2025. 1
[19] E. Hubinger, C. Denison, J. Mu, M. Lambert, M. Tong, M. MacDiarmid, T. Lanham, D. M. Ziegler, T. Maxwell, N. Cheng, et al. Sleeper agents: Training deceptive Ilms that persist through safety training. arXiv preprint urXiv:2401.05566 2024. 3
[20] M. llse, J. Tomezak, and M. Welling. Attention-based deep multiple instance learning. In International conference on machine learning, pages 2127-2136. PMLR, 2018. 22
[21] Z. Ji, D. Chen, E. Ishii, S. Cahyawijaya, Y. Bang, B. Wilie, and P. Fung. LLM internal states reveal hallucination risk faced with a query. In Y. Belinkov, N. Kim, Jumelet, H. Mohebbi. A. Mueller, and H. Chen, editors, Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 88-104, Miami, Florida, US, Nov. 2024. Association for Computational Linguistics. 2
[22] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38 2023. 1,2
[23] A. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. Chaplot, D. de las Casas, F. Bressand,
G. Lengyel, G. Lample, L. Saulnier, et al. Mistral 7b (2023). arXiv preprint arXiv:2310.06825.
2023. 6
[24] M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint /00i::/77..03000 2017.
[25] S. Kadavath, T. Conerly, A. Askell, Henighan, D. Drain, E. Perez, N. Schiefer, Z. Hatfield- Dodds, N. DasSarma, F.. Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint urXiv:2207.05222 2022. 1, 3, 6, 7
[26] Kossen, J. Han, M. Razzak, I.. Schut, Malik, and Gal. Semantic entropy probes: Robust and cheap hallucination detection in 1lms. arXiv preprint 14X.::2406.1522221 2024. 2,3
11

---

[27] A. Krithara, A. Nentidis, K. Bougiatiotis, and G. Paliouras. Bioasq-qa: A manually curated corpus for biomedical question answering. Scientific Data, 10(1):170, 2023.
[28] T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein,
I. Polosukhin, J. Devlin, K. Lee, et al. Natural questions: benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453-466, 2019.
[29] H. Lee, K.-H. Park, H. Byun, J. Yeom, J. Kim, G.-M. Park, and K. Song. Ced: Comparing embedding differences for detecting out-of-distributtin and hallucinated text. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14866-14882 2024. 6, 7
[30] K. Li, O. Patel, F. Viégas, H. Pfister, and M. Wattenberg, Inference-time intervention: Eliciting truthful answers from a language model, Advances in Neural Information Processing Systems, 36, 2024. 2,3
[31] S. Lin, J. Hilton, and O. Evans. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205. 14334, 2022. 3
[32] Q. Liu, X. Chen, Y. Ding. S. Xu, S. Wu, and L. Wang. Attention-uuide self-reflection for zero-shot hallucination detection in large language models. arXiv preprint arXiv:2501.09997 2025. 2
[33] S. Longpre, K. Perisetla, A. Chen, N. Ramesh, C. DuBois, and S. Singh. Entity-based knowledge conflicts in question answering. arXiv preprint arXiv:2109.05052. 2021. 1
[34] Manakul, A. Liusie, and M. Gales. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint 0rXiv:2303.08880 2023. 1, 3
[35] S. Marks and M. Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets, 2024. 3
[36] N. Mündler, J. He, S. Jenko, and M. Vechev. Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation. arXiv preprint arXiv:2305. /58502 2023. 3
[37] H. Orgad, M. Toker, Z. Gekhman, R. Reichart, I. Szpektor, H. Kotek, and Y. Belinkov. Llms know more than they show: On the intrinsic representation of 1lm hallucinations, 2024. 3
[38] L. Pacchiardi, A. J. Chan, S. Mindermann, I. Moscovitz, A. Y. Pan. Y. Gal, O. Evans, and
J. M. Brauner. How to catch an Al liar: Lie detection in black-box LLMs by asking unrelated questions. In The Twelfth International Conference on Learning Representations, 2024. 1,3
[39] A. Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint arXiv:1912.01703. 2019. 21
[40] E. Quevedo, J. Y. Salazar, R. Koerner, P. Rivas, and T. Cerny. Detecting hallucinations in large language model generation: A token probability approach. In World Congress in Computer Science Computer Engineering & Applied Computing, pages 154-173. Springer, 2024. 3
[41] P. Rajpurkar. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint urXiv:1006.05255 2016. 6
[42] Ren, Luo, Y. Zhao, K. Krishna, M. Saleh, B. Lakshminarayanan, and P. J. Liu. Out-of- distribution detection and selective generation for conditional language models. In The Eleventh International Conference on Learning Representations, 2023. 3, 6,
[43] T. Sellam, D. Das, and A. P. Parikh. Blcurt: Learning robust metrics for text generation. arXiv preprint 200:22004 2020. 6, 22
[44] B. Snyder, M. Moisescu, and M. B. Zafar. On early detection of hallucinations in factual question answering. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages
2024. 3
[45] G. Sriramanan, S. Bharti, V. Sadasivan, S. Saha, P. Kattakinda, and S. Feizi. Lim-check: Inves- tigating detection of hallucinations in large language models. Advances in Neural Information Processing Systems, 37:34188-344 34216, 2024. 6.7
[46] A. Talman, H. Celikkanat, S. Virpioja, M. Heinonen, and J. Tiedemann. Uncertainty-aware natural language inference with stochastic weight averaging. In T. Alumãe and M. Fishel, editors, Proceedings of the 24th Nordic Conference on Computational Linguistics (No), pages 358-365, Tórshavn, Faroe Islands, May 2023. University of Tartu Library. 3
12

---

[47] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, Bhargava, S. Bhosale, al. Llama 2: Open foundation and fine-tuned chat models. arXiv
preprini arXiv:2307.09288, 2023. 6
[48] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowiez, J. Davison, S. Shleifer, von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, '. L.. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush. Transformers: State-of-the-art natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online, Oct. 2020. Association for Computational Linguistics. 6, 21
[49] M. Yuksekgonul, V. Chandrasekaran, E. Jones, S. Gunasekar, R. Naik, H. Palangi, E. Kamar, and B. Nushi. Attention satisfies: A constraint-satiifaatiion lens on factual errors of language models. arXiv preprint arXiv:2309.1500 2023. 3
[50] Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, et al. Siren's song in the ai ocean: a survey on hallucination in large language models, arXiv preprint
arXiv:2309.012199 2023. 2
[51] K. Zhou, D. Jurafsky, and T. Hashimoto. Navigating the grey area: How expressions of uncertainty and overconfidence affect language models, arXiv preprint arX1v:2302.13449, 2023.
3
[52] N. Zucchet, J. Bornschein, S. Chan, A. Lampinen, R. Pascanu, and S. De. How do language models learn facts? dynamics, curricula and hallucinations. arXiv preprint arXiv:2503.21677 2025.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction accurately reflect our contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Appendix B.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We have fully disclosed the information needed to reproduce the main experi- mental results of our paper.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: We use public datasets for experiments and all references are provided. For the code, provide in the supplemental materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https: ///:://nii ht////ipsss cc/
publ c gui des CodeSulom ss onPol cyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (ht tnttpp: / /nips. cc/
wnips.cc/pbbic... httpppublllcggg/ele /wcc/uuisssss Codee Submi ss ssionnoooo oonoooolliio oolPolisee olicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify all the experimental setting details. Please refer to Section 5 and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Error bars are not reported because it would be too computationally expensive for some baselines. The average standard deviation of AUROC results of HaMI is 0.018.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
16

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide information on the computer resources in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: http::::///eeurre http:p/....cc.lllccccccccoooaaaa
Answer: [Yes]
Justification: Yes, our research with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification:
The broader impacts of our paper are discussed Section and Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Yes, all used assets respect the license and terms of use.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papers' raw thcode com/ dat tes has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Our paper describe the usage of LLMs. Details are in Section 5 and Appendix.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https: : / /t///nerrppppp// wttcc///ooooo cocf/ofeeeeeese 2025/ LLM) for what should or should not be described.
20

---

A Implementation Details
A.1 Setup
For the main results, our hallucination detector fo(-) utilíses two-layer MLP with hidden dimension of 256. The first linear layer is followed by BatchNorm and ReLU activations and the second linear layer is accompanied by sigmoid output. As shown in Figure 3b in paper, method relatively insensitive to variations in the number of layers and the hidden dimension size. At the training stage, representations extracted from single layer and the layer index determined by the results of the validation set.
Adam optimiser. For the input representations of our detector, utilise
We implement our method using PyTorch 2.6.0 [39] and transformers 4.51.3 [48] and conduct all experiments on NVIDIA 100 GPUs. Based on experimental records, the estimated total training and inference time for each dataset under the main experimental settings is as follows: 0.40 GPU-hours for LLaMA-3.1 0.45 GPU-hours for with 8-bit quantisation. is important to note that these results are based on the assumption that the semantic uncertainty score iincertiitt is readily available. Otherwise, computing this score involves generating multiple responses and evaluating entailment across them, which introduces nontrivial latency, approximately 7.6 seconds per question for L.aMA-3.1-88 1-BB 8.6 seconds for Meatral-Nemo-Insttie (I2B), and 46.1 seconds for 1..MA-3.3-Instruue work, we also propose lightweight variant, HaMI*, using only single generation, to climinate the influence computational efficiency while maintaining superior performance.
(12B) and 0.65 GPU-hours for
In this
A.2 Prompts for Generation and Evaluation
All prompts utilised in our experiments refer to those used by Farquhar el al. [17]. For the QA tasks, we prompt selected LLaMA and Mistral models to generate answers without context in a zero-shot manner. Figure 6 presents the used input prompt. To obtain response labels, we prompt GPT-4.1 [1] with OpenAl API to evaluate the quality the response. The prompts are illustrated in Figure 7. It is observed that GPT-4.1 I can make mistakes for the positive samples as checked with the gold answers. Therefore, we ask GPT-4.1 rejudge samples labelled as positive and discard samples if the result inconsistent with the first result.
Prompts for QA Tasks
Answer the following question in a single but complete sentence. An
Question: Which politician won the Nobel Peace Prize in 2009?\n
Answer:
Figure 6: Prompts for QA tasks without context.
B Study on the Effectiveness of the Smoothness Loss
The smoothness loss is designed under the assumption of continuity in hallucination scores between adjacent tokens. We investigate the effectiveness of smoothness loss four benchmarks with LLaMA- 3. 3-B model. Main results on each individual dataset and cross-datased results are respectively presented in Table 4 and Table 5 below. The results are averaged over three independent runs. As evidenced in the tables, incorporating the smoothness loss consistently yields improved performance across all the cases. On average, the smoothness loss consistently enhances HaMI, increasing the performance by 0.7% and 1.1% on the within- dataset and cross-dataset scenarios, respectively. This demonstrates that the contribution of the smoothness loss is more pronounced in the more challenging cross-dataset setting. These consistent improvements highlight that the smoothness loss plays rather positive role in helping achieve more robust and generalizable HaMI, which particularly valuable in doos-domain scenarios.
21

---

Prompts for GPT-4.1 Labelling
We assessing the quality of answers to the following question: (Qusetion) \n
The expected answer is: (Expected Answer).
or: The following are expected answers to this question: (Expected Answers). An
The proposed answer is: (predicted_ answer) \n
Within the context of the questicn and your own knowledge, is the proposed answer correct or the same as the expected answer?
Respond only with yes or no. \nkesponse:
Figure 7: GPT-411 evaluation prompts
CMIL
Cemooch
Trivia
SQuAD
NQ
BioASQ
0.802.1..005 0.898
0.825:10.002 0.835110.006
0.812 0.815
0.84010.0015 0.845
Table 4: Ablation study the MIL and smoothness losses.
CMIL
Demuothh
Trivia QA
SQuAD
NQ
BioASQ
0.852:10.012 0.863::00000
0.812:10.005 0.820:000002
0.000
0.790:10.007 0.795=00.002
0 0.80
Table 5: Ablation study on the MII. and smooth losses under the cross-dataset setting.
C
Study on Top-k Token Selection
HaMI employs a hard top-k selection strategy in its MIL formulation that functions similarly to max pooling during training. This design aligns with the core assumption in multi-instance learning (MIL) that if there are any positive instances, the bag should be labelled as positive. This principle is particularly relevant our detection task, where the salient hallucinated tokens appear sparsely within generated sequences. While there exist more sophisticated and learnable aggregation techniques, previous research has indicated that, given a sufficient number of training bags, different aggregation methods for bag tend to converge to similar performance [20]. Based on these observations and the remarkable performance of our method in our experimental results, despite being simple, the hard top-k strategy offers direct, yet effective solution for the detection task. As presented in Section 4.1, is dynamically determined by the generated token length as follows:
= Li x +
where is set to 0.1 by default. In this section, we investigate the sensitivity of our method to the choice of rı in the adaptive token selection process. shown in Figure HaMI exhibits stable performance across set of ranging from zero to 0.20, on all four datasets with ALaMA-3.1-88 and Mistral-Nemo--nstruue (12B). HaMI can perform stably across different choices of rx. Notably, smaller тı values already yield strong results, which implies that only few high-salience tokens sufficient for effective hallucination discrimination.
D Evaluation results with BLEURT score
In addition to labels identified by GPT-4.1 [1], we also use the BLEURT-threshold [43] method to generate labels and comparing HaMI with five representative baselines. Results are reported in Table 6, which shows that our methods HaMI and HaMI" can also consistently outperform the
22

---

0.00
0.05
0.10
0.15
0.20
0.9
AUROO
0.6
Trivia_QA
SQUAD
NQ
BioASQ
Trivia_QA
SQuAD
NQ
BioASQ
tettt--Nemo- Figure 8: Study different choices of four datasets with two LLMs.
LLaMA-3.1-8B
Method
Trivin QA
SQuAD
BloASQ
Perplexity
0.662
SE
0.709
0.813
0.704
0.714
CCS
0.682
0.657
0.738
0.683
SAPLMA
0.897
0.898
0.867
0.873
HaloScope
0.792
0.845
0,852
0,830
HaMP"
0.900
0.928
0.909
0.918
HaMI
0.909
0.937
0.920
0.922
Table 6: AUROC results with labels generated by the BLEURT method. The results are based on -LaMA-3.1-8B.
baselines using the BLEURT-basee labelled datasets. These results provide empirical evidence for the robustness of HaMI from another perspective (i.e., data labelling perspective).
E Limitations and Broader Impacts
Broader Impacts. With the growing deployment LLMs, ensuring the truthfulness of their outputs has become critical challenge, especially in high-stakes domains such as finance and healtheare. To enable effective hallucination detection, in this work, we propose a practical and robust approach that leverages internal representations to identify tokens with high likelihood of being hallucinated and enables the following detection. We hope that our proposed method, HaMI, can contribute to safer and more reliable real-world deployment of LLMs. Although HaMI can be extended to other tasks as well. For example, in the summarisation task, truthfulness can be assessed sentence by sentence. Verifying the correctness of each generated sentence can be reformulated as an entailment query against the source context, allowing hallucination detection to be performed in QA-like manner. Furthermore, the similarity between generated sentence and its corresponding context could serve as an uncertainty metric for internal representation enhancement.
research focuses on QA tasks,
Limitations. Apart from the aforementioned capabilities of HaMI, there remain some concerns. Unlike some black-box uncertainty-basee approaches, our method requires access to internal repre- sentations, limiting its deployments open-source LLMs. Moreover, we also explore integrating uncertainty metrics into the original representations to enhance their discriminative capability on correct and incorrect generations. While effective, the integration strategy is relatively simple and we acknowledge that more sophisticated fusion methods could be explored for better detection performance in future research.
23


Reviews:
[{"review_id": "jDlvertDIp", "reviewer": "Reviewer_NwzN", "summary": "The paper introduces Hallucination detection as Multiple Instance learning, short HaMI, which trains a hallucination detector that distinguishes between generations with hallucinations (positive bags of tokens) and generations without hallucinations (negative bags of tokens) by maximizing the distance between them in token-level representation space.", "questions": "- The authors \"choose the token instances with the top k highest hallucination scores as the salient tokens in each sequence\" (line 156). These hallucination scores are obtained from the hallucination detector. However, at the beginning of training, these are random scores, so maximizing the discriminative margin between these randomly selected tokens might not be optimal. How does this influence training?\n- How does the sequential smoothness loss impact the training and performance of the method? (Re. Weakness 1)\n- Would HaMI still achieve the best performance in Table 1 if a single hallucination detector were trained per model, rather than per dataset? (Re. Weakness 2)\n- How does the method compare to single-sequence uncertainty method both in performance and computational effectiveness? (Re. Weakness 3)", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. The paper proposes a an interesting approach to hallucination detection by jointly optimizing token selection and classification via a Multiple Instance Learning (MIL) formulation.\n2. HaMI demonstrates strong and consistent empirical performance, significantly outperforming state-of-the-art baselines across both uncertainty-based and internal-representation-based detection methods.\n3. The evaluation is thorough and well-structured, spanning three language models and four QA datasets.\n4. The ablation studies are well-designed and clearly show the contribution of each component to the overall performance.\n\n**Weaknesses:**\n1. The authors introduce a sequential smoothness loss based on the assumption that hallucination scores for neighboring tokens should be similar. However, this assumption has not been theoretically motivated or empirically validated. It would be helpful to further investigate this assumption.\n2. For the main results, the hallucination detector is both trained and evaluated on the same dataset, whereas the uncertainty estimation baselines are dataset-agnostic. A fairer comparison would be to train a single detector per model and evaluate it across multiple datasets. While Section 5.2 provides some insight into HaMI’s generalization ability, it remains unclear how its performance compares to that of the uncertainty estimation baselines. Additionally, it is not clear whether the method generalizes beyond the question answering domain.\n3. The baselines do not include sentence-level uncertainty measures that require generating only a single output sequence, such as G-NLL, which is computed using the most-likely output sequence only [1]. These methods are lightweight and widely applicable, thus should be considered standard in an evaluation.\n4. The authors claim that their method can identify the most hallucinated tokens within the positive bag. However, the ablation in Section 5.3 shows that semantically irrelevant words like “is” and “and” receive the highest hallucination scores..\n\n---\n[1] Lukas Aichberger, Kajetan Schweighofer, and Sepp Hochreiter. Rethinking uncertainty estimation in natural language generation. *arXiv preprint arXiv:2412.15176*, 2024.", "was_revised": true, "final_justification": "My questions and concerns have largely been resolved in the authors' rebuttal.\n\nThe authors confirmed that:\n- HaMI's sequential smoothness loss improves performance, even if only marginally.\n- HaMI remains effective when the detector is trained on a single dataset and evaluated on the other three.\n- HaMI improves upon single-sequence uncertainty methods.\n\nOverall, the rebuttal strengthened and clarified the contributions. I have therefore increased my score.", "rebuttal": "Thank you very much for your valuable and detailed feedback. We are grateful for your positive assessment of our technical contributions and empirical results. Please see our response to your comments one by one below.\n\n> **Question #1** How does randomly choosing tokens early in training affect the process?\n\nThank you very much for the question. The optimization in HaMI is analogous to an alternating-optimization approach, in which one of the two sub-problems, token selection and hallucination detection, is alternatively solved, with the parameters in the other one fixed. Please note that, to simplify the optimization, there are no learnable parameters in token selection; we select the tokens with the largest hallucination scores.\n\nIt is true that at the beginning of training, the token hallucination scores predicted by the detector are not properly optimized, and therefore, the selected tokens based on these scores may be not that relevant to hallucination detection. \nHowever, as more optimization epochs are done, the detector becomes more and more optimal, resulting in better token-level hallucination scores and thus better token selection, and vice versa.\nSpecifically, as training progresses on the datasets used in our experiments, the hallucination scores for tokens in positive bags increase gradually from below 0.5 (observed from the first 10 training steps) to values approaching 1.0, indicating stronger confidence in detecting hallucinated tokens. This phenomenon is observed on all our datasets and is consistently stable with different random parameter initialization seeds.\n\nTo further examine the effectiveness of token selection, we randomly selected 100 samples from the Trivia QA dataset and manually annotated the hallucinated tokens. We then tracked the recall rate in the token selection module at different stages of training. We observed that the recall rate of detecting the hallucinated tokens initially was relatively quite high (0.72 after just two training steps), then dropped to around 0.66 for several steps, and steadily improved and became stable thereafter, flattening at a recall rate of about 0.84. This recall rate is remarkable since the full training process is done without using token-level annotations.\n\nThese observations suggest that although the top-k token selection strategy may be random in the beginning, the MIL-based joint optimization framework enables HaMI to progressively refine its token selection and effectively learn to identify accurate hallucinated tokens. We will add these results and clarification to the final version to address this concern.\n\n> **Question #2 \\& Weekness #1**: How does the sequential smoothness loss impact the training and performance of the method? \n\nThank you very much for raising this question. We added ablation study on four benchmarks with LLaMA-3.1-8B model for the proposed loss functions, including both MIL loss and smoothness loss. Main results on each individual dataset and cross-dataset results are respectively presented in **Tables A1** and **A2** below. The results are averaged over three independent runs. As evidenced in the tables, incorporating the smoothness loss consistently yields improved performance across all the cases. On average, the smoothness loss consistently enhances HaMI, increasing the performance by 0.7\\% and 1.1\\% on the individual-dataset and cross-dataset scenarios, respectively. \nThis demonstrates that the contribution of the smoothness loss is more pronounced in the more challenging cross-dataset setting. These consistent improvements highlight that the smoothness loss plays a rather positive role in helping achieve a more robust and generalizable HaMI, which is particularly valuable in the cross-domain scenarios. We will add these results and this discussion in the final version to clarify the role of this loss.\n\n\n| $\\mathcal{L}_{MIL}$ | $\\mathcal{L}_{Smooth}$ | Trivia QA | SQuAD | NQ    | BioASQ |\n|-----------|------|-----------|-------|-------|--------|\n| ✓         |      | 0.892±0.005 |0.825±0.002 | 0.812±0.003 | 0.840±0.015|\n| ✓         | ✓    | 0.898±0.007 |0.835±0.006 | 0.815±0.004 | 0.845±0.012 |\n```\nTable A1. Ablation study on the two loss functions (AUROC).\n```\n\n| $\\mathcal{L}_{MIL}$ | $\\mathcal{L}_{Smooth}$ | Trivia QA | SQuAD | NQ    | BioASQ |\n|-----------|----|-----------|-------|-------|--------|\n| ✓         |    | 0.852±0.012  | 0.812±0.005 | 0.797±0.020 | 0.790±0.007  |\n| ✓         | ✓  | 0.865±0.008  | 0.820±0.002 | 0.808±0.012 | 0.795±0.002  |\n\n```\nTable A2. Ablation study on the two loss functions (AUROC) under the cross-dataset setting.\n```\n\n\n\n> **Question #3 \\& Weakness #2**: Would HaMI still achieve the best performance in Table 1 if a single hallucination detector were trained per model, rather than per dataset?... it is not clear whether the method generalizes beyond the question answering domain.\n\nThank you very much for the question. And yes, as shown in Figure 3(a) in our paper,  HaMI maintains superior performance when it is trained per model rather than per dataset, i.e., the detector is trained once on one single dataset and then evaluated on all the other three datasets. This setting is referred to as cross-dataset generalization in the paper. Here we present the experimental results with different settings on four datasets based with the LLaMA-3.1-8B model in **Table A3**. \nFor comparison, we also present the best results on each dataset from all non-learnable/training-free baselines: p(True), Perplexity, SE, MARS, MARS-SE (extracted from Table 1 of our paper), all of which consistently underperform HaMI across the datasets. \n\n\n|                                | Trivia QA     | SQuAD        | NQ           | BioASQ       |\n|--------------------------------|---------------|--------------|--------------|--------------|\n| HaMI - cross-dataset-performance | 0.865±0.008  | 0.820±0.002 | 0.808±0.012 | 0.795±0.002  |\n| Best AUROC for non-learnable methods | 0.828   | 0.787        | 0.777        | 0.757        |\n\n```\nTable A3. Cross-dataset generalization capability compared with non-learnable/training-free baselines.\n```\n\n\nFor your concern on HaMI's capability beyond QA domain in **Weakness #2**, we would like to clarify that HaMI can be directly employed in summarization tasks, where the target text and task instruction serve as the context prompt and question, respectively, and the summary is treated as the answer. For other tasks, such as consistency checking in storytelling, we can also reformulate them into a QA format for evaluation.\n\n\n> **Question #4 \\& Weakness #3**: How does the method compare to single-sequence uncertainty method both in performance and computational effectiveness? \n\nFollowing your suggestion, we include the single-sequence uncertainty method G-NLL [Ref1] as a baseline method. Since official implementation code is not available, we follow the methodology described in the original paper to reproduce their method. All experiments were conducted using the LLaMA-3.1-8B model across four benchmark datasets.\n\nAs shown in **Table A4** below, HaMI significantly outperforms G-NLL across all four datasets, with the improvement ranging from 20\\% to 30\\%. Notably, even HaMI*, a variant of HaMI that uses only a single generation, can substantially outperform G-NLL, further validating the superior performance of our formulation over G-NLL.\nThe other baseline *Perplexity*, with its performance reported in Table 1 in our paper, also relies on sentence-level aggregation of token-level negative log-likelihoods. Its performance shows similar trends as G-NLL.\n\nIn terms of computational efficiency, HaMI requires additional overhead due to the need to process high-dimensional representations and the training of the detector. In contrast, the single-sequence methods like G-NLL only requires predictive logits and does not involve additional training, making it computationally more efficient than HaMI. \nHowever, HaMI remains practical since the inference process operates at the token level without requiring large computational costs. We believe this trade-off between computational cost and substantial accuracy gains justifies the approach.\n\n|           Method | Trivia QA | SQuAD | NQ    | BioASQ |\n|------------------|-----------|--------|-------|--------|\n| G-NLL            | 0.732     | 0.683  | 0.612 | 0.692  |\n| HaMI*            | 0.854     | 0.783  | 0.788 | 0.823  |\n| HaMI             | 0.897     | 0.826  | 0.820 | 0.836  |\n\n```\nTable A4. Results for G-NLL and HaMI\n```\n\n> **Weakness #4**: Concerns regarding the ablation study results on token identification.\n\nThank you very much for the question. As mentioned in our paper, HaMI operates without access to token-level supervision. Consequently, while the model aims to identify tokens associated with hallucinated content, it relies on weak supervision signals derived from sequence/bag-level labels. This inevitably introduces some \"noise\". It means that the highlighted tokens may not always align precisely with the hallucinated words, but may appear before or after them, which can be meaningful for locating where hallucination appears. We have discussed this phenomenon in Section 5.3 (Lines 306-313). Nevertheless, as we discussed in our response to **Question #1**, given that the optimization of HaMI does not involve any token-level label information, its recall rate of detecting the true hallucinated tokens (~0.85) is rather high, offering strong supervisory signals for building an accurate hallucination detector.\n\n**References:**\n- [Ref1] Lukas Aichberger, Kajetan Schweighofer, and Sepp Hochreiter. \"Rethinking uncertainty estimation in natural language generation.\" arXiv preprint arXiv:2412.15176, 2024."}, {"review_id": "C5Wv9Mmhsx", "reviewer": "Reviewer_dBF6", "summary": "This paper proposes HaMI (Hallucination detection as Multiple Instance learning), a novel approach for detecting hallucinations in large language models through adaptive token selection. The key innovation is reformulating hallucination detection as a multiple instance learning problem where each response sequence is treated as a bag of token instances, enabling joint optimization of token selection and hallucination detection. The method adaptively identifies the most indicative tokens for hallucination rather than relying on predetermined token positions (first, last, etc.). Additionally, the authors introduce a representation enhancement module that integrates uncertainty information at multiple levels (token, sentence, and semantic consistency) to improve detection performance. Experimental results on four QA benchmarks (TriviaQA, SQuAD, Natural Questions, BioASQ) with three LLMs (LLaMA-3.1-8B, Mistral-Nemo-12B, LLaMA-3.13-70B) demonstrate significant improvements over state-of-the-art methods.", "questions": "- Can the authors provide more empirical analysis of when the smoothness assumption holds? Are there cases where abrupt changes in hallucination scores between adjacent tokens are actually beneficial (or would not degrade the performance much)?\n- What are the main failure modes of HaMI? When does adaptive token selection fail to identify the most relevant tokens?\n- Do different types of hallucinations (factual errors, nonsensical responses, contradictions) show different patterns in token selection? How does HaMI perform on fine-grained hallucination taxonomies?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Novel Problem Formulation: The MIL reformulation is intuitive and well-motivated. The assumption that only a few tokens contain hallucination information while most tokens in incorrect responses are neutral aligns well with the sparse nature of hallucinations.\n- End-to-End Joint Optimization: Unlike existing methods that separate token selection and detection into two stages, HaMI enables joint optimization, which is a methodological advancement.\n- Comprehensive Experimental Evaluation: The experiments cover multiple datasets, models, and baselines. \n- Strong Empirical Results: HaMI achieves substantial improvements over baselines, with gains of 8.1% to 11.9% AUROC over MARS-SE across three LLMs.\n- Thorough Ablation Studies: The paper provides detailed analysis of different components, uncertainty types, layer selection, and hyperparameter sensitivity.\n\nWeaknesses:\n- The best-performing variant (HaMI with semantic consistency) requires multiple generations and entailment evaluation, which significantly increases computational cost (7.6-46.1 seconds per question as noted in appendix).\n- The sequential smoothness loss (Eq. 3) assumes neighbor tokens should have similar hallucination scores, but this assumption may not always hold in practice (e.g., when hallucinations occur at specific entity mentions).\n- The paper misses discussion of when and why the method fails, which would be valuable for understanding its limitations.", "was_revised": false, "final_justification": "", "rebuttal": "We are grateful for your insightful comments and encouraging evaluation of our work. Below, we provide detailed responses to each of your comments.\n\n> **Question #1 \\& Weakness #2**: Can the authors provide more empirical analysis of when the smoothness assumption holds? Are there cases where abrupt changes in hallucination scores between adjacent tokens are actually beneficial (or would not degrade the performance much)?\n\nThank you very much for the question. The smoothness loss is designed under the assumption of continuity in hallucination scores between adjacent tokens. As shown by the added ablation study results in **Tables A1 and A2** below, this assumption holds well in practice since adding the smoothness loss consistently leads to improved or comparable performance across multiple benchmarks;  notably, the improvement is more pronounced in the cross-dataset setting, indicating its positive role in serving as a regularization term for better generalization across the domains. Please refer to our response to Reviewer **NwzN**'s **Question #2** for more detailed discussion of these ablation study results.\n\n| $\\mathcal{L}_{MIL}$ | $\\mathcal{L}_{Smooth}$ | Trivia QA | SQuAD | NQ    | BioASQ |\n|-----------|------|-----------|-------|-------|--------|\n| ✓         |      | 0.892±0.005 |0.825±0.002 | 0.812±0.003 | 0.840±0.015|\n| ✓         | ✓    | 0.898±0.007 |0.835±0.006 | 0.815±0.004 | 0.845±0.012 |\n```\nTable A1. Ablation study on the two proposed losses (AUROC).\n```\n\n| $\\mathcal{L}_{MIL}$ | $\\mathcal{L}_{Smooth}$ | Trivia QA | SQuAD | NQ    | BioASQ |\n|-----------|----|-----------|-------|-------|--------|\n| ✓         |    | 0.852±0.012  | 0.812±0.005 | 0.797±0.020 | 0.790±0.007  |\n| ✓         | ✓  | 0.865±0.008  | 0.820±0.002 | 0.808±0.012 | 0.795±0.002  |\n```\nTable A2. Ablation study on the two proposed losses under the cross-dataset setting.\n```\n\nThe smooth hallucination scores can enhance the optimization due to the following two main reasons. i) For non-hallucination sequences, all tokens are non-hallucination ones, and enforcing the smoothness over them results in consistently small hallucination scores for them (see Figure 4 for an example); for hallucination sequences, the smoothness loss does not stop the hallucination tokens from gaining large hallucination scores, but it can increase the scores of the neighborhood tokens of the hallucination tokens, i.e., to have a gradual rise and down at around the hallucination tokens while having consistently small scores for the non-hallucination tokens. This is typically observed in the experimental datasets. Additionally, since we pick the top-k tokens based on the hallucination scores of the tokens, the hallucinated tokens would be chosen as long as the margin between the scores of hallucinated and non-hallucinated tokens is sufficiently large; the token selection results would be similar to the cases that we have very large margins under the abrupt changes in hallucination scores. ii) The smoothness loss also effectively regularizes the model to avoid excessive attention to suspicious tokens, helping avoid overfitting the training data, and thereby enabling better generalization.\n\n> **Question #2 \\& Weakness #3**: What are the main failure modes of HaMI? When does adaptive token selection fail to identify the most relevant tokens?\n\nOne primary limitation of HaMI lies in its coarse supervision signals, i.e., the token selection in HaMI relies on sequence-level class labels. Since we do not have fine-grained, token-level hallucination labels, HaMI may sometimes fail to attribute the exact hallucinated tokens with the highest scores. Nonetheless, while the hallucination scores to the true hallucinated tokens may not be the absolute highest value, these tokens often still receive scores close to 1.0, indicating strong relevance. Moreover, although the highest score is not with the exact hallucinated token, it typically appears before or after the exact token. As such, HaMI effectively narrows the region where hallucinated content is likely to occur. This localization ability is valuable in practical settings, as it supports interpretability and downstream correction mechanisms, even in the absence of perfect token-level attribution.\n\nWe also notice that the effectiveness of the token selection module may be limited if the generated sequence is short, such as phrase-level generations. In these instances, the entire sequence is typically related to the exact hallucination, and the pre-defined final token may encapsulate the concentrated semantic better.\n\nThank you for your thoughtful question and we will include the discussion in our paper.\n\n\n> **Question #3**: Do different types of hallucinations (factual errors, nonsensical responses, contradictions) show different patterns in token selection? How does HaMI perform on fine-grained hallucination taxonomies?\n\nThank you very much for the question. In the current work, we did not explicitly evaluate HaMI on fine-grained hallucination taxonomies such as factual errors, nonsensical responses, or contradictions. Our primary evaluation focused on general hallucination detection. However, we conducted an empirical analysis using the TruthfulQA dataset [Ref1], which includes class labels for different hallucination types. Due to time limitations, we did not separate the training set by hallucination type;  HaMI is optimized on the training data with all hallucination types. At inference time, we included all response types and observed that HaMI's performance was relatively consistent across different hallucination categories. Hope this helps address your concern.\n\n> **Weakness #1**: The best-performing variant (HaMI with semantic consistency) significantly increases computational cost.\n\nWe include semantic entropy as uncertainty enhancement feature into HaMI because it provides a strong signal for hallucination detection. However, we agree that integrating the sementic entropy will introduce additional overhead due to the generation of multiple outputs and entailment evaluation.\nTo eliminate the influence on computational efficiency, we also evaluated HaMI*, which relies on only a single generation.\n\nAs shown in Table 1 in our paper, HaMI* achieves consistently superior performance on all four datasets, compared to all other single-generation-based baselines, including strong methods like SAPLMA and MARS. Even when compared with multiple-generation-based approaches such as SE, HaMI* outperforms or matches the best results across datasets, demonstrating that our method is not only effective but also adaptable to resource-constrained settings.\n\n**References:**\n- [Ref1] Lin, Stephanie, Jacob Hilton, and Owain Evans. \"Truthfulqa: Measuring how models mimic human falsehoods.\" 60th Annual Meeting of the Association for Computational Linguistics. 2022."}, {"review_id": "bIvEEpfr7d", "reviewer": "Reviewer_tguY", "summary": "This paper proposes HaMI, a novel hallucination detection method for large language models (LLMs) based on Multiple Instance Learning (MIL). Unlike prior work that relies on preselected tokens for detection (e.g., last token), HaMI introduces an adaptive token selection mechanism that jointly optimizes token scoring and hallucination classification in an end-to-end fashion. Furthermore, the model integrates token-level and sentence-level uncertainty into internal representations to improve discrimination between hallucinated and truthful outputs. The authors demonstrate state-of-the-art performance across four QA benchmarks (TriviaQA, SQuAD, NQ, and BioASQ) and three LLMs (LLaMA and Mistral variants). Empirical results show superior robustness and generalization over prior hallucination detection methods.", "questions": "How accurate are GPT-4.1 labels in practice? Could you report inter-agreement between the two judgment passes? Have you considered incorporating human annotations or knowledge-based QA datasets (e.g., where gold answer is guaranteed factual)?", "limitations": "See the weakness", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nTimely and relevant problem: Hallucination in LLMs is a critical safety issue, and detection methods remain underdeveloped.\n\nConceptual novelty: Reformulating hallucination detection as a MIL problem with adaptive token selection is original and well-motivated.\n\nClarity: The paper is well-written and well-structured, with clear methodological explanations and visualizations.\n\nWeaknesses:\n\n• Limited baseline comparison: Important recent hallucination detection methods such as LLM-Check and CED (Comparing Embedding Differences) are not included. These would provide stronger empirical context and help position the contribution more rigorously.\n\n[1] Sriramanan, Gaurang, et al. \"Llm-check: Investigating detection of hallucinations in large language models.\" Advances in Neural Information Processing Systems 37 (2024): 34188-34216.\n\n[2] Lee, Hakyung, et al. \"CED: Comparing Embedding Differences for Detecting Out-of-Distribution and Hallucinated Text.\" Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.\n\n• Label reliability: The ground truth annotations for hallucination are solely based on GPT-4.1 self-judgment, with only a two-pass validation. While practical, this approach lacks guarantees of correctness. Incorporating datasets with verified human or knowledge-grounded hallucination annotations would strengthen the empirical claims.\n\n• Implicit token selection: The MIL approach does not guarantee that selected high-scoring tokens are truly the ones responsible for hallucination. There is no interpretability or causal evidence to support their significance.\n\n• Lack of statistical/token-level validation: There is no human evaluation or statistical analysis demonstrating that the adaptively selected tokens indeed localize hallucinated content better than baseline tokens.", "was_revised": true, "final_justification": "The authors have conducted comprehensive comparisons and analyses against a broad set of baseline methods, providing solid empirical support for their approach. Overall, we consider this work to be a valuable and well-executed contribution that merits acceptance.", "rebuttal": "Thank you for your thoughtful suggestions. We sincerely appreciate your positive remarks on our contributions. Please see our response to your comments one by one below.\n\n> **Question #1 \\& Weakness #2**: How accurate are GPT-4.1 labels in practice? Could you report inter-agreement between the two judgment passes? Have you considered incorporating human annotations or knowledge-based QA datasets (e.g., where gold answer is guaranteed factual)?\n\nTo evaluate the correctness of free-form generations in QA tasks, GPT-4 labels have been widely adopted in recent hallucination detection studies, such as Semantic Entropy [Ref1] and TSV [Ref2], establishing a popular evaluation protocol with GPT-4 used as annotator in this field. In our work, we utilize updated GPT-4.1 as the labelling model. \nWe also observed that, when provided with gold answers, GPT-4.1 would give highly reliable annotation results for LLM-generated answers.\nThe inter-agreements between the two judgment passes are 96.1\\%, 82.3\\%, 91.2\\% and 92.1\\%, respectively for the datasets Trivia QA, SQuAD, NQ Open and BioASQ based on LLaMA-3.1-8B model, indicating a high level of agreement across datasets. In our experiments, we exclude samples where the two GPT-4.1 judgments are inconsistent, ensuring our evaluation is based on confident, aligned labels.\n\nWe also agree with the value of incorporating human annotations or leveraging knowledge-grounded QA datasets with gold factual references. To validate HaMI’s performance on such datasets, we conducted an additional experiment on TruthfulQA dataset [Ref3], which provides ground-truth true and false answers. \nHaMI achieves an AUROC score of 0.927 on this dataset, which is higher than SAPLMA (0.903) and HaloScope (0.685), as we tested. While human annotation provides high labelling fidelity, its high cost limits feasibility at scale. Due to time limitations, we haven't managed to obtain the results of all competing methods; we will include the full results on this dataset to provide empirical justification for scenarios with human-annotated labels.\n\n> **Weakness #1**: Limited baseline comparison: Important recent hallucination detection methods such as LLM-Check and CED (Comparing Embedding Differences) are not included. These would provide stronger empirical context and help position the contribution more rigorously.\n\nWe appreciate your suggestion and include the suggested two additional baselines LLM-Check and CED for a more comprehensive comparison. The comparison results across all four datasets (i.e., TriviaQA, SQuAD, NQ, and BioASQ) with LLaMA-3.1-8B are summarized in **Table A1** below. Observed from the original paper on LLM-Check, we choose hidden state and attention-based variations, which claim stronger performance. As presented in Table A1, our method consistently outperforms these added baselines by a significant margin. \n\nIn addition, as suggested by Reviewer **NwzN**, we have also added another recent method G-NLL [Ref4] as our baseline, and our method also obtains substantially better performance than G-NLL. Please refer to our response to Reviewer **NwzN**'s **Question 4 \\& Weakness #3** for more details.\n\n| Method                   | Trivia QA | SQuAD | NQ    | BioASQ |\n|--------------------------|-----------|--------|-------|--------|\n| LLM-Check-Hidden-Score   | 0.666     | 0.614  | 0.610 | 0.673  |\n| LLM-Check-Attention-Score| 0.651     | 0.644  | 0.626 | 0.675  |\n| CED                      | 0.745     | 0.776  | 0.710 | 0.695  |\n| HaMI*      | 0.854     | 0.783  | 0.788 | 0.823  |\n| HaMI       | 0.897     | 0.826  | 0.820 | 0.836  |\n```\nTable A1. Results for LLM-Check, CED and HaMI\n```\n\n\n> **Weakness #3 \\& Weakness #4**: \n> - Implicit token selection: The MIL approach does not guarantee that selected high-scoring tokens are truly the ones responsible for hallucination. There is no interpretability or causal evidence to support their significance.\n> - Lack of statistical/token-level validation: There is no human evaluation or statistical analysis demonstrating that the adaptively selected tokens indeed localize hallucinated content better than baseline tokens.\n\nThank you for raising this important point regarding the validation of our token selection module.\nHaMI relies on coarse supervision signals, i.e., sequence-level labels rather than token-level labels. As a result, the MIL-based approach cannot guarantee that the tokens with the highest scores always perfectly align with the exact hallucinated words. However, we would like to clarify that although the identified token may not always correspond precisely to the hallucinated token, it often appears close, for example, immediately before or after if such  inaccurate cases happen. Moreover, hallucinated tokens themselves tend to receive scores very close to 1.0, indicating strong relevance even in the absence of fine-grained labels. This property allows HaMI to effectively narrow down the region where hallucinated content is likely to occur, which is practically useful for interpretability.\n\nTo further support the validity of the token selection process, we conducted an additional study, a manual evaluation using 100 randomly sampled examples from the Trivia QA dataset. In this analysis, we manually annotated hallucinated tokens and measured the token detection recall rate of our method throughout training. Specifically, HaMI can achieve a recall rate at 0.84 as training progresses. This suggests that our approach enables the model to identify salient tokens even without token-level supervision.\n\nPlease kindly refer to our responses to Reviewer **NwzN**'s **Question #1**, **Weakness #4**, and Reviewer **dBF6**'s **Question #2 \\& Weakness #3** for detailed discussion and empirical evidence on the quality of the selected high-scoring tokens.\n\n**References:**\n- [Ref1] Farquhar, Sebastian, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. \"Detecting hallucinations in large language models using semantic entropy.\" Nature 630, no. 8017 (2024): 625-630. 2024\n- [Ref2] Park, Seongheon, Xuefeng Du, Min-Hsuan Yeh, Haobo Wang, and Yixuan Li. \"Steer LLM Latents for Hallucination Detection.\" Forty-Second International Conference on Machine Learning. 2025.\n- [Ref3] Lin, Stephanie, Jacob Hilton, and Owain Evans. \"Truthfulqa: Measuring how models mimic human falsehoods.\" 60th Annual Meeting of the Association for Computational Linguistics. 2022.\n- [Ref4] Lukas Aichberger, Kajetan Schweighofer, and Sepp Hochreiter. \"Rethinking uncertainty estimation in natural language generation.\" arXiv preprint arXiv:2412.15176, 2024."}, {"review_id": "cGdrgv84MM", "reviewer": "Reviewer_Z1J4", "summary": "The paper proposes HaMI, a hallucination detection framework for LLMs that formulates the task as a multiple instance learning (MIL) problem. Rather than relying on predetermined token positions (e.g., last token), HaMI adaptively selects salient tokens based on their hallucination scores and trains a detector end-to-end. The authors further enhance detection performance by incorporating predictive uncertainty into internal token representations. Extensive experiments across four QA benchmarks and multiple LLMs demonstrate strong improvements over state-of-the-art (SotA) baselines, particularly in cross-dataset generalization.", "questions": "1. The authors state that the model selects the top-k tokens based on hallucination scores. However, this operation is not differentiable. Do you simply treat top-k as a hard selection during training, or is there a differentiable approximation (e.g., using soft top-k, Gumbel-softmax, or similar techniques)? If it’s a hard selection, how is this really learning to select the right tokens? I would appreciate a clarification on that.\n2. In lines 275–276 You mention, \"For each dataset, we report the average AUROC scores of detectors trained on one of the other three datasets.\"\n     - Does this mean you trained three detectors, each on a different dataset and then averaged their performance on the target dataset?\n     - Or was only one source dataset chosen (randomly or fixed) for training, and results reported per target? If so, how did you choose it?\n Clarifying this setup is important to interpret generalization performance.\n3. In Figure 3(a), cross-dataset generalization is only meaningful if it outperforms non-learnable baselines, such as those based on token probabilities or logits. To strengthen their results, I suggest the authors provide a justification for why such probability/logit-based baselines would perform worse than their method. Ideally, and if time permits, an additional experiment could be included to compare against a simple heuristic—such as averaging the token probabilities or logits across a sentence—as a baseline for hallucination detection in the cross-dataset setup.\n4. In Table 2 (left), the performance gap between “ours” and probing the last token appears relatively small. While this might be acceptable, it’s important to note that “ours” includes additional information from predictive uncertainty. At this point, it’s unclear whether the observed improvement stems from the use of predictive uncertainty or from the top-k token selection itself. Clarification on this would be helpful. If feasible, I would also recommend adding a new row to Table 2 (left) showing results for a variant of HaMI without predictive uncertainty (e.g., HaMI*), to isolate the contribution of top-k selection.", "limitations": "The authors acknowledge that their method is limited to open-source LLMs due to its reliance on internal representations. They also note that their integration of uncertainty metrics is simple and could be improved with more advanced fusion techniques in future work.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n- The paper is well-written and easy to follow.\n- Recasting hallucination detection as a MIL problem is original and well-motivated.\n- The authors compare against both internal representation and uncertainty-based methods across a broad suite of datasets and models and show superior performance.\n\n\n**Weaknesses:**\n- The central claim of this work is \"adaptive token selection,\" but if I understand correctly, this adaptation is not actually learned. The method selects the top-k candidates, relying on the hope that the model will implicitly learn to choose the appropriate tokens during training. However, top-k selection is a non-differentiable operation, making it analogous to the max operation in convolutional layers—where backpropagation does not learn which input to pick, but merely propagates gradients through the selected maximum. This raises a concern: it's unclear how much the proposed \"adaptive token\" mechanism truly contributes to performance, especially when compared to simply probing the last token (see Question 4 below).\n- Minor concern: The proposed method appears to ignore the sequential nature of the tokens. If I understand correctly, it simply averages over the top-k candidates without considering their order or relative positions—information that could be valuable for detection tasks.", "was_revised": true, "final_justification": "I think this is a solid, incremental contribution. I would be happy to see it accepted, but I would also understand a rejection, given the areas for improvement outlined. As such, I will maintain my score as a borderline accept.", "rebuttal": "Thank you very much for the constructive comments. We are grateful for the positive comments on our innovation and empirical justification. Please see our response to your comments one by one below.\n\n> **Question #1 \\& Weakness #1**: Clarification on the top-k tokens based on hallucination scores. If it’s a hard selection, how is this really learning to select the right tokens?\n\nWe confirm that our current implementation employs a hard top-k selection strategy during training, without using a differentiable approximation. Although the selection operation itself is non-differentiable, the learning process remains effective due to the design of our Multiple Instance Learning (MIL) framework, where instance selection is treated as a latent and optimization is performed over the selected subset. Specifically, while the top-k selection is discrete, the hallucination scores for each token are generated by a fully differentiable detector network. Gradients from the MIL loss are backpropagated through this scoring function, allowing the model to learn how to assign more accurate scores with better supervision over time.\n\nTo validate the effectiveness of our approach, we manually annotated hallucinated tokens in 100 randomly selected samples and conducted a token-level recall analysis on the TriviaQA dataset. Our adaptive token selection achieved a recall rate of 0.84, indicating that the model is indeed learning to select tokens highly aligned with the exact hallucinations. This recall performance is remarkable considering that we do not have token-level hallucination labels. \n\nPlease also kindly refer to our responses to Reviewer **NwzN**'s **Question #1**, **Weakness #4**, and Reviewer **dBF6**'s **Question #2 \\& Weakness #3** for more discussion on addressing this concern.\n\n> **Question #2**: Clarification on \"For each dataset, we report the average AUROC scores of detectors trained on one of the other three datasets.\" (lines 275–276)\n> - Does this mean you trained three detectors, each on a different dataset and then averaged their performance on the target dataset?\n> - Or was only one source dataset chosen (randomly or fixed) for training, and results reported per target? If so, how did you choose it? Clarifying this setup is important to interpret generalization performance.\n\n\nAs correctly noted in your first point, given the four datasets, our cross-dataset evaluation follows the setup where detectors are trained separately on each of the three source datasets (leaving the target dataset out, one detector per source dataset), and we then evaluate the detectors on the target dataset and report the AUROC score averaged over the results of the three detectors. This setting is designed to assess the model's ability to generalize from various source datasets to the target dataset. We appreciate your attention to this part of experimental design, and we will clarify this point in the paper.\n\n\n> **Question #3**: In Figure 3(a), cross-dataset generalization is only meaningful if it outperforms non-learnable baselines, such as those based on token probabilities or logits. To strengthen their results, I suggest the authors provide a justification for why such probability/logit-based baselines would perform worse than their method. Ideally, and if time permits, an additional experiment could be included to compare against a simple heuristic—such as averaging the token probabilities or logits across a sentence—as a baseline for hallucination detection in the cross-dataset setup.\n\nThank you very much for the question. The predictive logits of the tokens reflect how likely a token is to be generated by an LLM, which is related to training distribution of the model. As such, probability- or logit-based measurements would be influenced by fluency and grammaticality while hallucinations often occur in fluent but factually incorrect text. This is often one main cause to their degraded performance in hallucination detection. In contrast, internal representations extracted from LLMs can encode deeper semantic and contextual information, including how the model encodes, processes and grounds knowledge, which allows for a more nuanced assessment of whether the model’s output is correct or not. These observations align with findings in previous studies [Ref1], which suggest that “LLMs know more than they express”.\n\nFor the comparison of cross-dataset generalization capability, we selected CCS, SAPLMA, and HaloScope because these methods access the internal states of LLMs, which is also applied in HaMI. We agree that including non-learnable/training-free baselines is also important for a comprehensive assessment of a method’s generalization ability. For the non-learnable methods, including p(True), Perplexity, SE, and MARS, their results in Table 1 reflect their cross-dataset performance since they involve no training on source datasets.\nComparing these results to the results of HaMI in Figure 3 (a), it is clear that HaMI consistently outperforms these state-of-the-art non-learnable baselines. Specifically, the cross-dataset performance of HaMI on four datasets are **0.858, 0.799, 0.796, 0.797**, which is higher than the best results on each individual dataset, **0.824, 0.787, 0.777, 0.757**, that can be obtained by these non-learnable baselines (these results are based on single running of HaMI; the same conclusion can be drawn for the averaged performance of HaMI over three independent runs in **Table A3** in our response to Reviewer **NwzN**'s **Question #3** & **Weakness #2**). Furthermore, the suggested simple logit baseline that averages token log-probabilities across the sentence achieves AUROC scores of only **0.732, 0.649, 0.659, 0.709** on the four datasets, which are significantly lower than the corresponding AUROC scores from HaMI.\n\nWe will add this additional justification and comparison in the revised version of the paper to address this concern.\n\n> **Question #4**: In Table 2 (left), the performance gap between “ours” and probing the last token appears relatively small. While this might be acceptable, it’s important to note that “ours” includes additional information from predictive uncertainty. At this point, it’s unclear whether the observed improvement stems from the use of predictive uncertainty or from the top-k token selection itself. Clarification on this would be helpful. If feasible, I would also recommend adding a new row to Table 2 (left) showing results for a variant of HaMI without predictive uncertainty (e.g., HaMI*), to isolate the contribution of top-k selection.\n\nWe would like to clarify that all the four variants of the method reported in Table 2 (left), including the “Last” and \"Ours\", are built upon the features enhanced with the uncertainty measure $P^{c}_{\\text{uncertainty}}$, i.e., the four methods are all variants of HaMI rather than HaMI*  in Table 1 in our paper.  Therefore, the performance improvements observed can be solely attributed to the effectiveness of our token selection strategy rather than the uncertainty modelling.\n\nAdditionally, please also refer to the results of Table 1, where we compare HaMI with HaMI* and SAPLMA using three different LLM models. HaMI is the model using both uncertainty modelling and adaptive token selection, HaMI* is trained directly on the raw features extracted from LLMs without using uncertainty modelling but using adaptive token selection, while SAPLMA relies solely on the last-token representations and does not use both uncertainty modelling and adaptive token selection. The differences between HaMI* and SAPLMA solely highlight the contribution of our adaptive token selection, while the differences between HaMI and HaMI* solely highlight the contribution of the uncertainty modelling.\n\n\n> **Weakness #2**: Minor concern: The proposed method appears to ignore the sequential nature of the tokens. If I understand correctly, it simply averages over the top-k candidates without considering their order or relative positions—information that could be valuable for detection tasks.\n\nThank you for your insightful question.\nOur method indeed directly averages over the top‑k token candidates, without explicitly modelling their sequential order or relative positions. This design choice is based on the primary motivation of HaMI, aiming to learn which tokens are most informative for hallucination detection, regardless of their position in the generated sequence.\nBy doing so, HaMI can robustly detect hallucination signals even when they appear sparsely or in varied locations across long generations. Therefore, the lack of explicit positional encoding in our MIL learning framework does not compromise performance, yet it helps our model adaptively select hallucination tokens in varying positions of the generated response. We agree that further incorporation of the sequence order may provide additional value; we will explore this addition under the MIL framework in our future work.\n\n**Reference:**\n- [Ref1] Azaria, Amos, and Tom Mitchell. The internal state of an LLM knows when it's lying. Findings of the Association for Computational Linguistics: EMNLP. 2023."}]

Author rebuttal:

Meta-review: This paper introduces HaMI, a novel approach for LLM hallucination detection that reformulates the problem as Multiple Instance Learning (MIL) with adaptive token selection, receiving consistent support from all reviewers. The core innovation lies in treating response sequences as "bags" of token instances and jointly optimizing token selection and classification, moving beyond predetermined token positions used in prior work. Reviewers praised the intuitive MIL formulation and strong empirical results. Authors provided comprehensive rebuttals including manual token annotation studies, extensive baseline comparisons and clarification that the lightweight variant maintains superior performance while being computationally efficient. All reviewers ultimately supported acceptance, recognizing this as a solid incremental contribution that opens new research directions in hallucination detection through the MIL framework.
Decision: Accept (poster)