Title: Robust Hallucination Detection in LLMs via Adaptive Token Selection
Venue: NeurIPS 2025
Accepted: yes
Keywords: Hallucination Detection; Large Language Models
Abstract: Hallucinations in large language models (LLMs) pose significant safety concerns that impede their broader deployment. Recent research in hallucination detection has demonstrated that LLMs' internal representations contain truthfulness hints, which can be harnessed for detector training. However, the performance of these detectors is heavily dependent on the internal representations of predetermined tokens, fluctuating considerably when working on free-form generations with varying lengths and sparse distributions of hallucinated entities. To address this, we propose HaMI, a novel approach that enables robust detection of hallucinations through adaptive selection and learning of critical tokens that are most indicative of hallucinations. We achieve this robustness by an innovative formulation of the Hallucination detection task as Multiple Instance (HaMI) learning over token-level representations within a sequence, thereby facilitating a joint optimisation of token selection and hallucination detection on generation sequences of diverse forms. Comprehensive experimental results on four hallucination benchmarks show that HaMI significantly outperforms existing state-of-the-art approaches.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes HaMI, a hallucination detection framework for LLMs that formulates the task as a multiple instance learning (MIL) problem. Rather than relying on predetermined token positions (e.g., last token), HaMI adaptively selects salient tokens based on their hallucination scores and trains a detector end-to-end. The authors further enhance detection performance by incorporating predictive uncertainty into internal token representations. Extensive experiments across four QA benchmarks and multiple LLMs demonstrate strong improvements over state-of-the-art (SotA) baselines, particularly in cross-dataset generalization.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The paper is well-written and easy to follow.\n- Recasting hallucination detection as a MIL problem is original and well-motivated.\n- The authors compare against both internal representation and uncertainty-based methods across a broad suite of datasets and models and show superior performance.\n\n\n**Weaknesses:**\n- The central claim of this work is \"adaptive token selection,\" but if I understand correctly, this adaptation is not actually learned. The method selects the top-k candidates, relying on the hope that the model will implicitly learn to choose the appropriate tokens during training. However, top-k selection is a non-differentiable operation, making it analogous to the max operation in convolutional layers—where backpropagation does not learn which input to pick, but merely propagates gradients through the selected maximum. This raises a concern: it's unclear how much the proposed \"adaptive token\" mechanism truly contributes to performance, especially when compared to simply probing the last token (see Question 4 below).\n- Minor concern: The proposed method appears to ignore the sequential nature of the tokens. If I understand correctly, it simply averages over the top-k candidates without considering their order or relative positions—information that could be valuable for detection tasks.\n\nQUESTIONS:\n1. The authors state that the model selects the top-k tokens based on hallucination scores. However, this operation is not differentiable. Do you simply treat top-k as a hard selection during training, or is there a differentiable approximation (e.g., using soft top-k, Gumbel-softmax, or similar techniques)? If it’s a hard selection, how is this really learning to select the right tokens? I would appreciate a clarification on that.\n2. In lines 275–276 You mention, \"For each dataset, we report the average AUROC scores of detectors trained on one of the other three datasets.\"\n     - Does this mean you trained three detectors, each on a different dataset and then averaged their performance on the target dataset?\n     - Or was only one source dataset chosen (randomly or fixed) for training, and results reported per target? If so, how did you choose it?\n Clarifying this setup is important to interpret generalization performance.\n3. In Figure 3(a), cross-dataset generalization is only meaningful if it outperforms non-learnable baselines, such as those based on token probabilities or logits. To strengthen their results, I suggest the authors provide a justification for why such probability/logit-based baselines would perform worse than their method. Ideally, and if time permits, an additional experiment could be included to compare against a simple heuristic—such as averaging the token probabilities or logits across a sentence—as a baseline for hallucination detection in the cross-dataset setup.\n4. In Table 2 (left), the performance gap between “ours” and probing the last token appears relatively small. While this might be acceptable, it’s important to note that “ours” includes additional information from predictive uncertainty. At this point, it’s unclear whether the observed improvement stems from the use of predictive uncertainty or from the top-k token selection itself. Clarification on this would be helpful. If feasible, I would also recommend adding a new row to Table 2 (left) showing results for a variant of HaMI without predictive uncertainty (e.g., HaMI*), to isolate the contribution of top-k selection.\n\nLIMITATIONS:\nThe authors acknowledge that their method is limited to open-source LLMs due to its reliance on internal representations. They also note that their integration of uncertainty metrics is simple and could be improved with more advanced fusion techniques in future work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes HaMI, a novel hallucination detection method for large language models (LLMs) based on Multiple Instance Learning (MIL). Unlike prior work that relies on preselected tokens for detection (e.g., last token), HaMI introduces an adaptive token selection mechanism that jointly optimizes token scoring and hallucination classification in an end-to-end fashion. Furthermore, the model integrates token-level and sentence-level uncertainty into internal representations to improve discrimination between hallucinated and truthful outputs. The authors demonstrate state-of-the-art performance across four QA benchmarks (TriviaQA, SQuAD, NQ, and BioASQ) and three LLMs (LLaMA and Mistral variants). Empirical results show superior robustness and generalization over prior hallucination detection methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nTimely and relevant problem: Hallucination in LLMs is a critical safety issue, and detection methods remain underdeveloped.\n\nConceptual novelty: Reformulating hallucination detection as a MIL problem with adaptive token selection is original and well-motivated.\n\nClarity: The paper is well-written and well-structured, with clear methodological explanations and visualizations.\n\nWeaknesses:\n\n• Limited baseline comparison: Important recent hallucination detection methods such as LLM-Check and CED (Comparing Embedding Differences) are not included. These would provide stronger empirical context and help position the contribution more rigorously.\n\n[1] Sriramanan, Gaurang, et al. \"Llm-check: Investigating detection of hallucinations in large language models.\" Advances in Neural Information Processing Systems 37 (2024): 34188-34216.\n\n[2] Lee, Hakyung, et al. \"CED: Comparing Embedding Differences for Detecting Out-of-Distribution and Hallucinated Text.\" Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.\n\n• Label reliability: The ground truth annotations for hallucination are solely based on GPT-4.1 self-judgment, with only a two-pass validation. While practical, this approach lacks guarantees of correctness. Incorporating datasets with verified human or knowledge-grounded hallucination annotations would strengthen the empirical claims.\n\n• Implicit token selection: The MIL approach does not guarantee that selected high-scoring tokens are truly the ones responsible for hallucination. There is no interpretability or causal evidence to support their significance.\n\n• Lack of statistical/token-level validation: There is no human evaluation or statistical analysis demonstrating that the adaptively selected tokens indeed localize hallucinated content better than baseline tokens.\n\nQUESTIONS:\nHow accurate are GPT-4.1 labels in practice? Could you report inter-agreement between the two judgment passes? Have you considered incorporating human annotations or knowledge-based QA datasets (e.g., where gold answer is guaranteed factual)?\n\nLIMITATIONS:\nSee the weakness", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes HaMI (Hallucination detection as Multiple Instance learning), a novel approach for detecting hallucinations in large language models through adaptive token selection. The key innovation is reformulating hallucination detection as a multiple instance learning problem where each response sequence is treated as a bag of token instances, enabling joint optimization of token selection and hallucination detection. The method adaptively identifies the most indicative tokens for hallucination rather than relying on predetermined token positions (first, last, etc.). Additionally, the authors introduce a representation enhancement module that integrates uncertainty information at multiple levels (token, sentence, and semantic consistency) to improve detection performance. Experimental results on four QA benchmarks (TriviaQA, SQuAD, Natural Questions, BioASQ) with three LLMs (LLaMA-3.1-8B, Mistral-Nemo-12B, LLaMA-3.13-70B) demonstrate significant improvements over state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Novel Problem Formulation: The MIL reformulation is intuitive and well-motivated. The assumption that only a few tokens contain hallucination information while most tokens in incorrect responses are neutral aligns well with the sparse nature of hallucinations.\n- End-to-End Joint Optimization: Unlike existing methods that separate token selection and detection into two stages, HaMI enables joint optimization, which is a methodological advancement.\n- Comprehensive Experimental Evaluation: The experiments cover multiple datasets, models, and baselines. \n- Strong Empirical Results: HaMI achieves substantial improvements over baselines, with gains of 8.1% to 11.9% AUROC over MARS-SE across three LLMs.\n- Thorough Ablation Studies: The paper provides detailed analysis of different components, uncertainty types, layer selection, and hyperparameter sensitivity.\n\nWeaknesses:\n- The best-performing variant (HaMI with semantic consistency) requires multiple generations and entailment evaluation, which significantly increases computational cost (7.6-46.1 seconds per question as noted in appendix).\n- The sequential smoothness loss (Eq. 3) assumes neighbor tokens should have similar hallucination scores, but this assumption may not always hold in practice (e.g., when hallucinations occur at specific entity mentions).\n- The paper misses discussion of when and why the method fails, which would be valuable for understanding its limitations.\n\nQUESTIONS:\n- Can the authors provide more empirical analysis of when the smoothness assumption holds? Are there cases where abrupt changes in hallucination scores between adjacent tokens are actually beneficial (or would not degrade the performance much)?\n- What are the main failure modes of HaMI? When does adaptive token selection fail to identify the most relevant tokens?\n- Do different types of hallucinations (factual errors, nonsensical responses, contradictions) show different patterns in token selection? How does HaMI perform on fine-grained hallucination taxonomies?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Hallucination detection as Multiple Instance learning, short HaMI, which trains a hallucination detector that distinguishes between generations with hallucinations (positive bags of tokens) and generations without hallucinations (negative bags of tokens) by maximizing the distance between them in token-level representation space.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n1. The paper proposes a an interesting approach to hallucination detection by jointly optimizing token selection and classification via a Multiple Instance Learning (MIL) formulation.\n2. HaMI demonstrates strong and consistent empirical performance, significantly outperforming state-of-the-art baselines across both uncertainty-based and internal-representation-based detection methods.\n3. The evaluation is thorough and well-structured, spanning three language models and four QA datasets.\n4. The ablation studies are well-designed and clearly show the contribution of each component to the overall performance.\n\n**Weaknesses:**\n1. The authors introduce a sequential smoothness loss based on the assumption that hallucination scores for neighboring tokens should be similar. However, this assumption has not been theoretically motivated or empirically validated. It would be helpful to further investigate this assumption.\n2. For the main results, the hallucination detector is both trained and evaluated on the same dataset, whereas the uncertainty estimation baselines are dataset-agnostic. A fairer comparison would be to train a single detector per model and evaluate it across multiple datasets. While Section 5.2 provides some insight into HaMI’s generalization ability, it remains unclear how its performance compares to that of the uncertainty estimation baselines. Additionally, it is not clear whether the method generalizes beyond the question answering domain.\n3. The baselines do not include sentence-level uncertainty measures that require generating only a single output sequence, such as G-NLL, which is computed using the most-likely output sequence only [1]. These methods are lightweight and widely applicable, thus should be considered standard in an evaluation.\n4. The authors claim that their method can identify the most hallucinated tokens within the positive bag. However, the ablation in Section 5.3 shows that semantically irrelevant words like “is” and “and” receive the highest hallucination scores..\n\n---\n[1] Lukas Aichberger, Kajetan Schweighofer, and Sepp Hochreiter. Rethinking uncertainty estimation in natural language generation. *arXiv preprint arXiv:2412.15176*, 2024.\n\nQUESTIONS:\n- The authors \"choose the token instances with the top k highest hallucination scores as the salient tokens in each sequence\" (line 156). These hallucination scores are obtained from the hallucination detector. However, at the beginning of training, these are random scores, so maximizing the discriminative margin between these randomly selected tokens might not be optimal. How does this influence training?\n- How does the sequential smoothness loss impact the training and performance of the method? (Re. Weakness 1)\n- Would HaMI still achieve the best performance in Table 1 if a single hallucination detector were trained per model, rather than per dataset? (Re. Weakness 2)\n- How does the method compare to single-sequence uncertainty method both in performance and computational effectiveness? (Re. Weakness 3)\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-06"}