HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detection
Authors: Min-Hsuan Yeh
Max Kamachee
Seongheon Park
Yixuan Li
Venue: Accepted by TMLR ()

Abstract: To mitigate the impact of hallucination nature of LLMs, many studies propose detecting hallucinated generation through uncertainty estimation. However, these approaches predominantly operate at the sentence or paragraph level, failing to pinpoint specific spans or entities responsible for hallucinated content. This lack of granularity is especially problematic for long-form outputs that mix accurate and fabricated information. To address this limitation, we explore entity-level hallucination detection. We propose a new data set, HalluEntity, which annotates hallucination at the entity level. Based on the dataset, we comprehensively evaluate uncertainty-based hallucination detection approaches across 17 modern LLMs. Our experimental results show that uncertainty estimation approaches focusing on individual token probabilities tend to over-predict hallucinations, while context-aware methods show better but still suboptimal performance. Through an in-depth qualitative study, we identify relationships between hallucination tendencies and linguistic properties and highlight important directions for future research.

HalluEntity: https://huggingface.co/datasets/samuelyeh/HalluEntity

Published in Transactions on Machine Learning Research (9/2025)
HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detection
Min-Hsuan Yeh
samuelyeh@cs.uiseee
University of WiscioninnMaaison
Max Kamachee University of Wiscomsin-Muddssnn
Ramachee@cs..uisee
Seongheon Park
remnhhonn_porkecs..re work@cs.wissc.el
University of Wisconsin-Madde
Yixuan Li University of Wisconsine-Madison
sharonli@cs.uuie wisc. edu
Reviewed on OpenReview:
Abstract
To mitigate the impact of hallucination nature of LLMs, many studies propose detecting hallucinated generation through uncertainty estimation. However, these approaches predomi- nantly operate at the sentence or paragraph level, failing to pinpoint specific spans or entities responsible for hallucinated content. This lack of gramularity is especially problematic for long-form outputs that mix accurate and fabricated information. To address this limitation, we explore entity-level hallucination detection. We propose a new data set, HALLUENTITY, which annotates hallucination at the entity level. Based on the dataset, we comprehensively evaluate uncertainty-based hallucination detection approaches across 17 modern LLMs. Our experimental results show that uncertainty estimation approaches focusing on individual token probabilities tend to over-predict hallucinations, while context-aware methods show better but still suboptimal performance. Through an in-depth qualitative study, we iden- tify relationships between hallucination tendencies and linguistic properties and highlight important directions for future research.
HalluEntity: https://iggiiiieeeeedddaaarrraar
1 Introduction
How can trust the facts generated by large language models (LLMs) when even a single hallucinated entity can distort an entire narrative? While LLMs have revolutionized text generation in various domains, from summarization to scientific writing (Liang et al., 2024), their tendency to produce hallucinations factually incorrect or unsupported content-remains remains a critical challenge (Xu et al., 2024). This issue particularly concerning in high-stakes applications, such as modical diagnostics (Chen et al., 2024b), legal document drasting (Lin & Cheng, 2024), and news generation (Odabaşı & Biricik, 2025), where inaccurate information cause harm to individuals and erosion of public trust. Detecting hallucinations is therefore a critical step toward ensuring the responsible deployment of LLMss
Various approaches have been proposed to detect hallucinations (Luo el al., 2024), with uncertainty-based methods emerging as a promising direction (Zhang et al., 2023a). However, current uncertainty-based hallucination detection approaches mainly operate the sentence or paragraph level, classifying the entire generation as either hallucinated or correct. While this provides coarso-grained assessment of factuality, it lacks the granularity needed to pinpoint which specific spans entities contribute to hallucination. This

---

Published in Transactions on Machine Learning Research (9/2025)
limitation is particularly problematic for long-form text, where both accurate and hallucinated information frequently coexist. For example, generated response about historical event might correctly state the date but fabricate details about the individuals involved, necessitating finer-grained detection.
To address these limitations, we present a first systematic exploration of entity-level hallucination detection by introducing a benchmark dataset, extensively evaluating uncertainty-based detection methods on this benchmark, and analyzing their strengths and limitations in identifying hallucinated entities. Specifically, we begin by constructing benchmark for entity-lovel hallucination detection, HALLUENTITY, which fills in the critical blank for the field. Constructing such dataset is challenging due to the labor-intensive nature of entity-level annotation, which requires annotators to segment mcaningful entitios and verify their factuality against reliable sources one by one. To overcome this, we develop a systematic pipeline that maps atomic facts from model-generater text to entity-level annotations, enabling structured hallucination detection at a finer granularity. HALLUENTIYY encompasses 18,785 annotated entities, and provides a foundation for evaluating hallucination detection methods with greater interpretability and precision.
Building HALLUENTITY, comprehensively evaluate the reliability of token-level uncertainty measure- ments in detecting hallucinated entities and their potential for localizing hallucinations within the generated text. Our evaluation broadly includes standard uncertainty estimators, such as token-level likelihood (Guer- reiro et al., 2023) and entropy scores (Malinin & Gales, 2021), as well as more advanced context-aware approaches that refine uncertainty estimation (Fadeeva et al., 2024; Duan et al., 2024; Zhang et al., 2023b). By aggregating token-level uncertainty to the entity level, we assess whether these methods can accurately distinguish hallucinated entities from factual ones. We experiment with 17 modern LLMs across different model families and capacities. The results reveal that methods solely relying on individual token probabilities (e.g., likelihood and entropy) tend to over-predict hallucinations, making them less reliable. In contrast, context-aware approaches (Duan et al., 2024; Zhang et al., 2023b) demonstrate better overall performance in entity-level hallucination detection. Additionally, model family and size have a limited impact on performance, compared to the choice of uncortainty cstimation method, emphasizing the need for improved uncortainty modeling.
Through in-depth qualitative analysis, we further identify relationships between hallucination tendencies and linguistic properties, such sentence positions and entity types. We found that calibrating uncertainty score with uallextual information in the best-performing method (Zhang et al., 2023b) helps reduce over-confidence tendencies in later sentence positions but can unintentionally penalize non non-hallueinated content. We also found that some uncertainty scores can frequently assign high uncertainty to informative content like named entities. These observations highlight critical areas for future research, including better modeling of contextual dependencies to maintain balanced precision-recall trade-offs. Our key contributions are summarized as follows:
1. We propose entity-level hallucination detection dataset, HALLUENTITY, which contains 18,785 annotated entities for Chat biographies.
2. We comprehensively evaluate uncertainty-based hallucination detection approaches across 17 LLMs on our proposed dataset.
3. We conduct an in-depth analysis to identify the strengths and weaknesses of current uncertainty-basee approaches, and provide insight design better uncertainty scores.
2 Related Work
Uncertainty-based hallucination detection methods. Various approaches have been proposed detect hallucinated content in LLM generation. Unlike other methods that require external knowledge sources for fact- checking (Gou et al., 2024; Chen et al, 2024a; Min et al., 2023; Huo et al., 2023), uncertainty- based approaches are reference-free and rely only on LLM internal states or behaviors to determine hallucination (Huang et al., 2024; Park et al., 2025). For instance, sampling- approaches generate multiple responses and measure the diversity in meaning among them (Fomicheva et al., 2020; Kuhn et al., 2023; Lin et al., 2024), while density-based approaches approximate the training data distribution and provide probabilitics or unnormalized scores to assess how likely a generated response belongs to the distribution (Yoo et al., 2022; Ren et al., 2023; Vazhentsev et al., 2023).
2

---

Published in Transactions on Machine Learning Research (9/2025)
In this paper, we focus on uncertainty quantification methods that rely on token-level likelihood tropy (Guerreiro et al., 2023; Malinin & Gales, 2021). Recent works have explored refining likelihood estimation by incorporating semantic relationships or reweighting token importance. For instance, Claim- Conditioned Probability (CCP) (Fadoeva et al., 2024) was introduced to recalculate likelihood according to semantical equivalence; while Zhang et al. (2023b) and Duan et al. (2024) adjust loken weights to better convey meaning in uncertainty aggregation. Although these approackes leverage token-level information, they are typically evaluated at the sentence level, raising questions about their reliability. To address this, we conduct comprehensive analysis of entity-level hallucination detection for finer-grained performance insights.
Fine-grained hallucination detection benchmark. Most hallucination detection benchmarks are at the sentence or paragraph level. For example, CoQA (Reddy et al., 2019), TriviaQA (Joshi et al., 2017), TruthfulQA (Lin et al., 2022), and HaluEval (Li et al., 2023). These benchmarks classify each generated response as either hallucinated or correct. However, instance-level detection cannot pinpoint specific hallucinated content, which is crucial for correcting misinformation (Cattan et al., 2024). This limitation becomes particularly problematic in long-form text, where a single response often combines supported and unsupported information, making binary quality judgnents inadequate (Min et al., 2023).
To address these challenges, recent works have advanced benchmarks for more gramular hallucination detection. For example, Min et al. (2023) introduced FACTSCORE, which decomposes LLM-generated text into atomic pets--short sentences conveying a single piece of information for more precise evaluation. In parallel, Cattan decompasing L.M--genrateed text with QA-SRI, a semantic
et al. (2024) introduced MASEECOnNISTENN formalism, to form simple QA pairs, where each QA pair represents one verifiable fact. However, these methods do not enable entity-level hallucination detection, as they lack explicit entity-level labeling (hallucinated or not) in the original generated text. Beyond decomposition- based approaches, datasets like HADES (Liu et al., 2022) and CLIFF (Cao & Wang, 2021) create token-level hallucinated content by perturbing human-written text, allowing token-level annotation on the same text. These perturbed hallucinated content, however, could be unrealistic, biased, and overly synthetic due to the limitations of the models they used to perturb words. To bridge this gap; we create a new dataset with entity-level hallucination labels on the same LLMs' generated text. This allows us to evaluate uncortainty-bascl hallucination dotection approaches on a finer-grained level and analyze their reliability.
3 HalluEntity: An Entity-Level Hallucination Dataset
3.1 Dataset Construction
Curating an entity-level hallucination detection dataset is challenging, requiring annotators to segment sentences into meaningful entities and verify the factual consistency of each entity against. reliable sources. This process is time-intensive, requires domain expertise, and is prone to subjectivity (Cao & Wang, 2021). To address these challenges, we first develop a data generation pipeline that maps atomic facts from FActScore (Min et al., 2023) back to the original generated lext.
Entity Label
Geneeratin
Lanny Tlanerty
Tell of Lanny Fiaherty
Tanny Fleerity American borm Decomber 1949, Ponsacols, Florida. has appeared numeroo fllme, tolovision theater productions...
car
Dessmber 16, 1946
Atomic
Panssrala
Lanny Flaterr luan fmorican. /
itan
Te
appasred
Lanig Platisty wos bornin Paraaccla, Parice, X
Lanny Elaturty horn December 1969
numerous fllms)
Ha laz uppoored nameraa Hirra ui
appoonal numerou inboraionssoooss.
Dotedd on sitioer
and
Thaeter productions_.
Entity segmentation and labeling. To construct our dataset HALLUENTITY!, we leverage biographies generated byGPT3.5 (OpenAI, 2023). Each data point consists of a name, a ChatGPT-gennratee biography, and a list of atomic facts labeled as either True or False. As illustrated in Figure 1, each atomic fact is short sentence that conveys single piece of information. Since atomic facts decompose a sentence into verifiable units, they provide a structured reference for identifying hallucinated entities.
Figure 1: Illustration of our entity-level dataset construction. We form entity-level hallucination labels according to the atomic facts extracted by FACTSCORE.
HALluENTIt publicly released under the MIT license.
3

---

Published in Transactions on Machine Learning Research (9/2025)
Hallucination
Not Hallucination
26
70-
15.0
60
Percentage (%)
Percentage (%)
Percentage (%)
12.5
16
10.0-
40
7.5
30-
88%
33%
6
2,5
11%
13%
21%
20%
5k
8%
10-
31%
13%
20%
36%
80%
40%
18%
SON
PRIPPN
NOKN
ADP
VERR
DET
Adu
PRON
CCONU
AUx
ADV
OTHER
Nonntamed
anilyy
Nomo arely
ORG
DATE
PERSON
WORK OF ART
OPE
NORe
GARDINAL
EVENT
LOC
ORDMAA
PRODUCT
OTHER
Tag
NER Tag
Figure 2: Distribution of POS (left), NER. (middle), and breakdown of NER. tags (right). Number on each
bar indicates the ratio of hallucination for each tag.
To derive entity-level labels, we first segment the original text into meaningful units rather than individual words. For instance, "strategic thinking" is treated as a single entity rather than two separate words. We call such meaningful units enfilies. Given that FACTSCORE decomposes multiple-fact sentences into independent. atomic facts, use these fact-level annotations to label entities. For example, in the sentence "Lanny Flaherty is an Americon actor born December 18, 1949," FACTSCORE produces atomic facts:
" Lanny Flaherty is an American." (True) Lanny Flaherty was born on December 18, (F499(False)
By aligning these atomic facts with the original text, we label " "American" as con-hallucinatod and "Deeember 18, 1949" hallueinated. To scale this process efficiently, we automatically identify and label these entities by instructing GPT-4o (OpenAI, 2024) with few-shot prompt. Specifically, manually annotate two examples, each containing an LLM-generated biography, list of atomic facts, and a corresponding entity-level annotation list. The prompt provides detailed description of our segmentation method, along with annotated examples. GPT-4o then generates entity labels, which we manually verify and refine to ensure correctness. Further details on the prompt design and annotation process are provided in Appendix A.
3.2 Data Analysis
Data statistics. HALLUENTITY comprises 157 instances containing a total of 18,785 entities, with 5,452 unique entities. Each entity averages 1.63 words length. On average, each instance contains 120 entitics, with 15% labeled hallucinated, and 85% non-hallucinated across the corpus.
Linguistic feature analysis. We analyze the relationship between the entity-level hallucination labels and linguistic features, e.g., part-of-speech (POS) and named entities recognition (NER) tags. Specifically, we identify these tags for each word with Spacy (Honnibal & Montani, 2017) and count their occurrence in hallucinated and non-hallucinated entities. The results for each of the part-of-speech (POS) and named entities recognition (NER) tags are shown in Figure 2.
Analysis of POS tags revcals significant patterns in the distribution between hallucinated and non-hallucinated content. Proper nouns (PROPN) constitute the most frequent category with 18.3% occurrences, followed by nouns (NOUN, 17.5%) and adpositions (ADP, 15.1%). Among them, proper nouns and nouns exhibit high hallucination rates of 30.9% and 33.6%, respectively, while adpositions have lower rate of 11.7%. Moreover, although adjectives (ADJ, 7.1%) and numbers (NUM, 4.4%) are less common, they suffer from a high hallucination rate of 28.9% and 36.2%.
4

---

Published in Transactions on Machine Learning Research (9/2025)
Non-named entities, which comprise 73.8% of total tokens, show low hallucination rate of 18.2%. In contrast, named entities entities-desiite accounting for only one-third of the tokens tokens-exhibit nearly double the hallucination rate, often exceeding 30%. Among these named entities, person names (PERSON) show the lowest hallucination rate of 13.4%, likely because ChatGPT was prompted to generate biographics for specific individuals.
Beyond POS and NER tagging, hallucination rates vary by position in sentences. The first six words of sentences have low hallucination rate (9%), but this significantly increases in the middle in the middle (25%) and peaks at the last six words (36%). This comprehensive analysis reveals systematic patterns in hallucination across linguistic features and entity types, providing crucial insights into the reliability of different categories of generated content. In Section 5.3, we see the connections between these linguistic features and the performance of mncertainty-based hallucination detection approaches.
4 Uncertainty Scores for Detecting Hallucinated Entities
Given the entity-level hallucination dataset we constructed, a key question arises: Can uncertainty scores efffectivel detect these hallucinated entities? In this section, we comprehensively explore uncertainty- methods, all of which measure uncertainly at the token level. These token-level scores can be conveniently aggregated to the entity level, allowing for a systematic evaluation of their effectiveness in identifying hallucinated entitics.
Notations and definitions. Let V be a vocabulary space, x= (Z1, consisting of tokens r; E V. The token-level hallucination scores are denoted as y (y1, ./...... .er), where Y1 E R. An entity in x is represented as éx = (æi, #i,),,,;;;; where and j are the start and end indices of the entity's tokens, satisfying < j S T. For a set entities fe,, ex} belong to æ, where K is the number of entities, their entity-level hallucination labels are defined as (4 (2, where 4 E (0, 1 indicates whether Gj hallucinated. The entity-level scores are computed as y" = (yf ...
.2...., IT) (2,, be sentence of length T
.k), where Di -0, Lizego Mi, which aggregates token-level scores to the entity level and 68,0 and 6R,1 the start and end indices of entity eg. We introduce five methods below to calculate the token-level uncertainty scores.
Likelihood (Guerreiro et al., 2023): The score is based on the negative log-likelihood of the i-th generated token:
y1 == - log p(g
Entropy (Malinin & Gales, 2021): The score is the entropy of the token probability distribution at position i:
1/4 := popelaaao
DEV
Claim-Conditiones Probability (CCP) (Fadceva et al., 2024): This method adjusts likelihood based semantic equivalence using natural language inference (NLI) model:
ExMIItreett p(a (F<)))
y1 loog Σ EN aoereed p(a (x<))*
where of is the k-th alternative of the i-th generated token, and NLI determines whether concatenating af with the preceding context entails ('e") or contradicts ('c") the original token. In our experiment, we use the top 10 alternatives and use LaBERTa-base (He et al., 2021) as the NLI model.
Shifting Attention to Relevance (SAR) (Duan et al., 2024): This method weights negative log- likelihood by semantic importance:
V/ log ( (x1,
5

---

Published in Transactions on Machine Learning Research (9/2025)
Approach
AUROC T
AUPRC f
Flops f
Precisionopt
Recallopt 1
Likelihood (Guerreiro et al., 2023)
0.57
0.17
0.29
0.18
0.74
Entropy (Malinin & Gales, 2021)
0.57
0.18
0.28
0.17
0.86
CCP (Fadeeva et al., 2024)
0.57
0.25
0.26
0.15
1.00
SAR Duan al. (2024)
0.67
0.27
0.34
0.26
0.51
Focus (Zhang et al., 2023b)
0.78
0.40
0.48
0.38
0.66
Table Performance comparison among five uncertainty scores using Llama3-81
where Rt is cosine similarity between the sentence embedding of x and x (r;). Following Duan et al. (2024), we use SentenceBERT (Reimers & Gurevych, 2019) with RoBERTa- (Liu et al., 2019) for embedding extraction.
Focus (Zhang et al., 2023b): This method refines log-likeliig reglliklihood and entropy using keyword selection, hallucination propagation, and probability correction:
yk I[xx E K.) (h+ vp:),
where I(.) indicator function and K. is keyword identified by Spacy (Homnibal & Montani, 2017). hy is the sum of the negative log-likelihood and entropy of Fi,
h log (og + Evey p(olues) log, p(v1zce)
Here, p(xx)) Jidf(a;) is the token probability adjusted by inverse document frequency (IDF),
and Pt is the hallucination score propagated from previous tokens,
att, Eizo att,, 1/j.
where att., the attention weight between H; and Hj after max-pooling for all the layers and attention heads. Following Zhang et al. (2023b), the token IDF is calculated based on 1M documents sampled from RedPajama dataset (Weber et al., 2024), and the hyperparameter for p. is set to be 0.9.
Besides these five approaches, we acknowledge that other uncertainty-based hallucination dctection approaches exist, such as Semantic Entropy (Kuhin al., 2023), Verbalized Uncertainty (Kadavath et al., 2022), Lexical Similarity (Fomicheva et al., 2020), FigValTaplacian (Lin et al., 2024), HaloScope (Du al., 2024), and TSV (Park et al., 2025) However, since these approaches do not produce token-level scores, they are not applicable to study detecting hallucinated entites.
5
Experiments
5.1
Experimental Setup
Models. To understand the impact of model family and capacity entity-level hallucination detection, experiment with 17 diverse LLMs, including Llama3-(8B, 70B} (Llama Team, 2024), Llama3.1-8BB Llama3.2-3B. Aquila2-(TT) (7B, 34B) (Zhang al., 2024), InternLM2- 20B) (Cai el al., 2024), Qwen2.5- {7B, 32B} (Qwen, 2025), (i- 34B} (01.AI, 2024), phi-2 (Gunasekar et al,, 2023), Mistral-7B (Jiang et al., 2023), Mixtral-8x22B (Jiang et al., 2024), and Gemma2-(99) 27B) (Gemma Team, 2024).
Evaluation Metrics. Entity-level hallucination detection can formulated as a binary classification task. To evaluate performance, we use (1) AUPRC and (2) AUROC, which assess the relationship between entity-level hallucination labels and scores y. These metrics are threshol--agnostic and better suited for comparing uncertainty-based scoring methods. AUPRC captures precision-recall trade-offs, while AUROC

---

Published in Transactions on Machine Learning Research (9/2025)
AUROC
AUPRC
Model
Likelihood
Entropy
CCP
SAR.
Focus Likelihood Entropy
CCP
SAR
Focus
meta-llama/ /eet//Meaalaaaaa M-Mett--laaa---8PP..
0.568
0.571
0.565
0.672
0.784
0.168
0.180
0.247
0.269
0.404
[2] meta-llama/ (aallllaaa/Mea---aa Laaa----aaa-00--
--70B
0.574
0.567
0,565
0.667
0.779
0.168
0.175
0.254
0.268
0.408
3 meta-llama/ Lww.maa...m.aamam
Batta-llamaLLlmmm-...1
0.584
0.592
0.564
0.684
0.783
0.173
0.193
0.246
0.274
0.412
[4] meta-llama/ (aaaa/mma/lammm3221022
Llama-3.2-33
0.577
0.591
0,564
0.685
0.772
0.170
0.191
0.235
0.269
0.368
55 BAAAI/Aquila2-7B
0.544
0.553
0.565
0.679
0.780
0.162
0.178
0.228
0.254
0.388
6 BAAI/Aquila2-3445
0.541
0.566
0,565
0.665
0.779
0.157
0.185
0.236
0.249
0.385
7 internlm//nneernnn//
0.586
0.584
0.562
0.678
0.777
0.173
0.185
0.232
0.264
0.38
8 (nterulm//nteernn2220h
0.579
0.573
0.561
0.674
0.77
0.171
0.179
0.233
0.267
0.349
99 Qwen/ /Qwen2.3-BB
0.557
0.571
0.558
0.675
0.767
0.164
0.185
0.220
0.255
0.346
[00 Qwen Sywen/Qwen.55-000 /wwn22.5-22
0.561
0.569
0.559
0.674
0.768
0.166
0.183
0.226
0.258
0.347
11] 0l-ai/Yi-9B
0.541
0.549
0.56
0.663
0.776
0.159
0.173
0.231
0.237
0.375
11 01-ai/Yii-3 Yi-34B
0.543
0.543
0.557
0.653
0.769
0.156
0.165
0.229
0.233
0.349
[3 microsoft/phi-2
0.619
0.656
0.571
0.705
0.775
0.190
0.236
0.238
0.279
0.371
[15] Bittraalii/Mistral----000
14 mistralai/
0.549
0.545
0.555
0.666
0.784
0.159
0.167
0.236
0.250
0.391
0.560
0.545
0,555
0.665
0.785
0.163
0.165
0.249
0.263
0.418
16 google/gemmm-22-00
0.574
0.575
0.561
0.680
0.744
0.172
0.187
0.234
0.264
0.281
[7 google/gemma-2227
2-277
0.576
0.566
0.557
0.673
0.780
0.174
0.177
0.232
0.263
0.397
Table 2: AUROC and AUPRC of five uncertainty scores across 17 LLMs.
Flopt
Precisonopt
Recallogs
Model Likelihood
Entropy
CCP
SAR
Focus
Likelihood
Entropy
CCP
SAR
Focus Likelihood
Entropy
CCP
SAR
Focus
[ 0.290
0.278
0.261
0.344
0.484
0.180
0.166
0.150
0.261
0.384
0.742
0.860
1.000
0.505
0.658
2 0.291
0.282
0.261
0.335
0.469
0,182
0.168
0.150
0.274
0.339
0.736
0.870
1.000
0.432
0.757
[3]
0.296
0.286
0.261
0.351
0.483
0.188
0.180
0.150
0.268
0.362
0.696
0.688
1.000
0.505
0.724
44 0.294
0.285
0.261
0.349
0.477
0.177
0.182
0.150
0.256
0.348
0.855
0.662
1.000
0.548
0.758
[5]
0.283
0.275
0.261
0.346
0.484
0.169
0.162
0,150
0,248
0.365
0.875
0.916
1.000
0.570
0.719
[)
0.283
0.277
0.261
0.329
0.484
0.171
0.168
0.15
0.222
0.361
0.838
0.786
1.000
0.632
0.732
7 0.304
0.286
0.261
0.348
0.475
0.192
0.174
0.150
0.260
0.385
0.736
0.797
1.000
0.525
0.620
8
0.293
0.281
0.261
0.348
0.467
0,181
0.169
0.150
0.272
0.332
0.771
0.820
1.000
0.481
0.783
99]
0,291
0.277
0,261
0.342
0.475
0.175
0.166
0.150
0.237
0.347
0.857
0.833
1.000
0.613
0,753
[10]
0.293
0. 0,280
0.261
0.341
0.482
0.179
0.166
0.150
0.251
0.356
0.817
0.883
1.000
0.535
0.749
[1]
0.285
0.276
0.261
0.332
0.482
0.172
0.162
0.150
0.233
0.362
0.843
0.936
1.000
0.580
0.721
[12] 0.289
0.279
0.261
0.323
0.478
0.175
0.166
0.150
0,227
0.353
0.831
0.880
1.000
0.559
0.739
13
0.315
0.323
0.266
0.361
0.477
0.195
0.215
0.261
0.254
0.348
U.8T0
0.650
0.270
0.622
0.760
[4
0.289
0.276
0.261
0.333
0.489
0.176
0.162
0.150
0.237
0.386
0.811
0.933
1.000
0.561
0,606
[15] 0.293
0.280
0.261
0.334
0.471
0.177
0.167
0.150
0.256
0.345
0.836
0.866
1.000
0.477
0.743
16
0.294
0.282
0.261
0.346
0.476
0,180
0.168
0.150
0.258
0.344
0.809
0.856
1.000
0.526
0.770
177
0.296
0.282
0.261
0.345
0.473
0.182
0.766
0.150
0.250
0.349
0.800
0.916
1.000
0.556
0.732
Table 3: Flopt, Precisionops: and Recallope of five uncertainty scores across 17 LLMs. Please see Table for models' name.
evaluates true and false positive rates. Unlike AUROC, AUPRC disregards true negatives, emphasizing false positive reduction-a key advantage for hallucination detection, where true negatives often involve less informative entities like prepositions and conjunctions. We complement these metrics by also reporting (3) Flopt, (4) Precisionopt: and (5) Recallopt: whore Flopt is the optimal F1 score among all possible threshold and Precisionopt, and Recallops are corresponding Precision and Recall values.
Computational resources. We conducted all experiments on a server equipped with eight Nvidia A100 GPUs. Depending on the model size, each LLM utilized between one to three GPUs. The time required compute uncertainty scores across the entire dataset varied from 30 seconds to minutes per approach and model, depending on the model size and the complexity of the chosen method.
7

---

Published in Transactions on Machine Learning Research (9/2025)
AUROC
AUPRC
Flopt
Lelihood Entropy
CCP
SAR
Focus
Precisionopt Recallopt
Flopt
0.50
1.0
0.40
0.40
0.75
0.45
0.35
0.8-
0.35
0.40
AUPRO
0.30
Score
0.6
Score
0.80
0.35
0.25
0.25
0.50
0.2
0.20
0.30-
0.15
0.26
lo
0.15
Simal
Large Simall
Logee
Smaah
Large
Simal
Loge
Smaa
Lare
<10%
10-20%
220%
110%
16-20%
>20%
-10%
10-20%
220%
Entrayy
Foous
cop
Sep
Fosua
Extragy
cge
Sep
Focus
Model Capacity
Hallucination Rate
LammaS-00, 700)
MamLMM-77 200)
10-00, 34t)
Likehhood
Foors
Approach
Approach
Approach
Aqu (78, 346)
Gwend 5-D8. 328)
Gommaz (98. 278)
Ergupy
(a) Different LLMs
(b) Different model sizes
(c) Different hallucination rates
Figure 3: For each method, show performance variation across 17 LLMs (3a). We also show AUPRC scores across LLMs with different capacities (3b), as well as performance data with different hallucination rates (3c). Note that the model used in Figure 3c is Llama3-8B.
5.2 Experimental Results
How do different uncertainty scores perform on entity-level hallucination detection? Table presents the evaluation results for five uncertainty-basee hallucination detection approaches using Llama3-8B. Likelihood, Entropy, and CCP exhibit low Precisionops ( overall hallucination rate) but achieve high Recallopt- This pattern suggests these methods over-predict hallucinations, making them less suitable for reliable detection. Their focus on individual token probabilities rather than contextual roles likely contributes to this limitation, indicating that hallucination detection is inherently contert-dependees and requires uncertainly scores calibruted with contextual information.
SAR and Focus, which incorporate context information, show better overall performance. However, their lower Recallopt indicates that the current mothods for modeling context remain suboptimal, failing to capture some hallucinated content. These findings highlight the challenges entity-level hallucination detection and the need for more advanced approaches that better integrate context ual information while achieving a balanced trade-off between precision and recall.
How do different LLM families and capacity impact performance? Table 2, and present the performance of five uncertainty scores across 17 LLMs. The results indicate that microsoft/phi-2 consistently achieves the highest performance across most scores and evaluation metrics, being the only model that avoids over-predicting hallucinations when using CCP. Additionally, models from Mistral AT 3 and mistrraaaiMixxral--xxx -xtall1-x22v-v..111 perform best when using Focus. Notably, phi-2 (2.7B parameters) and Mistral-7B are relatively small models, suggesting that a model's size does not strongly correlate with its ability to detect hallucinations based on token probabilities. While some models outperform others, the performance variations within the same uncertainty score are smaller than those across different scores, as shown in Figure 3a, which summarizes the AUROC, AUPRC, and Flopt scores vary across model families. This suggests that the method used to compute uncertainty scores has a more significant impact on performance.
Figure 3b presents the performance changes across different model sizes within six families: Llama3, Aquila2, I2ternLM2, Qwen2.5, Yi, and Gemma2, each comprising two size variants. The results reveal that, in most cases, using larger model docs not significantly enhance performance. The only exception using Gemma on Focus, where the AUROC score improves by 0.12 between the 27B and 9B versions. Performance improvements for other model families and approaches remain marginal, typically below 0.01. These findings
8

---

Published in Transactions on Machine Learning Research (9/2025)
Low
Medium
High
Hallucination Rate
<10%
10-20%
20%
# of Instance
54
59
44
naludiation
Figure 4: Distribution of entity- level hallucination rate. Most instances in our dataset have hallucination rate < 25%.
Table 4: Statistics of data grouped according to the hallucination rate. Each group has similar amount of data.
suggest that a larger LLM may not reflect its better capability of determining hallucination on ils loken probabilities.
How does performance vary across different hallucination levels? We categorize HALLU into three groups based on the hallucination rate-the proportion of hallucinated entities each generation. Figure 4 shows the distribution of hallucination rate. The results indicate that most biographies generated by ChatGPT have hallucination rate below 25%. Additionally, generations with hallucination rates below 10% and those between 10% to 20% occur at similar frequencies. Based on this observation, we categorize the data into three groups (< 10%, 10-20%, > 20%) to examine how hallucination rates impact detection performance. Table summarizes the group statistics, showing that each group contains a similar amount of data. Figure 3c shows the Precisionopt, Recallopt, and Flops scores across three groups. The results reveal that all methods struggle to detect hallucinated content when the hallucination rate is low, with Flopt scores around 0.2. Entropy and CCP exhibit a steep increase in Recallopt compared toPrecisionopt the hallucination rale increases, suggesting their lendency over-prediet hallucinations, particularly in highhhaallcinntionn scenario. In contrast, Focus achieves small difference between the Recallopt and Precisionopt when the hallucination rate is high, demonstrating its ability to balance precision-recall tradc-offs while also highlighting the challenge of detecting sparse hallucination.
5.3 In-depth Analysis
To better understand the strengths and limitations of uncertainty scores for detecting hallucinated entities, we analyze cases where (1) all scores failed or misidentified hallucinations, and (2) scores varied in performance. We classify entities using thresholds for Flopt and categorize false /nciivvvss by POS, NER tags, and sentence positions (first, middle, or last six words). We then identify tags and positions where approaches excel or falter, visualixing samples with color-coded uncertainty scores to uncover patterns behind detection discrepancies (See Table 5). Figure 5 shows the FPR and FNR across NER tags and sentence positions, and Figure 6 shows the FPR/FNR across POS tags. Our analysis focuses on Likelihood, SAR, and Focus, as SAR and Focus remonstraled the most effective performance in Section 5.2, and Likelihood serves as a straightforward baseline for comparison. Note that Entropy has a similar trend of FPR/FNR across POS and NER tags compared to Likelihood. Thus, we primarily report the result of Likelihood for simplicity.
SAR under-predicts hallucinations due to unreliable token importance weighting. The left and middle plots of Figure show that SAR has the lowest FPR but the highest FNR across most tags, particularly for named entities, indicating tendency to under-predict hallucinations. Consistent with Figure 5, Figure 6 shows that SAR demonstrates lower FPR but higher FNR across POS lags. This occurs because SAR weights token importance based on sentence similarity without the token, which often remains unchanged even if the token informative. The first case in Table illustrates this: SAR assigns lighter shades to entities like the second "Santa Cruz" since removing either "Santa" or "Cruz" barely affects sentence similarity, despite the term's 's informativeness.

---

Published in Transactions on Machine Learning Research (9/2025)
Case 1:
Under-prediction of SAR
Groundtruth
[...] Diaz startad his political carser
nembar of the Sangguniang Bayan (nunicipal council) of Santa Cruz in 1978.
He later becume the Vice Mayor of Santa Cruz in 1980 and was elected as the town's ys Nayor in 1980___.... [...]]
Likelifiodd
[..3 Bi8z started his political career BB nember of The SanggunIang Bayan (nunicipal council) of Santa Cruz An 1878
liee Later becune the Vice Mayor of Santa Cruz in 1988 and was elected as the town's Mayor 1988 [...]
BÄR
[...] Diaz started his political career as nember of the Sänggunlang Beyan (nunicipal council) of Santa Grüz in 1976
He Later becane the Vice Mayor of Santa Cruz in 1988 was elected as the town's Mayor 1988 L...]
Case 2: The type-fliter of Focus and the limitations of uncertainty scores
Groundtruth
Taral Hicks is an American actress and singer, born on Saptember 21, 1974, in The Bronx, New York. [...] She later
transitionss acting, appearing in films such as Dranx Tale" (1993), "Just Cause" (1995), and "Delly" (1998). [.
Likelifiood
Taral Ricks g an AnerIcan Betress and singer born oî Septerber 21, 1974 I In The Bronx, New York [..] Shie Tater
transitiones
sc acting appearing in Films such as [A Branx Tale" (1993),
"Just Couse" (1995), and "Belly" (1998).
[...]
Focus
Taral Ricks is än Anerican Betress and singer "born on September 21, 1974
In The Broix, New York [..]] She Tater
transitioned to acting appearing in Films such as Branx Tale" (1993))
Couse" (1985), and "Belly" (1 (1998).
[...]
Case 3: Uncertainty propagation of Focus
Groundtruth
[...] Fernandinho began his professional career with Atletico Paranaense in Brazil before noving Ukrainian club Shakhtar Donetsk in 2085. [...] He is known for his physicality, tackling ability, and passing range, and 15 widely regarded as one of the best defénsive midflelders in the world.
Likelifiod
[..]] Fernand1řhō Bagan his proressional Career with Atletico Parañaense Brazi berore noving Ukraimfan club
Shakhtar Donetsk in 2005 [l [...J Hie is known for his physicality tackling ability and passing renge and
Widely regarded as one of the best defensive midfielders in the world
Focus
[...] Fernandinho began his professional career wi with Ktleticn Paranaense in Brazil before noving Ukrainian club
Shakhtar Donetsk in 2025 [...] He is known for his physicality tackling ability and passing range and is
widely regarded as one of the best defensive midfielders in the world
Table 5: We sampled instances from our datasel to demonstrate the differences across uncertainty scores. For label, entities colored in red indicate hallucination. For uncertainty scores, entities boxed in red with different tints represent the degree of uncertainty. A lighter (darker) box indicates a lower (higher) uncertainty.
The type-filter of Focus reduces FNR on name entities but sheds light on a bigger limitation of uncortainty-based hallucination score. The left and middle plots of Figure 5 reveal that Focus performs differently for named and non-named entities. It achieves a low FNR but high FPR for named entities, and the opposite for non-named ones. This is because Focus filters for named entities based on POS and NER tags. While promising since named entities often hallucinate (as shown in Figure 2) its high FPR suggests that its base score (the sum of Likelihood and Entropy) poorly distinguishes hallucinations, frequently assigning high uncertainty to named entities. Such high uncertainty on correct named entities often arises when the correct information can be expressed in multiple valid ways. For example, in the generation, She later transitioned lo acling, oppeuring in films such as 'A Brons Tale' (1993), 'Just Cause' (1995), and 'Belly' (1998)," the order of the listed films can vary without affecting factual accuracy. In this scenario, there are several plausible next tokens following "appearing in films such as," cach with relatively low predicted probability, resulting in high entropy despite the content being correct.
Similarly, in Figure 6, Focus shows varying patterns depending on the POS tags. For proper nouns, nouns, and numbers- tags often associated with named entitics Focus has a higher FPR and lower FNR. However, for verbs, auxiliaries, and adverbs, Focus exhibits lower FPR but higher FNR. This highlights a limitation of Focus: by concentrating primarily on named entities, it tends to overlook hallucinations in other types of tokens. The 2nd case in Table 5 illustrates this: Focus ignores function words like "is" and "to," " reducing FPR, but indiscriminately highlights named entities like "American" and "A Bronx Tale," " even when accurate.
Uncertainty propagation of Focus alleviates the over-confidence nature of LLMs. The right plots in Figure 5 show that LLMs are less confident when generating the first few words of a sentence and become over-confident as generation progresses, as indicated by decrease in FPR and an increase in FNR for Likelihood. This contrasts with the typical distribution of hallucinations, which occur mostly in the middle and end of sentences (Section 3.2). Focus addresses this by propagating uncertainty scores based on attention, leading to a decrease in FNR over positions. However, its FPR increase over positions suggests that using attention scores to propagate uncertainty may wrongly penalize entities that are not over-confident. The 3rd case in Table 5 illustrates this: Likelihood assigns higher uncertainty to early words (e.g., "Fernandinho
10

---

Published in Transactions on Machine Learning Research (9/2025)
Likelhood
SAR
Focus
Likelihood
SAR
Focus
1,0
0.6
2 0.4
0e 0.6
02
044
DaTE
PERSION
WORK OF AR7
CANIONAL
EvenT
Loc
cocreset
PRoDuCT
onHER
Bepiosing
02
enery
0.
FROPH
NGUN
ADP
VERS
DET
ADU
PROW
coond
408
NCM
ADV
OTHER
Position
NER
NER Tag
PDS Tag
1.0
1.-
08
0.
ENR
0.6
0.4
0.
Hame
DarE
PERTION
Wore op ARy
oPE
NORe
CARDIEAL
Evewr
Loc
ORDINALL
PRODUCT
OTHER
Brgiosing
0.0
PROPH
NOUN
ADP
VERG
GET
PROM
cooRd
AUX
NaM
ADV
OTHER
Powlion
NER Tag
NER Tag
POS Tag
Figure 5: PPR/FNTS of uncertainty scores across NER tags (left and middle) and sentence positions (right).
Figure FPR/ENR of each uncertainty score across POS tags.
b)gan") and lower scores to later words (e.g., "Shakhtar Donetsk in 2005"), while Focus detects hallucinations at sentence ends by linking them to prior hallucinated content (e.g., "Ukrainian club").
6 Discussion
Throughout this study, we identify the limitations and strengths of current uncertainty-based hallucination detection approaches. In this section, we discuss two directions to improve the performance of uncertainty- based hallucination detection.
Uncertainty score calibration. In Section 5.3, we show how simple calibrations (lype-filter and uncertainty propagation) help improve the performance Focus. These calibrations were invented through the linguistic analysis of LLM-generated corpus, indicating the relationship between the tendency of hallucination and linguistic properties. Based on this finding, we recommend exploring more linguistic properties that can help determine the importance of generating content and the tendency of hallucination. This exploration would not only improve the performance of hallucination detection, but also help mitigate hallucination during generalion.
Utilizing information beyond token probabilities to estimate uncertainty. One major limitation of uncertainty-but based hallucination detection its over-predictio prediction nature. As shown in Section 5.2, all five approaches perform poorly when hallucinations are sparse. In Section 5.3, we further show that such over-prediction frequently happens informative content, such name entities. This suggests that token probabilities are not well separated between hallucinated and non-hallucinated content, and using uncertainty scores like Likelihood or Entropy to serve as the base score of hallucination detection is not reliable. Hence, we recommend investigating more sophisticated uncertainty estimation or integrating probing techniques that utilize other information from LLM's internal states to increase the reliability of hallucination detection.
Conclusion and future work. In this work, we comprehensively explore the promise of entity-level hallucination detection by curating HALLUENTITY, dataset tailored for fine-grained understanding and introducing evaluation metrics for the task. We benchmark five uncortainty-based approaches, finding that they struggle to localize hallucinated conlent, raising concerns about their reliability. Our qualitative analysis highlights their strengths and weaknesses and suggests two directions for improvement. Future work should explore more sophisticated techniques for incorporating context-aware uncertainty estimation and develop methods that adaptively propagate uncertainty across sentence positions enhance hallucination localization.
11

---

Published in Transactions on Machine Learning Research (9/2025)
Limitations
Dataset constructions. Since curating a hallucination delection dataset from scratch is challenging, especially when annotating hallucinations the entity level, we build our dataset upon the FActScore dataset to reduce cost and enhance quality. However, this means dataset would inherit limitations from it. For example, the FActScore dataset focuses only on biography generation but omits other common tasks such QA and data analysis. In addition, since the FActScore dataset verify claims with Wikipedia, which does not contain all the information to verify biography, some factual claims may be missclassified as "non-support." During our annotation process, we observe cases where the FActScore label was "non-support" while GPT-4o annotated the corresponding entity as "True." For these conflict. cases, we manually verified them through Google search and updated the labels if needed.
Evaluation of hallucination detection approaches. In this paper, we focus evaluating uncertainty- based hallucination delection approaches, where the uncertainly scores are estimated by token probabilities. For other types of uncertainty estimation that measure the diversity across samples, such as Semantic Entropy, since they estimate uncertainty at the sample level and do not output scores for cach token or entity, they can not be evaluated on HALLUENTITY. Although this incompatibility limits the usage of HALLUENTITY, it also shows the limitation of sample-based approaches -they are hard pinpoint hallucinated content.
Ethical Statement
This research addresses the critical challenge of hallucination detection in LLMs to enhance their safe and responsible use across high-stakes domains. By exploring entity-level hallucination detection and evaluating uncertainty-bant methods, we aim to improve the precision and reliability of identifying factual inaccuracies in generated content.. HALLDENNTTY and evaluation metrics are intended solely for research purposes, ensuring no sensitive or personal information included. We acknowledge the limitations of current approaches and advocate for continued improvements to promote transparency, accuracy, and responsible AI development.
Acknowledgement
Research is supported in part by the AFOSR Young Investigator Program under award number FA9550-23-1 0184, National Science Foundation (NSF) Award No. IIS-2237037 and IIS-233 2331669, Office Naval Research under grant number N00014-23-1-2648 Schmidt Sciences Foundation, Open Philanthropy, Alfred P. Sloan Fellowship, and gifts from Google and Amazon.
References
01.AI. Yi:
Open foundation models by Ol.ai. arXiv preprint arXic:2403.0460 2024.
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, Shuaibin Li, Wei Li, Yining Li, Hongwei Liu, Jiangning Liu, Jiawei Hong, Kaiwen Liu, Kuikun Liu, Xiaoran Liu, Chengqi Lv, Haijun Lv, Kai Lv, Li Ma, Runyuan Ma, Zerun Ma, Wenchang Ning, Linke Ouyang, Jiantao Qiu, Yuan Qu, Fukai Shang, Yunfan Shao, Demin Song, Zifan Song, Zhihao Sui, Peng Sun, Yu Sun, Huanze Tang, Bin Wang, Guoteng Wang, Jiaqi Wang. Jiayu Wang, Rui Wang, Yudong Wang, Ziyi Wang, Xingjian Wei, Qizhen Weng, Fan Wu, Yingtong Xiong, Chao Xu, Ruiliang Xu, Hang Yan, Yirong Yan, Xiaogui Yang, Haochen Ye, Huaiyuan Ying, Jia Yu, Jing Yu, Yuhang Zang, Chuyu Zhang, Li Zhang, Pan Zhang, Peng Zhang, Ruijie Zhang, Shuo Zhang, Songyang Zhang, Wenjian Zhang, Wenwei Zhang, Xingcheng Zhang, Xinyue Zhang, Hui Zhao, Qian Zhao, Xiaomeng Zhao, Fengzhe Zhou, Zaida Zhou, Jingming Zhuo, Yicheng Zou, Xipeng Qiu, Yu Qiao, and Dahua Lin. Internhm2 technical report. ar preprint Xiw:22403. 17297, 2024.
12

---

Published in Transactions on Machine Learning Research (9/2025)
Shuyang Cao and Lu Wang. CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021.
Arie Cattan, Paul Roit, Shiyue Zhang, David Wan, Roee Aharoni, Idan Szpektor, Mohit Bansal, and Ido Dagan. Localizing factual inconsistencies in attributable text generation. arXiv preprint S...:2410.07433 2024.
Jifan Chen, Grace Kim, Aniruddh Sriram, Greg Durrett, and Eunsol Choi. Complex claim verification with evidence retrieved in the wild. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024a.
Xiaolan Chen, Jiayang Xiang, Shanfu Lu, Yexin Liu, Mingguang He, and Danli Shi. Evaluating large language models in medical applications: survey. arxiv preprint 2rXiv:2405.000 05.07468, 2024b.
Xuefeng Du, Chaowei Xiao, and Yixuan Li. Haloscope: Harnessing unlabeled Ilm generations for hallucination detection. In Advances in Newral Information Processing Systems, 2024.
Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024.
Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, and Maxim Panov. Fact-checking the output of large language models via token-level uncertainty quantification. In Findings of the Association for Computational Linguistics ACL 2024, 2024.
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia. Specia. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8, 2020.
Google DeepMind Gemma Team. Gemma 2: Improving open language models at a practical size. arXiv preprint ar orxxv22408.000 001188, 2024.
Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC: Large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations, 2024.
Nuno M. Guerreiro, Elena Voita, and André Martins. Looking for a needle in a haystack: comprehensive study of hallucinations in neural machine translation. In Proceedings of the 17th Conference of the European Chapler of the Association for Compulational Linguisties, 2023.
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastion Bubock, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi Li. Textbooks are all you need. arXiv preprint arXiw:2306. 11644, 2023.
Pengcheng He, Xiaodong Liu, Jianseng Gao, and Weizhu Chen. Deberta: Decodiing-enhanced bert with disentangled attention. In International Conference on Learning Representationss, 2021.
Matthew Honnibal and Ines Montani. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing, 2017.
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst., 2024. ISSN 1046-8188.
13

---

Published in Transactions on Machine Learning Research (9/2025)
Siqing Huo, Negar Arabzadeh, and Charles L. A. Clarke. Retrieving supporting evidence for Ilms generated answers. Xin preprint. ArXiv:230.. 13781, 2023.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b. Xixio preprint
(al:2310006822 2023.
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanma, Florian Bressand, Gianna Lengyel, Guillaune Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie- Anniee-Anee Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang. Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mixtral of experts. arXiv preprint arXiv:2401.04088 2024.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguislies (Volume 1: Long Papers), 2017.
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jarod Kaplan. Language models (mostly) know what they know. arXiv preprint ar rxXte:227705222) 2022.
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncer- tainty estimation in natural language generation. In The Eleventh International Conference on Learning Representations, 2023.
Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian- Yun Nie, and Ji-Rong Wen. HaluEval:: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Diyi Yang, Christopher Potts, Christopher D Manning, and James Y. Zou. Mapping the increasing use of LLMs in scientific papers. In First Conference on Language Modeling, 2024.
Chun-Hsien Lin and Pu-Jen Cheng. Legal documents drafting with fine-tuned pre-trained large language model. arXiv preprint ar arXin:2406.04 04200. 2024.
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022.
Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantification for black-box large language models. Transactions on Machine Learning Reseorch, 2024. ISSN 2835-8856.
Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, and Bill Dolan. A Loken-level reference-free hallucination detection benchmark for free-form text generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zeullemoeer and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint trXiv:1907.11699 2019.
AT @ Meta Llama Team. The llama herd of models. Xrxiv preprint 2771222407.21783, 2024.
14

---

Published in Transactions on Machine Learning Research (9/2025)
Junliang Luo, Tianyu Li, Di Wu, Michael R. M. Jenkin, Steve Liu, and Gregory Dudck. Hallucination detection and hallucination mitigation: An investigation. arXiv preprinl axXo:2400.088558, 2024.
Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction. In International Conference on Learning Representalions, 2021.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-lau Yih, Pang Koh, Mohit Tyyer, Luke Zet Zee and Hannaneh Hajishirzi. FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
Abdurrahman Odabaşı and Göksel Biricik. Unraveling the capabilities of language models in news summa- rization. arXiv preprint 1811:2200111118,, 2025.
OpenAL. Chatgpt. https://chat..opeaal com// 2023. Large language model.
OpenAL. Gpt-40 system card. arXiv preprint arXiv:2410.21270 2024.
Seongheon Park, Xuefeng Du, Min-Hsuan Yeh, Haobo Wang, and Yixuan Li. How to steer Ilm latents for hallucination detection? In International Conference on Machine Learning, 2025.
Qwen. Qwen2.5 technical report. arXiv preprint arXfu:2412.15115, 2025.
Siva Reddy, Danqi Chen, and Christopher D. Manning. CoQA: A conversational question answering challenge. Transactions of the Association for Compulational Linguistics, 7, 2019.
Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using Siamese BERT-networkss In Proceedings of the 2019 Conference on Empirical Methods in Natural Lenguage Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)), 2019.
Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter J Liu. Out-of-distribution detection and selective generation for conditional language models. In The Eleventh International Conference on Learning Representations, 2023.
Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, and Artem Shelmanov. Hybrid uncertainty quantification for selective text classification in ambiguous tasks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), 2023.
Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, and Ce Zhang. Redpajama: an open dataset for training large language models. NewrIPS Dalasets and Benchmarks Track; 2024.
Ziwei Xu, Sanjay Jain, and Mohan S. Kankanhalli. Hallucination is inevitable: An innate limitation of large language models. arXiv preprint vrXiw:2401.18811 2024.
KiYoon Yoo, Jangho Kim, Jiho Jang, and Nojun Kwak. Detection of adversarial examples in text classification: Benchmark and baseline via robust density estimation. In Findings of the Association for Computational Linguistics: ACL 2022, 2022.
Bo-Wen Zhang, Liangdong Wang, Jijie Li, Shuhao Gu, Xinya Wu, Zhengduo Zhang, Boyan Gao, Yulong Ao, and Guang Liu. Aquila2 technical report. arXio preprint arXiv:2408.. 07410, 2024.
Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, and Luoyi Fu. Enhancing uncertainty-base hallucination detoction with stronger focus. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023a.
Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, and Luoyi Fu. Enhancing uncertainty- based hallucination detection with stronger focus. In Proceedings of the 2023 Conference Empiriaal Methods in Natural Language Processing, 2023b.
15

---

Published in Transactions on Machine Learning Research (9/2025)
A Details of Dataset Construction
Selection of data. From the FACTSCORE dataset, we select the set of biographies generated by ChatGPT to construct our entity-level hallucination detection dataset. The set of ChaiGPT-genneatedd biographies contains 183 samples. We filter out those that ChatGPT refuses to answer and end up with 157 instances.
Data labeling process. For each sample, FACTSCORE provides list of atomic facts short sentences conveying single pieces of information. These facts are labeled as Supported, Not-supported, or Irrelevant, where Irrelevant mcans the fact is unrelated to the prompt (í.e. a person's name), and Supported and Not-supported ted indicate whether the fact is supported by the person's 's Wikipedia page. Since only 8.3% of facts are labeled as Irrelevant, and most are related to Not-supported facts, we simplify the entity-level labeling procoss by merging both as False, treating only Supported facts as True
To assign entity-level labels, we first tokenize the biography into individual words. We then use the list of atomic facts to group words into meaningful units (entities) and assign labels based on fact types. Specifically, for atomic facts that share a similar sentence structure (e.g., "He was born on Mach 9, 1941." (True) and "He was born in Ramos Mejia." (False)), we label differing entities first-assis assigning True to "Mach 9, 1941" and False to "Ramos Mejia.". For those entities that are the same across atomic facts (e.g., "was born") or are neutral (e.g., "he," " "in," and "on"), we label them as True. In cases where no similar atomic fact exists, we identify the most informative entities in the sentence, label them based on the atomic fact, and treat the remaining entities True. Note that we assumee the pronouns in generations are always correct, in biography generation, they usually refer to the person specified in the prompt. When a claim is non-supported, we consider other information related to the pronouns as a hallucination.
GPT-40 prompt for data labeling. To scale the labeling process, we use GPT-4o to automatically identify and label entities with a few-shot prompt, as shown in Table 6. The system prompt includes detailed instructions on the labeling process, along with two manually created examples. In the user prompt, we maintain the same structured format used in the examples, inputting the biography and the corresponding list of atomic facts.
Data quality assessment. We performed three run annotations for a random subset of samples. We then compute the Jaccard similarity across the three entity sets. The similarity score is 0.813, suggesting a reasonably high similarity. We further compute Fleiss's Kappa annotatoo agreement entities' label (True/False) where the entities are the same across three runs. The Fleiss's Kappa across the three runs is 0.984, suggesting near-perfect agreement.
In addition to multiple-run annotation, we also conduct human verification to assess the annotation quality. The verification was conducted by the authors of the paper, who are well familiar with hallucination detection and LLM. To verify the GPT-4o annotation, we provided the generated biography and original FActScore annotation as references. The verification includes two parts: 1) whether a sentence is segmented into reasonable entities. For example, "December 18, 1949" should be one entity instead of three. 2) whether the label of an entity aligns with the original FActScore annotation. For example, given an original annotation, "Lanny Flaherty was born on December 18, 1949 (non-support)." the entity, "December 18, 1949," " should be labeled as Falsc
The authors verified the whole dataset independently. When a discrepancy between the verification results exists, the annotators would discuss it to reach a final consensus. The Jaccard similarity between the GPT-4o annotation and the annotation after verification is 0.945, and the Cohen's 's Kappa inner-annotaoor agreement entities' label (True/False, where the entities are the same before/ eefore/after /ater verification is 0.940. This result suggests that the GPT-4o annotation is reliable.
B Proxy LLM Setting
Our experiments are conducted under the proxy LLM setting, i.e., using different LLMs for hallucination detection and for response generation. This setting assumes the generator and detector LLMs are sufficiently
16

---

Published in Transactions on Machine Learning Research (9/2025)
System prompt
You helpful and precise assistant for segmenting and labeling sentences. We would like request your help curating dataset for entity-level hallucination detection.
We will give you a machine generated biography and a list of checked facts about the biography. Each fact consists of a sentence and a label (True/False). Please
following process. First, breaking down the biography into words. Second, by referring to the provided merging some broken down words in the previous step to form meaningful entities. For example, "strategic thinking" should be one entity instead of two. Third. according to the labels in the list of facts, labeling each entity as True or False. Specifically, for facts that share a similar sentence structure (e.g. "He born Mach 9, 1941." (True) and "He was born Ramos Mejia." (False)), please first assign labels entities that differ across atomic facts. For example, first labeling "Mach 9, 1941" (True) and "Ramos Mejia" (False) in the above case. For those entities that are the neutral (e.g., "he," "in," and "on"), please label them
facts,
same across atomic facts (e.g., "was born") atomic fact that shares the same sentence structure, please identify the most informative entities in the sentence and label them with the same label as the atomic fact while treating the rest of the entities as True, the end, output the entities and labels the following format: Entity 1 (Label 1)
For the cases that there is
Entity 2 (Label 2)
Entity (Label N)
Here are two examples:
[Example 1]
[he start of the biography|
Marianne McAndrew an American actress and singer, born on November 21, 1942, in Cleveland, Ohio. She began her acting career in the late 1960s, appearing various television shows and films.
[The end of the biography]
[The start of the list of checked facts]
[Marianne McAndrew is an American. (False): Marianne McAndrew is an actress. (True); Marianne McAndrew is singer. (False): Marlanne McAndrew born November 21, 1942. (False): Marianne McAndrew was born Cleveland, Ohio. (False): She began her acting career in the late 1960s. (True): She has appeared in various television shows. (True): She has appeared in various films. (True)] [The end of the list of checked facts]
[Te start of the ideal output]
[Marianne McAndrew (True); is (True): an (True); American (False): actress (True); and (True); singer (False); (True): born (True): (True): November 21, 1942 (False): , (True): in (True): Cleveland. Ohio (False): . (True): She (True): began (True): her (True): acting career (True): in (True): the late 1960s (True): (True); appearing (True); in (True): various (Truee; television shows (True): and (True); films (True): (True)] [The end of the ideal output]
[Example 2]
Te start of the biography]
Doug Sheehan is an American actor who was born on April 27, 1949, in Santa Monica, California, He is best known for his roles in soap operas, including his portrayal of Joe Kelly on "General Hospital" and Ben Gibson on "Knots Landing." [The end of the biography]
The start of the list checked facts] (oug Sheehan is an American. (True); Doug Sheehan is an actor. (True): Doug Sheehan was born on April 27, 1949. (True); Doug Sheehan was born Santa Monica, California. (False): He best known for his roles in soap operas. (True): portrayed Joe Kelly. ("True); Joe Kelly was General Hospital. (True); General Hospital soap opera. (True): He portrayed Ben Gibson. (True): Ben Gibson was Knots Landing, (True): Knots Landing is soap opera. (True)]
[The end of the list of checked facts]
TTe start of the ideal output]
[Doug Sheehan (True): is (True): an (True): American (True): actor (True): who (True): was born (True); on (True): April 27, 1949 (True); in (TTuee:: Santa Monica, California (False); (Truee:; He (True): (True); best known (True); (True): his rales soap operas (True); (True): including (True): in (True); his portrayal (True); of (True); Joe Kelly (True): on (True); "General Hospital" (True); and (True): Ben Gibson (True): on (True); "Knots Landing." (True)]
[The end of the ideal output]
User prompt
The start of the biography] (BIOGRAPHY)
The end of the biography]
[Th start of the list of checked facts] (LIST CHECKED FACTS)
[hhe end of the list of checked facts]
Table 6: GPT-40 prompt for labeling hallucinated entities.
similar in terms of training data and architecture. These assumptions are reasonable in practice. For archi- tectural similarity, most large language models today adopt the transformer-basoe decoder-only architecture, following the GPT paradigm. This includes both the target models we evaluate and the proxy models used for detection. Therefore, the structural inductive biases are highly aligned between the generator and the
17

---

Published in Transactions on Machine Learning Research (9/2025)
Proxy model
Llama3-SB
Llama3.1-8B
IntemLM2-7B
Qwen2.5-7B
Phi2
Mistral-7B
Gemma2-99
Target model
Llama3-8B
1.000
0.978
0.896
0.896
0.854
0.916
0.925
Mistral-7B
0.908
0.907
0.892
0.865
0.829
1.000
0.923
Gemma2-91
0.931
0.930
0.897
0.897
0.849
0.929
1.000
Table 7: Pearson's correlation of log- log-likelihood across models.
Proxy model
Llama3-8B
Llama3.1-8B
Internl.2-7BB
Qwen2.5-7B
Phi2
Mistral-7B
Gemma2-9B
Target model-
Llama3-88 8B
1.000
0.982
0.906
0.893
0.839
0.922
0.925
Mistral-7B
0.923
0.925
0.891
0.884
0.800
1.000
0,932
Gemma2-9h
0.941
0.938
0.901
0.911
0.847
0.941
1.000
Table 8: Pearson's correlation of entropy across models.
proxy. In addition, for training data similarity, the large-scale pretraining corpora used by many open-source LLMs show strong overlap, often including subsets of Common Crawl, Wikipedia, GitHub, and other widely shared internet-sourced datasets. While we may not have access to the exact training sets of closed models, this convergence in training data sources leads to similar token-level uncertainty patterns, especially on general-domain QA tasks.
Empirical validation. To validate this hypothesis more rigorously, we compute Pearson's correlation between uncertainty scores produced by the target model and those produced by proxy model, across multiple uncertainty scores (e.g., Likelihood and Entropy). Specifically, open-sourced LLM (e.g., Llama3-8B) and compute uncertainly scores using the same model as well as other proxy models. We observe consistently high Pearson's correlations in Table 7 and 8. This confirms that proxy uncertainty is a faithful surrogate.
generate biographies with an
18


Reviews:
[{"review_id": "Y1ngulEivn", "reviewer": "Reviewer_cHGr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Introduces the first large-scale, entity-level hallucination detection dataset with real LLM generations.\n2. Evaluates 5 distinct uncertainty-based methods over 17 LLMs, analyzing the effect of model family and size.\n3. The paper includes detailed qualitative and quantitative analysis.\n\nWeaknesses\n1. The paper only focuses on the performance of biography.\n2. While efficient, the annotation introduces the risk of propagation of GPT-4o’s own hallucination tendencies or missegmentations. There are no multi-agreements among multiple annotators (LLMs) or multiple runs of the same annotator (LLM).\n3. Also, the entity labeling process is derived from the atomic fact labels in the FACTSCORE dataset. Any biases or errors in the original FACTSCORE annotations would propagate into HALLUENTITY.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces HalluEntity, a novel dataset with 18,785 entity-level annotations based on biographies generated by ChatGPT, enriched with labels derived from atomic facts. It focuses on the challenge of fine-grained hallucination detection in large language models (LLMs) outputs. They evaluate five uncertainty-based methods—Likelihood, Entropy, CCP, SAR, and Focus—across 17 modern LLMs. The findings reveal that token-level uncertainty measures tend to over-predict hallucinations, while context-aware approaches (SAR, Focus) perform better but still struggle to maintain balanced precision and recall, particularly in low-hallucination scenarios.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please ensure the annotation process has multiple runs from the same or multiple annotators (LLMs).\n2. Please list inter-annotator agreements among multiple annotators (LLMs).\n3. Is there any relation between FPR/FNR and Entropy?\n4. The study notes that some scores assign high uncertainty to informative content like named entities, leading to false positives. What makes them \"uncertain\" to the model, even when correct? Is it low frequency in the training data, or something more complex?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "C4JYm7zmhp", "reviewer": "Reviewer_XtDP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**: This work conducts extensive experiments across 17 large language models and five uncertainty-based hallucination detection methods. The authors provide both quantitative and qualitative analyses, offering valuable insights into model behavior and hallucination patterns.\n\n**Weaknesses**: The dataset construction process is a little bit unclear, particularly how the labels for entities are obtained. The dataset's quality is not empirically validated, as the labeling is solely performed by GPT-4o. While the authors mention manual verification, the results of this process (e.g., user testing) are not presented in the paper. Finally, the study focuses exclusively on uncertainty-based hallucination detection methods, which limits the scope of the investigation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work introduces a novel dataset for entity-level hallucination detection. Leveraging this dataset, the authors evaluate 17 different large language models and explore 5 uncertainty scoring methods to assess their effectiveness in identifying hallucinated entities. The paper presents both quantitative and qualitative analyses of the experimental results, offering insights into the relationship between hallucination tendencies and various linguistic properties.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. **Clarify the entity-level labeling pipeline**: \nI am unclear about the process of generating labels for entities in Section 3.1. My understanding is that the authors use FActScore to extract facts from the original text, assign each fact a ‘True’ or ‘False’ label, and then propagate those labels to entities within the facts. I have several questions about this process:\n- In section 3.1, why is the fact “He was born on March 9, 1941” labeled as True while “He was born in Ramos Mejía” is labeled as False? Both seem supported by the original text. If the latter reflects hallucinated content, perhaps highlighting it in Figure 1 would clarify matters for readers. As presented, both facts appear correctly generated and factually plausible at first glance. Please also explain how fact labels are determined in the main text.\n- The authors state that “*neutral words like ‘he’ and ‘on’ are considered non-hallucinated*,” but this may have limitations. For example, pronouns such as ‘he’ could themselves be hallucinated in gender-sensitive contexts (e.g., referring to a female subject as ‘he’ or ‘Mr.’). It would be helpful to discuss this potential limitation.\n\n2. In second paragraph of Section 3.1, \"*we leverage biographies generated by ChatGPT (OpenAI, 2023).*\" For reproducibility, please specify which GPT model was used (e.g., GPT-4, GPT-3.5).\n3. At the end of Section 3.1, the authors mention that you “*manually verify and refine to ensure correctness.*” **Could you elaborate on this verification process**? For instance:\n- Who conducted the manual verification (e.g., recruited annotators or expert reviewers)?\n- What guidelines or criteria were applied?\n- Were any quantitative metrics recorded (e.g., correctness rate or inter-annotator agreement)?\n4. I feel the dataset’s quality depends heavily on FActScore’s fact-checking and on Wikipedia’s coverage. Is FActScore’s knowledge updated to reflect current information? Additionally, since Wikipedia may not cover every relevant fact, how do you address potential false negatives? I would appreciate the authors’ perspective on these points.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "w9VzIa3HjS", "reviewer": "Reviewer_3S6k", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n-----------\n\n* The study is thorough in its coverage of token-level uncertainty measures and language models. \n* The paper presents a lot of very interesting, additional experiments. From the composition of their dataset, to performances as grouped by uncertainty method, model size, and hallucination rate. The FPR and FNR analyses of the results for SAR, likelihood and Focus are also intriguing.\n\nWeaknesses\n---------------\n\n* For me, there is one glaring issue with this paper: For other types of hallucination detection, it makes sense to evaluate an external detector applied to some target model (in this case ChatGPT). The reason why uncertainty *might* be a good indicator for hallucinations is that it should be high when the model struggles to process ambiguous data, or generates plausible but unfounded claims that do not have any grounding in the training data. However, this study looks at the uncertainty of an external model to detect hallucinations in a target model. I can see that sometimes they might coincide, but especially for smaller models from completely different families this seems very questionable. I feel this study unfortunately does not possess much cogency unless it applies similar methods to the token-level uncertainties of the target model, and then potentially shows a relationship between uncertainties of the target and external models. I think that releasing the HalluEntity dataset together with token probability distributions and hidden representations of the target model would greatly improve their contribution.\n* Other minor weaknesses are elaborated on in the next section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors present a study on whether token-level uncertainty scores can detect hallucinated entities.\nFor this purpose, they create a dataset, HalluEntity, that is obtained by asking ChatGPT to write biographies about different people, which are split into verifiable facts that are manually verified.\nThe authors then propose a first analysis on the hallucination rates per PoS / NER tag.\nAfterwards, they test 5 token-level uncertainty quantification method (likelihood, entropy, claim-conditioned probability, shifting attention to relevance, and focus), across 17 different LLMs.\nThey follow-up by detailed analyses on the effect of false-positive and false-negative rates per NER tag and sentence position.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "What follows is an unordered list of requested changes / additional questions:\n\n* In the second paragraph on page 2, \\citep should be used for the citation.\n* The vertical spacing underneath FIgure 1 seems to be a bit off.\n* I personally found the ratio of hallucinations in Figure 2 with the secondary y-axis a bit confusing - maybe this could be replaced by a percentage value inside the bars of the stacked bar plot?\n* I would like to ask the authors to specify the exact version of ChatGPT they used to generate biographies.\n* The notation in section 4 appears a little cluttered; First of all, defining a sentence x with tokens v_i seems a bit unintuitive, but importantly clashes with the notation of the uncertainty metrics (e.g. likelihood should be -log p(v_i|v_{<i}) then). Secondly, it also feels unintuitive that a sentence x could be rewritten as a series of entities, rather I would define a set of entities belonging to a sentence x. Token-level halucination scores have a superscript t supposedly to distinguish them from the entitiy level hallucination scores. But t is easy to confuse for the timestep since sentences have length T. It might be easier to simply drop the superscript t for $y_t$. \n* For SAR, it looks the like the tilde for the relevance weight is over $R_T$ instead of just $R$.\n* For Focus,  the indicator function can be inferred from the context, but is notation is not introduced.", "broader_impact_concerns": "Any concerns are addressed in the Ethical Statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)