Evaluating Evaluation Metrics – The Mirage of Hallucination Detection (2025)

Abstract: Hallucinations pose a significant obstacle to the reliability and widespread adoption of language models, yet their accurate measurement remains a persistent challenge. While many task- and domain-specific metrics have been proposed to assess faithfulness and factuality concerns, the robustness and generalization of these metrics are still untested. In this paper, we conduct a large-scale empirical evaluation of 6 diverse sets of hallucination detection metrics across 4 datasets, 37 language models from 5 families, and 5 decoding methods. Our extensive investigation reveals concerning gaps in current hallucination evaluation: metrics often fail to align with human judgments, take an overtly myopic view of the problem, and show inconsistent gains with parameter scaling. Encouragingly, LLM-based evaluation, particularly with GPT-4, yields the best overall results, and mode-seeking decoding methods seem to reduce hallucinations, especially in knowledge-grounded settings. These findings underscore the need for more robust metrics to understand and quantify hallucinations, and better strategies to mitigate them.

## introduction hallucinations in language model generations are detrimental and, unfortunately, a pervasive phenomenon @xcite @xcite . as language models are rapidly adopted across various settings, addressing hallucinations has become a key research focus @xcite @xcite . however, before investing time and resources into devising its mitigation techniques, it is worthwhile to take a step back and ask: 1) are the existing metrics truly capturing the hallucinations effectively? 2) do these metrics generalize across different datasets, decoding * work done during internship at apple inc. the term 'hallucination' covers a spectrum of generation errors. in this work, we focus on its two most common manifestations: poor faithfulness and factuality, particularly in knowledge-grounded dialog @xcite and question-answering @xcite . faithfulness measures the consistency and truthfulness with the provided knowledge source, while factuality pertains to the accuracy wrt real-world facts or widely accepted knowledge @xcite . measuring these constructs is no simple task. in some cases, simple syntactic @xcite or sematic @xcite overlap with the input knowledge can provide an easy estimate of faithfulness. whereas other times, one has to resort to customtrained models @xcite , multi-step question answering pipelines @xcite , or llm-based evaluation @xcite . interestingly, while recent surveys have extensively explored the causes and mitigation techniques for hallucinations in language models @xcite @xcite @xcite , none have directly called into question the generalization capabilities of existing metrics. thus, in this work, we attempt to fill this gap, and conduct a rigorous empirical investigation of contemporary hallucination detection metrics. our study examines the above mentioned diverse sets of metrics from various perspectives -consistency, alignment with human judgments, variation across decoding methods, impact of post-training, and the effect of parameter scaling. our findings reveal that most metrics have limited inter-correlation and fail to consistently align with the human notion of hallucination. they seem to have a limited understanding of the problem, as they fail to generalize across datasets. anticlimactically, these metrics do not show a clear monotonic improvement with an increase in model size. on a positive note, we find that llm-based evaluation, particularly with gpt-4, offers the most reliable detection across diverse tasks and datasets. additionally, an ensemble of metrics also seems to be a good choice. instruction-tuning and mode-seeking decoding methods are also shown to reduce hallucinations. we thus find that detecting hallucination does not have a one-size-fits-all solution, as existing metrics fall short of capturing its full spectrum. language models. our study includes five llm families: opt @xcite , llama @xcite , olmo @xcite @xcite @xcite . we cover models ranging from 125m to 70b, including their instruction-tuned versions, totaling 37 models. evaluation spans five decoding methods of greedy, beam search (graves, 2012), ancestral, top-k @xcite , and top-p sampling @xcite . ## failure modes of the metrics to better understand the limitations and bottlenecks of evaluation metrics, we analyze how their performance varies with respect to instance difficulty. while 'difficulty' can be quantified in various ways, it is well-established that llms tend to struggle and exhibit increased hallucination rates, with longer responses @xcite . thus, we investigate the potency of different metrics across varying response lengths. specifically, we calculate the accuracy of each metric for different response lengths, as shown in figure 5 . appendix c contains additional error analysis and discussion. unieval suite's consistency performs reasonably well on longer responses in the begin cmu and begin tc datasets, where accuracy rises steadily with response length -reaching values above 0.9. however, its performance on begin wow starts quite low (0.4 in short completions), and only becomes competitive with length. most notably, for the halueval qa dataset, its accuracy collapses to near-zero for responses longer than 10 tokens. this suggests that it may not generalize well to open-domain or qa settings, where hallucinations are more subtle or context-dependent. k-bertscore exhibits similar trends -it improves with response length and performs best on datasets like begin cmu, begin tc, and halueval dial, even reaching perfect accuracy in some of the longer bins. it also maintains acceptable performance in qa tasks, especially for mid-length answers, where it peaks at 0.95. however, its reliability drops for very short outputs. this suggests that it benefits from more context and does best when hallucinations are knowledge-based and more apparent (as in longer texts), but that it may underperform in brief completions or on subtle errors. q 2 nli stands out in the qa task. it consistently achieves high accuracy (≥ 0.85), going up to 1.0 in longer bins. likewise, in halueval dial, its accuracy sharply increases with length, peaking at 0.95 in the longest bin. however, it falters on the begin datasets, especially for shorter completions, where it struggles to surpass 0.4. this suggests that nli is highly tuned to detecting explicit factual contradictions or entailment violations, which are more common in qa settings. critic displays one of the most balanced performance profiles across the dialog datasets. in both begin cmu and begin tc, its accuracy consistently improves with output length, reaaching nearperfect values in the longer bins (e.g., 0.97). it is also strong in wow and halueval dial, @xmath0.8 accuracy for nearly all lengths. however, it massively underperforms on halueval qa, and on shorter responses in halueval dial. this shows that critic is well-suited for dialogues and is relatively robust to variation in output length, but might not be the best choice for qa. gpt-4 performs strongly overall, especially on longer outputs. for most datasets, it achieves @xmath1.75 across different response lengths, often peaking above 0.9. however, for begin tc, its accuracy is unexpectedly lower (∼ 0.65) for midlength completions. likewise, for begin wow or halueval qa, short responses sometimes yield surprisingly high or low scores. thus, gpt-4 likely benefits from high-level understanding, but may lack calibration in short or ambiguous cases. ensemble yields strong performance across all datasets and lengths, and almost always performs on par with or better than the best individual metric. it reaches near-perfect accuracy for halueval qa. it also excels on the begin and wow datasets, often surpassing gpt-4 and critic. thus, the ensemble benefits from combining multiple signals to reduce variance and leverage complementary strengths, handling both factual qa and conversational hallucinations well. ## conclusion hallucination detection in llm-generated text is a tricky task. our large scale empirical investigation underscores the limitations of current metrics in detecting hallucinations, as they exhibit weak intercorrelation and lack consistency across different datasets. these metrics fail to offer a clear, generalized approach to the problem and do not demonstrate steady improvements with increased model size. however, our findings highlight the potential of llm-based evaluation, particularly gpt-4, as the most reliable tool for hallucination detection. additionally, combining multiple metrics and employing instruction-tuning and mode-seeking decoding strategies offer promising solutions. ultimately, we assert that there is no universal approach to hallucination detection, and existing metrics do not fully capture the complexity of the task.