Do My Eyes Deceive Me? A Survey of Human Evaluations of Hallucinations in NLG (2025)

Abstract: Hallucinations are one of the most pressing challenges for large language models (LLMs). While numerous methods have been proposed to detect and mitigate them automatically, human evaluation continues to serve as the gold standard. However, these human evaluations of hallucinations show substantial variation in definitions, terminology, and evaluation practices. In this paper, we survey 64 studies involving human evaluation of hallucination published between 2019 and 2024, to investigate how hallucinations are currently defined and assessed. Our analysis reveals a lack of consistency in definitions and exposes several concerning methodological shortcomings. Crucial details, such as evaluation guidelines, user interface design, inter-annotator agreement metrics, and annotator demographics, are frequently under-reported or omitted altogether.

## introduction the popularity of large language models (llms) has led to an increase in human evaluations assessing the degree to which a model's outputs diverge from its inputs -in other words, the number of hallucinations or confabulations generated by the given language model. this is also reflected in the increased number of papers covering the topic (figure 1 ). human evaluations are commonly viewed as the more reliable way to evaluate natural language generation (nlg) systems (in contrast to, e.g., using automatic metrics). following on from recent nlp surveys that have looked at various human and automatic evaluation practices @xcite @xcite @xcite , this paper takes a more focused look at the challenge of evaluating the faithfulness of output from llms. we build on corresponding author: schmidtova@ufal.mff.cuni.cz figure 1 : there is an exponentially growing trend of papers concerned with hallucination, both in absolute and in relative terms. the numeric values represented by this chart can be found in hallucinations the term hallucination is used in diverse and sometimes conflicting ways across the literature, making it difficult to assess model performance in a systematic way @xcite . recent works anchored in factuality frame hallucinations as content that fails to align with accepted truths. @xcite treat such outputs as content inconsistent with world knowledge or the user-provided context. @xcite highlight wholly fabricated or misleading details with respect to world knowledge. @xcite describe the phenomenon broadly as the generation of ungrounded, factually erroneous text across varied domains. @xcite similarly emphasise outputs that appear correct but are not grounded in fact. a complementary line of research emphasises faithfulness to the source input. @xcite characterise hallucination as text that diverges from the input, distinguishing intrinsic (contradicts the source) from extrinsic (unverifiable) cases, and contrasting the notions of faithfulness and factuality. @xcite extend this stance with a finegrained taxonomy covering entity-and relationlevel errors, factual fabrication, overclaims, and instruction, context, and logical inconsistencies. other studies focus on grounding in the model's own discourse. @xcite distinguish input-, context-, and fact-conflicting hallucinations, while rawte et al. (2023a) differentiate factual mirage (distortions based on an otherwise correct prompt) from silver lining (elaborate narratives generated from an incorrect prompt). @xcite similarly move away from external verification, framing hallucinations as meaning-level deviations where fluent outputs misinterpret or distort the intended content. these definitions vary along three principal axes: (i) the grounding criterion (input context, external knowledge, or self-consistency), (ii) the verifiability standard (direct contradiction vs. unverifiability), and (iii) the granularity of error types (binary vs. multi-class taxonomies). this heterogeneity makes it difficult to achieve reproducible evaluation and impedes the development of comparable metrics. we discuss the prevalence of these different definition types in our surveyed papers in sections 4 and 5. meta-evaluations in nlg over the past two decades, the nlg community has increasingly recognised inconsistencies in the evaluation of generated text. @xcite analysed 165 nlg papers employing human evaluation, documented the diversity of quality dimensions, and introduced standardised evaluation sheets and definitions to enhance consistency. additionally, @xcite proposed an 18-property classification for human evaluation methods in nlg to support comparability, meta-evaluation, and reproducibility testing. gehrmann et al. (2023) subsequently reviewed two decades of human and automatic evaluation practices, assessed the extent to which 66 contemporary studies adhered to recommended guidelines, and proposed concrete reporting standards and template evaluation reports to strengthen methodological rigour. addressing the quality of the studies, @xcite found that only 30% of nlg papers release human evaluation guidelines, and 77% of those contain vulnerabilities. we present similar findings, with a more specific focus on human evaluation of hallucinations, offering a deeper analysis within this scope. ## discussion human evaluation shows absolute growth, relative decline we analyzed 1,405 papers that mention hallucinations, tracking over time how many also discuss human evaluation, both in absolute numbers and relative proportions. figure 10 shows that while the absolute number of hallucination papers mentioning human evaluation continues to grow, the proportion of such papers is declining dramatically. this decline is indicative of a rapid overall increase in hallucination research outpacing the use of human evaluation methods. we offer our hypothesis of possible factors that could have contributed to this trend. first, running a properly designed human evaluation requires a considerable amount of effort @xcite . as the absence of human insight into the model's errors is becoming standard, with less than 50% of hallucination papers providing it, many au- figure 10 : the count of hallucination papers that use human evaluation is growing (solid blue line). however, their relative proportion is falling, i.e., a larger % of papers do not consider human evaluation (dashed blue line). we observe a similar trend in the abstracts (yellow), where a lower proportion of authors deem human evaluation important enough to mention. this information is also available as second, llm-as-a-judge evaluations @xcite have emerged as an alternative to human evaluation due to their lower cost. this approach, made popular in machine translation evaluation @xcite , has grown in the hallucination literature from just one paper in 2023 to 40 papers (5% of hallucination papers) in 2024. while we cannot establish a direct causal relationship, this growth coincides with the relative decline in human evaluation usage. third, researchers may increasingly substitute benchmark evaluation for human evaluation, viewing automated metrics on standard datasets as sufficient. we discuss why this practice is problematic in our next point. robust human evaluation methodologies matter as in the end "a careful and well-designed human evaluation is usually the best way to meaningfully evaluate an nlg system" (reiter, 2024). the continued importance of human evaluation despite trends toward automated evaluation, human judgment remains essential for hallucination assessment. human evaluation provides irreplaceable insights that automated approaches cannot capture: humans can apply specialized do-main expertise (e.g., medical, legal knowledge), assess contextual appropriateness and cultural sensitivity, and evaluate real-world applicability from an end-user perspective @xcite . moreover, human evaluation often reveals novel failure modes and error patterns not captured by predefined metrics, which is crucial for understanding model limitations and improving systems. the persistent misalignment between automatic metrics and human judgment @xcite has intensified as evaluation tasks have grown more complex -hallucinations require nuanced assessment that goes beyond simpler, previously evaluated factors @xcite . current automated alternatives suffer from significant limitations: benchmarks are plagued by llm training data contamination @xcite , dataset errors @xcite , and poor real-world applicability @xcite , while llm-as-a-judge approaches require human validation for new datasets or tasks @xcite and may inherit training data biases. finally, developing better automated metrics itself requires human-annotated gold standards, and certain domains demand human validation for safety or regulatory compliance. hallucination definition hallucination definitions vary greatly in the papers we reviewed. some authors used standardised definitions previously published in the literature, while others grounded their definitions in the specific context of the task, such as medical decision-making. for instance, foot_2 one paper defined hallucination in the context of clinical safety as: factual accuracy, specifically looking for missing or incorrect information that could lead to errors in medical treatment after discharge however, we also found vague or difficult-tointerpret definitions, such as: hallucinations -0: no stuff that is not factual. there is also an additional need for researchers to be more proactive in releasing research data. only 20 papers (43%) in our survey have actually released any annotation data. model outputs with error annotations would not only be useful for further error analysis, but also for the development and improvement of new automatic evaluation methods, such as comet @xcite was developed for machine translation. the issues observed in our results are not new issues. the same lack of reporting was observed by @xcite in their survey of nlg human evaluations. however, it is disappointing, but not surprising, that no progress on this front has been made over the past five years. are responsible nlp checklists used responsibly? our findings reveal a troubling inconsistency between the formal requirements of the responsible nlp checklist and actual reporting practices in papers published at premier nlp venues. despite the checklist being integrated into reviewer guidelines and made mandatory for submissions to the acl rolling review since naacl 2022, a significant proportion of papers failed to report critical information about human evaluation practices. this pattern is surprising when contrasted with papers from inlg, a venue not formally bound by the checklist, which nonetheless reported these attributes more consistently. this discrepancy raises concerns about the seriousness with which authors and reviewers are taking the responsible nlp checklist. it suggests that mere formal inclusion of reporting guidelines may not be sufficient; instead, stronger enforcement via higher peer review quality may be necessary to ensure broader compliance and more transparent reporting of information. the checklist itself may need simplification in order to increase its practical adoption. ## recommendations defining clear hallucination definitions authors should use standardised definitions from existing literature whenever possible to promote consistency and facilitate comparison. definitions must be clear, unambiguous, and should include concrete examples. in order to avoid ambiguity, authors should explicitly specify the grounding criterion, verifiability standard, and granularity of error types, as discussed in section 2. when introducing new definitions, authors should justify why existing definitions are inadequate and pilot them with annotators to confirm understanding. inter-annotator agreement reporting authors should consistently report iaa results from their human evaluations, as this information is cru-cial for assessing the reliability and reproducibility of findings. in addition to reporting iaa scores, authors should specify the quality assurance steps taken, in order to provide proper context for interpreting these results. we urge reviewers to avoid rejecting papers with low iaa when proper quality assurance measures were implemented and the agreement is adequately addressed. some tasks are inherently subjective, and using low iaa as a "rejection shortcut" reduces transparency and loses valuable insights about task difficulty and subjectivity that benefit the broader research community. use evaluation reporting sheets standardised evaluation reporting sheets, such as heds @xcite , allow for evaluation comparability and reproducibility by ensuring that all relevant evaluations are recorded. while such reporting sheets may seem overwhelming at first, they can help practitioners to better understand what details should be reported. at the very least, we urge authors to read through them to understand which information should always be reported. ## conclusion our survey of 64 human evaluation studies uncovered key insights across multiple aspects of how llms' hallucinations are assessed. although human evaluation remains the gold standard for nlg researchers @xcite , we observed a concerning decline in the proportion of papers conducting such evaluations. moreover, methodological reporting is often lacking; critical details, such as inter-annotator agreement, annotator demographics, and annotation guidelines, are frequently omitted. definitions and categorisations of hallucinations vary widely across tasks and papers. we argue that adopting standardised definitions addressing the three axes we propose in section 2 would support a more unified understanding of hallucinations. finally, it is troubling that even papers published at top nlp venues often fail to report essential information about their human evaluation procedures, despite being prompted to include the responsible nlp checklist. these findings underscore the urgent need for more rigorous, transparent, and standardised practices in human evaluations of hallucinations.