F aith B ench: A Diverse Hallucination Benchmark for Summarization by M odern LLM s (2025)

Abstract: Summarization is one of the most common tasks performed by large language models (LLMs), especially in applications like Retrieval-Augmented Generation (RAG). However, existing evaluations of hallucinations in LLM-generated summaries, and evaluations of hallucination detection models both suffer from a lack of diversity and recency in the LLM and LLM families considered. This paper introduces FaithBench, a summarization hallucination benchmark comprising challenging hallucinations made by 10 modern LLMs from 8 different families, with ground truth annotations by human experts. “Challenging” here means summaries on which popular, state-of-the-art hallucination detection models, including GPT-4o-as-a-judge, disagreed on. Our results show GPT-4o and GPT-3.5-Turbo produce the least hallucinations. However, most state-of-the-art hallucination detection models have near 50% accuracies on FaithBench, indicating lots of room for future improvement.

## introduction with the increasing use of large language @xcite to process textual data, ensuring their trustworthiness has become a critical concern. in applications such as retrieval augmented generation (rag) @xcite , llms are used to generate answers or summaries from textual input. when the generated text includes unsupported information, it is considered a hallucination, which can be misleading or harmful. understanding the state of hallucinations in llms is crucial but hard. existing hallucination leaderboards, such as vectara's hallucination leaderboard * and galileo's hallucination index * , detect hallucinations using models such as google's trueteacher @xcite ), vectara's hhem-2.1-open @xcite , or even gpt series models in a zero-shot, llm-as-ajudge fashion @xcite . these detection models are known to have an accuracy below 80% on benchmarks such as @xcite and ragtruth @xcite . moreover, existing benchmarks often rely on a narrow selection of llms, many of which are outdated and lack diversity across model families. if we assume llms hallucinate differentlydue to variations in training methods, datasets, and architectures, as well as changes in behavior as models scale up-then conclusions drawn from such benchmarks are incomplete, capturing only specific types of hallucinations. to address this gap, the industry and research community need a hallucination benchmark that includes modern llms across diverse model families, along with human-annotated ground truth for more reliable evaluation. this paper presents faith-bench, a summarization hallucination benchmark built on top of vectara's hallucination leaderboard which is popular in the community @xcite because it contains summaries generated by dozens of modern llms. we add human annotations, including justifications at the level of individual text spans, to summaries from 10 llms belonging to 8 llm families. to make the best use of our annotators' time, we focus on labeling challenging samples where hallucination detectors disagree the most, as obvious hallucinations can be reliably detected automatically. the majority of our annotators are experts in the field of hallucination detection, with half of them having published hallucination-related papers at major nlp conferences. faithbench allows us to evaluate both the hallucination rates of llms and the accuracy of hallucination detection models. to the best of our knowledge, this is the first evaluation of hallucina-tions across 10 llms and 8 llm families using human-annotated ground truth. gpt-4o has the lowest hallucination rate, followed by gpt-3.5-turbo, gemini-1.5-flash, and llama-3-70b. all hallucination detectors are found to correlate poorly with human-annotated ground truth, with the best balanced accuracy and f1-macro score at 62% and 57% respectively. this highlights our limited understanding of hallucinations and the challenges ahead. we hope that faithbench can catalyze research into detecting and mitigating hallucinations in llms. in contrast with existing benchmarks, faith-bench 1) covers a wide array of llm families and diverse hallucination characteristics, 2) factors the subjectivity of hallucination perception, by expanding binary consistent vs. unfaithful labels to include two new "gray-area" labels: "questionable" and "benign", 3) includes only challenging hallucination samples. the repo is https: //github.com/vectara/faithbench 2 the benchmark