Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers (2024)

Abstract: The increased use of large language models (LLMs) across a variety of real-world applications calls for mechanisms to verify the factual accuracy of their outputs. In this work, we present Factcheck-Bench, a holistic end-to-end framework for annotating and evaluating the factuality of LLM-generated responses, which encompasses a multi-stage annotation scheme designed to yield detailed labels for fact-checking and correcting not just the final prediction, but also the intermediate steps that a fact-checking system might need to take. Based on this framework, we construct an open-domain factuality benchmark in three-levels of granularity: claim, sentence, and document. We further propose a system, Factcheck-GPT, which follows our framework, and we show that it outperforms several popular LLM fact-checkers. We make our annotation tool, annotated data, benchmark, and code available at https://github.com/yuxiaw/Factcheck-GPT.

## introduction large language models (llms) have demonstrated impressive capabilities to generate naturally sounding answers over a broad range of human inquiries @xcite . yet, they often generate answers that contradict real-world facts @xcite @xcite @xcite . this degrades llm's performance and undermines its reliability, representing a significant obstacle for deployment, especially in high-stake settings, e.g., clinical, legal, and financial @xcite @xcite . before llms, most prior work investigated hallucinations of conditional text generation for specific tasks, such as abstract summarization, dialogue generation, and machine translation @xcite . this prior research was either highly taskspecific, with gold standard references, or focusing on short statements, in which automatic evaluation by rule-based matching or semantic similarity measurements with references was feasible @xcite . however, in the case of free-form llm generations over open domains, there is no single gold standard reference answer that can be used to assess the factual correctness of a long free-text model output @xcite . this makes the factual evaluation of open-domain llm responses non-trivial, either depending on manual verification or on automatic fact-checkers, e.g., factscore and factool @xcite . human assessment is costly @xcite , while existing automatic factchecking systems are often unreliable. how to evaluate and to improve the accuracy of automated fact-checkers is critical for producing dependable llm factuality evaluations. recent work on fact-checking compared the system predictions to human annotations. this only assesses the final verification results, i.e., whether a claim or a document is true or false @xcite @xcite . thus, the system is regarded as a black box. the lack of evaluation of the intermediate steps that the system has taken, makes it difficult to understand which specific system component(s) led to an erroneous factual judgement. moreover, prior studies used disparate evaluations, rendering them hard to compare @xcite . see more related work in appendix a. to address these issues, we introduce factcheck-bench, a benchmark to evaluate systems for automatic fact-checking of llm output, with each model response annotated with labels covering eight steps that might occur in the automatic detection and correction of factual errors (see figure 1 ). this enables the evaluation of the results of the intermediate component modules in various automatic fact-checkers in a unified framework. we make the following contributions: ## fine-grained annotation framework to evaluate all possible intermediate results of automatic fact-checking systems, the framework should cover as many fine-grained steps as possible. based on our review of most prior systems and datasets, we frame the automated detection and correction of factual errors for outputs of llms into eight subtasks: (1) decomposition; (2) decontextualisation; the concept of "context-independent" is straightforward, while the notion of "atomic" is subjective and ambiguous. this poses challenges: how to determine the granularity of an atomic claim? when and where to break down a response? for example, s1: canada is a constitutional monarchy, and as such, it does not have a king, can be fact-checked as one statement, or be decomposed into two claims: canada is a constitutional monarchy and canada does not have a king. in our work, we first split a document into sentences, and then from sentence to claims, with each claim containing only one property or fact to verdict. we specifically classify a statement into four categories: factual claim, opinion, not a claim (e.g. questions, exclamations, imperatives), and others (e.g. as a language model, i cannot...). afterwards, a set of checkworthy factual claims needs to be verified by retrieving and collecting evidence. note that for checkworthiness, we not only take account of objective fact against subjective judgement, other aspects such as the role (importance) of the claim to the response is also a crucial criterion for its checkworthiness. for example, the sentence s1 needs more attention than the last sentence s4 in figure 1 . we label the importance level of both decomposed sentences and claims by labels: most important, intermediate, and less important. (4) retrieve and collect evidence evidence can be retrieved by a search engine like google, or deep retrieval from a closed document collection such as wikipedia, or using the parametric knowledge of a llm. search queries can be questions covering different aspects of the claim, entities in the claim, or even the claim itself @xcite . we used google search, considering the quality and coverage. however, some evidence may neither refute nor fully support a claim. this mainly results from the fact that it is always possible that the evidence supports part of the claim. for example, for the claim elon musk is the founder, ceo, and chief engineer of spacex, evidence elon musk is the ceo of spacex, tesla, and twitter falls into this category. the evidence supports elon musk is the ceo of spacex, but it does not provide information regarding whether elon musk is the founder and chief engineer of spacex. we may argue the claim above is not an atomic claim. in practice, it is hard to ensure all claims to be atomic, though theoretically they should be. for some cases, it is more reasonable to to retain certain elements together rather than decomposing them into atomic components. for example, in the sentence from a recipe "i tried using a double boiler to gently cook the eggs and sugar before adding them to the mascarpone", it is preferable to keep "eggs and sugar" as a unit rather than separating them, despite this conflicting with the definition of atomic claims. if the evidence mentions "eggs and honey", partial support can be applied. therefore, we incorporate partially support in addition to support, refute and irrelevant @xcite . support means that the evidence entails the claim. partial support refers to the scenario where part of the information presented in a claim appears in the evidence. refute means that the evidence mentions the same event as the claim, but a clear opposite fact contrasting to a part or the whole facts presented in a claim. irrelevant refers to the situation that the evidence does not mention anything about the fact described in the claim, such that it neither supports nor refutes the claim. sometimes, it is ambiguous to distinguish between refute and irrelevant. we highlight that the evidence shows a clear opposite fact under refute stance, while the evidence does not include relevant facts mentioned in the claim under irrelevant. a label often used is not-enough-evidence if there is insufficient information to make the veracity prediction, e.g., all retrieved evidence is irrelevant or intricate contradictory evidence @xcite . so we set three labels in terms of factuality: true, false, and not-enough-evidence. ## dataset construction felm @xcite annotated sentence-level true or false labels without correction and showed that factual error detection performance tends to be improved when utilising claim-based segmentation methods compared with sentences. therefore, we annotate a claim-based document-level factchecking dataset. this dataset serves for a benchmark evaluating the effectiveness of approaches for llm fact-checking subtasks or the whole pipeline, and few-shot demonstration examples. ## unit test for fact-checkers in this section, we compare the results of automatic methods that are commonly used in current fact-checking systems (e.g., rarr, factscore, factool) for subtasks with human annotations. we first compare the automatic and human-annotated decomposition of atomic claim, and then evaluate five subtasks: (1) identify whether the sentence contains a factual statement; (2) detect the checkworthiness of a claim by categories of factual, opinion, not a claim and other; (3) judge the stance of a given evidence against a claim, whether it supports, partially supports, refutes or is irrelevant to the claim; (4) determine whether a claim is factually true or false, give a claim without "gold evidence", if false, revise it into a correct one; (5) edit a list of originally-true or revised claims into a new response, given the original response, to correct the factual errors while preserving the linguistic features and style of the original. other steps are excluded because they are either relatively easy for current techniques (e.g., splitting a document into sentences), or results of automatic approaches have been compared against human annotations in data analysis, such as the relevance or quality of the automatically-retrieved evidence.