SlowGuess/NAACL_2022_OCR huggingface dataset:
Or Honovich $^{T*}$  Roee Aharoni $^{G}$  Jonathan Herzig $^{G}$  Hagai Taitelbaum $^{G}$  Vered Cohen $^{G}$  Doron Kukliansky $^{G}$  Thomas Scialom $^{M}$  Idan Szpektor $^{G}$  Avinatan Hassidim $^{G}$  Yossi Matias $^{G}$
$^{T}$ Tel Aviv University  ${}^{G}$ Google Research  ${}^{M}$ Meta AI Tel Aviv, Israel Paris, France or.honovich@gmail.com {roeeaharoni, szpektor}@google.com
Grounded text generation systems often generate text that contains factual inconsistencies, hindering their real-world applicability. Automatic factual consistency evaluation may help alleviate this limitation by accelerating evaluation cycles, filtering inconsistent outputs and augmenting training data. While attracting increasing attention, such evaluation metrics are usually developed and evaluated in silo for a single task or dataset, slowing their adoption. Moreover, previous meta-evaluation protocols focused on system-level correlations with human annotations, which leave the example-level accuracy of such metrics unclear. In this work, we introduce TRUE: a comprehensive survey and assessment of factual consistency metrics on a standardized collection of existing texts from diverse tasks, manually annotated for factual consistency. Our standardization enables an example-level meta-evaluation protocol that is more actionable and interpretable than previously reported correlations, yielding clearer quality measures. Across diverse state-of-the-art metrics and 11 datasets we find that large-scale NLI and question generation-and-answering-based approaches achieve strong and complementary results. We recommend those methods as a starting point for model and metric developers, and hope TRUE will foster progress towards even better evaluation methods.<sup>1</sup>
A core issue in deploying text generation models for real-world applications is that they often generate factually inconsistent text with respect to the input they are conditioned on, or even completely "hallucinate" (Lee et al., 2018; Rohrbach et al., 2018; Maynez et al., 2020; Zhao et al., 2020) as exemplified in Table 1.
Summarization (Wang et al., 2020)
Input	Phyllis schlafly, a leading figure in the us conservative movement, has died at her home in missouri, aged 92...
Summary	Us conservative activist phyllis schlafly has died at the age of 87.
Fact Verification (Thorne et al., 2018)
Evidence	Ronald Bilius “Ron” Weasley is a character in J. K. Rowling’s Harry Potter fictional series.
Claim	Ron Weasley is a President.
Paraphrasing (Zhang et al., 2019)
Input	The tracks were produced by Tommy Lee, and feature Michael Beinhorn on drums.
Paraphrase	The tracks were produced by Michael Beinhorn and have Tommy Lee on drums.
Knowledge-Grounded Dialogue (Honovich et al., 2021)
Knowledge	The first flip trick called a kickflip, originally called a "magic flip," was invented by professional skateboarder Rodney Mullen.
Response	I remember the first one was called magic flip. It was called a magic flip and was invented in the 60’s.
Table 1: Factual inconsistencies (in red) from various tasks which are part of the TRUE study. The corresponding parts in the input/grounding are in blue.
To tackle such inconsistencies, one would like to detect them automatically by predicting whether a generated text is factually consistent with respect to a grounding text (frequently referred to as the "input", or the "knowledge"). Such automatic methods attract increasing attention (Zhou et al., 2021; Deng et al., 2021) as they enable both better evaluation and better generation models by automatically filtering training data (Gehrmann et al., 2021) or by augmenting training data for controlled generation (Rashkin et al., 2021b).
While automatically evaluating factual consistency is an active line of work, there is no single agreed-upon meta-evaluation protocol for measuring the quality of such methods, and labeling schemes vary in their granularity. Works are usual
1Our code is publicly available at http://www. github.com/google-research/true
Human Language Technologies, pages 3905 - 3920
ally done in silo, introducing new datasets and methods that target a specific task or domain, such as summarization (Falke et al., 2019; Kryscinski et al., 2020; Wang et al., 2020; Scialom et al., 2021; Deutsch et al., 2021; Xie et al., 2021) or dialogue (Dziri et al., 2021; Honovich et al., 2021; Nie et al., 2021; Qin et al., 2021). Comparing the robustness of such methods across tasks and datasets is therefore difficult, impeding progress on this subject.
In this work, we present TRUE: a comprehensive survey and assessment of factual consistency evaluation methods, covering various metrics, tasks and datasets. We consolidate 11 existing datasets annotated for factual consistency into a unified format, including pairs of a target text and a grounding source text, with a binary annotation of whether the target text is factually consistent w.r.t its source. TRUE<sup>2</sup> covers summarization, knowledge-grounded dialogue, paraphrasing and fact verification.<sup>3</sup> The proposed standardization enables us to properly compare consistency evaluation methods in a robust manner across these various tasks and domains.
Previous works on automatic factual consistency evaluation have mainly focused on measuring system-level correlations of the proposed metrics with human judgements (Pagnoni et al., 2021). Yet, these correlations are not useful for estimating the performance of a measured metric when making example-level, binary decisions, decoupled from specific system implementations (see recent discussion by Deutsch et al. (2022) on the limitations of reporting correlations). Instead, we aim to measure how well a method detects inconsistent texts (recall) and how often it falsely disregards consistent texts (precision), which can be easily computed using the aforementioned binary labeling scheme. Therefore, as a meta-evaluation protocol we report the Area Under the ROC Curve (ROC AUC) with respect to inconsistent example detection for each evaluation metric and dataset.
Our thorough survey and assessment of 12 metrics draws a clearer picture on the state of evaluating factual consistency. We show that Natural Language Inference (NLI) approaches, as well as Question Generation and Answering (QG-QA) ap
proaches achieve significantly better $^4$  results on a wide variety of tasks and datasets. We also show that NLI and QG-QA are complementary: combining the two yields even better results and hints at room for further improvement. Finally, we perform both quantitative and qualitative analysis of our results, finding that all approaches struggle with long inputs, labeling issues and personal statements – paving interesting avenues for future work.
To summarize, our contributions are as follows: (1) We argue that work on factual consistency evaluation should be unified and generalized across tasks, and standardize 11 published datasets into a single labeling scheme to corroborate this. (2) We propose a meta-evaluation protocol that allows more actionable and interpretable quality measures than previously reported correlations. (3) We survey and evaluate 12 diverse metrics in this unified perspective, showing that large-scale NLI and QG-QA-based approaches achieve strong and complementary results across tasks. (4) We analyze our results both qualitatively and quantitatively, pointing at challenges like long inputs and personal statements to be addressed in future work.
2 Standardizing Factual Consistency
In this section we elaborate on our re-evaluation setup. We first formally define what factual consistency refers to in this work. We then detail the datasets we consider and how we standardize them. Finally, we discuss the meta-evaluation protocol we propose for measuring the performance of evaluation methods on the standardized datasets.
2.1 Definitions and Terminology
We define a text to be factually consistent w.r.t its grounding text if all the factual information it conveys is consistent with the factual information conveyed by the grounding text.5 While some previous works distinguished between inconsistent erroneous text to inconsistent correct text (Maynez et al., 2020), we take a strict approach, requiring the text to be faithful to its grounding text, regardless of the "correctness" w.r.t the "real world". In other words, we consider only the information present in the input text, not external knowledge, to assess faithfulness. This enables a more well-defined task, since determining the truthfulness of a fact w.r.t a
2The name is a homage to GLUE (Wang et al., 2018) and not an acronym.
3We focus on English text-to-text tasks, and leave data-to-text (Parikh et al., 2020; Reiter and Thomson, 2020), multilingual and multimodal tasks to future work.
4We conduct significance testing, see section 4.
<sup>5</sup>We exclude personal and social statements such as opinions and chit-chat from the scope of factual information.
Task	# Examples	Open Test	Cons.
Summarization
- FRANK (Pagnoni et al., 2021)	671	+	33.2%
- SummEval (Fabbri et al., 2021a)	1,600	-	81.6%
- MNBM (Maynez et al., 2020)	2,500	-	10.2%
- QAGS-CNNDM (Wang et al., 2020)	235	-	48.1%
- QAGS-XSum (Wang et al., 2020)	239	-	48.5%
Dialogue
- BEGIN (Dziri et al., 2021)	836	+	33.7%
- Q2 (Honovich et al., 2021)	1,088	-	57.7%
- DialFact (Gupta et al., 2021)	8,689	+	38.5%
Fact Verification
- FEVER (Thorne et al., 2018)	18,209	-	35.1%
- VitaminC (Schuster et al., 2021)	63,054	+	49.9%
Paraphrasing
- PAWS (Zhang et al., 2019)	8,000	+	44.2%
Table 2: Statistics for the datasets incorporated in TRUE. Cons. is the ratio of consistent examples.
general "real world" is subjective and depends on the knowledge, values and beliefs of the subject (Heidegger, 2001). This definition follows similar strictness in Textual Entailment, Question Answering, Summarization and other tasks where comprehension is based on a given grounding text, irrespective of contradiction with other world knowledge. This is also in line with recent work on evaluating attribution in text generation (Rashkin et al., 2021a), where humans are required to judge whether a generated text is attributable to a grounding text. We use the terms consistent, grounded, faithful and factual interchangeably.
2.2 Standardization Process
We include 11 datasets that contain human annotations w.r.t factual consistency in diverse tasks (Table 2). Other than the importance of covering a wide variety of error types, this also alleviates issues of rating quality which may vary across datasets (Denton et al., 2021).
To allow a unified evaluation framework we convert all annotations to binary labels that correspond to whether the entire target text is factually consistent w.r.t the given grounding text or not. We note that a fine-grained annotation scheme, i.e., a typology of errors, was proposed for factual consistency (Pagnoni et al., 2021). While useful, most existing datasets do not include such labels. Moreover, while Machine Translation (MT) evaluation also showed value in fine-grained annotations (Freitag et al., 2021), it was proposed after years of improving MT to the level where coarse-grained annotation is insufficient. We argue that current grounded generation models are still at early stages w.r.t factual consistency, making binary labeling more beneficial now as it enables easier standardization across tasks and domains, with the goal of bringing researchers to collaborate on a shared methodology. Binary annotation also corresponds
to practical applications where filtering out unfaithful predictions is desired, and is in-line with the recommendations for human evaluation of attribution in text generation by Rashkin et al. (2021a).
We next detail the 11 datasets included in TRUE.
2.2.1 Abstractive Summarization
FRANK Pagnoni et al. (2021) proposed a typology of factual errors, grounded in frame semantics (Fillmore, 1976; Palmer et al., 2005) and linguistic discourse theory (Brown and Yule, 1983). Based on this typology, they collected annotations for model-generated summaries on the CNN/DailyMail (CNN/DM; Hermann et al., 2015) and XSum (Narayan et al., 2018) datasets, resulting in 2250 annotated system outputs. Each summary sentence was annotated by three annotators. We take the majority vote for each sentence to get a sentence-level label and consider a summary as consistent if all sentences are consistent.
SummEval SummEval (Fabbri et al., 2020) is a comprehensive study of evaluation metrics for text summarization. The authors collected human judgments for 16 model outputs on 100 articles taken from the CNN/DM dataset, using both extractive and abstractive models. Annotators were asked to rate summaries on a Likert scale from 1 to 5, over 4 dimensions: consistency, coherence, fluency and relevance. Each summary was scored by 5 crowd-workers and 3 expert annotators. We label summaries as consistent only if all the expert annotators gave a consistency score of 5.
MNBM Maynez et al. (2020) annotated system outputs for the XSum dataset (Narayan et al., 2018). They sampled 500 articles and annotated summaries generated by four different systems, as well as the gold summaries. Annotators were asked to assess whether the summary includes hallucinations. Judgments from three different annotators were collected for each document-summary pair. To convert to a binary-label format, we use the binary consistency decision of whether a summary contains no hallucinations, and assign a label by taking the majority vote of the three annotators.
QAGS Wang et al. (2020) collected judgments of factual consistency on generated summaries for CNN/DM and XSum. Annotators were presented with the summaries one sentence at a time, along with the article, and determined whether each sentence is factually consistent w.r.t the article. Each
sentence was annotated by 3 annotators, using the majority vote as the final score. To convert to binary-label format, we consider a summary consistent only if all its sentences are consistent.
2.2.2 Dialogue Generation
BEGIN (Dziri et al., 2021) is a dataset for evaluating groundedness in knowledge-grounded dialogue systems, in which system outputs should be consistent with a grounding knowledge provided to the dialogue agent. BEGIN frames the task as textual entailment (Dagan et al., 2006; Bowman et al., 2015), adopting the entailment and contradiction labels, and splitting the neutral label into three subcategories: hallucination, off-topic responses and generic responses. Dialogue responses were generated by fine-tuning two systems on the Wizard of Wikipedia (WoW) dataset (Dinan et al., 2019), in which responses should be grounded in a span of text from Wikipedia. The generated responses were split into sentences, and each sentence was annotated separately. To convert to a binary-label format, we treat entailed sentences as consistent and all others as inconsistent.
$Q^{2}$  Honovich et al. (2021) annotated 1,088 generated dialogue responses for binary factual consistency w.r.t the knowledge paragraph provided to the dialogue model, for two dialogue models trained on WOW. Responses were annotated using binary labels by 3 of the paper authors, one annotator per response. We use  $Q^{2}$ 's labels without changes.
DialFact Gupta et al. (2021) introduced the task of fact-verification in dialogue and constructed a dataset of conversational claims paired with pieces of evidence from Wikipedia. They define three tasks: (1) detecting whether a response contains verifiable content (2) retrieving relevant evidence and (3) predicting whether a response is supported by the evidence, refuted by the evidence or if there is not enough information to determine. We use the verifiable (i.e., factual, rather than personal) responses annotated for the third task, treating supported annotations as consistent and the rest as inconsistent. In cases where several evidence were marked as required for verification, we concatenate all evidence sentences to be the grounding text.
2.2.3 Fact Verification
FEVER Thorne et al. (2018) introduced FEVER (Fact Extraction and VERIFICATION), a dataset for fact verification against textual sources. FEVER
was constructed by extracting information from Wikipedia, generating claims using annotators, then labeling whether each claim is supported or refuted by Wikipedia. Claims can also be labeled with NotEnoughInfo, meaning that there is not enough information in Wikipedia to either verify or refute the claim. Given a claim, the task defined by FEVER is to first extract evidence, then to determine whether it supports or refutes the claim. In a slightly different framing, the latter stage in FEVER is to determine whether the claim is factually consistent or not w.r.t the evidence, which is aligned with what we aim to measure in TRUE. We use the development set of the NLI version of FEVER (Nie et al., 2019, 2020), treating supported claims as consistent and the rest as inconsistent.
VitaminC Schuster et al. (2021) derived a large-scale fact verification dataset from factual revisions to Wikipedia pages. Each example includes an evidence text from Wikipedia and a fact, with an annotation of whether the fact is supported, refuted or neutral w.r.t the evidence. The authors collected factual revisions to Wikipedia articles (pairs of "before" and "after" sentences), and asked annotators to write two facts for each pair: one that is supported by the first sentence and refuted by the second, and vice versa. When no explicit contradiction was present, the annotators wrote facts that are neutral w.r.t the evidence. Additional examples were created by revising examples from FEVER. We treat examples that include supported facts as consistent, and refuted or neutral facts as inconsistent.
2.2.4 Paraphrase Detection
PAWS Zhang et al. (2019) constructed a dataset for paraphrase identification with 108,463 paraphrase and non-paraphrase pairs with high lexical overlap, generated by controlled word swapping and back-translation, followed by judgments from human raters. Source sentences were drawn from Wikipedia and the Quora Question Pairs (QQP) corpus. We only use the examples with Wikipedia source sentences and view the binary paraphrase labels as consistency labels. We note that the definition of paraphrase is not equivalent to the definition of factual consistency, as a subset of a source text is not a paraphrase but may still be factually consistent with the source. However, PAWS was constructed such that non-paraphrases usually have contradicting meanings and is therefore relevant.
2.3 Meta-Evaluation
Previous work on evaluating factual consistency focused on measuring correlation with human judgements (Pagnoni et al., 2021) to compare different metrics. However, such system-level numbers are not very informative when one is interested in evaluating the absolute performance of inconsistency detection methods that perform a binary decision w.r.t each input. Deutsch et al. (2022) also recently discuss various issues in measuring system-level correlations to assess the validity of automatic evaluation metrics for summarization.
To conduct a more fine-grained evaluation at the single example level, we report the Receiver Operating Characteristic Area Under the Curve (ROC AUC) w.r.t binary detection of inconsistent examples. The ROC curve is created by plotting the true positive rate (TPR, a.k.a. the recall) against the false positive rate (FPR, a.k.a. the fallout) at different possible thresholds for each tested metric. Measuring ROC AUC evaluates the different metrics without setting a specific decision threshold.
For datasets with existing development/test split, we also tune a threshold for the binary consistency/inconsistency decision on the development set and report the test set accuracy using this threshold. We tune the thresholds by optimizing the geometric mean of the TPR and 1-FPR:  $\sqrt{\mathrm{TPR}*(1 - \mathrm{FPR})}$ .
We compare various standard as well as state-of-the-art approaches to measure factual consistency. This comparison should draw a clear picture of current research on this subject and raise directions for future work. For example, we expect that robust metrics should perform well across various tasks and datasets. We next describe the different metrics we assess as part of this study. We note that for all reference-based metrics, we use the grounding text as the reference. For metrics where the scores are not in the [0,1] range, we normalize the scores to be in that range.
3.1 N-Gram Based Metrics
Standard N-Gram matching metrics such as BLEU (Papineni et al., 2002), ROUGE (Lin, 2004) and token-level F1 were shown to have weak correlation with factual consistency (Maynez et al.,
2020; Honovich et al., 2021). We add them as baselines to this study mainly to corroborate this claim on a wide set of datasets and tasks.
3.2 Model-Based Metrics
BERTScore (Zhang et al., 2020) aggregates similarity scores between the BERT contextual embedding of tokens in candidate and reference sentences. We report results for the BERTScore-precision variant as it showed better results in preliminary experiments. We use BERTScore version 0.3.11 with the DeBERTa-xl-MNLI model (He et al., 2021; Nangia et al., 2017), which is the recommended model as of the time of writing this paper. $^{7}$
BLEURT (Sellam et al., 2020a,b) is a learned metric based on BERT (Devlin et al., 2019) for evaluating text generation. BLEURT includes additional pretraining on synthetic data followed by fine-tuning on human judgements to train a model that scores system outputs. We use the recommended BLEURT-20 checkpoint (Pu et al., 2021).
FactCC (Kryscinski et al., 2020) is a BERT-based metric for verifying the factual consistency of summaries. It is trained on synthetically generated data obtained by applying rule-based transformations to generate consistent and inconsistent summaries.
BARTScore (Yuan et al., 2021) evaluates text using probabilities from force-decoding with a BART model (Lewis et al., 2020). We use the version finetuned on the ParaBank2 dataset (Hu et al., 2019).
CTC (Deng et al., 2021) measures the average token-level alignment of the generated text w.r.t the grounding text using a BERT sequence tagging model. The model is trained to detect hallucinated tokens generated by a BART model in a self-supervised manner.
3.3 Natural Language Inference Metrics
ANLI The task of Textual Entailment (Dagan et al., 2006) or Natural Language Inference (NLI; Bowman et al., 2015) is to determine, given two
$^{7}$ https://github.com/Tiiger/bert_score $^{8}$ https://github.com/google-research/bleurt/blob/master/checkpoints.md
9This metric has come to our attention after the paper was accepted, so we add the results to the appendix to avoid adding unreviewed results in the camera ready version. See https://github.com/tanyuqian/ctc-gen-eval/blob/master/factual-consistency.md for implementation details and Table 9 for results.
This is equivalent to the AUC w.r.t detecting consistent examples.
sentences, a hypothesis and a premise, whether the hypothesis in entailed by the premise, contradicts it, or is neutral w.r.t to it. The resemblance<sup>10</sup> of NLI to factual consistency evaluation has led to utilizing NLI models for measuring factual consistency (Thorne et al., 2018; Welleck et al., 2019; Maynez et al., 2020; Dziri et al., 2021). We trained an NLI model by fine-tuning T5-11B (Raffel et al., 2020) on the Adversarial NLI (ANLI; Nie et al., 2020) dataset. As suggested by Maynez et al. (2020), we compute the entailment probability with the grounding text as the premise and the generated text as the hypothesis and use it as the example-level factual consistency score.<sup>11</sup>