Venue: NAACL 2022

# TRUE: Re-evaluating Factual Consistency Evaluation

Or Honovich\({}^{T}\)1 Roee Aharoni\({}^{G}\) Jonathan Herzig\({}^{G}\) Hagai Taitelbaum\({}^{G}\)

**Vered Cohen\({}^{G}\) Doron Kulkiansky\({}^{G}\) Thomas Scialom\({}^{M}\) Idan Szpektor\({}^{G}\) Avinatan Hassidim\({}^{G}\) Yossi Matias\({}^{G}\) \({}^{T}\)Iel Aviv University \({}^{G}\)Google Research \({}^{M}\)Meta AI Tel Aviv, Israel Paris, France or.honovich@gmail.com**

Footnote 1: Work done during an internship at Google Research.

###### Abstract

Grounded text generation systems often generate text that contains factual inconsistencies, hindering their real-world applicability. Automatic factual consistency evaluation may help alleviate this limitation by accelerating evaluation cycles, filtering inconsistent outputs and augmenting training data. While attracting increasing attention, such evaluation metrics are usually developed and evaluated in silo for a single task or dataset, slowing their adoption. Moreover, previous meta-evaluation protocols focused on system-level correlations with human annotations, which leave the example-level accuracy of such metrics unclear. In this work, we introduce TRUE: a comprehensive survey and assessment of factual consistency metrics on a standardized collection of existing texts from diverse tasks, manually annotated for factual consistency. Our standardization enables an example-level meta-evaluation protocol that is more actionable and interpretable than previously reported correlations, yielding clearer quality measures. Across diverse state-of-the-art metrics and 11 datasets we find that large-scale NLI and question generation-and-answering-based approaches achieve strong and complementary results. We recommend those methods as a starting point for model and metric developers, and hope TRUE will foster progress towards even better evaluation methods.2

Footnote 2: Our code is publicly available at [http://www.github.com/google-research/true](http://www.github.com/google-research/true)

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3905 - 3920 July 10-15, 2022 ©2022 Association for Computational Linguistics

## 1 Introduction

A core issue in deploying text generation models for real-world applications is that they often generate factually inconsistent text with respect to the input they are conditioned on, or even completely "hallucinate" Lee et al. (2018); Rohrbach et al. (2018); Maynez et al. (2020); Zhao et al. (2020) as exemplified in Table 1.

To tackle such inconsistencies, one would like to detect them automatically by predicting whether a generated text is factually consistent with respect to a grounding text (frequently referred to as the "input", or the "knowledge"). Such automatic methods attract increasing attention Zhou et al. (2021); Deng et al. (2021) as they enable both better evaluation and better generation models by automatically filtering training data Gehrmann et al. (2021) or by augmenting training data for controlled generation Rashkin et al. (2021b).

While automatically evaluating factual consistency is an active line of work, there is no single agreed-upon meta-evaluation protocol for measuring the quality of such methods, and labeling schemes vary in their granularity. Works are usu

\begin{table}
\begin{tabular}{|p{426.7pt}|p{142.3pt}|} \hline  & \multicolumn{1}{c|}{**Summarization Wang et al. (2020)**} \\ \cline{2-3} Input & Phyllis schldify, a leading figure in the us conservative movement, has died at her home in missouri, aged 92... \\ \hline Summary & Us conservative activist phylils schldify has died at the age of 87. \\ \hline \end{tabular} \begin{tabular}{|p{426.7pt}|p{142.3pt}|} \hline  & Ronald Bilius “Ron” Weasley is a character in J. K. Rowling’s Harry Potter fictional series. \\ \hline Claim & Ron Wesley is a President. \\ \hline \end{tabular} \begin{tabular}{|p{426.7pt}|p{142.3pt}|} \hline  & \multicolumn{1}{c|}{**Paraphrasing Zhang et al. (2019)**} \\ \hline Input & The tracks were produced by Tommy Lee, and feature Michael Beinhorn on drums. \\ \hline Paraphrase & The tracks were produced by Michael Beinhorn and have Tommy Lee drums. \\ \hline \end{tabular} 
\begin{tabular}{|p{426.7pt}|p{142.3pt}|} \hline  & \multicolumn{1}{c|}{The first flip trick called a kickflip, originally called a ”magic flip,” was invented by professional skateboard Rodney Mullen.} \\ \hline Response & I remember the first one was called magic flip. It was called a magic flip and was inverted in the 60’s. \\ \hline \end{tabular}
\end{table}
Table 1: Factual inconsistencies (in red) from various tasks which are part of the TRUE study. The corresponding parts in the input/grounding are in blue.



ally done in silo, introducing new datasets and methods that target a specific task or domain, such as summarization (Falke et al., 2019; Kryscinski et al., 2020; Wang et al., 2020; Scialom et al., 2021; Deutsch et al., 2021; Xie et al., 2021) or dialogue (Dziri et al., 2021; Honovich et al., 2021; Nie et al., 2021; Qin et al., 2021). Comparing the robustness of such methods _across_ tasks and datasets is therefore difficult, impeding progress on this subject.

In this work, we present TRUE: a comprehensive survey and assessment of factual consistency evaluation methods, covering various metrics, tasks and datasets. We consolidate 11 existing datasets annotated for factual consistency into a unified format, including pairs of a target text and a grounding source text, with a binary annotation of whether the target text is factually consistent w.r.t its source. TRUE2 covers summarization, knowledge-grounded dialogue, paraphrasing and fact verification.3 The proposed standardization enables us to properly compare consistency evaluation methods in a robust manner across these various tasks and domains.

Footnote 2: The name is a homage to GLUE (Wang et al., 2018) and not an acronym.

Previous works on automatic factual consistency evaluation have mainly focused on measuring system-level correlations of the proposed metrics with human judgements (Pagnoni et al., 2021). Yet, these correlations are not useful for estimating the performance of a measured metric when making _example-level, binary_ decisions, decoupled from specific system implementations (see recent discussion by Deutsch et al. (2022) on the limitations of reporting correlations). Instead, we aim to measure how well a method detects inconsistent texts (_recall_) and how often it falsely disregards consistent texts (_precision_), which can be easily computed using the aforementioned binary labeling scheme. Therefore, as a meta-evaluation protocol we report the Area Under the ROC Curve (ROC AUC) with respect to inconsistent example detection for each evaluation metric and dataset.

Our thorough survey and assessment of 12 metrics draws a clearer picture on the state of evaluating factual consistency. We show that Natural Language Inference (NLI) approaches, as well as Question Generation and Answering (QG-QA) approaches achieve significantly better4 results on a wide variety of tasks and datasets. We also show that NLI and QG-QA are complementary: combining the two yields even better results and hints at room for further improvement. Finally, we perform both quantitative and qualitative analysis of our results, finding that all approaches struggle with long inputs, labeling issues and personal statements - paving interesting avenues for future work.

Footnote 3: We focus on English text-to-text tasks, and leave data-to-text (Parikh et al., 2020; Reiter and Thomson, 2020), multilingual and multimodal tasks to future work.

To summarize, our contributions are as follows: (1) We argue that work on factual consistency evaluation should be unified and generalized across tasks, and standardize 11 published datasets into a single labeling scheme to corroborate this. (2) We propose a meta-evaluation protocol that allows more actionable and interpretable quality measures than previously reported correlations. (3) We survey and evaluate 12 diverse metrics in this unified perspective, showing that large-scale NLI and QG-QA-based approaches achieve strong and complementary results _across_ tasks. (4) We analyze our results both qualitatively and quantitatively, pointing at challenges like long inputs and personal statements to be addressed in future work.

## 2 Standardizing Factual Consistency

In this section we elaborate on our re-evaluation setup. We first formally define what factual consistency refers to in this work. We then detail the datasets we consider and how we standardize them. Finally, we discuss the meta-evaluation protocol we propose for measuring the performance of evaluation methods on the standardized datasets.

### Definitions and Terminology

We define a text to be factually consistent w.r.t its grounding text if all the factual information it conveys is consistent with the factual information conveyed by the grounding text.5 While some previous works distinguished between inconsistent erroneous text to inconsistent correct text (Maynez et al., 2020), we take a strict approach, requiring the text to be faithful to its grounding text, regardless of the "correctness" w.r.t the "real world". In other words, we consider only the information present in the input text, not external knowledge, to assess faithfulness. This enables a more well-defined task, since determining the truthfulness of a fact w.r.t a 

general "real world" is subjective and depends on the knowledge, values and beliefs of the subject Heidegger (2001). This definition follows similar strictness in Textual Entailment, Question Answering, Summarization and other tasks where comprehension is based on a given grounding text, irrespective of contradiction with other world knowledge. This is also in line with recent work on evaluating attribution in text generation Rashkin et al. (2021), where humans are required to judge whether a generated text is attributable to a grounding text. We use the terms _consistent_, _grounded_, _faithful_ and _factual_ interchangeably.

### Standardization Process

We include 11 datasets that contain human annotations w.r.t factual consistency in diverse tasks (Table 2). Other than the importance of covering a wide variety of error types, this also alleviates issues of rating quality which may vary across datasets Denton et al. (2021).

To allow a unified evaluation framework we convert all annotations to binary labels that correspond to whether the entire target text is factually consistent w.r.t the given grounding text or not. We note that a fine-grained annotation scheme, i.e., a typology of errors, was proposed for factual consistency Pagnoni et al. (2021). While useful, most existing datasets do not include such labels. Moreover, while Machine Translation (MT) evaluation also showed value in fine-grained annotations Freitag et al. (2021), it was proposed after years of improving MT to the level where coarse-grained annotation is insufficient. We argue that current grounded generation models are still at early stages w.r.t factual consistency, making binary labeling more beneficial now as it enables easier standardization across tasks and domains, with the goal of bringing researchers to collaborate on a shared methodology. Binary annotation also corresponds to practical applications where filtering out unfaithful predictions is desired, and is in-line with the recommendations for human evaluation of attribution in text generation by rashkin2021natural.

We next detail the 11 datasets included in TRUE.

#### 2.2.1 Abstractive Summarization

##### Frank

Pagnoni et al. (2021) proposed a typology of factual errors, grounded in frame semantics Fillmore (1976); Palmer et al. (2005) and linguistic discourse theory Brown and Yule (1983). Based on this typology, they collected annotations for model-generated summaries on the CNN/DailyMail (CNN/DM; Hermann et al. (2015) and XSum Narayan et al. (2018) datasets, resulting in 2250 annotated system outputs. Each summary sentence was annotated by three annotators. We take the majority vote for each sentence to get a sentence-level label and consider a summary as consistent if all sentences are consistent.

SummEvalSummEval Fabbri et al. (2020) is a comprehensive study of evaluation metrics for text summarization. The authors collected human judgments for 16 model outputs on 100 articles taken from the CNN/DM dataset, using both extractive and abstractive models. Annotators were asked to rate summaries on a Likert scale from 1 to 5, over 4 dimensions: _consistency_, _coherence_, _fluency_ and _relevance_. Each summary was scored by 5 crowd-workers and 3 expert annotators. We label summaries as consistent only if all the expert annotators gave a _consistency_ score of 5.

MnbmMaynez et al. (2020) annotated system outputs for the XSum dataset Narayan et al. (2018). They sampled 500 articles and annotated summaries generated by four different systems, as well as the gold summaries. Annotators were asked to assess whether the summary includes hallucinations. Judgments from three different annotators were collected for each document-summary pair. To convert to a binary-label format, we use the binary consistency decision of whether a summary contains no hallucinations, and assign a label by taking the majority vote of the three annotators.

QagsWang et al. (2020) collected judgments of factual consistency on generated summaries for CNN/DM and XSum. Annotators were presented with the summaries one sentence at a time, along with the article, and determined whether each sentence is factually consistent w.r.t the article. Each

\begin{table}
\begin{tabular}{l|c|c|c}
**Task** & **\# Examples** & **Open Test** & **Cons.** \\ \hline
* & \(81.6\%\) \\ \cline{2-
 

sentence was annotated by 3 annotators, using the majority vote as the final score. To convert to binary-label format, we consider a summary consistent only if all its sentences are consistent.

#### 2.2.2 Dialogue Generation

Begin(Dziri et al., 2021) is a dataset for evaluating groundedness in knowledge-grounded dialogue systems, in which system outputs should be consistent with a grounding knowledge provided to the dialogue agent. BEGIN frames the task as textual entailment (Dagan et al., 2006; Bowman et al., 2015), adopting the _entailment_ and _contradiction_ labels, and splitting the neutral label into three subcategories: _hallucination_, _off-topic_ responses and _generic_ responses. Dialogue responses were generated by fine-tuning two systems on the Wizard of Wikipedia (WoW) dataset (Dinan et al., 2019), in which responses should be grounded in a span of text from Wikipedia. The generated responses were split into sentences, and each sentence was annotated separately. To convert to a binary-label format, we treat entailed sentences as consistent and all others as inconsistent.

\(Q^{2}\)Honovich et al. (2021) annotated 1,088 generated dialogue responses for binary factual consistency w.r.t the knowledge paragraph provided to the dialogue model, for two dialogue models trained on WoW. Responses were annotated using binary labels by 3 of the paper authors, one annotator per response. We use \(Q^{2}\)'s labels without changes.

DialFactGupta et al. (2021) introduced the task of fact-verification in dialogue and constructed a dataset of conversational claims paired with pieces of evidence from Wikipedia. They define three tasks: (1) detecting whether a response contains verifiable content (2) retrieving relevant evidence and (3) predicting whether a response is _supported_ by the evidence, _refuted_ by the evidence or if there is _not enough information_ to determine. We use the verifiable (i.e., factual, rather than personal) responses annotated for the third task, treating _supported_ annotations as consistent and the rest as inconsistent. In cases where several evidence were marked as required for verification, we concatenate all evidence sentences to be the grounding text.

#### 2.2.3 Fact Verification

FeverThorne et al. (2018) introduced FEVER (Fact Extraction and VERfication), a dataset for fact verification against textual sources. FEVER was constructed by extracting information from Wikipedia, generating claims using annotators, then labeling whether each claim is _supported_ or _refitted_ by Wikipedia. Claims can also be labeled with _NotEnoughInfo_, meaning that there is not enough information in Wikipedia to either verify or refute the claim. Given a claim, the task defined by FEVER is to first extract evidence, then to determine whether it supports or refutes the claim. In a slightly different framing, the latter stage in FEVER is to determine whether the claim is actually consistent or not w.r.t the evidence, which is aligned with what we aim to measure in TRUE. We use the development set of the NLI version of FEVER (Nie et al., 2019, 2020), treating _supported_ claims as consistent and the rest as inconsistent.

VitaminCSchuster et al. (2021) derived a large-scale fact verification dataset from factual revisions to Wikipedia pages. Each example includes an evidence text from Wikipedia and a fact, with an annotation of whether the fact is supported, refuted or neutral w.r.t the evidence. The authors collected factual revisions to Wikipedia articles (pairs of "before" and "after" sentences), and asked annotators to write two facts for each pair: one that is _supported_ by the first sentence and _refuted_ by the second, and vice versa. When no explicit contradiction was present, the annotators wrote facts that are _neutral_ w.r.t the evidence. Additional examples were created by revising examples from FEVER. We treat examples that include _supported_ facts as consistent, and _refuted_ or _neutral_ facts as inconsistent.

#### 2.2.4 Paraphrase Detection

PawsZhang et al. (2019) constructed a dataset for paraphrase identification with 108,463 paraphrase and non-paraphrase pairs with high lexical overlap, generated by controlled word swapping and back-translation, followed by judgments from human raters. Source sentences were drawn from Wikipedia and the Quora Question Pairs (QQP) corpus. We only use the examples with Wikipedia source sentences and view the binary paraphrase labels as consistency labels. We note that the definition of paraphrase is not equivalent to the definition of factual consistency, as a subset of a source text is not a paraphrase but may still be factually consistent with the source. However, PAWS was constructed such that non-paraphrases usually have contradicting meanings and is therefore relevant 



### Meta-Evaluation

Previous work on evaluating factual consistency focused on measuring correlation with human judgements (Pagnoni et al., 2021) to compare different metrics. However, such system-level numbers are not very informative when one is interested in evaluating the absolute performance of inconsistency detection methods that perform a _binary_ decision w.r.t each input. Deutsch et al. (2022) also recently discuss various issues in measuring system-level correlations to assess the validity of automatic evaluation metrics for summarization.

To conduct a more fine-grained evaluation at the single example level, we report the Receiver Operating Characteristic Area Under the Curve (ROC AUC) w.r.t binary detection of inconsistent examples.6 The ROC curve is created by plotting the _true positive rate_ (TPR, a.k.a. the recall) against the _false positive rate_ (FPR, a.k.a. the fallout) at different possible thresholds for each tested metric. Measuring ROC AUC evaluates the different metrics without setting a specific decision threshold.

Footnote 6: This is equivalent to the AUC w.r.t detecting consistent examples.

For datasets with existing development/test split, we also tune a threshold for the binary consistency/inconsistency decision on the development set and report the test set accuracy using this threshold. We tune the thresholds by optimizing the geometric mean of the TPR and 1-FPR: \(\sqrt{\text{TPR}*(1-\text{FPR})}\).

## 3 Evaluation Metrics

We compare various standard as well as state-of-the-art approaches to measure factual consistency. This comparison should draw a clear picture of current research on this subject and raise directions for future work. For example, we expect that robust metrics should perform well across various tasks and datasets. We next describe the different metrics we assess as part of this study. We note that for all reference-based metrics, we use the grounding text as the reference. For metrics where the scores are not in the [0,1] range, we normalize the scores to be in that range.

### N-Gram Based Metrics

Standard N-Gram matching metrics such as BLEU Papineni et al. (2002), ROUGE Lin (2004) and token-level F1 were shown to have weak correlation with factual consistency Maynez et al. (2020); Honovich et al. (2021). We add them as baselines to this study mainly to corroborate this claim on a wide set of datasets and tasks.

### Model-Based Metrics

BERTScoreZhang et al. (2020) aggregates similarity scores between the BERT contextual embedding of tokens in candidate and reference sentences. We report results for the BERTScore-precision variant as it showed better results in preliminary experiments. We use BERTScore version 0.3.11 with the DeBERTa-xl-MNLI model He et al. (2021); Nangia et al. (2017), which is the recommended model as of the time of writing this paper.7

BLEURSTSellam et al. (2020, 2020) is a learned metric based on BERT Devlin et al. (2019) for evaluating text generation. BLEURT includes additional pretraining on synthetic data followed by fine-tuning on human judgements to train a model that scores system outputs. We use the recommended BLEURT-20 checkpoint Pu et al. (2021).8

Footnote 8: [https://github.com/google-research/bleurt/blob/master/checkpoints.md](https://github.com/google-research/bleurt/blob/master/checkpoints.md)

Footnote 9: This metric has come to our attention after the paper was accepted, so we add the results to the appendix to avoid adding unreviewed results in the camera ready version. See [https://github.com/tanyuqian/ctc-gen-eval/blob/master/factual-consistency.md](https://github.com/tanyuqian/ctc-gen-eval/blob/master/factual-consistency.md) for implementation details and Table 9 for results.

FactCCKryscinski et al. (2020) is a BERT-based metric for verifying the factual consistency of summaries. It is trained on synthetically generated data obtained by applying rule-based transformations to generate consistent and inconsistent summaries.

BARTScoreYuan et al. (2021) evaluates text using probabilities from force-decoding with a BART model Lewis et al. (2020). We use the version fine-tuned on the ParaBank2 dataset Hu et al. (2019).

CTCDeng et al. (2021) measures the average token-level alignment of the generated text w.r.t the grounding text using a BERT sequence tagging model. The model is trained to detect hallucinated tokens generated by a BART model in a self-supervised manner. 10

Footnote 10: This is equivalent to the AUC w.r.t detecting consistent examples.

### Natural Language Inference Metrics

ANliThe task of Textual Entailment Dagan et al. (2006) or Natural Language Inference NLI; Bowman et al. (2015) is to determine, given two

sentences, a _hypothesis_ and a _premise_, whether the _hypothesis_ in entailed by the _premise_, contradicts it, or is neutral w.r.t to it. The resemblance10 of NLI to factual consistency evaluation has led to utilizing NLI models for measuring factual consistency Thorne et al. (2018); Welleck et al. (2019); Maynez et al. (2020); Dziri et al. (2021). We trained an NLI model by fine-tuning T5-11B Raffel et al. (2020) on the Adversarial NLI (ANLI; Nie et al. (2020) dataset. As suggested by Maynez et al. (2020), we compute the entailment probability with the grounding text as the premise and the generated text as the hypothesis and use it as the example-level factual consistency score.11

Footnote 10: One example where the definition of NLI and factual consistency differs is in dialog, where subjective or opinionated statements of the dialog agent are not evaluated as factual statements, while NLI models consider all the text in the premise.

SummaC(Summary Consistency; Laban et al. 2021) is focused on evaluating factual consistency in summarization. They use NLI for detecting inconsistencies by splitting the document and summary into sentences and computing the entailment probabilities on all document/summary sentence pairs, where the premise is a document sentence and the hypothesis is a summary sentence. They aggregate the NLI scores for all pairs by either taking the maximum score per summary sentence and averaging (SC\({}_{\text{ZS}}\)) or by training a convolutional neural network to aggregate the scores (SC\({}_{\text{Conv}}\)). We use the publicly available implementation.12

Footnote 11: More implementation details on the NLI model are available in Section B in the appendix.

### Qg-QA Based Metrics

Durmus et al. (2020) and Wang et al. (2020) proposed to use Question Generation (QG) and Question Answering (QA) models to automatically evaluate factual consistency in abstractive summarization, showing promising results. Honovich et al. (2021) employed a similar approach for evaluating knowledge-grounded dialogue generation.

The steps of the QG-QA approach are as follows: (1) Questions are automatically generated for spans in the generated text, such that the answer to a question is its respective input span. (2) The generated questions are answered using a QA model on the grounding text, resulting in an answer span or a "no-answer" output. (3) For each question, the two answer spans from the grounding and the generated text are compared to get a score. (4) The scores for all questions are aggregated into a final score.

Q2(Honovich et al., 2021) is a QG-QA method that employs an NLI model to compare the two answers for each question, where the grounding text answer is the premise and the generated text answer is the hypothesis. We report results for a re-implementation of \(Q^{2}\) using T5-11B as the backbone for the QG, QA and NLI models. While Honovich et al. (2021) validate each generated question by answering it using a QA model and comparing to the original extracted answer candidate using exact match, we relax this and instead use F1 token-overlap with a predefined threshold.13

Footnote 13: More implementation details are available in Section B in the appendix.

QuestEval(Scialom et al., 2021) is a QG-QA method that measures both factual consistency and relevance (by reversing the roles of the generated and grounding texts). The authors trained a model that weights each generated question according to the relevance of its answer to appear in the generated text. Their results showed high correlation with human judgments in comparison to prior work on the SummEval benchmark Fabbri et al. (2021). We use the publicly available version.14

Footnote 14: [https://github.com/ThomasScaliom/QuestEval](https://github.com/ThomasScaliom/QuestEval)

## 4 Results

We report the ROC AUC15 of various metrics on the standardized datasets in Table 3. The ROC curves can be found in Figure 2 in the appendix. SC\({}_{\text{ZS}}\) was trained on VitaminC which includes examples from FEVER, so we exclude those datasets from the average AUC calculation for a more fair comparison. As all metrics operate in a "zero-shot" manner on all datasets (except for SC\({}_{\text{ZS}}\) on VitaminC and FEVER) and no threshold tuning is required, we report results on the development sets.16

Footnote 15: Multiplied by 100 for better readability.

Footnote 16: AUC for the test sets and accuracy for the dev and test sets are provided in Tables 10, 11 and 12 in the appendix.

The results show that the NLI-based models (ANLI, SC\({}_{\text{ZS}}\)17) outperformed the other approaches on 6 datasets, with average AUC of 81.5 and 81.4 for ANLI and SC\({}_{\text{ZS}}\), respectively. \(Q^{2}\) outperformed the other approaches on 4 datasets, with an average AUC of 80.7. The next best method, BARTScore, 

had lower average AUC of 72.2. All other approaches scored 72 or lower on average across all datasets (excluding FEVER and VitaminC). As expected, the simple token-matching based metrics did not perform well, and for completeness, we report their performance in Table 9 in the appendix. We keep the F1 score in Table 3 for convenient comparison to the other metrics.

One outlier is BEGIN, which is the only dataset where simple metrics like F1 token overlap achieved scores higher than 80. We measured the average overlap between the grounding and target texts per dataset, and found that BEGIN exhibits a high difference between grounded and ungrounded texts in comparison to other datasets (Table 8 in appendix A), which explains this.

We follow Laban et al. (2021) and perform significance testing through bootstrap resampling Efron (1982), comparing the best method to the second-best method on each dataset. We perform interval comparison at \(p=0.05\) and \(p=0.01\) and find significantly best results on 6 datasets, 3 achieved by \(Q^{2}\) and 3 by the ANLI-based model.

Given that no single method outperformed the rest on all datasets, we hypothesize that the NLI and QG-QA based metrics are complementary. We test this by averaging the \(Q^{2}\), ANLI and SCZS scores per example18 (_Ensemble_ in Table 3). Indeed, averaging the three methods yields better results on most datasets and on average, with an increase of 4.5 in ROC AUC from the best single-metric result.

Footnote 18: Pairwise ensembles are reported in the appendix, Table 9.

Our results show that a single metric can do well across all tasks and datasets, with all 3 best metrics scoring higher than 80 on average on the 11 datasets. This corroborates our hypothesis that evaluating factual consistency can be unified, and we hope such unified perspective will be adopted in future work to accelerate progress on the subject.

## 5 Analysis

Input LengthAs QA and NLI models may struggle with long inputs Kocisky et al. (2018); Pang et al. (2021); Yin et al. (2021); Shaham et al. (2022), metrics based on them may fail when handling long text. To study the effect of input length on the metrics performance, we unify all datasets19 and split examples into 6 bins according to the grounding length.20 We focus on the grounding as the target texts are usually short (see Table 7 in Appendix A). We measure AUC of the best 3 metrics according to their overall score for each length bin, sampling 1,000 examples per bin.

Footnote 19: Excluding VitaminC as it is much larger than other datasets and might therefore distort results. Statistics regarding the grounding and target text lengths per dataset is in Appendix A.

Footnote 20: We measure length in tokens (before subword splitting) as different metrics use different subword tokenizations.

The results are shown in Figure 1. We find that there is a consistent degradation for texts longer than 200 tokens for all metrics, including SCZS which is designed to better handle long text. We find it surprising that the ANLI-based model and \(Q^{2}\) still do relatively well on the longest bin (with AUC > 0.825) as they perform end-to-end QA and NLI on text with more than 500 tokens.

Model SizeModel-based metrics are expected to benefit from increasing the model size. To quantify this we study the effect of using smaller models for the ANLI, BLEURT and BERTScore metrics. We compare the average ROC AUC of larger and

\begin{table}
\begin{tabular}{|l||c|c|c|c|c|c|c|c|c|} \hline  & **Ensemble** & **QU\({}_{\text{metrics}}\)** & **ANLI** & **SCZS** & **F1** & **BLEURT** & **QuestEval** & **FactCC** & **BERT\({}_{\text{semo}}\)** & **BERT\({}_{\text{score}}\)** \\ \hline
**FRNK** & 91.2 & 87.8 & **89.4** & 89.1 & 76.1 & 82.8 & 84.0 & 76.4 & 86.1 & 84.3 \\ \hline
**SummEval** & 82.9 & 78.8 & 80.5 & **81.7** & 61.4 & 66.7 & 70.1 & 75.9 & 73.5 & 77.2 \\ \hline
**MNBM** & 76.6 & 68.7 & **77.9*** & 71.3 & 64.2 & 64.5 & 65.3 & 59.4 & 60.9 & 62.8 \\ \hline
**QAGS-C** & 87.7 & **83.5** & 82.1 & 80.9 & 63.8 & 71.6 & 64.2 & 76.4 & 80.9 & 69.1 \\ \hline
**QAGS-X** & 84.8 & 70.9 & **83.8** & 78.1 & 51.1 & 57.2 & 56.3 & 64.9 & 53.8 & 49.5 \\ \hline
**BEGIN** & 86.2 & 79.7 & 82.6 & 82.0 & 86.4 & 86.4 & 84.1 & 64.4 & 86.3 & **87.9** \\ \hline
**Q\({}_{\text{dataset}}\)** & 82.8 & **80.9* & 72.7 & 77.4 & 65.9 & 72.4 & 72.2 & 63.7 & 64.9 & 70.0 \\ \hline
**DialFeat** & 90.4 & **86.1*** & 77.7 & 84.1 & 72.3 & 73.1 & 77.3 & 55.3 & 65.6 & 64.2 \\ \hline
**PAWS** & 91.2 & **89.7*** & 86.4 & 88.2 & 51.1 & 68.3 & 69.2 & 64.0 & 77.5 & 77.5 \\ \hline
**FEVER** & 94.7 & **88.4** & **93.2*** & 93.2 & 51.8 & 59.5 & 72.6 & 61.9 & 64.1 & 63.3 \\ \hline
**VitaminC** & 96.1 & 81.4 & **88.3*** & 97.9 & 61.4 & 61.8 & 66.5 & 56.3 & 63.2 & 62.5 \\ \hline \hline
* +/no NIC, FEVER*
* & 86.0 & 80.7 & **81.5*
* & 81.4 & 63.8 & 71.4 & 71.4 & 66.7 & 72.2 & 71.4 \\ \hline \end{tabular}
\end{table}
Table 3: ROC AUC results for the different metrics on the TRUE development set. We exclude VitaminC and FEVER from the average calculation as SCZS was trained on VitaminC that includes examples from FEVER. The highest score in each row (excluding the Ensemble) is in bold and the aforementioned SC results are in strikethrough. Statistically significant results are indicated using
* and *
* for \(p<0.05\) and \(p<0.01\) respectively.
 

smaller model variants for each metric. The ablation results are in Table 4. We find an advantage of 4.7, 3.7 and 1.3 average ROC AUC for the larger ANLI, BLEUWt and BERTScore variants respectively, showing that larger models indeed allow for better factual consistency evaluation metrics, and hinting at potential improvements from using even larger models.

Qualitative Analysis.We conduct manual error analysis to point at weaknesses of the different metrics and present challenges posed by the task. We analyze 80 examples that were misclassified by all three best metrics, as well as 100 examples that were correctly classified by one or two of the three.

Out of the analyzed examples, many seem to have a wrong label. This is especially true for cases in which all best metrics failed, with annotation errors in 35/80 cases. For the cases where one or two metrics failed, we found annotation errors in 27/100 cases. To verify that the high annotation error rate is indeed a result of inspecting the "hardest" examples and not a general issue in the datasets we used, we uniformly sample 100 additional examples, finding that only 10 had annotation errors. We therefore stress that the high misannotation rate indeed characterizes "hard" examples only, and is not a general property of the datasets we used. This is inline with the findings of Freitag et al. (2021), who showed that in some cases, metrics may be "better" than non-expert annotators. These findings demonstrate the potential of automatic methods in "cleaning" training data by filtering factually inconsistent examples.

Despite showing impressive results, the best-performing metrics fail to detect subtle inconsistencies, as presented in Table 5. This was the case for 21/180 analyzed examples. Metrics that aggregate scores across parts of a target text, such as \(Q^{2}\) or SCZS, might assign a high score for texts in which all but a small part is consistent. End-to-end NLI should predict "contradiction" even when only a small part of the text contradicts the grounding, but it may fail to do so. Applying a strict approach in the aggregation step, like taking the minimum instead of the average, could potentially remedy this - with the price of having more false-negatives. Other errors are caused by domain-specific challenges, such as handling personal statements in dialogues. As shown in Table 5, such statements may be falsely classified as ungrounded. This was the case for 10/62 analyzed dialogue responses. A possible way to alleviate this would be to automatically exclude non-factual parts from the evaluation.

Ensemble Analysis.As shown in SS4, a simple averaging ensemble using the three best metrics achieves strong results, outperforming individual metrics on most datasets. To understand this further, we analyze cases in which at least one of the best three metrics failed, while the ensemble succeeded. Overall, there were 25,761 such cases, where in 85.2% of these cases, two out of the three metrics succeeded, and only one failed. In 14.6% of these cases, one metric succeeded while the other two failed, and only in 0.2% of the cases, the ensemble succeeded while all metrics failed. These cases are a result of the different threshold used for the ensemble model vs. the thresholds for the individual metrics. We sample 100 of these examples and manually analyze them. Out of the sampled examples, 47% were misclassified by one metric only, where this metric assigned a borderline score - i.e., close to the decision threshold. 36% of these examples were misclassified by one metric only, and also with a non-borderline score - i.e., the metric was far from a correct prediction. Other cases include two, or even three, erroneous metrics.

## 6 Related Work

Adding to the related work mentioned throughout the paper, works on unified evaluation of text

Figure 1: ROC AUC when splitting TRUE’s data according to the grounding length.

 

generation across tasks include GEM (Gehrmann et al., 2021), where the focus is on evaluating system outputs and not the factual consistency evaluation methods as in TRUE. BEAMetrics (Scialom and Hill, 2021) proposes meta-evaluation protocols across tasks, but does not focus on factual consistency. When discussing consistency ("correctness") they measure correlations, which are not sufficient as mentioned in Section 2.3. Chen et al. (2021) present an adversarial meta-evaluation for factual consistency evaluators, focused on summarization. Other works on meta-evaluation of factual consistency across datasets include GO-FIGUREE (Gabriel et al., 2021) FRANK (Pagnoni et al., 2021) SummaC (Laban et al., 2021) and QAFactEval (Fabbri et al., 2021b), however they all focus solely on summarization. Yeh et al. (2021) conduct a thorough assessment of dialog metrics, however not specifically around factual consistency. To the best of our knowledge, our work is the first to generalize the discussion on evaluating factual consistency across tasks and datasets, and the first to show that large-scale QG-QA and NLI are strong and highly complementary - setting better baselines and meta-evaluation methodology for future work.

## 7 Discussion and Future Work

We discuss the main takeaways of the TRUE study, pointing at actionable insights for future work. First, as QG-QA and NLI-based methods show better performance than other approaches, especially when combined together, we recommend model developers to use those methods for evaluation when factual consistency is a priority. As for metric developers, we recommend using those methods and the datasets in TRUE when evaluating new metrics.

We also suggest reporting ROC AUC rather than correlations, as it is more interpretable and actionable. Our proposed binary annotation scheme allows to easily test new metrics across tasks and datasets, which would be useful for future work.

Finally, we encourage data curators to use the binary annotation scheme, which is inline with the recommendations of Rashkin et al. (2021a). Having said that, we do not rule out more detailed labeling schemes - but rather ask to provide a protocol for converting such labels into the more general binary format. Future work may also address the challenges of long inputs and personal statements in dialogue, which we point out in our analysis.

## 8 Conclusions

We presented TRUE, a survey and assessment of automatic factual consistency evaluation methods. We standardized various datasets from diverse tasks into a unified labeling scheme to perform a thorough comparison of automatic evaluation methods, showing that large scale NLI and QG-QA based approaches perform well _across_ multiple tasks and datasets. We further show that these methods are highly complementary - hinting at additional headroom for improvement while pointing on current limitations. We hope our results and methodology will encourage a more unified perspective in future work to foster progress towards more factually-consistent NLP applications.

## Acknowledgements

We thank Dipanjan Das, Sebastian Gehrmann and Joshua Maynez for their valuable comments and suggestions for this work.

\begin{table}
\begin{tabular}{|p{96.7pt}|l|p{145.5pt}|} \hline
**Grounding** & **Generated Text** & **Explanation** \\ \hline The word “bilutely” is the English version of the French word ”philatelie”, coined by Georges Herpin in 1864. & The word philately is actually a fresh word coined by george herpin. & The word philately is an English word _based_ on a French word, but not French. All best metrics misclassified. \\ \cline{2-3}   French police have interviewed presidential candidate francois filidate francois fillon and his wife penolope over claims she was paid for fake work. They provided information that would help find the ’truth’, mr film said... & French presidential candidate francois flion has said he and his wife penolope have been questioned by police over claims she worked illegally. & Most details are correct and the hallucination is subtle. In the case of \(Q^{2}\), most of the generated questions have the same answer based on the grounding and the generated text, therefore the overall score was high. \\ \cline{2-3}  & Stamp collecting is generally accepted as one of the areas that make up the wider subject of philately, which is the study of stamps. & I've never heard of stamps, but I do know that the word ”philately” refers to the study of stamps. & The personal statement “I’ve never heard of stamps” is notfactual and should not be evaluated. \\ \hline Evidence suggests that cognitive behavioral therapy and a gradual increase in activity suited to individual capacity can be beneficial in some cases. & It has been suggested that cognitive behavioural therapy and gradual increase in exercise could help in some cases so I’m going to try that for now. & Similar to the previous examples – SummaC and ANL falsely marked the text as inconsistent, probably due to the personal statement. \\ \hline \end{tabular}
\end{table}
Table 5: Examples for the error analysis. The first two rows show cases of challenging inconsistencies, while the last two show dialogue responses containing non-factual personal statements.

 



## References

* B. Bowman, G. Angeli, C. Potts, and C. D. Manning (2015)A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 632-642. External Links: Link Cited by: SS1.
* G. Brown and G. Yule (1983)Discourse analysis. Cambridge University Press.. Cited by: SS1.
* Y. Chen, P. Liu, and X. Qiu (2021)Are factuality checkers reliable? adversarial meta-evaluation of factuality in summarization. In Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic, pp. 2082-2095. External Links: Link Cited by: SS1.
* I. Dagan, O. Glickman, and B. Magnini (2006)The pascal recognising textual entailment challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Textual Entailment, Berlin, Heidelberg, pp. 177-190. External Links: ISBN 978-3-319-142-4, Document, Link Cited by: SS1.
* M. Deng, B. Tan, Z. Liu, E. Xing, and Z. Hu (2021)Compression, transduction, and creation: a unified framework for evaluating natural language generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 7580-7605. External Links: Link Cited by: SS1.
* E. Denton, M. Diaz, I. Kivlichan, V. Pradhakaran, and R. Rosen (2021)Whose ground truth? accounting for individual and collective identities underlying dataset annotation. arXiv preprint arXiv:2112.04554. Cited by: SS1.
* D. Deutsch, T. Bedrax-Weiss, and D. Roth (2021)Towards question-answering as an automatic metric for evaluating the content quality of a summary. Transactions of the Association for Computational Linguistics9, pp. 774-789. External Links: ISSN 0018-921X, Document, Link Cited by: SS1.
* D. Deutsch, R. Dror, and D. Roth (2022)Re-examining system-level correlations of automatic summarization evaluation metrics. arXiv preprint arXiv:2204.10216. Cited by: SS1.
* J. Devlin, M. Chang, K. Lee, and K. Toutanova (2019)BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171-4186. External Links: Link Cited by: SS1.
* Emily Dinan, S. Roller, K. Shuster, A. Fan, M. Auli, and J. Weston (2019)Wizard of Wikipedia: knowledge-powered conversational agents. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: SS1.
* E. Durmus, H. He, and M. Diab (2020)FEQA: a question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp. 5055-5070. External Links: Link Cited by: SS1.
* N. Dziri, H. Rashkin, T. Linzen, and D. Reitter (2021)Evaluating groundedness in dialogue systems: The begin benchmark. Cited by: SS1.
* Bradley Efron (1982)The jackknife, the bootstrap and other resampling plans. SIAM. Cited by: SS1.
* Alexander R Fabbri, W. Kryscinski, B. McCann, C. Xiong, R. Socher, and D. Radev (2020)Summeval: re-evaluating summarization evaluation. arXiv preprint arXiv:2007.12626. Cited by: SS1.
* Alexander R. Fabbri, W. Kryscinski, B. McCann, C. Xiong, R. Socher, and D. Radev (2021)SummEval: re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics9, pp. 391-409. External Links: Link Cited by: SS1.
* A. R. Fabbri, W. Kryscinski, B. McCann, C. Xiong, R. Socher, and D. Radev (2021)Summeval: re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics9, pp. 391-409. External Links: Link Cited by: SS1.
* T. Falke, L. F. R. Ribeiro, P. Ajie Utama, I. Dagan, and I. Gurevych (2019)Ranking generated summaries by correctness: an interesting but challenging application for natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 2214-2220. External Links: Link Cited by: SS1.
* C. Fillmore (1976)Frame semantics and the nature of language *. Annals of the New York Academy of Sciences280. Cited by: SS1.
* Markus Freitag, G. Foster, D. Grangier, V. Ratnakar, Q. Tan, and W. Macherey (2021)Experts, errors, and context: a large-scale study of human evaluation for machine translation. arXiv preprint arXiv:2104.14478. Cited by: SS1.
* Saadia Gabriel, A. Celikyilmaz, R. Jha, Y. Choi, and J. Gao (2021)GO FIGURE: a meta evaluation of factuality in summarization. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online, pp. 478-487. External Links: Link Cited by: SS1.



Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammannachi, Anoluwapo Aremu, Antoine Bosselut, Khyathi Raghavich Chandu, Miruna-Adriana Clinciu, Dipjanjan Das, Kaustubb Dhole, Wanyu Du, Esin Durmus, Ondrej Dusek, Chris Chinenye Emzeue, Varun Gangal, Cristina Garbacea, Tatsumori Hashimoto, Yufang Hou, Yacine Jernite, Harsh Jhamtani, Yangfeng Ji, Shailza Jolly, Mihir Kale, Drhur Kumar, Faisal Ladhak, Aman Madaaan, Mounica Maddela, Khryati Mahajan, Saad Mahamood, Bodhisistwra Prasad Majumder, Pedro Henrique Martins, Angelina McMillanMajor, Simon Mille, Emiel van Miltenburg, Moin Nadeem, Shashi Narayan, Vitaly Nikolaev, Andre Niyongabo Rubango, Salomey Osei, Ankur Parikh, Laura Perez-Beltrachini, Niranjan Ramesh Rao, Vikas Raunak, Juan Diego Rodriguez, Sashank Santhanara, Joao Sedoc, Thibault Stellam, Samira Shaikh, Anastasia Shimorina, Marco Antonio Sobrevilla Cabezudo, Hendrik Strobelt, Nishant Subramani, Wei Xu, Diyi Yang, Akhila Yerukola, and Jiawei Zhou. 2021. The GEM benchmark: Natural language generation, its evaluation and metrics. In _Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)_, pages 96-120, Online. Association for Computational Linguistics.
* Gupta et al. [2021] Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2021. Dialfact: A benchmark for fact-checking in dialogue. _arXiv preprint arXiv:2110.08222_.
* He et al. [2021] Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: Decoding-enhancedbert with disentangled attention. In _International Conference on Learning Representations_.
* Heidegger [2001] Martin Heidegger. 2001. On the essence of truth. _The Nature of Truth: Classic and Contemporary Perspectives_, 1:295-316.
* Hermann et al. [2015] Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In _Advances in Neural Information Processing Systems_, volume 28. Curran Associates, Inc.
* Honovich et al. [2021] Or Honovich, Leshem Choshen, Roee Aharoni, Ella Neeman, Idan Szpekltor, and Omri Abend. 2021. Evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 7856-7870, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Hu et al. [2019] J. Edward Hu, Abhinav Singh, Nils Holzenberger, Matt Post, and Benjamin Van Durme. 2019. Large-scale, diverse, paraphrastic bitexts via sampling and clustering. In _Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)_, pages 44-54, Hong Kong, China. Association for Computational Linguistics.
* Kocisky et al. [2018] Tomas Kocisky, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gabor Melis, and Edward Grefenstette. 2018. The NarrativeQA reading comprehension challenge. _Transactions of the Association for Computational Linguistics_, 6:317-328.
* Kryscinski et al. [2020] Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 9332-9346, Online. Association for Computational Linguistics.
* Laban et al. [2021] Philippe Laban, Tobias Schnabel, Paul N Bennett, and Marti A Hearst. 2021. Summmac: Re-visiting misbased models for inconsistency detection in summarization. _arXiv preprint arXiv:2111.09525_.
* Lan et al. [2019] Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Simpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. _arXiv preprint arXiv:1909.11942_.
* Lee et al. [2018]Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannijiang, and David Sussillo. 2018. Hallucinations in neural machine translation. _NeurlPS 2018 Workshop on Interpretability and Robustness for Audio, Speech, and Language_.
* Lewis et al. [2020] Mike Lewis, Yinhan Liu, Naman Goyal, Marijan Ghazvininejad, Abdelrahaman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7871-7880, Online. Association for Computational Linguistics.
* Lin [2004] Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In _Text Summarization Branches Out_, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
* Liu et al. [2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. _arXiv preprint arXiv:1907.11692_.
* Maynez et al. [2020] Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 1906-1919, Online. Association for Computational Linguistics.
* Nangia et al. [2017] Nikita Nangia, Adina Williams, Angelixi Lazaridou, and Samuel Bowman. 2017. The RepEval 2017 shared task: Multi-genre natural language inference 

with sentence representations. In Proceedings of the 2nd Workshop on Evaluating Vector Space Representations for NLP, Copenhagen, Denmark, pp. 1-10. External Links: Link Cited by: SS3.
* S. Narayan, S. B. Cohen, and M. Lapata (2018)Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 1797-1807. External Links: Link Cited by: SS4.
* Y. Nie, H. Chen, and M. Bansal (2019)Combining fact extraction and verification with neural semantic matching networks. In Association for Advancement of Artificial Intelligence (AAAI), Cited by: SS5.
* Y. Nie, A. Williams, E. Dinan, M. Bansal, J. Weston, and D. Kiela (2020)Adversarial nli: a new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp. 4885-4901. External Links: Link, Document Cited by: SS6.
* Y. Nie, M. Williamson, M. Bansal, D. Kiela, and J. Weston (2021)I like fish, especially dolphins: addressing contradictions in dialogue modeling. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, pp. 1699-1713. External Links: Link, Document Cited by: SS1, SS2.
* A. Pagnoni, V. Balachandran, and Y. Tsvetkov (2021)Understanding factuality in abstractive summarization with rrank: a benchmark for factuality metrics. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online, pp. 4812-4829. External Links: Link, Document Cited by: SS1, SS2.
* M. Palmer, D. Gildea, and P. Kingsbury (2005)The Proposition Bank: an Annotated Corpus of Semantic Roles. Computational Linguistics31 (1), pp. 71-106. External Links: ISSN 0002-9993, Link, Document Cited by: SS1, SS2.
* R. Yuanzhe Pang, A. Parrish, N. Joshi, N. Nangia, J. Phang, A. Chen, V. Padmakumar, J. Ma, J. Thompson, He He, et al. (2021)Quality: question answering with long input texts, yes!. arXiv preprint arXiv:2112.08608. External Links: Link, 2112.08608 Cited by: SS1, SS2.
* K. Papineni, S. Roukos, T. Ward, and W. Zhu (2002)Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, pp. 311-318. External Links: Link, Document Cited by: SS1, SS2.
* A. Parikh, X. Wang, S. Gehrmann, M. Farauqui, B. Dhingra, D. Yang, and D. Das (2020)ToTPC: a controlled table-to-text generation dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online, pp. 1173-1186. External Links: Link, Document Cited by: SS1, SS2.
* A. Pu, H. W. Chung, A. Parikh, S. Gehrmann, and T. Sellam (2021)Learning compact metrics for mt. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 751-762. External Links: Link, Document Cited by: SS1, SS2.
* L. Qin, T. Xie, S. Huang, Q. Chen, X. Xu, and W. Che (2021)Don't be contradicted with anything! cd-toD: towards benchmarking consistency for task-oriented dialogue system. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, pp. 2357-2367. External Links: Link, Document Cited by: SS1, SS2.
* Colin Raffel, Noam Shazeer, A. Roberts, Katherine Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2020)Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research21 (140), pp. 1-67. External Links: Link, Document Cited by: SS1, SS2.
* Hannah Rashkin, V. Nikolaev, M. Lamm, M. Collins, D. Das, S. Petrov, G. Tomer, I. Turc, and D. Reitter (2021)Measuring attribution in natural language generation models. arXiv preprint arXiv:2112.12870. External Links: Link, 2112.12870 Cited by: SS1, SS2.
* H. Rashkin, D. Reitter, G. Singh Tomar, and D. Das (2021)Increasing faithfulness in knowledge-grounded dialogue with controllable features. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online, pp. 704-718. External Links: Link, Document Cited by: SS1, SS2.
* E. Reiter and C. Thomson (2020)Shared task on evaluating accuracy. In Proceedings of the 13th International Conference on Natural Language Generation, Dublin, Ireland, pp. 227-231. External Links: Link, Document Cited by: SS1, SS2.
* A. Rohrbach, L. A. Hendricks, K. Burns, T. Darrell, and K. Saenko (2018)Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4035-4045. External Links: Link, Document Cited by: SS1, SS2.
* Tal Schuster, A. Fisch, and R. Barzilay (2021)Get your vitamin c! robust fact verification with contrastive evidence. In Proceedings of the 2021 Conference of the North American Chapter of the Asso 

ciation for Computational Linguistics: Human Language Technologies_, pages 624-643, Online. Association for Computational Linguistics.
* Scialom et al. (2021) Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, Jacopo Staiano, Alex Wang, and Patrick Gallinari. 2021. QuestEval: Summarization asks for fact-based evaluation. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 6594-6604, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Scialom and Hill (2021) Thomas Scialom and Felix Hill. 2021. Beametrics: A benchmark for language generation evaluation evaluation. _arXiv preprint arXiv:2110.09147_.
* Sellam et al. (2020a) Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020a. BLEURT: Learning robust metrics for text generation. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7881-7892, Online. Association for Computational Linguistics.
* Sellam et al. (2020b) Thibault Sellam, Amy Pu, Hyung Won Chung, Sebastian Gehrmann, Qijun Tan, Markus Freitag, Dipanjan Das, and Ankur Parikh. 2020b. Leaming to evaluate translation beyond English: BLEUt submissions to the WMT metrics 2020 shared task. In _Proceedings of the Fifth Conference on Machine Translation_, pages 921-927, Online. Association for Computational Linguistics.
* Shaham et al. (2022) Uri Shaham, Elad Segal, Maor Ivyi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, and Omer Levy. 2022. Scrolls: Standardized comparison over long language sequences.
* Thorne et al. (2018) James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2018. The fact extraction and VERification (FEVER) shared task. In _Proceedings of the First Workshop on Fact Extraction and VERifaction (FEVER)_, pages 1-9, Brussels, Belgium. Association for Computational Linguistics.
* Wang et al. (2020) Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the factual consistency of summaries. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 5008-5020, Online. Association for Computational Linguistics.
* Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_, pages 353-355, Brussels, Belgium. Association for Computational Linguistics.
* Welleck et al. (2019) Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019. Dialogue natural language inference. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 3731-3741, Florence, Italy. Association for Computational Linguistics.
* Xie et al. (2021) Yuexiang Xie, Fei Sun, Yang Deng, Yaliang Li, and Bolin Ding. 2021. Factual consistency evaluation for text summarization via counterfactual estimation. In _Findings of the Association for Computational Linguistics: EMNLP 2021_, pages 100-110, Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Yeh et al. (2021) Yi-Ting Yeh, Maxine Eskenazi, and Shikib Mehri. 2021. A comprehensive assessment of dialog evaluation metrics. In _The First Workshop on Evaluations and Assessments of Neural Conversation Systems_, pages 15-33, Online. Association for Computational Linguistics.
* Yin et al. (2021) Wenpeng Yin, Dragomir Radev, and Caiming Xiong. 2021. DocNLI: A large-scale dataset for document-level natural language inference. In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_, pages 4913-4922, Online. Association for Computational Linguistics.
* Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. BARTScore: Evaluating generated text as text generation. In _Advances in Neural Information Processing Systems_.
* Zhang et al. (2020) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. In _International Conference on Learning Representations_.
* Zhang et al. (2019) Yuan Zhang, Jason Baldridge, and Luheng He. 2019. PAWS: Paraphrase adversaries from word scrambling. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 1298-1308, Minneapolis, Minnesota. Association for Computational Linguistics.
* Zhao et al. (2020) Zheng Zhao, Shay B Cohen, and Bonnie Webber. 2020. Reducing quantity hallucinations in abstractive summarization. _arXiv preprint arXiv:2009.13312_.
* Zhou et al. (2021) Chunting Zhou, Graham Neubig, Jiatao Gu, Mona Diab, Francisco Guzman, Luke Zettlemoyer, and Marjan Ghazvininejad. 2021. Detecting hallucinated content in conditional neural sequence generation. In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_, pages 1393-1404, Online. Association for Computational Linguistics 

Additional Data Statistics

Tables 6 and 7 presents statistics regarding the length of the grounding text and the generated text for TRUE's datasets, respectively.

## Appendix B Implementation Details

We train all models using the t5x library.21

Footnote 21: [https://github.com/google-research/t5x](https://github.com/google-research/t5x)

QG-QAFor our reimplementation of \(Q^{2}\)Honovich et al. (2021) we use T5-11B as the pretrained model for QG, QA and NLI, while Honovich et al. (2021) used T5-Base, ALBERT Lan et al. (2019), and RoBERTa Liu et al. (2019) for the QG, QA and NLI models, respectively. We use a maximum length of 2048 tokens for the input. We set the F1 token overlap threshold to 0.54 by tuning it on a held-out dataset. We use beam search with a beam size of 4 to generate multiple questions, and use the first question that passes the validation threshold.

NliWe fine-tune a T5-11B model on ANLI Nie et al. (2020) for \(25K\) steps with a learning rate of \(10^{-4}\) and a batch size of 32. During inference we use a maximum input length of 2048 tokens.

## Appendix C ROC Curves

Figure 2 presents the ROC curves for the different datasets studied in TRUE, using the best-performing metrics.

 

* [14] M. C. Krizhevsky, A. Vedaldi, and R. Fergus. Learning from deep networks: a survey of machine learning models. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 257-260, June 2013.
* [15] J. L. Friedman, S. Ghahramani, and D. Ramanan. Deep residual learning for image recognition. In _Advances in neural information processing systems_, volume 89 of _Lecture Notes in Computer Science_, pages 1-10, 2002.
* [16] P. F. Donoho, E. Rabinovich, and I. Sutskever. Very accurate representations of tiny images by exploiting features. _IEEE Transactions on Image Processing Systems_, 28(1):127-132, 2002.
* [17] T. Darrell, N. Torr, and Y. Bengio. Generative adversarial nets with convolutional layers. In _NIPS conference on computer vision_, pages 1388-1390, 2013.
* [18] H. Chen, X. Li, and Z. Sun. Deep residual learning for image recognition. In _International Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 235-240, July 2013.

[MISSING_PAGE_POST]

 



\begin{table}
\begin{tabular}{|l||c c|c|c|c|c|c|c|} \hline  & **Ensemble** & **Q2** & **ANLI** & **SCzS** & **BLEURT** & **QuestEval** & **FactCC** & **BART\({}_{\text{seme}}\)** & **BERT\({}_{\text{score}}\)** \\ \hline
**FRANK** & 89.6 & 78.8 & **89.2** & 88.6 & 83.2 & 86.4 & 73.9 & 88.3 & 86.0 \\ \hline
**EEGIN** & 85.9 & 78.0 & 82.8 & 84.2 & 82.2 & 81.4 & 65.0 & 83.7 & **86.0** \\ \hline
**DialFact** & 88.6 & **85.0** & 75.9 & 82.1 & 72.2 & 76.3 & 55.1 & 65.5 & 64.3 \\ \hline
**PAWS** & 92.4 & **90.1** & 87.3 & 89.7 & 67.1 & 70.1 & 65.1 & 77.3 & 76.4 \\ \hline
**VitaminC** & 96.7 & 83.4 & **89.6** & **98.4** & 63.0 & 67.8 & 56.8 & 64.1 & 63.5 \\ \hline \hline
**Avg. += vtc,**voc & 89.4 & 85.2 & 83.8 & **86.2** & 76.2 & 78.5 & 64.8 & 78.7 & 78.2 \\ \hline \end{tabular}
\end{table}
Table 10: ROC AUC results for the different metrics on the TRUE test set. We exclude Vitamin from the average calculation as SC\({}_{\text{ZS}}\) was trained on VitaminC. The highest score in each row (excluding the Ensemble) is in bold and the aforementioned SC results are in striktethrough.

\begin{table}
\begin{tabular}{|l||c|c|c|c|c|c|c|c|c|} \hline  & **Ensemble** & **Q2** & **ANLI** & **SCzS** & **BLEURT** & **QuestEval** & **FactCC** & **BART\({}_{\text{seme}}\)** & **BERT\({}_{\text{score}}\)** \\ \hline
**FRANK** & 83.2 & 82.9 & **83.5** & 80.9 & 67.5 & 76.0 & 74.2 & 73.9 & 80.2 & 77.0 \\ \hline
**SummEval** & 79.4 & **77.3** & 72.9 & 75.0 & 64.2 & 66.4 & 67.0 & 65.3 & 68.9 & 73.7 \\ \hline
**MNBM** & 69.6 & 66.5 & **66.7** & 64.5 & 40.4 & 53.7 & 66.1 & 62.9 & 61.0 & 60.1 \\ \hline
**QAGS-C** & 81.3 & **78.3** & 75.3 & 72.8 & 62.6 & 68.9 & 62.6 & 70.6 & 74.5 & 66.8 \\ \hline
**OAGGS-X** & 77.4 & 63.6 & **80.3** & 72.8 & 51.0 & 56.5 & 56.9 & 65.3 & 54.4 & 52.3 \\ \hline
**BEGIN** & 77.9 & 73.0 & 76.0 & 79.7 & 80.0 & 78.1 & 75.5 & 66.6 & 77.4 & **80.5** \\ \hline
**Q2\({}_{\text{attr}}\)** & 76.8 & **76.1** & 66.6 & 71.3 & 61.4 & 67.0 & 65.8 & 60.5 & 59.7 & 66.1 \\ \hline
**DIaIet** & 83.4 & **80.5** & 70.9 & 75.8 & 66.4 & 67.0 & 69.9 & 53.4 & 60.8 & 61.2 \\ \hline
**FAWS** & 83.4 & **83.5** & 80.5 & 81.5 & 50.2 & 64.2 & 64.3 & 60.2 & 72.2 & 71.1 \\ \hline
**FFVER** & 90.4 & 82.7 & **90.2** & 87.6 & 52.3 & 57.7 & 69.4 & 57.2 & 60.5 & 58.7 \\ \hline
**VitiminC** & 90.8 & 76.0 & **82.3** & 93.5 & 58.0 & 58.0 & 62.2 & 55.0 & 59.1 & 58.6 \\ \hline
**Avg. += vtc,**voc & 79.2 & **75.7** & 74.7 & 74.9 & 60.4 & 66.4 & 66.9 & 64.3 & 67.6 & 67.6 \\ \hline \end{tabular}
\end{table}
Table 11: Accuracy results for the different metrics on the TRUE development set. Note that thresholds were tuned on the development set itself. We exclude VitaminC and FEVER from the average calculation as SC\({}_{\text{ZS}}\) was trained on VitaminC that includes examples from FEVER. The highest score in each row (excluding the Ensemble) is in bold and the aforementioned SC results are in strikethrough.

\begin{table}
\begin{tabular}{|l||c|c|c|c|c|c|c|c|} \hline  & **ENemble** & **Q2** & **ANLI** & **SCzS** & **BLEURT** & **QuestEval** & **FactCC** & **BART\({}_{\text{seme}}\)** & **BERT\({}_{\text{score}}\)** \\ \hline
**FRANK** & 89.6 & 91.1 & 90.4 & 87.5 & 88.9 & 80.1 & 78.0 \\ \hline
**SummEval** & 80.7 & 83.0 & 82.0 & 76.0 & 79.8 & 68.8 & 60.2 \\ \hline
**MNBM** & 75.6 & 77.1 & 74.6 & 72.3 & 67.2 & 47.5 & 49.3 \\ \hline
**QAGS-C** & 86.0 & 84.7 & 86.4 & 73.4 & 79.6 & 67.1 & 63.9 \\ \hline
**QAGS-X** & 81.8 & 85.1 & 79.3 & 73.1 & 76.1 & 52.9 & 48.6 \\ \hline
**BFGEN** & 85.7 & 82.1 & 85.7 & 77.9 & 81.6 & 86.4 & 84.6 \\ \hline Q^2 & 83.0 & 76.9 & 83.9 & 85.3 & 77.5 & 66.8 & 64.3 \\ \hline
**DiMartect** & 89.4 & 84.5 & 90.2 & 83.5 & 81.2 & 71.2 & 72.5 \\ \hline
**PAWS** & 90.5 & 89.7 & 91.4 & 86.0 & 88.2 & 82.2 & 77.3 \\ \hline
**FEVER** & 94.0 & 94.6 & 92.9 & 84.8 & 86.7 & 49.9 & 51.1 \\ \hline
**VItaminC** & 90.3 & 96.4 & 96.5 & 84.9 & 94.5 & 59.9 & 59.6 \\ \hline \hline
**Avg. += vtc,**voc & 84.7 & 83.8 & 84.9 & 79.4 & 80.0 & 69.2 & 66.5 \\ \hline \end{tabular}
\end{table}
Table 9: ROC AUC results for metrics that were not reported in Table 3.