SlowGuess/NAACL_2022_OCR huggingface dataset:
FALSE: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization
Prasetya Ajie Utama†Joshua Bambrick† Nafise Sadat Moosavi‡Iryna Gurevych
† Bloomberg, London, United Kingdom
$\diamond$  UKP Lab, Technical University of Darmstadt, Germany
$\ddagger$  Department of Computer Science, The University of Sheffield
{putama,jbambrick7}@bloomberg.net
Neural abstractive summarization models are prone to generate summaries which are factually inconsistent with their source documents. Previous work has introduced the task of recognizing such factual inconsistency as a downstream application of natural language inference (NLI). However, state-of-the-art NLI models perform poorly in this context due to their inability to generalize to the target task. In this work, we show that NLI models can be effective for this task when the training data is augmented with high-quality task-oriented examples. We introduce FALSESUM, a data generation pipeline leveraging a controllable text generation model to perturb human-annotated summaries, introducing varying types of factual inconsistencies. Unlike previously introduced document-level NLI datasets, our generated dataset contains examples that are diverse and inconsistent yet plausible. We show that models trained on a FALSESUM-augmented NLI dataset improve the state-of-the-art performance across four benchmarks for detecting factual inconsistency in summarization.<sup>1</sup>
Recent advances in conditional text generation and the availability of large-scale datasets have given rise to models which generate highly fluent abstractive summaries (Lewis et al., 2019; Zhang et al., 2019). However, studies indicate that such models are susceptible to generating factually inconsistent outputs, i.e., where the content of the summary is not semantically entailed by the input document (Kryscinski et al., 2019; Goodrich et al., 2019). This motivates a new line of research for recognizing factual inconsistency in generated summaries (Kryscinski et al., 2020; Pagnoni et al., 2021; Wang et al., 2020; Fabbri et al., 2021).
This factual consistency problem is closely related to the task of natural language inference (NLI) whereby a hypothesis sentence is classified as either entailed, neutral, or contradicted by a given premise sentence (Condoravdi et al., 2003; Dagan et al., 2006; Bowman et al., 2015). Using an input document as the premise and a corresponding generated summary as the hypothesis, earlier solutions have adopted out-of-the-box NLI models to detect factual inconsistency, albeit with limited success (Falke et al., 2019; Kryscinski et al., 2020).
This poor performance largely stems from the fact that most NLI datasets are not designed to reflect the input characteristics of downstream tasks (Khot et al., 2018). Such datasets may not always capture the kinds of entailment phenomena which naturally arise from neural abstractive summarization. More importantly, there is also a discrepancy in terms of the input granularity, i.e., the premises in this consistency classification task consist of multi-sentence documents while common NLI datasets use single-sentence premises.
In this work, we introduce FALSESUM, a data generation pipeline that produces NLI examples consisting of documents paired with gold summaries as positive examples and automatically generated inconsistent summaries as negative examples. We propose a novel strategy to train a text generation model to render false summaries of a given document using only supervision from an existing summarization dataset (Nallapati et al., 2016). In addition, our generator supports switchable input control codes to determine the type of factual error exhibited in the generated output. This design allows FALSESUM to compose diverse and naturalistic outputs which more closely resemble the inconsistent summaries generated by summarization models (Maynez et al., 2020). This contrasts with previous solutions (e.g., Kryscinski et al., 2020; Yin et al., 2021), which synthesize NLI examples using rule-based transformations
The code to obtain the dataset is available online at https://github.com/joshbambrick/Falsesum
Human Language Technologies, pages 2763 - 2776
Figure 1: Overview of the FALSESUM generation framework. FALSESUM preprocesses and formats the source document (A) and a gold summary (B) before feeding it to a fine-tuned generator model. The model produces a factually inconsistent summary, which can then be used to obtain (A, D) or (A, E) as the negative (non-entailment) NLI premise-hypothesis example pair. We also use the original (A, B) as a positive NLI example (entailment).
or language model-based replacements, limiting their diversity and ability to reflect realistic factual errors in summarization. Overall, our contributions in this paper are the following:
First, we present a novel training pipeline to create a text generation model which takes as input a pair of a document and a corresponding gold summary. It then perturbs the summary such that it is no longer factually consistent with the original document. Our strategy obviates the need for explicit examples of inconsistent summaries, using only an existing summarization dataset. We use this model to generate a large-scale NLI dataset for the task of recognizing factually inconsistent summaries. The resultant dataset consists of pairs with documents as the premise and naturalistic summaries as the hypotheses, each labeled as either entailment or non-entailment.
Second, we demonstrate the utility of our generated data for augmenting existing NLI datasets. We show that on four benchmark datasets, NLI models trained on FALSESUM-augmented data outperform those trained on previous document-level NLI datasets. We conduct an analysis to show that FALSESUM-generated summaries are plausible and hard to distinguish from human-written summaries. Lastly, we show that the improvement over the benchmarks is largely attributable to the diversity of factual errors that FALSESUM introduces.
This work is related to the growing body of research into factual consistency and hallucination in text generation models, particularly for summa
rization (Cao et al., 2018). Research has found that around  $30\%$  of summaries generated by abstractive summarization models contain information which is inconsistent with the source document (Kryscinski et al., 2019). This motivates the development of an automatic approach to assess factual consistency in generated summaries, in addition to the benchmark datasets to measure the progress in this task (Falke et al., 2019; Kryscinski et al., 2020; Pagnoni et al., 2021; Fabbri et al., 2021).
Earlier work by Goodrich et al. (2019) proposes to use an information extraction model to extract relation tuples from the ground-truth summary text and the generated summary and then count the overlap as the measure of factuality. Eyal et al. (2019); Durmus et al. (2020); Wang et al. (2020) use a question-answering model to detect factual inconsistency by matching the predicted answers using the document and the summary as the context.
Concurrently, researchers have drawn a connection between factual consistency and natural language inference (NLI), observing that all information in a summary should be entailed by the source document. While this approach enables the summary to be directly evaluated without first extracting its intermediate semantic structure, earlier attempts were largely unsuccessful. Falke et al. (2019) use the probabilities assigned to the entailment label by NLI models to re-rank the summary candidates given by beam search but found no improvement in the consistency errors. Kryscinski et al. (2020) evaluate out-of-the-box NLI models on the task of inconsistency detection in a binary classification setting and show that the performance is only slightly better than majority voting.
In the same paper, Kryscinski et al. (2020) pro
pose FactCC, a synthetic NLI data generation process which applies a set of transformation rules to obtain examples of inconsistent summaries (e.g., sentence negation, entity swapping). They demonstrate that the resulting NLI model performs well on realistic test cases which are obtained by manually annotating the output of several summarization models. This highlights the importance of NLI examples beyond sentence-level granularity and which more closely resemble the input characteristics of the downstream tasks (Mishra et al., 2021).<sup>2</sup>
While the FactCC model is moderately effective for detecting factual inconsistency, subsequent work indicates that it only performs well on easier test cases, where highly extractive summaries (i.e., those with high lexical overlap between a summary and the source document) tend to be factually consistent and more abstractive summaries are likely to be inconsistent (Zhang et al., 2020). Furthermore, Goyal and Durrett (2021) show that the synthetic and rule-based nature of FactCC leads to lack of diversity of consistency error types and it poorly aligns with the error distribution found in more abstractive summaries.
FALSEism addresses these limitations using controlled natural language generation to construct an NLI dataset which better targets the summarization domain. Inspired by the recent work on controllable generation (Keskar et al., 2019; Ross et al., 2021), we employ a generation model conditioned on an input code which controls the type of consistency errors induced. We further use the generated document-level NLI examples for augmentation and show that NLI models can benefit from the additional data without hurting their existing inference ability (Min et al., 2020).
3 FALSESUM Approach
3.1 Design Overview
FALSEsum takes as an input a source document D and a corresponding reference summary  $\mathbf{S}^{+}$ . The framework then preprocesses and formats D and  $\mathbf{S}^{+}$  and feeds them into a generation model  $\mathcal{G}$  which outputs a factually inconsistent summary  $\mathbf{S}^{-}$ . For each summarization example, we then have both positive (entailment) and negative (non
entailment) NLI tuples  $(\mathrm{D},\mathrm{S}^{+},Y = 1)$ ,  $(\mathrm{D},\mathrm{S}^{-},Y = 0)$ , which consist of a document-level premise, a summary sentence, and the consistency label (1 indicates entailment).
FALSESUM aims to produce a naturalistic  $\mathbf{S}^{-}$  which is contrastive with respect to its corresponding  $\mathrm{S^{+}}$ . This means that  $\mathrm{S^{+}}$  and  $\mathrm{S^{-}}$  should be indistinguishable in their surface characteristics (e.g., style, length, vocabularies) and only differ in their factual consistency with respect to D. This ensures that the resulting NLI model learns the correct notion of factual consistency rather than discriminating based on surface features (McCoy et al., 2019). In addition to naturalness, we consider the diversity of the consistency error types exhibited by  $\mathrm{S^{-}}$ . We follow the consistency error typology introduced by Maynez et al. (2020), which categorizes consistency errors as either intrinsic, i.e., errors due to incorrect consolidation of information from the source document, or extrinsic, i.e., errors due to assuming new information not directly inferable from the contents of the source document.
As illustrated in Figure 1, a generation model  $\mathcal{G}$  is trained to imitate the consistency mistakes of summarization models. Specifically, it generates perturbed summaries by either (1) incorrectly inserting pieces of information from the source document into random spans of the original summary; or (2) amending pieces of information in the summary by hallucinating new "facts" not present in the source document.
To this end, the framework identifies  $(\diamond i)$  what information or "facts" in the source document are available to the generator; and  $(\diamond ii)$  where the incorrect information can be inserted into the gold summary, which is indicated by span masking. We obtain both by subsequently performing input preprocessing and formatting steps (§3.2 and §3.3).
Next, we define the following seq2seq task to train the model  $\mathcal{G}$ : "Given  $(\diamond i)$  a list of shuffled and formatted pieces of information extracted from source document and gold summary and  $(\diamond ii)$  a partially masked gold summary, fill in the blanks and generate the original gold summary." Note that using gold summaries means that we can apply the existing summarization corpus to train  $\mathcal{G}$  to generate more coherent and plausible sentences.
3.2 Input Preprocessing
Following Goodrich et al. (2019), "facts" in the source document and the gold summary are de
${}^{2}$  Contemporaneous work by Laban et al. (2022) attempts to improve the application of sentence-level NLI models to detect document-level factual inconsistencies using a learn-able aggregation of sentence-level predictions. Our work is orthogonal since they can benefit from better quality training examples to train their aggregation weights.
fined as an open information extraction (OpenIE) tuple, which represents the predicate and argument structures found in a sentence. We denote each relation tuple as  $(\mathrm{ARG}_0,\mathrm{PRED},\dots ,\mathrm{ARG}_n)$ , where predicate PRED describes the event (what happened) and its complementing semantic arguments ARG represent the who, to whom, where, or how of the event. Predicates are usually the main verb of a clause. Both predicates and their arguments consist of spans of tokens (Fader et al., 2011).
We use an OpenIE implementation of PredPatt (White et al., 2016; Zhang et al., 2017), a pattern-based framework for predicate-arguments extraction. As illustrated in the top half of Figure 2, we extract the relation tuples from each source document and its corresponding reference summaries. To minimize the risk of  $\mathcal{G}$  inadvertently generating consistent summaries, we corrupt each extracted "fact" by removing one randomly chosen argument from each tuple. For instance, OpenIE may extract the following tuple from a sentence:
$$
\left(\frac {\text {J o}}{\arg 0}, \frac {\text {p l a n s t o g i v e}}{\text {P R E D}}, \frac {\text {A l e x}}{\arg 1}, \frac {\text {a p p l e s}}{\arg 2}\right)
$$
We then randomly choose apples $_{\mathrm{ARG}_2}$  to be removed from the tuple. We additionally lemmatize the dependency root word of each argument and predicate span, e.g., plans to give  $\Rightarrow$  plan to give. This forces the model to learn to correct for grammaticality by inflecting the spans when inserting them to the masked spans. Once all such spans are extracted and processed, they are grouped and shuffled into two lists (predicates and arguments).
3.3 Input Formatting
Let  $\mathrm{P} = (\mathrm{PRED}_1,\dots ,\mathrm{PRED}_n)$  and  $\mathrm{A} = (\mathrm{ARG}_1,\dots ,$ $\mathrm{ARG}_m)$  be the unordered lists of extracted predicates and arguments from a source document D and the summary sentence  $\mathbf{S}^{+}$  . Additionally, we assume a masked summary sentence M (described later), derived from  $\mathbf{S}^{+}$  , and a control code variable  $c\in \{\mathrm{intrinsic},\mathrm{extrinsic}\}$  .Generator  $\mathcal{G}$  is trained to compute  $p(\mathrm{S}^{+}|\mathrm{P},\mathrm{A},\mathrm{M},c)$  .As illustrated in the bottom half of Figure 2, we encode all the conditional variables into the following format:
Predicates: P; Arguments: A; Code: c; Summary: M
In the following, we describe the key steps in the input formatting process:
Figure 2: Input format design of FALSESUM. The framework first extracts the predicate and argument spans from the source document and the gold summary. The spans are then corrupted, lemmatized, and shuffled before being inserted into the input template.
Step 1: Span Removal Initially, P and A include predicate and argument spans from the original summary which may be used to reconstruct  $\mathbf{S}^{+}$ . However, at test time we remove these "gold" spans from the two lists to force the  $\mathcal{G}$  to make consistency mistakes. The removal is also done when training the model for control code extrinsic to train  $\mathcal{G}$  to predict plausible unseen spans. We summarize the different input formatting in Table 1.
Step 2: Span Reduction To encourage  $\mathcal{G}$  to generate fine-grained errors (Pagnoni et al., 2021; Goyal and Durrett, 2021), we also train it to hallucinate incorrect modifiers into spans from P and A. To this end, we randomly drop adjectives and adverbs from  $10\%$  of the gold predicate and argument spans. For instance, an argument span "recently elected prime minister" will be reduced to "minister". This teaches the model to generate the remaining part of the span given only the context provided in the formatted input.
Step 3: Control Code To control the type of consistency errors generated by  $\mathcal{G}$ , we append the string "code:" followed by either "intrinsic" or "extrinsic" into the input tokens. The code is determined randomly with equal probability of 0.5.
<sup>4</sup>It is possible that some spans from the source document are duplicates of gold ones. For instance, the document may mention "The Queen of England", while the gold span from the summary is "The Queen". We use a simple heuristic to remove such duplicates by searching for other spans whose (lemmatized) dependency root token is the same.
<sup>3</sup>We note that the quality of the OpenIE extractions may impact the overall quality of our data generation framework.
Mode	Input	Expected Output	Description
train intrinsic	Predicates: caught, plead guilty to, ..., appear before, face; Arguments: the corruption scandal, Two Pennsylvania judges, ..., many children, the U.S. Code: intrinsic; Summary: <span_1> <span_0> federal fraud charges.	Two Pennsylvania judges plead guilty to federal fraud charges.	Model learns to combines listed spans to produce most plausible summary.
test intrinsic	Predicates: caught, plead guilty to, ..., appear before, face; Arguments: the corruption scandal, Two Pennsylvania judges, ..., many children, the U.S. Code: intrinsic; Summary: <span_1> <span_0> federal fraud charges.	Many of the children face federal fraud charges.	Model consolidates incorrect information.
train extrinsic	Predicates: is pressing for, limit, ..., is being erode, is fight; Arguments: panelist, action, ..., sea level, Arctic melt, at the climate change conference Code: extrinsic; Summary: The Alliance <span_0> <span_1> <span_2>.	The Alliance is pressing for action at the climate change conference.	Model learns to hallucinate new unsupported information.
test extrinsic	Predicates: is pressing for, limit, ..., is being erode, is fight; Arguments: panelist, action, ..., sea level, Arctic melt, at the climate change conference Code: extrinsic; Summary: The Alliance <span_0> <span_1> <span_2>.	The Alliance is planning to impose limits on emissions.	Model hallucinates new unsupported information.
Table 1: Examples of input formatting on two different summarization instances for both intrinsic and extrinsic error types during training and testing. Gold input spans (indicated by **boldface**), which are extracted from the gold summary, are only visible to the model during intrinsic training. They are removed from the input in all other settings, as indicated by **strikethrough** text.
Once the code is chosen, we perform the remaining formatting steps accordingly (see Table 1).
Step 4: Summary Masking We derive masked summary M by replacing the spans of randomly selected predicates and arguments with a special token <span_i>, where  $i = 0$  is reserved for the predicate, and  $i > 0$  for their arguments. These tokens control where the incorrect information should be inserted by the generator model into the original summary (see Table 1).
3.4 Training FALSESUM
We run the FALSESUM data generation pipeline on the train split of the CNN/DailyMail corpus (Hermann et al., 2015), originally collected for question answering, but subsequently reformulated for summarization by Nallapati et al. (2016). This dataset contains English news documents paired with human-written summaries, each consisting of multiple sentences. We break the summaries down such that each FALSESUM example consists of the document text and a single sentence summary. We then run the preprocessing and formatting steps on each document-summary pair. The resulting pairs of formatted input and target output are subsequently split into train and test sets which consist of 394,774 and 262,692 instances, respectively.
We use the T5-base model (Raffel et al., 2020) as generator  $\mathcal{G}$  and fine-tune it on the seq2seq task described in §3.1. The NLI examples are produced by running the fine-tuned generator on the preprocessed and formatted test split. This renders an equal number of positive and negative examples. In our experiments, we randomly sample 100,000 FALSE examples to augment the NLI dataset.
Our experiments aim to demonstrate the effectiveness of FALSESUM-generated document-level examples for NLI dataset augmentation. We evaluate the downstream performance of the NLI models by testing them against several benchmarks for determining the factual inconsistency of generated summaries. In this section, we describe the training setup of the NLI models, including the model and both the sentence- and document-level datasets.
NLI models We train several NLI models by fine-tuning RoBERTa-base (Liu et al., 2019) on either the original or the augmented MNLI dataset (Williams et al., 2018). The MNLI dataset consists of 392,702 train instances, each labeled
See Appendix A for the hyperparameter details.
as either "entailment", "neutral", or "contradiction". To enable the application of NLI data to this factual consistency task, we use a binary formulation of NLI, where the "neutral" and "contradiction" labels are combined into "non-entailment". The document-level inputs are formatted similarly to sentence-level examples, i.e., the document premise D and summary hypothesis  $(\mathrm{S}^{+}$  or  $\mathrm{S}^{-}$ ) are concatenated and a special classification token ([CLS]) is used (Devlin et al., 2019).
Document-level NLI datasets We conduct augmentation comparisons with several multi-sentence NLI datasets which obtain examples from news or summarization domains. We consider the following datasets: ANLI (Nie et al., 2020), a paragraph-level NLI dataset collected via an iterative and adversarial human-in-the-loop annotation protocol. It consists of mostly Wiki data but also includes a small portion of news text; DocNLI (Yin et al., 2021), a document-level NLI dataset containing multi-sentence premise and hypothesis sentences, collected by converting QA examples to NLI instances (Demszky et al., 2018) and replacing words and sentences in news summaries using a language model; FactCC (Kryscinski et al., 2020), a large-scale dataset specifically generated for training summary factual correctness classification models. The positive examples in FactCC are obtained by backtranslating a random sentence from a CNN/DailyMail news story, while negative examples are obtained by perturbing the sentence using predefined rules, e.g., entity swapping. For fair comparison, we sample 100,000 examples from each augmentation dataset in our experiments.
4.2 Benchmark Datasets
We evaluate these NLI models on four benchmark datasets to classify the factual consistency of abstractive summaries. These datasets differ in terms of the annotation protocol, the granularity of the summaries (single- or multi-sentence), the summarization corpus used, and the models used to generate the summaries that are annotated. The tasks are formulated as a binary classification with the labels "consistent" and "inconsistent". We evaluate NLI models on these tasks by mapping the predicted label "entailment" to "consistent" and "non-entailment" to "inconsistent". The benchmarks datasets are detailed in the following:
FactCC In addition introducing a synthetic training dataset for the task, Kryscinski et al. (2020)
introduce a manually annotated test set. It contains 1,431 document and single-sentence summary pairs generated by various neural abstractive summarization models trained on CNN/DailyMail corpus. $^6$