Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization (2022)

Abstract: Neural abstractive summarization models are prone to generate summaries that are factually inconsistent with their source documents. Previous work has introduced the task of recognizing such factual inconsistency as a downstream application of natural language inference (NLI). However, state-of-the-art NLI models perform poorly in this context due to their inability to generalize to the target task. In this work, we show that NLI models can be effective for this task when the training data is augmented with high-quality task-oriented examples. We introduce Falsesum, a data generation pipeline leveraging a controllable text generation model to perturb human-annotated summaries, introducing varying types of factual inconsistencies. Unlike previously introduced document-level NLI datasets, our generated dataset contains examples that are diverse and inconsistent yet plausible. We show that models trained on a Falsesum-augmented NLI dataset improve the state-of-the-art performance across four benchmarks for detecting factual inconsistency in summarization.

## introduction recent advances in conditional text generation and the availability of large-scale datasets have given rise to models which generate highly fluent abstractive summaries @xcite . however, studies indicate that such models are susceptible to generating factually inconsistent outputs, i.e., where the content of the summary is not semantically entailed by the input document @xcite . this motivates a new line of research for recognizing factual inconsistency in generated summaries @xcite @xcite . this factual consistency problem is closely related to the task of natural language inference (nli) whereby a hypothesis sentence is classified as either entailed, neutral, or contradicted by a given premise sentence @xcite @xcite . using an input document as the premise and a corresponding generated summary as the hypothesis, earlier solutions have adopted out-of-the-box nli models to detect factual inconsistency, albeit with limited success @xcite . this poor performance largely stems from the fact that most nli datasets are not designed to reflect the input characteristics of downstream tasks @xcite . such datasets may not always capture the kinds of entailment phenomena which naturally arise from neural abstractive summarization. more importantly, there is also a discrepancy in terms of the input granularity, i.e., the premises in this consistency classification task consist of multi-sentence documents while common nli datasets use single-sentence premises. in this work, we introduce falsesum, a data generation pipeline that produces nli examples consisting of documents paired with gold summaries as positive examples and automatically generated inconsistent summaries as negative examples. we propose a novel strategy to train a text generation model to render false summaries of a given document using only supervision from an existing summarization dataset @xcite . in addition, our generator supports switchable input control codes to determine the type of factual error exhibited in the generated output. this design allows falsesum to compose diverse and naturalistic outputs which more closely resemble the inconsistent summaries generated by summarization models @xcite . this contrasts with previous solutions (e.g., @xcite , which synthesize nli examples using rule-based transformations or language model-based replacements, limiting their diversity and ability to reflect realistic factual errors in summarization. overall, our contributions in this paper are the following: first, we present a novel training pipeline to create a text generation model which takes as input a pair of a document and a corresponding gold summary. it then perturbs the summary such that it is no longer factually consistent with the original document. our strategy obviates the need for explicit examples of inconsistent summaries, using only an existing summarization dataset. we use this model to generate a large-scale nli dataset for the task of recognizing factually inconsistent summaries. the resultant dataset consists of pairs with documents as the premise and naturalistic summaries as the hypotheses, each labeled as either entailment or non-entailment. second, we demonstrate the utility of our generated data for augmenting existing nli datasets. we show that on four benchmark datasets, nli models trained on falsesum-augmented data outperform those trained on previous document-level nli datasets. we conduct an analysis to show that falsesum-generated summaries are plausible and hard to distinguish from human-written summaries. lastly, we show that the improvement over the benchmarks is largely attributable to the diversity of factual errors that falsesum introduces. ## related work this work is related to the growing body of research into factual consistency and hallucination in text generation models, particularly for summa-rization @xcite . research has found that around 30% of summaries generated by abstractive summarization models contain information which is inconsistent with the source document @xcite . this motivates the development of an automatic approach to assess factual consistency in generated summaries, in addition to the benchmark datasets to measure the progress in this task @xcite @xcite . earlier work by @xcite proposes to use an information extraction model to extract relation tuples from the ground-truth summary text and the generated summary and then count the overlap as the measure of factuality. @xcite al. (2019); durmus et al. (2020); wang et al. (2020 ) use a question-answering model to detect factual inconsistency by matching the predicted answers using the document and the summary as the context. concurrently, researchers have drawn a connection between factual consistency and natural language inference (nli), observing that all information in a summary should be entailed by the source document. while this approach enables the summary to be directly evaluated without first extracting its intermediate semantic structure, earlier attempts were largely unsuccessful. @xcite use the probabilities assigned to the entailment label by nli models to re-rank the summary candidates given by beam search but found no improvement in the consistency errors. @xcite evaluate out-of-the-box nli models on the task of inconsistency detection in a binary classification setting and show that the performance is only slightly better than majority voting. in the same paper, @xcite pro-pose factcc, a synthetic nli data generation process which applies a set of transformation rules to obtain examples of inconsistent summaries (e.g., sentence negation, entity swapping). they demonstrate that the resulting nli model performs well on realistic test cases which are obtained by manually annotating the output of several summarization models. this highlights the importance of nli examples beyond sentence-level granularity and which more closely resemble the input characteristics of the downstream tasks @xcite . 2 while the factcc model is moderately effective for detecting factual inconsistency, subsequent work indicates that it only performs well on easier test cases, where highly extractive summaries (i.e., those with high lexical overlap between a summary and the source document) tend to be factually consistent and more abstractive summaries are likely to be inconsistent @xcite . furthermore, @xcite show that the synthetic and rule-based nature of factcc leads to lack of diversity of consistency error types and it poorly aligns with the error distribution found in more abstractive summaries. falsesum addresses these limitations using controlled natural language generation to construct an nli dataset which better targets the summarization domain. inspired by the recent work on controllable generation @xcite , we employ a generation model conditioned on an input code which controls the type of consistency errors induced. we further use the generated document-level nli examples for augmentation and show that nli models can benefit from the additional data without hurting their existing inference ability @xcite . ## experimental settings our experiments aim to demonstrate the effectiveness of falsesum-generated document-level examples for nli dataset augmentation. we evaluate the downstream performance of the nli models by testing them against several benchmarks for determining the factual inconsistency of generated summaries. in this section, we describe the training setup of the nli models, including the model and both the sentence-and document-level datasets.