On Faithfulness and Factuality in Abstractive Summarization (2020)

Abstract: It is well known that the standard likelihood training and approximate decoding objectives in neural text generation models lead to less human-like responses for open-ended tasks such as language modeling and story generation. In this paper we have analyzed limitations of these models for abstractive document summarization and found that these models are highly prone to hallucinate content that is unfaithful to the input document. We conducted a large scale human evaluation of several neural abstractive summarization systems to better understand the types of hallucinations they produce. Our human annotators found substantial amounts of hallucinated content in all model generated summaries. However, our analysis does show that pretrained models are better summarizers not only in terms of raw metrics, i.e., ROUGE, but also in generating faithful and factual summaries as evaluated by humans. Furthermore, we show that textual entailment measures better correlate with faithfulness than standard metrics, potentially leading the way to automatic evaluation metrics as well as training and decoding criteria.

## introduction current state of the art conditional text generation models accomplish a high level of fluency and coherence, mostly thanks to advances in sequenceto-sequence architectures with attention and copy @xcite @xcite , fully attention-based transformer architectures @xcite and more recently pretrained language modeling for natural language understanding @xcite @xcite . there has been a growing interest in understanding how maximum likelihood training and approximate beam-search decoding in these models lead to less human-like text in open-ended text generation such as language modeling and story generation @xcite @xcite . in this paper we investigate how these models are prone to generate hallucinated text in conditional text generation, specifically, extreme abstractive document summarization @xcite . document summarization -the task of producing a shorter version of a document while preserving its information content @xcite -requires models to generate text that is not only human-like but also faithful and/or factual given the document. the example in figure 1 illustrates that the faithfulness and factuality are yet to be conquered by conditional text generators. the article describes an event of "conservative mp zac smith winning the primary for 2016 london mayoral election", but summaries often forge entities (e.g., "nigel goldsmith" or "zac goldwin") or information (e.g., "ukip leader nigel goldsmith", "nigel goldsmith winning the mayoral election", "sadiq khan being the former london mayor" or "zac goldwin being the labour's candidate") that are not supported by the document or are factually wrong. interestingly, all summaries are topical and fluent, and perform well in terms of rouge scores @xcite . we conducted a large-scale human evaluation of hallucinated content in systems that use recurrent neural network (rnn) @xcite , convolutional neural network (cnn) @xcite , and transformers @xcite , as well as human written summaries for the recently introduced extreme summarization task (xsum, @xcite . we seek to answer the following questions: (i) how frequently do abstractive summarizers hallucinate content?; (ii) do models hal-gold zac goldsmith will contest the 2016 london mayoral election for the conservatives, it has been announced. document: the richmond park and north kingston mp said he was "honoured" after winning 70% of the 9,227 votes cast using an online primary system. he beat london assembly member andrew boff, mep syed kamall and london's deputy mayor for crime and policing stephen greenhalgh. mr goldsmith's main rival is likely to be labour's sadiq khan. (2 sentences with 59 words are abbreviated here.) mr goldsmith, who was the favourite for the tory nomination, balloted his constituents earlier this year to seek permission to stand. at the very point of his entry into the race for london mayor, zac goldsmith's decision revealed two big characteristics. ## hallucinations in summarization open-ended generation -the task of generating text that forms a natural continuation from the input text -requires the model to hallucinate text; hence the focus has been to ensure that the model learns to generate text that is more human-like (i.e., less repetitive or dull with more content-related words) @xcite @xcite . in contrast, tasks such as document summarization @xcite @xcite and data-to-text generation @xcite which are not open-ended, require models to be factual and/or faithful to the source text. despite recent improvements in conditional text generation, most summarization systems are trained to maximize the log-likelihood of the reference summary at the word-level, which does not necessarily reward models for being faithful. moreover, models are usually agnostic to the noises or artifacts of the training data, such as reference divergence, making them vulnerable to hallucinations @xcite @xcite . thus, models can generate texts that are not consistent with the input, yet would likely have reasonable model log-likelihood. ## extreme document summarization we focus on the recently introduced extreme summarization dataset (xsum, narayan et al., 2018a) foot_2 which comprises 226,711 british broadcasting corporation (bbc) articles paired with their singlesentence summaries, provided by the journalists writing the articles. the dataset is split into three subsets: training (90%, 204,045), validation (5%, 11,332), and test (5%, 11,334) sets. all models in §4 trained to generate abstractive summaries are trained and evaluated using this standard split, provided by the authors. we choose to focus our study on extreme summarization for the following reasons: first, this task aims to create a single-sentence summary of a news article; these shorter summaries are relatively easier to annotate and analyze than longer summaries such as story highlights from the cnn/dailymail dataset @xcite or abstracts from the ny times @xcite or the wikisum @xcite dataset. secondly, the gold summary in the extreme summarization dataset is an introductory sentence prefacing each article. by virtue of this property, the extreme summarization task is not amenable to extractive strategies and requires an abstractive modeling approach. hence, it provides us a better benchmark to assess abstractive models' abilities to produce abstractions which are faithful and factual. finally, since we conclude that hallucination is a problem on this dataset, then we can safely conclude it is a problem for summarization datasets with longer summaries, as modeling longer-distance dependencies and discourse structures make the task harder. ## abstractive summaries we evaluate summaries from rnn, cnn and transformer-based state-of-the-art abstractive summarization methods and the reference human writ-ten summaries. see the appendix for hyperparameter and decoding details for all models. human written reference summaries. the single-sentence summaries contained in the extreme summarization dataset (xsum) are also evaluated as part of this study. these summaries were written by journalists as introductions to the news articles they precede. these summaries, therefore, often have true additional information not found in the document. such divergence issue between source and target is not uncommon in conditional text generation @xcite @xcite . 2018a ) is an abstractive system which is conditioned on the article's topics and based entirely on convolutional neural networks @xcite . tconvs2s is better suited for extreme summarization, as convolution layers capture long-range dependencies between words in the document more effectively than rnns. simultaneously, the convolutional encoder associates each word with a topic vector, capturing whether it is representative of the document's content. transformer-based abstractive methods. we experiment with three transformer-based model variants, all of which have 12 layers, a hidden size of 768, filter size of 3072, and 12 attention heads. ## experiments and results the main focus of this work is not to propose a solution to hallucination related issues, but to achieve a better understanding of hallucinations in abstractive summarization through their human assessment. we randomly sampled 500 articles from the test set to facilitate our study. using the full test set was unfeasible given its size and the cost of human judgments. we have trained annotators (fluent in english) specifically for our assessment. our annotators went through two pilot studies to have a better understanding of intrinsic and extrinsic hallucinations, and factuality of summaries. documents used in the pilot studies were not used in the final annotation. we also report on rouge (lin and hovy, 2003) scores, bertscore @xcite and semantic inference metric such as textual entailment @xcite @xcite and question answering @xcite . ## related work following the document understanding conference (duc; @xcite , a majority of work has focused on evaluating the content and the linguistic quality of summaries @xcite . most popular among them is the automatic metric rouge @xcite that measures the unigram and bigram overlap (rouge-1 and rouge-2) as a proxy for assessing informativeness and the longest common subsequence (rouge-l), for fluency. rouge, however, can be misleading when used as the only means to assess the informativeness of summaries @xcite . hence, the rouge score is often complemented with subjective human assessment of summaries. more objective measures have been proposed to improve agreement among human annotators. pyramid method @xcite requires summaries to be annotated by experts for salient information. @xcite used a questionanswering based approach where a summary is used as context to answer questions which were written based on its reference summary. @xcite proposed a reference-less approach where a summary is assessed against the source document, highlighted with its pertinent content. there has not been much work on evaluating faithfulness and truthfulness of abstractive summaries. the automatic evaluation such as rouge and the human evaluation of saliency and linguistic quality of summaries are not sufficient due to the complex nature of the task. recently, @xcite asked human annotators to assess the summary relevance measuring both the saliency and the presence of contradictory/unrelated information. @xcite proposed a new automatic metric, parent, for data-to-text generation @xcite which aligns n-grams from the reference and generated texts to the source table to measure the accuracy of n-grams that are entailed from the source table. @xcite proposed a modelbased automatic metric to assess the faithfulness of wikipedia summaries; they trained an end-to-end model to extract a complete set of openie-style @xcite facts from both the source text and the generated summary. the summary is faithful if it is precise in generating facts from the source text. in our experiments with openiebased measures, we found that they are not suited for evaluating extreme summarization models; all models perform poorly on these metrics without any significant differences. like ours, few recent works (some in parallel) have explored natural language inference and question answering models to detect factual consistency in generated text @xcite @xcite . in line with our findings, @xcite observed that the bert-based nli models substantially improved summaries reranking in terms of their correctness. @xcite proposed an nli-based fact checking model that is trained on a dataset tailored for detecting factual inconsistencies in generated text. @xcite proposed a question answering and generation based automatic evaluation protocol that is designed to identify factual inconsistencies in a generated summary. future work will likely investigate better ways of generating questions and measuring factual consistency to address poor correlation with faithfulness and factuality annotations. finally, others have used reinforcement learning to improve informativeness and reduce contradictory information in abstractive summaries, e.g., @xcite used a textual entailment-based reward and @xcite , a question-answering based reward. however, these approaches don't evaluate if these rewards improve faithfulness of summaries. ## conclusion we conducted a large-scale study of hallucinations in abstractive document summarization. we found that (i) tackling hallucination is a critical challenge for abstractive summarization, perhaps the most critical, (ii) nlu-driven pretraining in neural text generators is key to generate informative, coherent, faithful and factual abstracts, but it is still far from solving the problem; and (iii) measures such as rouge or bertscore will not be sufficient when studying the problem; semantic inference-based automatic measures are better representations of true summarization quality.