SlowGuess/NAACL_2021_OCR huggingface dataset:
Table 7 outlines our results. The results show that models trained on Ent-C perform slightly better than those trained on Gen-C, but many of the systems are in the same range, with accuracy values of around  $75\%$ . However, the reported accuracy values on held-out Ent-C/Gen-C examples are consistently over  $90\%$  (results included in Appendix B). This demonstrates that while models trained on these factuality datasets are able to fit the synthetic data distributions well, these are inherently different from actual generation errors. The Appendix also includes graphs of how the human annotated dev set performance varies with training iterations, showing that constant performance on the held-out training set corresponds with highly fluctuating performance on the human annotated data, further
Table 2: Label-balanced accuracy of factuality models when trained on synthetic factuality training datasets in the CNN/DAILYMAIL domain. Performance is reported on the human-annotated test set from Kryscinski et al. (2020).
Train Data	Majority	Sent-Fact	DAE	DAE-Weak
Ent-C	50	50.9	51.2	53.6
Gen-C	50	54.2	53.0	51.6
Table 3: Performance of factuality models trained on synthetic factuality datasets in the XSUM domain. Label-balanced accuracy is reported on 500 examples from the human-annotated test set from Maynez et al. (2020).
indicating that these settings are not identical.
XSUM Next, we similarly evaluate the synthetic datasets and factuality models on the more challenging XSUM domain. Again, we evaluate on a human annotated dataset collected by prior work (Maynez et al., 2020). The dataset contains span highlights indicating hallucinated/incorrect content or information with respect to the source article for 4 different summarization models trained on the XSUM domain (as well as for gold summaries). Figure 1 illustrates this. Similar to prior work, if any word in a summary is marked as hallucinated, we mark the sentence as non-factual. Therefore, for XSUM-HUMAN, the annotation is available at both the sentence-level and span-level.
In total, this dataset contains  $2500(A,S)$  pairs (along with their factuality labels). We use 500 examples from these to construct our test dataset. The remaining 2000 examples are used to train models, explained in Section 5.2.
Table 3 outlines the results. Unlike on CNN/DM, we see that all models trained on synthetic factuality datasets perform very poorly, achieving close to the majority label baseline. Again, the performance on the held-out synthetic datasets was observed to be very high (see Appendix B). There is a fundamental difference between the errors that are produced by XSUM summarization models and those introduced by artificial data corruption mechanisms. Other data that more closely resembles the generation errors is needed to train factuality models in this setting.
5.2 Human Annotated Dataset Evaluation
To investigate whether human annotated data is useful to train factuality models, we train our 3 factuality models on the remaining 2000 human annotated examples from XSUM-HUMAN. In order to train DAE model on this dataset, we use the span highlights to derive dependency-level gold annotations, using the same strategy from 2.3 (illustrated
5This techniques resembles posterior regularization (Ganchev et al., 2010); however, these constraints are enforced in a hard way on individual examples rather than in expectation at the corpus level. It can also be viewed as an instance of constraint-driven learning (Chang et al., 2007).
Model	Balanced-Acc
Sent-Factuality	65.6
DAE	78.7
DAE-Weak	70.9
in Figure 1).
The results are shown in Table 4. Comparing these with results from Table 3, we see that a small number of human annotated examples can outperform large auto-generated training datasets by a large margin. Notably, we see that availability of fine-grained factuality annotations significantly boosts performance, with models that leverage that information (DAE) significantly outperforming sentence-level models. Even in the absence of fine-grained annotations, we see that the DAE-Weak model that decomposes the error computation and explicitly tries to localize errors is better than the sentence-level model.
However, these factuality models do not transfer to CNN/DM: the best model achieves an accuracy of 55.9, substantially lower than  $76.7\%$  in Table 7. This demonstrates that summarization models make different types of errors on different domains, and data collection and modelling efforts for factuality should account for these differences.
6 Localization of errors
Our evaluation so far has focused on the sentence-level performance of factuality models. Next, we evaluate the models' ability to localize errors within the generated summary as well as show how such a capability can be leveraged to train less error-prone summarization models.
6.1 Localizing Factuality on XSUM
We evaluate the error localization performance of the models at two granularity levels: (1) Dependency arc-level and (2) Word-level. Table 5 outlines the results of our experiments.
The DAE model outperforms the DAE-Weak model at both levels of granularity. This reiterates our earlier claim that fine-grained annotations lead to better factuality models with more
Table 4: Comparison of different factuality models when trained on human annotated data and evaluated on XSUM (compare to Table 3). Fine-grained annotations provide a big boost in performance.
Model	Precision	Recall	F1
Dependency-level
DAE	69.7	78.2	73.7
DAE-Weak	54.9	76.6	63.9
Word-level
DAE	57.5	74.7	65.0
DAE-Weak	56.2	62.3	59.1
DAE (best-ckpt)	62.0	83.9	71.3
Table 5: Error localization comparison of the different factuality models. The DAE model achieves high recall for both word-level and dependency-level factuality.
reliable localization. However, the DAE-Weak model is able to achieve comparable recall at the dependency-level; both models are more recall-oriented, which is desirable for certain applications.
For Section 6.2, we select our DAE model's best checkpoint on the test data (best-ckpt), which achieves a recall of 83.9, a significant gain if we directly optimize for this metric.
6.2 Downstream Applications
Localizing errors potentially allows for post-hoc correction (Zhao et al., 2020; Cao et al., 2020); however, repairing a summary to be fully factual is a very hard problem and past work has focused on a subset of errors as a result. Instead, we show that even our imperfect error localization techniques can be used to meaningfully improve the training data for summarization. We use our DAE model to identify unsupported facts in the XSUM training data and ignore the corresponding tokens when training our summarization model.
Training on a subset of tokens Summaryization models are trained to maximize the log likelihood of the summary given the source article:  $\mathcal{L} = \sum_{i = 1:|S|}\log p(S_i|D,S_{1:i - 1})$  . When a word in the summary is non-factual, training on it encourages the model to hallucinate new content. In our approach, we modify the training objective to only maximize the likelihood of factual words in the summary, factuality being determined by the DAE model from the previous sections:  $\mathcal{L} = \sum_{i = 1:|S|}M_{i}\log p(S_{i}|D,S_{1:i - 1})$  where  $M_{i} = 1$  if the word  $w_{i}$  is factual, otherwise  $M_{i} = 0$  . A similar objective has been used by prior work (Song et al., 2020b) to encourage the model to copy words present in the source.
We compare our approach with two systems: a baseline model trained without this masking and
<sup>6</sup>We can approximately extract word-level decision from the dependency-level predictions: if any arc containing word  $w$  is non-factual, then  $w$  is non-factual; otherwise, it is factual.
Model	Word-ER ↓	Sent-ER ↓	Human ↑
Baseline	32.2	74.0	37.4
Loss trunc	31.1	70.9	39.1
DAE-based (ours)	23.7	61.4	46.5
Table 6: Comparison of the different summarization models. Our proposed approach achieves significantly lower word error rates, sentence error rates and are rated higher by human annotators.
a model using the loss truncation technique well-suited for noisy datasets from Kang and Hashimoto (2020). All models are trained on 50k examples using BART summarization model initialized from the BART-XSUM-LARGE checkpoint. For all these approaches, summaries generated on the original XSUM test set (11k examples) are compared. $^7$
Evaluation First, we use our trained DAE model to evaluate the performance of our summarization models. That is, we generate summaries for all examples in the test set using the three models; the DAE model is then used to compute the word error rate (fraction of words determined to be non-factual according to the DAE model) and the sentence error rate (fraction of sentences determined to be non-factual). Table 6 outlines the results, which show that our DAE-masked training leads to better factuality performance.
Next, we perform human evaluation to compare the factuality of summaries generated by the three models using Amazon Mechanical Turk. We randomly sampled 50 articles from the test set and generated summaries corresponding to the 3 models. We asked 7 human annotators to classify each (article, summary) pair as either factual (score = 1) or non-factual (score = 0). An average score is computed for each summary by aggregating the 7 annotator scores. Table 6 reports the average summary scores for the 50 (article, summary) pairs across the 3 summarization models. The results show that the proposed approach outperforms both the baseline model and the loss truncation approach. This demonstrates that factuality models trained on a small number of annotated examples can be used to train factual summarization models, even when the underlying summarization dataset is noisy.
Earlier work on abstraction (Barzilay et al., 1999; Carenini and Cheung, 2008) and compression (Knight and Marcu, 2000; Berg-Kirkpatrick et al., 2011; Woodsend and Lapata, 2012; Durrett et al., 2016) in summarization has typically focused evaluation on content selection and grammaticality, with little heed paid to factuality. Human evaluation similarly focused on content selection (Gillick and Liu, 2010). Methods such as Pyramid (Nenkova and Passonneau, 2004) that could have in principle been used to evaluate factuality were primarily used to understand content selection.
Recent work has explored different methods for enforcing factuality: modifying the model, such as encoding SRL structures in the input (Cao et al., 2018), post-hoc correction (Dong et al., 2020), or constrained decoding (Song et al., 2020a; Mao et al., 2020). However, these techniques fundamentally struggle to handle the whole range of factual errors; factuality is a fuzzy notion and cannot be easily encapsulated into a set of discrete rules.
Faithfulness and factuality have also been tackled in related tasks, including summarizing radiology reports (Zhang et al., 2020b) and data-to-text generation tasks (Tian et al., 2019). Another recent line of work has looked at fact verification (Thorne et al., 2018; Nie et al., 2019; Atanasova et al., 2020). In this literature, the claims are usually human-authored and a straightforward statement of a fact, whereas generated summaries might feature claims buried in nominal modifiers like two-time winner.
In this work, we showed that existing synthetic datasets are not well-suited to factuality evaluation of recent summarization models (like BART) in challenging domain (like XSUM). Models trained on human-annotated data, especially those that leverage fine-grained annotations, can enable training of more factual summarization models. We hope future work will explore better modeling and data creation to address the pressing issues in current systems.
This work was partially supported by NSF Grant IIS-1814522, a gift from Salesforce Inc, and an equipment grant from NVIDIA. Thanks as well to Jiacheng Xu and the anonymous reviewers for their helpful comments.
To ensure fair comparison between the different models, we removed the examples from XSUM-HUMAN used to train the factuality models from our test set.
See Appendix E for more details about the task design.
Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, and Isabelle Augenstein. 2020. Generating fact checking explanations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7352-7364.
Regina Barzilay, Kathleen R. McKeown, and Michael Elhadad. 1999. Information fusion in the context of multi-document summarization. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 550-557, College Park, Maryland, USA. Association for Computational Linguistics.
Taylor Berg-Kirkpatrick, Dan Gillick, and Dan Klein. 2011. Jointly Learning to Extract and Compress. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 481-490.
Meng Cao, Yue Dong, Jiapeng Wu, and Jackie Chi Kit Cheung. 2020. Factual error correction for abstractive summarization models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6251-6258.
Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018. Faithful to the original: Fact aware neural abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
Giuseppe Carenini and Jackie C. K. Cheung. 2008. Extractive vs. NLG-based abstractive summarization of evaluative text: The effect of corpus controversiality. In Proceedings of the Fifth International Natural Language Generation Conference, pages 33-41, Salt Fork, Ohio, USA. Association for Computational Linguistics.
Ming-Wei Chang, Lev Ratinov, and Dan Roth. 2007. Guiding semi-supervision with constraint-driven learning. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 280-287.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171-4186.
Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, and Jingjing Liu. 2020. Multi-fact correction in abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9320-9331.
Esin Durmus, He He, and Mona Diab. 2020. FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization. In Proceedings of the 58th Annual Meeting
of the Association for Computational Linguistics, pages 5055-5070.
Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. 2016. Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 1998-2008.
Alexander R Fabbri, Wojciech Krysciński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021. SummEval: Re-evaluating summarization evaluation. Transactions of the Association for Computational Linguistics (TACL).
Tobias Falke, Leonardo FR Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking generated summaries by correctness: An interesting but challenging application for natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2214-2220.
Kuzman Ganchev, Joao Graça, Jennifer Gillenwater, and Ben Taskar. 2010. Posterior regularization for structured latent variable models. The Journal of Machine Learning Research, 11:2001-2049.
Dan Gillick and Yang Liu. 2010. Non-expert evaluation of summarization systems is risky. In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, pages 148-151, Los Angeles. Association for Computational Linguistics.
Tanya Goyal and Greg Durrett. 2020a. Evaluating factuality in generation with dependency-level entailment. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 3592-3603.
Tanya Goyal and Greg Durrett. 2020b. Neural syntactic preordering for controlled paraphrase generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 238-252, Online. Association for Computational Linguistics.
Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in neural information processing systems, pages 1693-1701.
Daniel Kang and Tatsunori Hashimoto. 2020. Improved natural language generation via loss truncation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 718-731, Online. Association for Computational Linguistics.
Kevin Knight and Daniel Marcu. 2000. Statistics-Based Summarization—Step One: Sentence Compression. In Proceedings of the National Conference on Artificial Intelligence (AAAI) and Confer
ence on Innovative Applications of Artificial Intelligence (IAAI), pages 703-710.
Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332-9346.
Logan Lebanoff, John Muchovej, Franck Dernoncourt, Doo Soon Kim, Seokhwan Kim, Walter Chang, and Fei Liu. 2019. Analyzing sentence fusion in abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, Hong Kong, China. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. Association for Computational Linguistics.
Klaus-Michael Lux, Maya Sappelli, and Martha Larson. 2020. Truth or Error? Towards systematic analysis of factual errors in abstractive summaries. In Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, pages 1-10, Online. Association for Computational Linguistics.
Yuning Mao, Xiang Ren, Heng Ji, and Jiawei Han. 2020. Constrained Abstractive Summarization: Preserving Factual Consistency with Constrained Generation. In arXiv preprint 2010.12723.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan Thomas Mcdonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of The 58th Annual Meeting of the Association for Computational Linguistics (ACL).
Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gulcehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 280-290.
Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807.
Ani Nenkova and Rebecca Passonneau. 2004. Evaluating content selection in summarization: The pyramid method. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational
Linguistics: HLT-NAACL 2004, pages 145-152, Boston, Massachusetts, USA. Association for Computational Linguistics.
Yixin Nie, Haonan Chen, and Mohit Bansal. 2019. Combining fact extraction and verification with neural semantic matching networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6859-6866.
Kaiqiang Song, Logan Lebanoff, Qipeng Guo, Xipeng Qiu, Xiangyang Xue, Chen Li, Dong Yu, and Fei Liu. 2020a. Joint parsing and generation for abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8894-8901.
Kaiqiang Song, Bingqing Wang, Zhe Feng, Ren Liu, and Fei Liu. 2020b. Controlling the amount of verbatim copying in abstractive summarization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8902-8909.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERIFICATION. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, Louisiana. Association for Computational Linguistics.
Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P Parikh. 2019. Sticking to the facts: Confident decoding for faithful data-to-text generation. arXiv preprint arXiv:1910.08684.
Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and Answering Questions to Evaluate the Factual Consistency of Summaries. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumont, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R'emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface's transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771.
Kristian Woodsend and Mirella Lapata. 2012. Multiple Aspect Summarization Using Integer Linear Programming. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing (EMNLP) and Computational Natural Language Learning (CoNLL).
Fang-Fang Zhang, Jin-ge Yao, and Rui Yan. 2018. On the abstractiveness of neural document summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 785-790.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020a. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the International Conference on Machine Learning.
Yuhao Zhang, Derek Merck, Emily Tsai, Christopher D. Manning, and Curtis Langlotz. 2020b. Optimizing the factual correctness of a summary: A study of summarizing radiology reports. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. Association for Computational Linguistics.
Zheng Zhao, Shay B Cohen, and Bonnie Webber. 2020. Reducing quantity hallucinations in abstractive summarization. In *Findings of EMNLP*.
A Manual Annotation of Errors
In Section 3, we outline the error distributions of multiple factuality datasets. The distributions were obtained by combing manual annotations from two authors of this paper. On a common set of 50 summaries annotated by both authors, we observe the following: (1) Both authors agreed on what spans/hallucinations within a summary constitute an error  $74\%$  of the times. (2) In cases when both authors marked a common span as erroneous, they agreed on the error category  $84\%$  of the time.
B Synthetic Dataset Performance on held-out samples
Section 5.1 evaluates the performance of models trained on the synthetic datasets on human annotated test sets for two summarization domains. Here, we report the model performance on heldout tests datasets that are constructed in the same way as the training datasets. Table 7 presents these results. For both domains, we see that the models report very high performance indicating that they are able to fit the distribution of the synthetic domain. However, we see in Section 5.1 that the performance is significantly lower on actual generation outputs, with close to majority label baseline performance on the more challenging XSUM domain. This means that the two datasets have inherently different error distributions.