SlowGuess/NAACL_2022_OCR huggingface dataset:
Ranksum Falke et al. (2019) formulate the factual consistency problem in summarization as a ranking task. They introduce a dataset consisting of 107 documents, each paired with a set of five ranked summary candidates obtained from the beam search of a summarization model. Given the manually annotated consistency label on summary candidates, the task is to re-rank the list such that the top-1 summary is factually consistent.
Summeval Fabbri et al. (2021) introduce a comprehensive benchmark for factual consistency detection in summarization. It includes summaries generated by seven extractive models and sixteen abstractive models, which are judged by three annotators using a 5-point Likert scale. $^7$
QAGS The dataset collected by Wang et al. (2020) consists of 239 test set instances from XSUM (Narayan et al., 2018) and 714 instances from CNN/DailyMail. Each instance consists of a pair of a source document and a single-sentence summary, which is labeled via majority voting on three annotators' labels.
Performance on FactCC, QAGS, and SummEval is measured using balanced accuracy, which is suitable for class imbalanced settings, since the factually consistent label is the majority in some benchmark datasets. It is defined as the average recall of the two classes, such that majority label voting obtains only a  $50\%$  score. To measure ranking performance in Ranksum, we calculate the average Precision@1, which computes the fraction of times a factually consistent summary is ranked highest on each test instance. We perform five training runs for each setup using different random seeds and take the mean to address performance instability (Reimers and Gurevych, 2017).
6We merge the test and validation sets into a single test set.
<sup>7</sup>We aggregate the label as "consistent" if all annotators rated the summary as a 5 and "inconsistent" otherwise.
This is the number of instances after we split multisentence summaries into separate single-sentence summary test instances, where an individual factuality judgement is available.
Dataset	Augmentation	Benchmark Datasets	Overall
FactCC	Ranksum	QAGS	SummEval
Majority voting	-	50.00	50.46	50.00	50.00	50.11
MNLI-128	-	57.39	57.01	59.72	54.11	57.06
[split-doc] MNLI-128	-	72.07	68.03	71.08	55.32	66.63
MNLI-512	-	57.93	51.40	52.73	48.75	51.43
MNLI-512	ANLI	53.91	55.76	53.54	49.56	53.19
MNLI-512	DocNLI	58.13	53.58	57.10	52.59	55.35
MNLI-512	FactCC	73.87	67.29	73.50	60.04	69.02
MNLI-512	FALSESUM (ours)	83.52	72.90	75.05	65.18	74.17
Table 2: Performance of MNLI models with different augmentation data across benchmarks to classify the factual consistency of summaries. MNLI-128 and MNLI-512 are RoBERTa-base models trained using maximum token length of 128 and 512, respectively.
Training Dataset	Overall	Δ
MNLI+FALSESUM	74.17
MNLI+FALSESUM -Contrastive	73.11	-1.06
MNLI+FALSESUM -Extrinsic	71.95	-2.22
MNLI+FALSESUM -Intrinsic	69.14	-5.03
Table 3: Model performance when trained on ablated FALSESUM dataset. Excluding the contrastive, extrinsic, and intrinsic examples results in lower overall performance, indicating each property is beneficial.
From the results in Table 2, we observe the following: (1) Models trained on sentence-level MNLI datasets perform poorly when evaluated directly on document-level benchmarks, even after we increase the maximum input token length from 128 to  $512;^{9}$  (2) This limitation can be alleviated by the sentence-wise prediction strategy ([split-doc]MNLI-128), $^{10}$  which achieves 66.63. Note, however, that this improvement comes at the expense of compute cost which is multiplied by a significant factor; (3) DocNLI and ANLI perform poorly even though they contain longer premise sentences, indicating that the length mismatch may not be the primary issue; (4) FALSESUM obtains substantial improvement over the previous state-of-the-art FactCC, despite being derived from the same summarization dataset (CNN/DailyMail). This indicates that FALSESUM provides higher quality examples and includes more types of entailment phenomena that occur naturally in this task.
5.2 Ablation Analysis on FALSESUM Data
We perform an ablation analysis to study how each component of our data generation pipeline
contributes to the final performance. We first remove the contrastive property of the FALSESUM data by randomly including only either the positive  $(\mathrm{D},\mathrm{S}^{+},Y = 1)$  or negative  $(\mathrm{D},\mathrm{S}^{-},Y = 0)$  NLI examples obtained from a single  $(\mathrm{D},\mathrm{S}^{+})$  pair. Next, we filter out the negative NLI instances that are generated using either intrinsic or extrinsic code. We refer to the three ablated datasets as -contrastive, -intrinsic and -extrinsic, respectively. We set the sampled training size to 100,000 for the three ablation setups and aggregate the results from five training runs.
Table 3 shows the performance of the ablated models. We observe that removing contrastive pairs in the augmented training data results in a  $1.06\%$  drop on the overall benchmarks score. We also see that removing intrinsic error examples results in the highest performance loss,  $-5.03\%$  compared to  $-2.22\%$  by -extrinsic. This is explained by the fact that intrinsic consistency errors are more dominant on benchmarks that are built on the CNN/DailyMail corpus (Goyal and Durrett, 2021). We conclude that all the above properties are important for the overall improvements obtained by FALSESUM.
5.3 Fine-grained Evaluation
Previous work has shown that NLI models are prone to relying on fallible heuristics which associate lexical overlap with entailment labels (McCoy et al., 2019). In the factual consistency task, this corresponds to models associating highly extractive summaries with the "consistent" label. This raises a question about whether FALSESUM data alleviates this tendency in the resulting NLI models.
To answer this question, we partition the FactCC annotated test examples into five ordered subsets based on the lexical overlap between their
${}^{9}$  Average context word count is only 22 in MNLI and 546 in FactCC.
See details in Appendix B
Figure 3: Comparison between NLI models augmented with FALSESUM and FactCC across different measures of summary extractiveness. The x-axis shows the median overlap score of each test subset.
summary hypothesis and the source document premise. We define an overlap score using the NORMALIZED COVERAGE and DENSITY summary extractiveness scores introduced by Grusky et al. (2018). Both measures have the range [0.0, 1.0], where DENSITY = 1.0 indicates that all words in a summary are also present in the source document and NORMALIZED COVERAGE = 1.0 indicates that the summary is obtained by copying a continuous fragment of the source document. We then define OVERLAP = NORMALIZED COVERAGE × DENSITY.
Figure 3 shows the comparison of FactCC and FALSESUM augmentation performance across varying lexical overlap scores. We see that FALSESUM performs better on all subsets of the FactCC test set with the greatest performance gap appearing on the 0.9 overlap subset. Upon closer inspection, we see that the FactCC model makes mostly false positive classification errors on this subset, i.e., it tends to predict highly extractive summaries as "consistent", leading to near majority voting performance of  $50\%$ . FALSESUM, on the other hand, better discriminates the factual consistency of examples without over-relying on lexical overlap.
5.4 Data Quality Analysis
We conduct both manual and automatic quality evaluation of the FALSESUM-generated dataset. First, we sample 200 generated negative examples and manually verify whether (i) the perturbed summary  $\mathbf{S}^{-}$  is indeed factually inconsistent; (ii) the type of consistency error follows the specified control code; (iii) the incorrect "fact" is inserted at the specified missing span. Following Kryscinski
Code	Label √	Type √	Span √
Intrinsic	86%	94%	94%
Extrinsic	81%	65%	95%
Table 4: Manual verification of FALSESUM-generated NLI examples. Label, type, and span indicate the percentage of generated summaries with correct inconsistency label, error type, and error span, respectively.
FactCC	DocNLI	FALSESUM
Majority voting	50.84	53.55	50.00
CBOW-GloVe	60.36	70.38	56.13
BiLSTM-GloVe	68.26	73.04	57.62
RoBERTA-base	82.15	78.46	69.38
Table 5: Hypothesis-only model performance (accuracy) to measure the presence of artifacts and naturalness of FALSESUM dataset (lower is better).
et al. (2020), the authors perform this annotation to avoid high disagreement by crowd annotators in this task (Falke et al., 2019). The results in Table 4 show that about  $86\%$  of intrinsic  $81\%$  of extrinsic generated error examples are factually inconsistent, which happen due to several reasons, e.g., generator model chooses a span from the list that is similar to the original span, or generator model correctly guesses the original missing span. This further suggests that pre-trained language models such as RoBERTa-base can be robust against the induced label noise and can still learn a performant classifier. While  $\mathcal{G}$  almost always inserts the incorrect "fact" at the specified positions, we observe that it often fails to follow the specified extrinsic code correctly. We suspect that this is because the model prefers the easier task of copying the input over generating novel phrases.[11]
Following Gururangan et al. (2018), we also evaluate the naturalness of the generated dataset. We train an NLI model using positive examples from CNN/DailyMail and FALSESUM-generated negative examples. The model receives no premise so must distinguish between entailed and non-entailed hypotheses using semantic plausibility or spurious surface features, e.g., grammatical mistakes or fluency errors. The relatively low accuracy of these models on FALSESUM data (shown in Table 5) suggests that, compared to FactCC and DocNLI, FALSESUM-generated summaries are relatively hard to distinguish from the gold ones.
<sup>11</sup>We include more examples of generated NLI instances as well as the inadvertently consistent output in Appendix D.
NLI models present a promising solution for automatic assessment of factual consistency in summarization. However, the application of existing models for this task is hindered by several challenges, such as the mismatch of characteristics between their training dataset and the target task data. This mismatch includes the difference in terms of the input granularity (sentence vs. document level premises) and the types of (non-)entailment phenomena that must be recognized.
In this work, we present FALSESUM, a data generation pipeline which renders large-scale document-level NLI datasets without manual annotation. Using our training strategy, we demonstrate that it is possible to learn to generate diverse and naturalistic factually inconsistent (non-entailed) summaries using only existing (entailed) consistent summaries for training. We show that the resultant data is effective for augmenting NLI datasets to improve the state-of-the-art performance across four summary factual inconsistency benchmarks.
We would like to thank Marco Ponza, Marco Fiscato, Umut Topkara and other colleagues from Bloomberg AI for the thoughtful discussion and feedback throughout this project. We also thank Leonardo Ribeiro for comments on the earlier version of this work and the anonymous reviewers for their constructive feedback. The authors affiliated with UKP were supported by the German Research Foundation through the research training group "Adaptive Preparation of Information from Heterogeneous Sources" (AIPHES, GRK 1994/1) and by the German Federal Ministry of Education and Research and the Hessian State Ministry for Higher Education, Research and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal. Association for Computational Linguistics.
Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018. Faithful to the original: Fact aware neural abstrac
tive summarization. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 4784-4791. AAAI Press.
Cleo Condoravdi, Dick Crouch, Valeria de Paiva, Reinhard Stolle, and Daniel G. Bobrow. 2003. Entailment, intensionality and text understanding. In Proceedings of the HLT-NAACL 2003 Workshop on Text Meaning, pages 38-45.
Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pages 177-190, Berlin, Heidelberg. Springer Berlin Heidelberg.
Dorottya Demszky, Kelvin Guu, and Percy Liang. 2018. Transforming question answering datasets into natural language inference datasets. CoRR, abs/1809.02922.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Esin Durmus, He He, and Mona Diab. 2020. FEQA: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5055-5070, Online. Association for Computational Linguistics.
Matan Eyal, Tal Baumel, and Michael Elhadad. 2019. Question answering as an automatic evaluation metric for news article summarization. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3938-3948, Minneapolis, Minnesota. Association for Computational Linguistics.
Alexander R. Fabbri, Wojciech Krysciński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021. SummEval: Re-evaluating Summarization Evaluation. Transactions of the Association for Computational Linguistics, 9:391-409.
Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing,
pages 1535-1545, Edinburgh, Scotland, UK. Association for Computational Linguistics.
Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan, and Iryna Gurevych. 2019. Ranking generated summaries by correctness: An interesting but challenging application for natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2214-2220, Florence, Italy. Association for Computational Linguistics.
Ben Goodrich, Vinay Rao, Peter J. Liu, and Mohammad Saleh. 2019. Assessing the factual accuracy of generated text. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, page 166-175, New York, NY, USA. Association for Computing Machinery.
Tanya Goyal and Greg Durrett. 2021. Annotating and modeling fine-grained factuality in summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1449-1462, Online. Association for Computational Linguistics.
Max Grusky, Mor Naaman, and Yoav Artzi. 2018. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 708-719, New Orleans, Louisiana. Association for Computational Linguistics.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 107-112, New Orleans, Louisiana. Association for Computational Linguistics.
Karl Moritz Hermann, Tomás Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 1693-1701.
Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. CTRL: A conditional transformer language model for controllable generation. CoRR, abs/1909.05858.
Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. Scitail: A textual entailment dataset from science question answering. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence,
(AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 5189-5197. AAAI Press.
Wojciech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Neural text summarization: A critical evaluation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 540-551, Hong Kong, China. Association for Computational Linguistics.
Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332-9346, Online. Association for Computational Linguistics.
Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-Visiting NLIBased Models for Inconsistency Detection in Summarization. Transactions of the Association for Computational Linguistics, 10:163-177.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. BART: denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. CoRR, abs/1910.13461.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906-1919, Online. Association for Computational Linguistics.
Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428-3448, Florence, Italy. Association for Computational Linguistics.
Junghyun Min, R. Thomas McCoy, Dipanjan Das, Emily Pitler, and Tal Linzen. 2020. Syntactic data augmentation increases robustness to inference heuristics. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2339-2352, Online. Association for Computational Linguistics.
Anshuman Mishra, Dhruvesh Patel, Aparna Vijayakumar, Xiang Lorraine Li, Pavan Kapanipathi, and Kartik Talamadupula. 2021. Looking beyond sentence-level natural language inference for question answering and text summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1322-1336, Online. Association for Computational Linguistics.
Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, Caglar Gucehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning, pages 280-290, Berlin, Germany. Association for Computational Linguistics.
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807, Brussels, Belgium. Association for Computational Linguistics.
Yuyang Nie, Yuanhe Tian, Yan Song, Xiang Ao, and Xiang Wan. 2020. Improving named entity recognition with attentive ensemble of syntactic information. In *Findings of the Association for Computational Linguistics: EMNLP* 2020, pages 4231-4245, Online. Association for Computational Linguistics.
Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. 2021. Understanding factuality in abstractive summarization with FRANK: A benchmark for factuality metrics. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4812-4829, Online. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1-140:67.
Nils Reimers and Iryna Gurevych. 2017. Reporting score distributions makes a difference: Performance study of LSTM-networks for sequence tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 338-348, Copenhagen, Denmark. Association for Computational Linguistics.
Alexis Ross, Tongshuang Wu, Hao Peng, Matthew E. Peters, and Matt Gardner. 2021. Tailor: Generating and perturbing text with semantic controls. CoRR, abs/2107.07150.
Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the factual consistency of summaries. In Proceedings of