SlowGuess/NAACL_2022_OCR huggingface dataset:
We compare REFLECT with several strong baselines (Liu and Lapata, 2019a; Gehrmann et al., 2018; Fabbri et al., 2019; Perez-Beltrachini and Lapata, 2021; Li et al., 2020; Liu et al., 2021a; Zhong et al., 2020; Zhang et al., 2020a; Pasunuru et al., 2021) on Multi-News (Fabbri et al., 2019), Multi-XScience (Lu et al., 2020) and WikiCatSum (Perez-Beltrachini et al., 2019) corpora, derived from news, academic domains and Wikipedia, respectively. Due to space limit, the results of Multi-XScience and WikiCatSum are provided in the Appendix A. For evaluation, we use ROUGE F1 metrics (Lin, 2004), BERTScore (Zhang et al., 2020b), and factual consistency evaluated with FactCC (Kryscinski et al., 2020) to investigate performance from different perspectives. In our architecture, the hierarchical extractor is initialized by RoBERTa-base (Liu et al., 2020) containing 12 attention layers. We take the first  $l$  layers and the rest layers  $(12 - l)$  as the token- and sentence-level encoder, respectively. The input length limit  $M$  is set to 512. The abstractor is a sequence-to-sequence model initialized by BART (Lewis et al., 2020). To make the CASC computationally efficient, we use the BART-base as the abstractor that provides rewards for the extractor during training, while exploiting the BART-large in the testing. We provide more implementation details in Appendix C.
Table 1 summarizes the performance of RE-FLECT with various baselines on Multi-News
corpus with the rows sorted by the average of ROUGE 1, 2, and L scores (Lin, 2004). The results demonstrate that REFLECT outperforms all baselines. PG-BRNN and Hi-MAP both use an LSTM-based point generator for sentence selection and apply a decoder for the generation. However, the performance is limited since LSTM could still suffer long-term dependency problem (Trinh et al., 2018). Hierarchical-Transformer (row 1) further uses an LSTM-based ranker to select paragraphs through predicting the ROUGE scores with the summaries for each paragraph, and uses a Transformer-based hierarchical encoder to capture the local and global information. However, the estimation of the ROUGE scores could be difficult since there are multiple valid summaries. In contrast, REFLECT further applies pre-trained language models to extract the hierarchical features, and the proposed SR method explicitly provides the extractor with the evidence for selecting more salient contents to be rewritten from the abstractor. Moreover, GraphSum (row 5 & 6) leverages graph structures to explore the relations of paragraphs in the encoder, and Highlight-Transformer (row 7) specifically assigns higher attention weights for key phrases. Both the methods implicitly provide additional selective information for the decoder. REFLECT realizes such a content selection pro
2We do not compare ROUGE-L results with HighlightTransformer and MatchSum because 1) HighlightTransformer does not report ROUGE-L results and 2) MatchSum reports summary-level ROUGE-L (ROUGE-LSum) results which are different from sentence-level ROUGE-L used here (Lin, 2004). Nevertheless, ROUGE-LSum results of REFLECT in Table 3 still show the superiority over MatchSum (45.05 v.s. 41.89).
cess in the data-level through the two-stage design. The proposed POR and SR methods alleviate the negative effects from the pseudo oracles, and thus provide complete information for the abstractor to rewrite. The results of PEGASUS (row 9) show the benefits of pre-trained sequence-to-sequence language models, which are also leveraged in REFLECT. The BART-Long-Graph (row 10 & 11) further combines the previous advantages on graph structure and pre-trained language models to achieve strong performance. REFLECT outperforms it in terms of the ROUGE-2 and ROUGE-L scores especially, probably due to the local attention mechanism used in BART-Long-Graph. Although such a design reduces the complexity to accommodate longer sequences, the trade-off for the attention capacity still restricts the model from generating more coherent summaries. Due to the space limit, please refer to Appendix A for results on Multi-XScience and WikiCatSum.
In addition to the evaluation of lexical overlaps, Table 2 shows the performance of semantic and factual consistency, which are important for summarization applications that will influence the public. The results demonstrate that REFLECT can improve factual consistency through the hierarchical architecture, i.e., the architecture enables selection of useful information from multiple documents, while still maintaining the semantics in generations.
Model	BERTScore	Factual Consistency
BART-base	0.870	79.7
CTF-DPP	0.852	81.9
REFLECT	0.871	82.2
Table 2: BERTScore and factual consistency evaluated with FactCC on Multi-News corpus.
4.3 Ablations and Analyses
In this subsection, we perform ablations and analyses for REFLECT, and more results can be found in Appendix B.
Effect of Pseudo Oracle Relaxation (POR). We start the discussions with the results using MLE objectives. As shown in Table 3, when applying POR to the base method (row 1 & 2), the extraction recall increases while the precision decreases. Pseudo oracles generally contain sentences with high ROUGE scores to the summaries, and POR encourages the selection for such sentences by as
signing smaller loss weights to increase the recall. From the abstraction performance, it shows that such selection preference could provide more concise information (even though not in pseudo oracles) as the input for improvements. Under the combination of SR (row 3 & 4), the precision is further improved and achieves a higher overall F1 score due to more information for selection. However, the abstraction results show that the performance is actually inferior to the one only using SR, which meets our suggestion that the training objective of the extractor is inconsistent in the test-time. The proposed CASC learning further overcomes such problems and improves the performance by integrating POR and SR (row 6 & 10).
Effect of Summary Referencing (SR). We first investigate the MLE results. With SR, the extractor is learned to select sentences given the approximations of summaries generated from the abstractor. The extraction recall is thus enhanced with such references. However, the trade-off between recall and precision still exists due to the discrepancy between ground-truth summaries and generation results (row 1 & 3). Although ground-truth can be used as references to increase extraction performance during training, it fails to generalize in testing due to the distributional difference between them. The abstraction performance also shows SR makes significant improvements (row 3) and also benefits from the combination with CASC (row 6 & 8).
Effect of Credit-Aware Self-Critic (CASC) Learning. Consider the case that applying CASC with POR and SR respectively (row 5 & row 6, row 7 & row 8), the results demonstrate that CASC substantially outperforms the Self-Critic (SC) learning methods, even when combining the usage of POR and SR (row 9 & row 10). We could further investigate the difference between MLE and RL methods through the visualization in Figure 2. It shows that, although having lower extraction performance, the RL methods can improve over the MLE methods in final abstraction performance due to the consistent objective between training and testing. CASC further improves over SC through explicitly assigning advantage to the actions that have the credits for exploration. Such design is critical for the long input sequences in multi-document summarization and can potentially be applied to other applications.
MLE	RL	Extraction Performance	Abstraction Performance
SR	POR	SR	CA	Precision	Recall	F1	R-1	R-2	R-L	R-LSum	Average
-	-	0.6489	0.4218	0.5113	47.60	18.48	23.89	43.22	33.29
✓	-	-	0.4618	0.7684	0.5769	47.92	18.76	23.57	43.87	33.53
✓	-	-	0.4755	0.6275	0.5410	48.40	18.95	24.08	44.03	33.86
✓	✓	-	-	0.4926	0.7112	0.5820	48.16	18.87	23.61	44.06	33.68
✓	0.6161	0.4704	0.5335	48.59	19.04	24.20	44.21	34.01
✓	✓	0.5344	0.5948	0.5630	48.91	19.77	24.46	44.80	34.49
✓	✓	0.5933	0.5328	0.5614	48.83	19.50	24.36	44.57	34.32
✓	✓	✓	0.5442	0.5942	0.5681	49.04	19.80	24.50	44.94	34.57
✓	✓	✓	0.6007	0.5170	0.5557	48.84	19.41	24.34	44.56	34.29
✓	✓	✓	✓	0.5873	0.5108	0.5464	49.27	19.96	24.76	45.04	34.76
Figure 2: Comparisons of MLE and RL results. For clarity, the displayed range for average ROUGE scores and extraction F1 scores are limited in [33,35] and [0.5,0.59], respectively.
Table 3: Ablation of REFLECT with Pseudo Oracle Relaxation (POR), Summary Referencing (SR), and Credit-Aware Self-Critic (CASC) learning. ROUGE score is abbreviated as  $R$  .
Reference Source	R-1	R-2	R-L	Average
PEGASUS	49.28	19.99	24.84	34.78
BART-large-A	49.29	19.99	24.83	34.79
BART-large-O	49.29	19.98	24.81	34.78
Ground-Truth	49.38	20.05	24.88	34.85
Table 4: Performance of different summary referencing strategies in REFLECT. We also present the results of BART models trained by truncated articles (A) and pseudo oracles (O). ROUGE score is abbreviated as  $R$  .
4.4 Effects of Summary Reference
REFLECT uses the BART-large fine-tuned with truncated articles (BART-large-A) to provide summary references for both training and testing. Here, we study the effect of applying different summary referencing strategies in test-time. To investigate such effect across models, we leverage PEGASUS (Zhang et al., 2020a) and fine-tune it with Multi-News corpus to produce summary references. Furthermore, we also conduct experiments that use the BART-large fine-tuned with pseudo oracle sentences (BART-large-O) and the ground-truth summaries. Note that these two methods are prohibited in practical usage since they require annotations on testing data. The results in Table 4 manifest that using references from another model that is different from the training one does not significantly affect the performance. For the case using oracle inputs, the performance is similar to the ones directly using articles. This suggests that our method is stable for the practical scenario. The result of the ground-truth also shows that the reference quality could still affect the performance, which is left as a future research direction.
In this work, we present an effective extract-thenabstract framework, named REFLECT, for MDS. We utilize large pre-trained language models to construct a hierarchical extractor for solving the long sequence problem. Moreover, we investigate current learning paradigms for such frameworks and find that entirely relying on the pseudo oracles produced via a greedy process could hinder the performance. Therefore, we propose three corresponding techniques (POR, SR, and CASC) to overcome the issues. The experimental results not only show that REFLECT outperforms the state-of-the-art models on Multi-News, Multi-XScience, and WikiCatSum corpora, but also demonstrate that bridging the gap between training and testing is significant. Also, we present extensive studies to motivate more investigations. Finally, we consider further exploring the interactions between the extractor and abstractor, including iteratively providing a better reference or reusing the extraction predictions as the training signals for the abstractor, and study how to build a more efficient reference for providing the extraction evidence.
This work is supported in part by the Ministry of Science and Technology (MOST) of Taiwan under the grants MOST-109-2221-E-009-114-MY3, MOST-110-2221-E-001-001 and MOST-110-2221-E-A49-164. This work was also supported by the Higher Education Sprout Project of the National Yang Ming Chiao Tung University and Ministry of Education (MOE), Taiwan. We are grateful to the National Center for High-performance Computing for computer time and facilities.
Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
Arthur Bražinskas, Mirella Lapata, and Ivan Titov. 2021. Learning opinion summarizes by selecting informative reviews. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9424-9442, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Wei Chen, Wei Hu, Fu Li, Jian Li, Yu Liu, and Pinyin Lu. 2016. Combinatorial multi-armed bandit with general reward functions. In Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.
Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 675-686, Melbourne, Australia. Association for Computational Linguistics.
Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 615-621, New Orleans, Louisiana. Association for Computational Linguistics.
Günes Erkan and Dragomir R. Radev. 2004. Lexrank: Graph-based lexical centrality as salience in text summarization. J. Artif. Int. Res., 22(1):457-479.
Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. 2019. Multi-news: A large-scale multi-document summarization dataset and abstractive hierarchical model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1074-1084, Florence, Italy. Association for Computational Linguistics.
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1243-1252. PMLR.
Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018. Bottom-up abstractive summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4098-4109, Brussels, Belgium. Association for Computational Linguistics.
Jade Goldstein, Vibhu Mittal, Jaime Carbonell, and Mark Kantrowitz. 2000. Multi-document summarization by sentence extraction. In Proceedings of the 2000 NAACL-ANLP Workshop on Automatic Summarization - Volume 4, NAACL-ANLP-AutoSum '00, page 40-48, USA. Association for Computational Linguistics.
Hanqi Jin, Tianming Wang, and Xiaojun Wan. 2020. Multi-granularity interaction network for extractive and abstractive multi-document summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6244-6254, Online. Association for Computational Linguistics.
Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332-9346, Online. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computational Linguistics.
Wei Li, Xinyan Xiao, Jiachen Liu, Hua Wu, Haifeng Wang, and Junping Du. 2020. Leveraging graph to improve abstractive multi-document summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6232-6243, Online. Association for Computational Linguistics.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
Peter J. Liu, Mohammad Saleh, Etienne Pot, Ben Goodrich, Ryan Sepassi, Lukasz Kaiser, and Noam Shazeer. 2018. Generating wikipedia by summarizing long sequences. In International Conference on Learning Representations.
Shuaiqi Liu, Jiannong Cao, Ruosong Yang, and Zhiyuan Wen. 2021a. Highlight-transformer: Leveraging key phrase aware attention to improve abstractive multi-document summarization. In *Findings of the Association for Computational Linguistics: ACLIJCNLP* 2021, pages 5021-5027, Online. Association for Computational Linguistics.
Yang Liu and Mirella Lapata. 2019a. Hierarchical transformers for multi-document summarization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5070-5081, Florence, Italy. Association for Computational Linguistics.
Yang Liu and Mirella Lapata. 2019b. Text summarization with pretrained encoders. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3730-3740, Hong Kong, China. Association for Computational Linguistics.
Yang Liu, Sheng Shen, and Mirella Lapata. 2021b. Noisy self-knowledge distillation for text summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 692-703, Online. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2020. RoBERTa: A robustly optimized BERT pretraining approach.
Yao Lu, Yue Dong, and Laurent Charlin. 2020. MultiXScience: A large-scale dataset for extreme multi-document summarization of scientific articles. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8068-8074, Online. Association for Computational Linguistics.
Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The pagerank citation ranking: Bringing order to the web. Technical Report 1999-66, Stanford InfoLab.
Richard Yuanzhe Pang, Adam Lelkes, Vinh Tran, and Cong Yu. 2021. AgreeSum: Agreement-oriented multi-document summarization. In *Findings of the Association for Computational Linguistics: ACLIJCNLP* 2021, pages 3377-3391, Online. Association for Computational Linguistics.
Ramakanth Pasunuru, Mengwen Liu, Mohit Bansal, Sujith Ravi, and Markus Dreyer. 2021. Efficiently summarizing text and graph encodings of multi-document clusters. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4768-4779, Online. Association for Computational Linguistics.
Laura Perez-Beltrachini and Mirella Lapata. 2021. Multi-document summarization with determinantal point process attention. J. Artif. Int. Res., 71:371-399.
Laura Perez-Beltrachini, Yang Liu, and Mirella Lapata. 2019. Generating summaries with topic templates and structured convolutional decoders. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5107-5116, Florence, Italy. Association for Computational Linguistics.
Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. 2020. On extractive and abstractive neural document summarization with transformer language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9308-9319, Online. Association for Computational Linguistics.
Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1179-1195, Los Alamitos, CA, USA. IEEE Computer Society.
Trieu Trinh, Andrew Dai, Thang Luong, and Quoc Le. 2018. Learning longer-term dependencies in RNNs with auxiliary losses. In Proceedings of the 35th International Conference on Machine Learning (PMLR), volume 80, pages 4965-4974.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Xiaojun Wan and Jianwu Yang. 2006. Improved affinity graph based multi-document summarization. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 181-184, New York City, USA. Association for Computational Linguistics.
Xiaojun Wan and Jianwu Yang. 2008. Multi-document summarization using cluster-based link analysis. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '08, page 299-306, New York, NY, USA. Association for Computing Machinery.
Yuxiang Wu and Baotian Hu. 2018. Learning to extract coherent summary via deep reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1).
Baosong Yang, Zhaopeng Tu, Derek F. Wong, Fandong Meng, Lidia S. Chao, and Tong Zhang. 2018. Modeling localness for self-attention networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4449-4458, Brussels, Belgium. Association for Computational Linguistics.
Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020a. PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 11328-11339. PMLR.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020b. *Bertscore: Evaluating text generation with bert*. In International Conference on Learning Representations.
Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, and Xuanjing Huang. 2020. Extractive summarization as text matching. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6197-6208, Online. Association for Computational Linguistics.
Hao Zhou, Weidong Ren, Gongshen Liu, Bo Su, and Wei Lu. 2021. Entity-aware abstractive multi-document summarization. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 351-362, Online. Association for Computational Linguistics.
Fangwei Zhu, Shangqing Tu, Jiaxin Shi, Juanzi Li, Lei Hou, and Tong Cui. 2021. TWAG: A topic-guided Wikipedia abstract generator. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4623-4635, Online. Association for Computational Linguistics.
A Results on Other Corpora
A.1 REFLECT on Multi-XScience Corpus
In this subsection, we additionally experiment REFLECT with Multi-XScience (Lu et al., 2020) corpus, which is a multi-document summarization corpus for scientific articles. The source of this corpus is a tuple of a paper abstract and abstracts of the citation papers, while the generation target is the corresponding related-work paragraph. The performances of baselines are from Lu et al. (2020)<sup>3</sup>. Table 5 shows that REFLECT achieves a better performance comparing to all baselines, and makes a substantial improvement especially in ROUGE-2 score. We also report the results of BART-large which serves as our abstractor to summarize from the extracted sentences. The improvement shows the benefit of the extraction process after the Credict-Aware Self-Critic Learning.
Model	R-1	R-2	R-L	R-LSum
LEAD	27.46	4.57	-	-
LEXRANK	30.19	5.53	-	-
TEXTRANK	31.51	5.83	-	-
Hi-MAP	31.66	5.91	-	-
PG	34.11	6.76	-	-
BART-large	33.29	8.07	17.31	29.04
REFLECT (MLE)	33.87	8.13	17.20	29.69
REFLECT (CASC)	34.18	8.20	17.42	29.73
Table 5: Performance of REFLECT and baselines on Multi-XScience corpus. We also present the results of BART models trained by truncated articles. The results of baselines are from Lu et al. (2020), where PG represents Pointer-Generator and the Hi-MAP represents the model proposed by Fabbri et al. (2019). ROUGE score is abbreviated as  $R$  .
A.2 REFLECT on WikiCatSum Corpus