Improving Multi-Document Summarization through Referenced Flexible Extraction with Credit-Awareness (2022)

Abstract: A notable challenge in Multi-Document Summarization (MDS) is the extremely-long length of the input. In this paper, we present an extract-then-abstract Transformer framework to overcome the problem. Specifically, we leverage pre-trained language models to construct a hierarchical extractor for salient sentence selection across documents and an abstractor for rewriting the selected contents as summaries. However, learning such a framework is challenging since the optimal contents for the abstractor are generally unknown. Previous works typically create pseudo extraction oracle to enable the supervised learning for both the extractor and the abstractor. Nevertheless, we argue that the performance of such methods could be restricted due to the insufficient information for prediction and inconsistent objectives between training and testing. To this end, we propose a loss weighting mechanism that makes the model aware of the unequal importance for the sentences not in the pseudo extraction oracle, and leverage the fine-tuned abstractor to generate summary references as auxiliary signals for learning the extractor. Moreover, we propose a reinforcement learning method that can efficiently apply to the extractor for harmonizing the optimization between training and testing. Experiment results show that our framework substantially outperforms strong baselines with comparable model sizes and achieves the best results on the Multi-News, Multi-XScience, and WikiCatSum corpora.

## introduction neural multi-document summarization has drawn a lot of attention due to the wide applications, e.g., wikipedia generation @xcite , news digest @xcite , or related-work section generation @xcite . through con-catenating the document clusters, it is applicable to adopt the approaches from single-document summarization @xcite under multi-document settings. however, the input length from multiple documents is typically long, which might be computationally infeasible for transformer-based models @xcite . one possible solution is to set a fixed length and truncate the input sequence. however, the truncation leads to information loss and performance drop. moreover, even if the model can take such a long sequence as input, the attentions could be dispersed over long sequences @xcite and further degrade the performance @xcite @xcite . to tackle the long sequence problem, another possible solution is to leverage the extract-thenabstract architecture in single document summarization @xcite @xcite . in such an architecture, an extractor first selects salient contents from the documents, and an abstractor is applied to rewrite the selected contents to coherent summaries. however, several issues arise for directly applying the procedure in mds. 1) long-sequence problem for the extractor. although the input length for the abstractor can be controlled through the content selection and thus enables the usage of transformer models @xcite , it is still inevitable for the extractor to process the complete input documents. previous methods mainly use lstm-based extractors @xcite @xcite . however, this fails to leverage knowledge of pre-trained language models. 2) suboptimal pseudo oracles. most summarization corpora do not contain the oracle for extraction. as an alternative, previous works typically generate pseudo extraction oracle, or simply called pseudo oracle, through a greedy process @xcite . specifically, for each iteration in the process, candidate sentences are individually concatenated with previously-selected sentences and compute the rouge @xcite scores with the humanwritten summaries. the top-scored sentence is iteratively selected until no sentence could further improve the rouge score. in practice, there are different ways for scoring. for examples, @xcite use the average of rouge-1 and rouge-2 f1 scores while @xcite use rouge-l recall. the variants of design could cause the extractor to behave differently. in our study, we also found that using the rouge precision metric leads to much less extraction than using the recall metric. this implies that the pseudo oracles are suboptimal, and learning the extractor fully relies on the pseudo oracles could restrict the performance. how to alleviate the negative effects of pseudo oracles remains open. 3) insufficient information for the extractor. even if the pseudo oracles are good enough to train the abstractor well, learning a precise extractor is still challenging. the problem lies in that a pseudo oracle is derived from a specific summary. however, there are potentially multiple valid summaries given the documents. to select salient sentences, the extractor is required to implicitly infer the underlying summary used for oracle construction, which is difficult due to the lack of evidence. 4) inconsistent objectives for the extractor. with pseudo oracles, the extractor is learned to select a set of sentences that has high lexical similarity to the summaries without redundancy. however, the goal of the extractor in test-time is to provide inputs for the abstractor that non-overlapping lexical may still be valuable. in other words, the objective for the extractor in training is inconsistent with the one in testing. to address these issues, we propose the referenced flexible extraction with credit-awareness (reflect) for mds. for the first problem, we propose a transformer-based hierarchical extractor that contains the token-and sentencelevel feature encoders. both the encoders are initialized with pre-trained language models to utilize the pretext knowledge. for the second problem, we propose pseudo oracle relaxation (por) to render the model aware of the unequal importance for the non-oracle sentences. this mechanism encourages the model to emphasize the precision for critical sentences with either high or low lexical similarity to the summaries, and avoids the confusion arising from the different labels for similar sentences. for the third problem, we propose summary referencing (sr) to leverage the fine-tuned abstractor for providing additional learning signals to evidence extraction prediction. the summary reference serves as an approximation for the humanwritten summary while being able to generalize for testing. for the fourth problem, we propose credit-aware self-critic (casc) learning to fine-tune for matching the objective between training and testing. different from previous methods that assign an identical reward for all actions, we reallocate the rewards based on the impacts of actor explorations. the contributions are summarized as follows: ## related work early attempts for mds focus on extracting sentences through statistical methods @xcite @xcite . for example, @xcite extend maximal marginal relevance (mmr) method to select sentences that are relevant to the query and novel across different documents. @xcite leverage sentence relations with graph structures that represent pairwise sentence similarities, and apply pagerank @xcite algorithm to extract sentences given the query document. however, extractive methods often suffer the coherence problem @xcite . therefore, instead of directly extracting sentences from the articles, abstractive methods that can rewrite the articles achieve great success with the advantages of large annotated corpora @xcite @xcite @xcite . directly operating on the long inputs in mds often leads to model degradation @xcite . one of the promising solutions is to leverage the extract-then-abstract architectures in single document summarization @xcite . for example, @xcite propose an lstm-based word-level extractor to choose phrases from the document, and apply an abstractor with the copy mechanism to generate summaries given the selected contents. pilault et al. (2020) also apply an lstm-based sentence extractor to select contents, but further leverage a transformer decoder to improve the performance. although introducing transformer architectures provides improvements, the input length is typically limited according to the computation and memory overhead. a recent line of studies has been proposed to alleviate such problems @xcite @xcite . @xcite propose to rank the candidate input paragraphs, and only concatenate the top few as the inputs for the abstractor. jin et al. (2020) encode multiple documents in different granularity including token-level, sentence-level, and document-level. @xcite design a graph encoder parallel with standard encoder to provide inter-document information and integrate the pre-trained bart @xcite with local attention mechanism @xcite to overcome the long input problem. @xcite apply policy gradient optimization to learn a train-time selector using lexical features pre-computed from source texts and gold summaries as inputs. due to the lack of gold summaries in testing, a test-time selector is learned using lexical features solely from source texts as inputs and predictions from the train-time selector as learning targets. comparatively, our methods thoroughly leverage the language models with an extract-and-abstract framework that enjoys the full capacities for both the extractor and the abstractor. the introduced summary referencing further improves the extractor with additional signals in both train-and test-phase. ## methodology utilizing large pre-trained language models brings great benefits for text generation problems. however, the input length of multi-document summarization is typically long, which makes large pre-trained language models, e.g., transformer-based models, inefficient for processing. to match the length constraint, document-level truncation @xcite has been widely-used. however, the truncation could inevitably cause information loss and degrade the performance. to solve the problem, we propose to leverage the extract-thenabstract framework. however, as described in the introduction, four challenges arise while learning such a framework. we first present our architecture to solve the long sequence problem and elaborate on the proposed methods to tackle the challenges. ## conclusion in this work, we present an effective extract-thenabstract framework, named reflect, for mds. we utilize large pre-trained language models to construct a hierarchical extractor for solving the long sequence problem. moreover, we investigate current learning paradigms for such frameworks and find that entirely relying on the pseudo oracles produced via a greedy process could hinder the performance. therefore, we propose three corresponding techniques (por, sr, and casc) to overcome the issues. the experimental results not only show that reflect outperforms the stateof-the-art models on multi-news, multi-xscience, and wikicatsum corpora, but also demonstrate that bridging the gap between training and testing is significant. also, we present extensive studies to motivate more investigations. finally, we consider further exploring the interactions between the extractor and abstractor, including iteratively providing a better reference or reusing the extraction predictions as the training signals for the abstractor, and study how to build a more efficient reference for providing the extraction evidence.