Abstractive Multi-Document Summarization via Joint Learning with Single-Document Summarization (2020)

Abstract: Single-document and multi-document summarizations are very closely related in both task definition and solution method. In this work, we propose to improve neural abstractive multi-document summarization by jointly learning an abstractive single-document summarizer. We build a unified model for single-document and multi-document summarizations by fully sharing the encoder and decoder and utilizing a decoding controller to aggregate the decoder’s outputs for multiple input documents. We evaluate our model on two multi-document summarization datasets: Multi-News and DUC-04. Experimental results show the efficacy of our approach, and it can substantially outperform several strong baselines. We also verify the helpfulness of single-document summarization to abstractive multi-document summarization task.

## introduction document summarization aims at producing a fluent, condensed summary for the given document or document set. it involves identifying important information and filtering out redundant information from input sources. while single-document summarization takes a single source document as input, multi-document summarization requires producing a summary from a cluster of thematically related documents. there are two primary methodologies for document summarization: extractive and abstractive. extractive methods directly select important sentences from the original documents, which are relatively simple but face the drawbacks of information redundancy and incoherence between sentences. abstractive methods enable generating new words, phrases, and sentences, which are able to generate better summaries with higher readability and conciseness. in this paper, we focus on abstractive document summarization. empowered by large parallel datasets automatically harvested from online news websites, sequence-to-sequence learning has shown promising results on abstractive single-document summarization @xcite @xcite . compared with single-document summarization, annotated multi-document summarization datasets are often scarce. several works have explored adapting the neural encoder-decoder model trained for single-document summarization to multi-document summarization. @xcite add a document set encoder to extend the neural abstractive model trained on large scale single-document summarization corpus to the multi-document summarization task. @xcite incorporate the maximal marginal relevance method into a neural encoder-decoder model trained for singledocument summarization to address the information redundancy for multi-document summarization. single-document and multi-document summarizations are very closely related in both task definition and solution method @xcite . both tasks need to deal with document-level input, identify the important content of documents, and paraphrase the important information to generate the summary, while the main difference is that multi-document summarization involves summarizing multiple input documents. since the two tasks are closely related, it is promising to learn for two summarization tasks jointly. compared with single-document summarization, multi-document summarization needs to handle multiple input documents. a simple method is to concatenate multiple documents into a long flat text and treat it as a long sequence-tosequence task. however, it blurs the boundaries between documents and loses the hierarchy within the document cluster. it is natural to regard multidocument summarization as a two-stage process of summarizing every single document and then merging multiple summaries. nevertheless, this process is quite trivial, and it is difficult to utilize multi-document summarization corpus to train the single-document summarization model. furthermore, the synthesis of multiple summaries involves eliminating redundant parts and organizing related paragraphs or sentences, which are also challenges to be solved. in this work, we propose a joint learning approach to improve neural abstractive multi-document summarization by using singledocument summarization corpus to address these issues. our approach first uses a shared document encoder to encode each document in the document set, then uses a shared decoder to predict the word probabilities for each document, and finally applies a decoding controller to aggregate all output probabilities from the summary decoder to make the final prediction at each decoding step. the shared encoder and decoder are jointly trained on the single document summarization data. in this way, we can unify single-document and multi-document summarizations into one architecture simultaneously, and make better use of single-document and multi-document corpora, so that both tasks can benefit from joint learning, especially for the multidocument summarization task. we evaluate our approach on the benchmark multi-document summarization datasets, multi-news and duc-04, and it brings substantial improvements over several strong baselines for multi-document summarization. we leverage cnn/dailymail, a single-document summarization dataset, to perform joint learning with multi-news. we also test the performance on cnn/dailymail test set, and joint learning also brings certain performance improvement for the single-document summarization baselines. in summary, we make the following contributions in this paper: ## conclusion and future work in this paper, we propose a joint learning approach to improve neural abstractive multi-document summarization by using single-document summarization dataset. specifically, we use the shared document encoder and summary decoder to process each document in the document set, and apply a decoding controller to aggregates all output probabilities from the summary decoder for multi-document summarization. the shared encoder and decoder are jointly trained on the single document sum-marization dataset. experimental results show that our approach substantially outperforms several strong multi-document summarization baselines and achieves state-of-the-art or very competitive performances on multi-news and duc-04 datasets. in the future, we will incorporate bert or other pre-trained language models into our model to further improve the performance.