M em S um: Extractive Summarization of Long Documents Using Multi-Step Episodic M arkov Decision Processes (2022)

Abstract: We introduce MemSum (Multi-step Episodic Markov decision process extractive SUMmarizer), a reinforcement-learning-based extractive summarizer enriched at each step with information on the current extraction history. When MemSum iteratively selects sentences into the summary, it considers a broad information set that would intuitively also be used by humans in this task: 1) the text content of the sentence, 2) the global text context of the rest of the document, and 3) the extraction history consisting of the set of sentences that have already been extracted. With a lightweight architecture, MemSum obtains state-of-the-art test-set performance (ROUGE) in summarizing long documents taken from PubMed, arXiv, and GovReport. Ablation studies demonstrate the importance of local, global, and history information. A human evaluation confirms the high quality and low redundancy of the generated summaries, stemming from MemSum’s awareness of extraction history.

## introduction automatic text summarization is the task of automatically summarizing a long document into a relatively short text while preserving most of the information @xcite ). text summarization methods can be categorized into abstractive and extractive summarization @xcite . given a document d consisting of an ordered list of n sentences, extractive summarization aims to pick up m (m ≪n ) sentences as the summary of the document. the extracted summaries tend to be both grammatically and semantically more reliable than abstractive summaries @xcite @xcite , as they are directly selected from the source text. extractive summarization is usually modeled as two sequential phases @xcite : 1) sentence scoring and 2) sentence selection. in the sentence scoring phase, an affinity score is computed for each sentence by neural networks such as bidirectional rnns @xcite @xcite or @xcite . in the sentence selection phase, sentences are selected by either i) predicting a label (1 or 0) for each sentence based on its score, and selecting sentences with label 1 @xcite @xcite , or ii) ranking sentences based on their scores and selecting the top k sentences as the summary @xcite , or iii) sequentially sampling sentences without replacement, where the normalized scores of the remaining sentences are used as sampling likelihoods @xcite . in these approaches, sentence scores are generally not updated based on the current partial summary of previously selected sentences, indicating a lack of knowledge of extraction history. we deem extractive summarizers that are not aware of the extraction history to be susceptible to redundancy in a document, because they will repeatedly add sentences with high scores to a summary, regardless of whether similar sentences have been selected before. and, redundancy leads to performance decreases evaluated by rouge f1. in this paper, we propose to model extractive summarization as a multi-step episodic markov decision process (mdp). as shown in figure 1 , at each time step in an episode, we define a sentence state composed of three sub-states: 1) the local content of the sentence, 2) the global context of the sentence within the document, and 3) information on the extraction history, including the previously selected set of unordered sentences and the remaining sentences. at each time step, the policy network (agent) takes the current sentence state as input and produces scores used to select an action of either stopping the extraction process or selecting one of the remaining sentences into the candidate summary. unlike one-step episodic mdp-based models @xcite @xcite that encode the state information only once at the beginning of the episode, in our multi-step policy, the agent updates at each time step the extraction history before selecting an action. such a step-wise state-updating strategy enables the agent to consider the content of the partial summary when selecting a sentence. to efficiently encode local and global sentence states, we design an extraction agent based on lstm networks @xcite . to encode the extraction history and to select actions, we use a reduced number of attention layers @xcite of relatively low dimensionality. these choices enable our model to be easily trainable and to summarize long documents such as scientific papers @xcite or reports @xcite . the contributions of our work are as follows: 1) we propose to treat extractive summarization as a multi-step episodic mdp that is aware of the extraction history. 2) we show that extraction-history awareness allows our model to extract more compact summaries than models without history awareness and behave more robustly to redundancies in documents. 3) our model outperforms both extractive and abstractive summarization models on pubmed, arxiv @xcite , and govreport @xcite datasets. 4) finally, human evaluators rate the memsum summaries to be of higher quality than those from a competitive approach, especially by virtue of lower redundancy foot_0 . ## related work extraction history awareness was previously considered in neusum @xcite , where a gru encoded previously selected sentences into a hidden vector that then was used to update the scores of the remaining sentences to bias the next selection. neusum contains no stopping mechanism and therefore it can only extract a fixed number of sentences, which likely is sub-optimal. also, the potential benefits of extraction history have not been quantified and so the idea remains unexplored to a large extent. recently, bert-based extractors such as match-sum (zhong et al., 2020) achieved sota performance in extractive summarization of relatively short documents from the cnn/dm @xcite dataset. however, the quadratic computational and memory complexities @xcite of such models limit their scalability for summarizing long documents with thousands of tokens, which is common for scientific papers and government reports. although large pre-trained transformers with efficient attention @xcite have been adapted for abstractive summarization of long documents, we believe that extractive summarization is more faithful in general, which is why we chose an extractive approach. ## model this section outlines the multi-step episodic mdp policy for extractive summarization. ## results and discussion here we present the results on various extractive summarization tasks and analyze the contribution of different modules via ablation studies. ## conclusion extractive summarization can be achieved effectively with a multi-step episodic markov decision process with history awareness. using encoders of local sentence, global context, and extraction history, memsum is given information that is intuitively also used by humans when they summarize a document. awareness of the extraction history helps memsum to produce compact summaries and to be robust against redundancy in the document. as a lightweight model (appendix c), mem-sum outperforms both extractive and abstractive baselines on diverse long document summarization tasks. because memsum achieves sota performance on these tasks, mdp approaches will be promising design choices for further research.