GS um: A General Framework for Guided Neural Abstractive Summarization (2021)

Abstract: Neural abstractive summarization models are flexible and can produce coherent summaries, but they are sometimes unfaithful and can be difficult to control. While previous studies attempt to provide different types of guidance to control the output and increase faithfulness, it is not clear how these strategies compare and contrast to each other. In this paper, we propose a general and extensible guided summarization framework ( GSum ) that can effectively take different kinds of external guidance as input, and we perform experiments across several different varieties. Experiments demonstrate that this model is effective, achieving state-of-the-art performance according to ROUGE on 4 popular summarization datasets when using highlighted sentences as guidance. In addition, we show that our guided model can generate more faithful summaries and demonstrate how different types of guidance generate qualitatively different summaries, lending a degree of controllability to the learned models.

## introduction modern techniques for text summarization generally can be categorized as either extractive methods @xcite @xcite , which identify the most suitable words or sentences from the input document and concatenate them to form a summary, or abstractive methods @xcite @xcite , which generate summaries freely and are able to produce novel words and sentences. compared with extractive algorithms, abstractive algorithms are more flexible, making them more likely to produce fluent and coherent summaries. however, the unconstrained nature of abstractive summarization can also result in problems. first, it can result in unfaithful summaries @xcite , containing factual errors as well as hallucinated content. second, it can be difficult to control the content of summaries; it is hard to pick in advance which aspects of the original content an abstractive system may touch upon. to address the issues, we propose methods for guided neural abstractive summarization: methods that provide various types of guidance signals that 1) constrain the summary so that the output content will deviate less from the source document; 2) allow for controllability through provision of user-specified inputs. there have been some previous methods for guiding neural abstractive summarization models. for example, @xcite specify the length of abstractive summaries, @xcite provide models with keywords to prevent the model from missing key information, and @xcite propose models that retrieve and reference relevant summaries from the training set. while these methods have demonstrated improvements in summarization quality and controllability, each focuses on one particular type of guidance -it remains unclear which is better and whether they are complementary to each other. in this paper, we propose a general and extensible guided summarization framework that can take different kinds of external guidance as in-work guidance form tokens triples sentences summaries @xcite (length tokens) @xcite (retrieved sums.) @xcite (keywords) @xcite (highlighted sents.) @xcite (length tokens) @xcite (length, entity, style tokens) @xcite al. (2020 put. like most recent summarization models, our model is based on neural encoder-decoders, instantiated with contextualized pretrained language models, including bert @xcite and bart @xcite . with this as a strong starting point, we make modifications allowing the model to attend to both the source documents and the guidance signals when generating outputs. as shown in figure 1 , we can provide automatically extracted or user-specified guidance to the model during test time to constrain the model output. at training time, to encourage the model to pay close attention to the guidance, we propose to use an oracle to select informative guidance signals -a simple modification that nonetheless proved essential in effective learning of our guided summarization models. using this framework, we investigate four types of guidance signals: (1) highlighted sentences in the source document, (2) keywords, (3) salient relational triples in the form of (subject, relation, object), and (4) retrieved summaries. we evaluate our methods on 6 popular summarization benchmarks. our best model, using highlighted sentences as guidance, can achieve stateof-the-art performance on 4 out of the 6 datasets, including 1.28/0.79/1.13 rouge-1/2/l improvements over previous state-of-the-art model on the widely-used cnn/dm dataset. in addition, we perform in-depth analyses of different guidance signals and demonstrate that they are complementary to each other in that there is potential to aggregate their outputs together and obtain further improvements. an analysis of the results also reveals that our guided models can generate more faithful summaries and more novel words. finally, we demonstrate that we can control the output by providing user-specified guidance signals, with different provided signals resulting in qualitatively different summaries. several techniques have been proposed to improve the model architecture. for example, models of copying @xcite @xcite allow words to be copied directly from the input to the output, and models of coverage discourage the model from generating repetitive words @xcite . guidance can be defined as some variety of signal g that is fed into the model in addition to the source document x: within this overall framework, the types of information that go into g and the method for incorporating this information into the model may vary. while there are early attempts at non-neural guided models @xcite , here we focus on neural approaches and summarize recent work in there are also works on controlling the summary length @xcite and styles @xcite by explicitly feeding the desired features to the model. in addition, @xcite and @xcite follow a twostage paradigm, in which a subset of the source document {x i 1 , • • • , x in } will first be selected by a pretrained extractor as highlighted sentences and then be fed into the model encoder in the second stage with the rest of the text discarded. ## methods figure 2 illustrates the general framework of our proposed method. we feed both the source docu-ments and various types of guidance signals to the model. specifically, we experiment with guidance signals including highlighted sentences, keywords, relations, and retrieved summaries, although the framework is general and could be expanded to other varieties of guidance as well. ## conclusion we propose a general framework for guided neural summarization, using which we investigate four types of guidance signals and achieve state-of-theart performance on various popular datasets. we demonstrate the complementarity of the four guidance signals, and find that our models can generate more novel words and more faithful summaries. we also show that we can control the output by providing user-specified guidance signals. given the generality of our framework, this opens the possibility for several future research directions including 1) developing strategies to ensemble models under different guidance signals; 2) incorporating sophisticated techniques such as copy or coverage over the source document, the guidance signal, or both; and 3) experimenting with other kinds of guidance signals such as salient the highest rouge scores with the reference summary. we use a similar algorithm to select the relation triples as well. concretely, we flatten each relational triple (s, r, o) by concatenating its elements together and treat each concatenated text as a source sentence, then use algorithm 1 to select the relation triples greedily.