Controllable Abstractive Sentence Summarization with Guiding Entities (2020)

Abstract: Entities are the major proportion and build up the topic of text summaries. Although existing text summarization models can produce promising results of automatic metrics, for example, ROUGE, it is difficult to guarantee that an entity is contained in generated summaries. In this paper, we propose a controllable abstractive sentence summarization model which generates summaries with guiding entities. Instead of generating summaries from left to right, we start with a selected entity, generate the left part first, then the right part of a complete summary. Compared to previous entity-based text summarization models, our method can ensure that entities appear in final output summaries rather than generating the complete sentence with implicit entity and article representations. Our model can also generate more novel entities with them incorporated into outputs directly. To evaluate the informativeness of the proposed model, we develop a fine-grained informativeness metrics in the relevance, extraness and omission perspectives. We conduct experiments in two widely-used sentence summarization datasets and experimental results show that our model outperforms the state-of-the-art methods in both automatic evaluation scores and informativeness metrics.

## introduction automatic text summarization is a task of shortening and creating a concise version of a text that represents the most important or relevant information within the original text. the task can be divided into two subtasks: extractive and abstractive summarization. recently, the research trend @xcite @xcite has shifted towards abstractive summarization since it can produce more natural and coherent texts than extractive methods. sequence-to-sequence models with attention mechanism @xcite ) have attracted growing attention due to their state-of-the-art performance on abstractive summarization. however, most previous seq2seq models have two obvious limitations: the first limitation is that existing methods are difficult to identify salient entities and related events in the original articles @xcite . as a result, they are frequently found to be unfaithful to the input and lack of topic coherence. according to our observations, entities are important for summary generation in three points: (1) summaries are mainly composed by entities extracted from the original texts. 77.1% and 71.5% of the noun phrases on the gigaword and duc-2004 datasets contain at least one entity, respectively. (2) entities can reveal the main topics in an article @xcite , which can be utilized to generate the summaries of highly topic coherence. for example, the per entity "kobe-bryant" and the org entity "nba" are mostly associated with topic "games" and "players". such implicit representations cannot guarantee entities to appear in final outputs. including a novel entity in the summary is even harder since the entity has not been seen in training data. as a result, it is necessary to develop a controllable framework which incorporates entity information and generates summaries with selected entities. to address above limitations, we propose a controllable neural model which generates summaries with guiding entities. as shown in figure 1 , we first extract entities in the original texts using a pretrained named entity recognition model @xcite . considering that only a few of these entities should appear in final summaries, an entity selector will identify the most important entities and send the entity representations to the summary generation phase. previous works @xcite generates summaries from left to right, the entity information is encoded implicitly and may not appear in generated summaries. we handle this problem by combining two lstms to generate the left part and the right part of the sequence around the entity. each lstm encodes information of the other part of the sequence and then generates a summary based on the encoded article and entity representations. this allows the two lstm decoders to connect with the selected entities fluently. as a result, we can guarantee that important entities appear in output summaries. since the novel entities are incorporated into the generated sentences directly, our model can also generate more novel entities without going through the encoding-decoding process. in summary, we make the following major contributions in this paper: entity-based text summarization. entities have been studied on the summarization tasks in the preprocessing steps to anonymize text data @xcite or to mitigate oov problems @xcite . @xcite find it beneficial to use linked entities in decoder phase to improve the summarization performance. their method fails in capturing the entities that do not exist in the knowledge bases. @xcite propose a two-step method which combines the salient content selector and entity-based decoder to improve the coherence for abstractive summarization. however, the above methods are based on the sequence-to-sequence framework which generates summaries from left to right and cannot guarantee that the entities appear in final output summaries. to the best of our knowledge, we are the first to use two separate lstms to generate summaries from guiding entities. ## abstractive summarization with guiding entities in this paper, we propose a neural abstractive summarization model which leverages named entity information in original articles to generate informative summaries. the architecture is illustrated in figure 2 . given an input article, our model is able to incorporate the selected entities into generated summaries. unlike previous works generating summaries from left to right and decoding with implicit entity representations, our model combines two lstms to generate the partial summaries on two sides of the guiding entities. the overall architecture is built on the widely-used encoder-decoder framework. in encoding phase, we extract original article representations with a bi-lstm layer. the named entities are identified by a pretrained bert-ner @xcite model. we select the top k important entities and transform them into word embeddings. in decoding phase, we use lstm-l to denote the lstm layer generating the left part of the summary and lstm-r to generate the right part. the two lstm layers are training simultaneously with a multitask loss, which can share the same article features and entity information. we will describe each part of our model in detail. ## results and discussion in this section, we evaluate the proposed model from both quantitative and qualitative perspectives. ## conclusion in this paper, we propose an abstractive summarization model which is more informative than previous works with the guiding entities. our model can ensure that the important entities appear in final output texts with lstm-l and lstm-r decoders. experimental results demonstrate that our model can generate entities with better fluency and coherence. we also develop a fine-grained informativeness metric to evaluate our model compared to previous state-of-the-art methods. our model achieves better informativeness results than other methods in a comprehensive and diverse manner.