Knowledge and Keywords Augmented Abstractive Sentence Summarization (2021)

Abstract: In this paper, we study the abstractive sentence summarization. There are two essential information features that can influence the quality of news summarization, which are topic keywords and the knowledge structure of the news text. Besides, the existing knowledge encoder has poor performance on sparse sentence knowledge structure. Considering these, we propose KAS, a novel Knowledge and Keywords Augmented Abstractive Sentence Summarization framework. Tri-encoders are utilized to integrate contexts of original text, knowledge structure and keywords topic simultaneously, with a special linearized knowledge structure. Automatic and human evaluations demonstrate that KAS achieves the best performances.

## introduction with the increasing of computing power and model capacity, it is possible to generate mostly grammatical summarization of natural language text. in general, there are two essential information features of summarization: (1) topic keywords in text (2) the knowledge structure of the text. these features can basically cover all the information in summary generation, especially in sentence or short text summarization. therefore, considering this reason, we are building a neural network model that integrates both topic keyword context and knowledge structure context. knowledge augmented summarization has been intensively studied recently, most of which are about document summarization. however, there is not much research on knowledge-based sentence summarization. the main reason is that the existing methods are not applicable to sentence summarization. the knowledge-based summarization frameworks usually use gnn (graph 1 code is in https://github.com/seang-325/kas neural networks) as the knowledge structure encoder. gnn is designated for encoding graph structure, and it has gained good performances on integrating knowledge graph contexts for document and multi-paragraph summarization. however, the knowledge graph of sentence is usually sparse, and gnn has poor performance in sparse knowledge structure. specifically, gnns may cause over-smoothing problem when training on the sparse graphs @xcite , especially for gcns (graph convolutional networks) @xcite , decreasing the robustness and performance of the model. therefore, we are creating a new knowledge-augmented sentence summarization model considering these problems. besides, considering most of the knowledge based summarization models are only applicable to english, we are aiming at making our model applicable to multiple languages. in order to address these issues, we propose a special linearized knowledge sequence structure that are applicable to sentence summarization. correspondingly, we propose a novel triencoder framework kas integrating three separate encoders, considering contexts of original text, topic keywords and knowledge structure simultaneously based on their salience. evaluations demonstrate that kas framework and the corresponding linearized knowledge structure enhances the performances significantly. besides, the structure of kas can be applied to summarization on multiple languages. we have conducted experiments on english and chinese corpus and achieved best performances on both. the pointer mechanism pointer mechanism has drawn much attention in text generation @xcite @xcite . in text summarization, pointer-generator network model @xcite is proposed to keep the generation ability while using pointer mechanism. @xcite proposed a method for using pointer mechanism with multiple separate encoders. the idea of pointer mechanism is setting soft or hard gates to select from predefined vocabulary or input sequences to generate tokens. ## conclusion in this work, we propose kas, an abstractive summarization framework augmented by knowledge and topic keywords that supports multiple languages. experimental results show that kas generates more qualified summaries and achieves the best performances. in the future, we aim at enhancing attractiveness of sentence summarization based on our structure.