BASS : Boosting Abstractive Summarization with Unified Semantic Graph (2021)

Abstract: Abstractive summarization for long-document or multi-document remains challenging for the Seq2Seq architecture, as Seq2Seq is not good at analyzing long-distance relations in text. In this paper, we present BASS, a novel framework for Boosting Abstractive Summarization based on a unified Semantic graph, which aggregates co-referent phrases distributing across a long range of context and conveys rich relations between phrases. Further, a graph-based encoder-decoder model is proposed to improve both the document representation and summary generation process by leveraging the graph structure. Specifically, several graph augmentation methods are designed to encode both the explicit and implicit relations in the text while the graph-propagation attention mechanism is developed in the decoder to select salient content into the summary. Empirical results show that the proposed architecture brings substantial improvements for both long-document and multi-document summarization tasks.

## introduction nowadays, the sequence-to-sequence (seq2seq) based summarization models have gained unprecedented popularity @xcite @xcite . however, complex summarization scenarios such as long-document or multi-document summarization (mds), still bring great challenges to seq2seq models @xcite . in a long document numerous details and salient content may distribute evenly @xcite while multiple documents may contain repeated, redundant or contradictory information @xcite . these problems make seq2seq models struggle with content selection and organization which mainly depend explanation cop nomod:of ## unified semantic graph in this section, we introduce the definition and construction of the unified semantic graph. ## summarization model in this section, we introduce our graph-based abstractive summarization model, which mainly consists of a graph encoder and a graph decoder, as shown in figure 2 . in the encoding stage, our albert einstein was a theoretical physicist . he was born in germany text encoder albert einstein was born in a theoretical physicist was germany graph @xmath0: illustration of our graph-based summarization model. the graph node representation is initialized from merging token representations in two-level. the graph encoder models the augmented graph structure. the decoder attends to both token and node representations and utilizes graph structure by graph-propagation attention. model takes a document or the concatenation of a set of documents as text input (represented as @xmath1, and encodes it by a text encoder to obtain a sequence of local token representations. the graph encoder further takes the unified semantic graph as graph input (represented as @xmath2, e) in section 3.1), and explicitly model the semantic relations in graph to obtain global graph representations. based on several novel graph-augmentation methods, the graph encoder also effectively taps the implicit semantic relations across the text input. in the decoding stage, the graph decoder leverages the graph structure to guide the summary generation process by a novel graph-propagate attention, which facilitates salient content selection and organization for generating more informative and coherent summaries. ## experiment setup in this section, we describe the datasets of our experiments and various implementation details. ## conclusion and future work in this paper, we propose to leverage the unified semantic graph to improve the performance of neural abstractive models for long-document summarization and mds. we further present a graph-based encoder-decoder model to improve both the document representation and summary generation process by leveraging the graph structure. experiments