Extractive Summarization Considering Discourse and Coreference Relations based on Heterogeneous Graph (2021)

Abstract: Modeling the relations between text spans in a document is a crucial yet challenging problem for extractive summarization. Various kinds of relations exist among text spans of different granularity, such as discourse relations between elementary discourse units and coreference relations between phrase mentions. In this paper, we propose a heterogeneous graph based model for extractive summarization that incorporates both discourse and coreference relations. The heterogeneous graph contains three types of nodes, each corresponds to text spans of different granularity. Experimental results on a benchmark summarization dataset verify the effectiveness of our proposed method.

## introduction automatic summarization aims to condense the information of the input document into a shorter summary. the task has two main paradigms: extractive summarization and abstractive summarization. generating summary sentences from scratch, abstractive summarizers can generate concise and flexible summaries. however, they also suffer from the problem of not being able to reproduce factual details correctly @xcite . on the other hand, extractive summarization aims to select salient text spans (mostly sentences) from the input document. compared to abstractive summarizers, extractive summarizers have the advantage of being efficient and factually reliable. in this paper, we will focus on extractive summarization. for extractive summarization, it is crucial to model the relations between text spans throughout the document. between text spans of different granularity, there exist many different kinds of relations (figure 1 ). for example, coreference relations exist between mention phrases of the same entity, and discourse relations exist between elementary discourse units (edus) within a document. due to its complex nature, modeling the various relations among text spans of a document remains an open challenge. to capture inter-sentential relations, some recent works utilize recurrent neural networks (rnns) or transformer @xcite based encoders on top of the acquired sentence representations @xcite @xcite . however, empirical observations show that these sentence-level encoders do not bring much performance gain @xcite . graph structure is an intuitive way to model long-range dependencies among text spans throughout a document. early works build connectivity graphs based on content similarity between sentences @xcite . some recent works incorporate discourse or coreference relations into the graph structure and utilize graph neural networks (gnns) to obtain a high-level representation of text spans @xcite @xcite . most of these works operate on homogeneous graphs with only one type of nodes, such as approximate discourse graph (adg) @xcite or rhetorical structure theory (rst) @xcite dependency graph. as illustrated in figure 1 , the various types of relations exist between text spans of different granularity. thus, homogeneous graphs may not be an ideal way to encode the various types of relations between text spans. in this paper, we propose a novel heterogeneous graph based model for extractive summarization. heterogeneous graphs are graphs that contain multiple node types and/or multiple edge types, which is in contrast to homogeneous graphs that have only one type of nodes and one type of edge. heterogeneous graphs have been widely studied and applied to model data structures such as citation networks @xcite , recommendation systems @xcite , etc. in this work, we use heterogeneous graph to model the document structure with three types of nodes of different granularity: sentence nodes, edu nodes, and entity nodes. we also try to encode both discourse and coreference relations into the graph structure. we encode the discourse relations with the edges between edu nodes. as for the coreference relations, edges between edu nodes and entity nodes are introduced. instead of extracting salient sentences like most existing extractive summarizers, our model extracts salient edus. to identify salient edus in a certain sentence, we add edges between sentence nodes and their constituent edu nodes. to the best of our knowledge, we are the first to utilize heterogeneous graph to incorporate multiple types of relations simultaneously for extractive summarization. ## proposed method given an input document d with n edus {d 1 , d 2 , ..., d n }, we formulate extractive summarization as a sequence labeling problem. the model predicts a sequence of binary labels @xmath0, y 2 , ..., y n }, where y @xmath1. figure 2 provides an overview of our proposed model. first, a bert encoder is used to embed the input document d. with the edu and entity encoders, we acquire the initial node representation of the heterogeneous graph. we then apply a heterogeneous graph encoder to obtain high-level node representations. finally, we make predictions based on the edu node representations. ## conclusion in this paper, we proposed a novel heterogeneous graph based model for extractive summarization. by introducing nodes of different granularity, the heterogeneous graph has the capacity to embed various types of relations between text spans. experiments on cnn/dailymail benchmark dataset illustrated the effectiveness of our proposed method.