Multiplex Graph Neural Network for Extractive Text Summarization (2021)

Abstract: Extractive text summarization aims at extracting the most representative sentences from a given document as its summary. To extract a good summary from a long text document, sentence embedding plays an important role. Recent studies have leveraged graph neural networks to capture the inter-sentential relationship (e.g., the discourse graph) within the documents to learn contextual sentence embedding. However, those approaches neither consider multiple types of inter-sentential relationships (e.g., semantic similarity and natural connection relationships), nor model intra-sentential relationships (e.g, semantic similarity and syntactic relationship among words). To address these problems, we propose a novel Multiplex Graph Convolutional Network (Multi-GCN) to jointly model different types of relationships among sentences and words. Based on Multi-GCN, we propose a Multiplex Graph Summarization (Multi-GraS) model for extractive text summarization. Finally, we evaluate the proposed models on the CNN/DailyMail benchmark dataset to demonstrate effectiveness of our method.

## introduction numerous documents from a variety of sources are uploaded to the internet or database everyday, such as news articles @xcite , scientific papers @xcite and electronic health records @xcite . how to effectively digest the overwhelming information has always been a fundamental question in natural language processing @xcite . this question has sparked the research interests in the task of extractive text summarization, which aims to generate a short summary of a document by extracting the most representative sentences from it. most of the recent methods @xcite @xcite @xcite formulate the task of extractive text summarization as a sequence labeling task, where the labels indicate whether a sentence should be included in the summary. to extract sentence features, existing approaches generally use recurrent neural networks (rnn) @xcite @xcite @xcite , convolutional neural networks (cnn) @xcite @xcite or transformers @xcite . endeavors have been made to develop models to capture various sentence-level relations. early studies, such as lexrank @xcite and textrank @xcite , built similarity graphs among sentences and leverage pagerank @xcite to score them. later, graph neural networks e.g., graph convolutional network (gcn) (kipf and welling, 2016) have been adopted on various inter-sentential graphs, such as the approximate discourse graph @xcite , the discourse graph @xcite and the bipartite graph between sentences and words @xcite . albeit the effectiveness of the existing methods, there are still two under-explored problems. firstly, the constructed graphs of the existing studies only involve one type of edges, while sentences are often associated with each other via multiple types of relationships (referred to as the multiplex graph in the literature @xcite ). two sentences with some common keywords are considered to be naturally connected (we refer this type of graph as the natural connection graph). for example, in figure 1 , the first and the last sentence exhibit a natural connection (green) via the shared keyword "city". although the two sentences are far away from each other, they can be jointly considered as part of the summary since the entire document is about the keyword "city". however, such a relation can barely be captured by traditional encoders such as rnn and cnn. two sentences sharing similar meanings are also considered to be connected (we refer this type of graph as the semantic graph). in figure 1 , the second and the third sentence are semantically similar since they express a similar meaning (yellow). the semantic similarity graph maps the semantically similar sentences into the same cluster and thus helps the model to select sentences from different clusters, which could improve the coverage of the summary. different relationships provide relational information from different aspects, and jointly modeling different types of edges will improve model's performance @xcite @xcite @xcite . secondly, the aforementioned methods fall short in taking advantage of the valuable relational information among words. both of the syntactic relationship @xcite and the semantic relationship among words @xcite @xcite @xcite have been proven to be useful for the downstream tasks, such as text classification @xcite , information retrieval @xcite ) and text summarization @xcite . we summarize our contributions as follows: ## methodology we first present multi-gcn to jointly model different relations, and then present the multi-gras approach for extractive text summarization. ## sensitivity experiments to check the performance on the number of selected sentences, we conduct a sensitivity experiment for both cnn and dailymail datasets. the results in figure 3 show that the multi-gras performs the best when the number of the selected sentences is 2 for the cnn dataset and 3 for the dailymail dataset. ## conclusion in this paper, we propose a novel multi-gcn to jointly model multiple relationships among words and sentences. based on multi-gcn, we propose a novel multi-gras model for extractive text summarization. experimental results on the benchmark cnn/dailymail dataset demonstrate the effectiveness of the proposed methods.