A Survey of Automatic Text Summarization Using Graph Neural Networks (2022)

Abstract: Although automatic text summarization (ATS) has been researched for several decades, the application of graph neural networks (GNNs) to this task started relatively recently. In this survey we provide an overview on the rapidly evolving approach of using GNNs for the task of automatic text summarization. In particular we provide detailed information on the functionality of GNNs in the context of ATS, and a comprehensive overview of models utilizing this approach.

## introduction the advent of the internet has led to an explosion in the amount of textual information available online. the extensive availability of textual information paired with a need to quickly understand it has led to major efforts in the field of automatic text summarization (ats). for a comprehensive review and survey of ats as a task we recommend a recent survey by @xcite . the goal of ats is to produce a concise, correct and fluent summary of a given text. although this definition is intuitively understandable, there is no commonly agreed upon formal definition for these qualities. this is in part due to the difficulty of the task itself as producing a summary with the above properties is challenging, even for humans. although the field of ats has made major steps forward, generally, we differentiate between two basic approaches to the task of ats. the first being extractive ats and the second being abstractive ats. extractive ats involves extracting text spans from the original document such that a summary is generated. thus the main challenge consists in identifying useful spans of text from the original document. this approach is popular as it eliminates the non-trivial task of generating factually correct and coherent sentences. abstractive ats on the other hand involves the challenging task of generating novel sentences for a summary. despite impressive advancements, generating factually correct and fluent sentences is still a major challenge in ats @xcite . in addition to the above, one differentiates between the task of singledocument and multi-document summarization. ## graph neural networks graph based methods for ats are not a new innovation, with methods such as textrank @xcite being first presented in the early 2000s. in fact, the core idea of early graph-based approaches such as textrank is similar to the one used by gnns designed for the ats task. the idea is to encode the structural and semantic information contained within a text document into an explicit form with the help of a graph. deep learning has by now become a common tool for solving tasks throughout many domains, with various end-to-end paradigms such as recurrent-neural-networks (rnns) or convolutional-neural-networks (cnns) emerging as versatile and powerful tools. in particular deep learning has shown extraordinary power for data which is euclidean in nature. as an example, images can be represented as regular grids in euclidean space. using such a representation cnns are able to extract meaningful local representations with the help of convolution. however, for many domains and applications such a representation is often either inconvenient or not even directly possible. the most obvious example being the field of chemistry in which molecules should be modeled as graphs. the complexity of translating such complex data into existing deep learning paradigms has led, in part, to the development of gnns which attempt to leverage the power of deep learning for non-euclidean data. ## graph neural network models for automatic text summarization the current state of research clearly shows that the usage of gnns for ats follows one of two patterns, either the gnn is used directly for the ats task or it is used to support a larger system. these trends justify the taxonomy as seen in figure 2 . more precisely, we classify standalone ats gnns as gnn models which are directly responsible for generating a summary. embedded gnns on the other hand are used to support a larger system, and are not directly or solely responsible for producing the summary. in addition to this difference, we further differentiate between embedded gnns used in an encoder-decoder setting and other embedded gnns. note that we do not specifically differentiate between single and multi-document gnn summarization models as some models are by design capable of handling both tasks. additionally, we do not make a distinction between abstractive and extractive approaches, since embedded gnns can be part of either approach, while abstractive approaches have not yet been developed for standalone gnns, to the best of our knowledge. ## conclusions and outlook for future we have surveyed in this paper the main developments in the area of gnns applied to the automatic text summarization task, first describing how gnns work and then discussing the prominent models used for ats. we have also provided simple categories of the models. finally, in we have already highlighted some general advantages of gnns for ats. now considering the models presented and the results shown in