GUSUM : Graph-based Unsupervised Summarization Using Sentence Features Scoring and Sentence- BERT (2022)

Abstract: Unsupervised extractive document summarization aims to extract salient sentences from a document without requiring a labelled corpus. In existing graph-based methods, vertex and edge weights are usually created by calculating sentence similarities. In this paper, we develop a Graph-Based Unsupervised Summarization(GUSUM) method for extractive text summarization based on the principle of including the most important sentences while excluding sentences with similar meanings in the summary. We modify traditional graph ranking algorithms with recent sentence embedding models and sentence features and modify how sentence centrality is computed. We first define the sentence feature scores represented at the vertices, indicating the importance of each sentence in the document. After this stage, we use Sentence-BERT for obtaining sentence embeddings to better capture the sentence meaning. In this way, we define the edges of a graph where semantic similarities are represented. Next we create an undirected graph that includes sentence significance and similarities between sentences. In the last stage, we determine the most important sentences in the document with the ranking method we suggested on the graph created. Experiments on CNN/Daily Mail, New York Times, arXiv, and PubMed datasets show our approach achieves high performance on unsupervised graph-based summarization when evaluated both automatically and by humans.

## introduction text summarization is the process of compressing a long text into a shorter version while preserving key information and significance of the content. researchers have examined two summarization models as extractive and abstractive summarization @xcite . extractive summarization creates summaries by extracting text from source documents, whereas abstractive summarization rewrites documents by paraphrasing or deleting some words or phrases. modern text summarization approaches focus on supervised neural networks, which adapt sequenceto-sequence translation, reinforcement learning and large-scale pre-training techniques. these approaches have accomplished favourable results thanks to the availability of large-scale datasets @xcite @xcite @xcite . nevertheless, a major limitation of those supervised methods is that their success is strongly reliant on the availability of large training corpora with human-generated high-quality summaries which are both expensive to produce and difficult to obtain. we focus on unsupervised summarization in this study, where we simply need unlabeled documents. the fundamental issue with unsupervised summarizing is determining which sentences in a document are important. graph-based algorithms, in which each vertex is a sentence and the weights of the edges are measured by sentence similarity, are the most prevalent approaches among these studies. the relevance of each sentence is then estimated using a graph ranking approach. a vertex's centrality is often measured using graph-based ranking algorithms such as pagerank @xcite to decide which sentence to include in the summary. we observe that the importance of the sentences in the document should be emphasized in addition to the semantic similarity of the sentences in the summary. accordingly, we suggest in this study that the centrality measure can be enhanced in two significant ways. first, we define an initial score that specifies the importance of the sentence that each vertex represents. second, we use sentence-bert @xcite which is a modification of the pre-trained bert network @xcite that uses siamese and triplet network structures to derive semantically meaningful sentence embeddings to better capture the sentence meaning and calculate sentence similarity. in this paper, we propose a novel approach, gusum (as shorthand for graph-based unsupervised summarization) which is a simple and powerful approach to improving graph-based unsupervised extractive text summarization. we evaluate the gusum on the cnn/daily mail and new york times short document summarization datasets and arxiv and pubmed long document summarization datasets. for graph-based summarization tasks, pre-trained embeddings are generally used only for measuring sentence similarities in graph-based summarization systems. however, this situation causes the importance of the sentences in the document to be ignored. in our approach, we applied a ranking method that combines sentence similarities and sentence features to calculate sentence centrality. our experiments show that better results are obtained by creating weighted graphs in which the main features of the sentence are represented in the ordering stage based on sentence centrality. our code is available at https: //github.com/tubagokhan/gusum ## related work the proposed method is based on graph-based, unsupervised extractive text summarization techniques. in this section, we introduce work on graphbased summarization, unsupervised summarization and pre-training. ## methodology in this section, we describe our unsupervised summarization method gusum. the system is composed of four main steps: first, we calculate sentence features for defining vertex weight; second, we produce sentence embeddings by sentence-bert to measure sentence similarities; next, we create a graph by comparing all the pairs of sentence embeddings obtained; finally, we rank the sentences by their degree centrality in this graph. figure 1 gives an overview of the whole proposed method. ## experimental setup in this section we assess the performance of gusum on the document summarization task. we first introduce the datasets that we used, then give our pre-processing and implementation details. ## discussion there are two basic stages in document summarization: (1) identification of the most salient sentences in the document, (2) removal of similar sentences from the summary. generally in graphbased approaches, graphs are created based on only sentence similarity, and then the most salient sentences are selected. on the contrary, in gusum we included these two basic steps in our approach. along with the semantic similarity, we also embedded the attributes of the sentences in our graph. furthermore, gusum advocates the idea that the most important sentence in a document is the sentence most similar to the others. for this reason, the total similarity value for each sentence is evaluated in the ranking stage. the experimental results of gusum, which is a simple and effective method based on these ideas, prove the validity of our ideas. as seen in the experimental results, gusum showed high performance on all datasets. however, the limitation of gusum is that sentence features scoring does not have a significant impact on long documents as can be see in the most difficult part of this study is the evaluation stage. evaluating the performance of summarization systems poses a problem for many researchers @xcite . it is a known fact by researchers that human evaluation is the best summary performance evaluation method. for this reason, we included human evaluation as a performance evaluation method in our study. however, what we noticed in our study is that the questions used for human evaluation based on the qa paradigm in other studies published to date have not been shared by the researchers. as a result of this situation, researchers prepare their own questions and the results cannot be compared with the literature. as a solution to this problem, we publish the questions and answers that we prepared from the cnn/dm and nyt datasets based on the qa paradigm for use in future studies (see 5.2). ## conclusions and future works in this paper, we have proposed a graph-based single-document unsupervised extractive summarization method. we revisited traditional graphbased ranking algorithms and refined how sentence centrality is computed. we defined values indicating the importance of the sentences in the document for the node weights in the graphs and we built graphs with undirected edges by employing sentence-bert to better capture sentence similarity. experimental results on four summarization benchmark datasets demonstrated that our method outperforms other recently proposed extractive graph-based unsupervised methods and achieves performance comparable to many stateof-the-art supervised approaches which shows the effectiveness of our method. in the future, we would like to remove the limitations that would increase the performance of gusum in long document summarization with the ideas introduced in this study and explore the performance of gusum in multi-document summarization.