S umm VD : An efficient approach for unsupervised topic-based text summarization (2022)

Abstract: This paper introduces a new method, SummVD, for automatic unsupervised extractive summarization. This method is based on singular value decomposition, a linear method in the number of words, in order to reduce the dimensionality of word embeddings and propose a representation of words on a small number of dimensions, each representing a hidden topic. It also uses word clustering to reduce the vocabulary size. This representation, specific to one document, reduces the noise brought by several dimensions of the embeddings that are useless in a restricted context. It is followed by a linear sentence extraction heuristic. This makes SummVD an efficient method for text summarization. We evaluate SummVD using several corpora of different nature (news, scientific articles, social network). Our method outperforms in effectiveness recent extractive approaches. Moreover, SummVD requires low resources, in terms of data and computing power. So it can be run on long single documents such as scientific papers as much as large multi-document corpora and is fast enough to be used in live summarization systems.

## introduction research on automatic summarization has recently focused on supervised approaches. since pointer generator by @xcite , there has been considerable advances in the supervised generative summarization field @xcite @xcite . however, these approaches need substantial learning corpora composed of a large amount of documents and summary pairs, and despite recent advances on fine-tuning and transfer learning, are limited to specific domains. thus research on unsupervised summarization methods cannot be left out. in this paper, we tackle the problem of unsupervised extractive summarization, which aims to select sentences from one or multiple documents and put them together in order to build a summary. this extraction is often based on centrality and diversity notions : how much is a sentence central to the input text, and how many of the central information is present in the output summary. inspired by the work of @xcite on long texts similarity computation, we assume that hidden topics specific to a text can emerge from word embeddings computed from a general corpus. each topic stands for a particular aspect of the text semantics. these hidden topics allow to remove unnecessary information from word representations and can be viewed as a new representation of the text. words can be matched against a hidden topic, and this way, we can derive word centrality scores from a text, originally represented as a word embeddings matrix. given these word scores, a sentence extraction heuristic can be applied to generate an extractive summary. we propose a new efficient method for unsupervised extractive summarization, called summvd, whose code is available online foot_0 . we present recent unsupervised methods in section 2. after, we describe our method in section 3.1. section 4 presents our experiments led on a large variety of summarization corpora combining single and multi-document benchmarks, in order to test its generalization. the results shown in section 5 outperform recent unsupervised methods on most of the evaluation corpora, and get sometimes close to supervised methods. we then discuss in section 6 complexity and scalability of our method. sum-mvd's ability to run on long and multi-documents makes it an efficient method to summarize any kind of document, like scientific articles. ## results in order to evaluate our method, we use the common known rouge f1 measure @xcite . the python library that we use can be found here 7 . this is equivalent to calling the perl rouge script as: "rouge-1.5.5.pl -m -e ./data -n 2 -a /tmp/rouge/settings.xml". ## conclusion this article presents a method, summvd, based on word embedding and unsupervised methods which achieves fast and reliable summaries. we presented an extraction heuristic able to exploit the reduced document matrix that deals with single or multidocument and conducted an evaluation as complete as possible, led on heterogeneous corpora. the empirical study shows interesting results according to the state-of-the-art whether in terms of rouge effectiveness or in computation time. compared to the most recent approaches, summvd is better in average rouge scores while being around 1000 times faster on the datasets with the longest documents. this is achieved without any domain adaptation of the word embeddings; so there is room for improvement on domains such as medical/scientific or social media because they use a specific vocabulary that could be handled better. its versatility on documents regardless of their type or size, paves the way to much more exploration on huge multi-document datasets, like google, tri-padvisor or amazon for example.