An Efficient Coarse-to-Fine Facet-Aware Unsupervised Summarization Framework Based on Semantic Blocks (2022)

Abstract: Unsupervised summarization methods have achieved remarkable results by incorporating representations from pre-trained language models. However, existing methods fail to consider efficiency and effectiveness at the same time when the input document is extremely long. To tackle this problem, in this paper, we proposed an efficient Coarse-to-Fine Facet-Aware Ranking (C2F-FAR) framework for unsupervised long document summarization, which is based on the semantic block. The semantic block refers to continuous sentences in the document that describe the same facet. Specifically, we address this problem by converting the one-step ranking method into the hierarchical multi-granularity two-stage ranking. In the coarse-level stage, we proposed a new segment algorithm to split the document into facet-aware semantic blocks and then filter insignificant blocks. In the fine-level stage, we select salient sentences in each block and then extract the final summary from selected sentences. We evaluate our framework on four long document summarization datasets: Gov-Report, BillSum, arXiv, and PubMed. Our C2F-FAR can achieve new state-of-the-art unsupervised summarization results on Gov-Report and BillSum. In addition, our method speeds up 4-28 times more than previous methods.

## introduction the text summarization task aims to condense a document or a set of documents into several sentences and keep the primary information. recent years, both supervised @xcite @xcite and unsupervised @xcite @xcite methods have made significant improvements over short documents with the development of semantic representations from pre-trained language models (plms). due to the noise and complexity of the increased input and output length, long-form document summarization is still a challenge @xcite @xcite . compared with supervised one, unsupervised methods do not rely on large amounts of labeled data and have no limitation on input length. in addition, unsupervised methods can be easily adapted to data from different domains, types, and languages. in this paper, we focus on unsupervised extractive methods for long document summarization. most unsupervised extractive methods are graphbased @xcite @xcite . they represent document sentences as nodes in a graph, where the edge value is the similarity between sentences. then, they measure the importance of each node via computing the degree centrality @xcite or running pagerank @xcite algorithm. @xcite pointed out that centrality-based methods always tend to select sentences within the same facet (i.e. aspect, sub-topic) and proposed a facet-aware ranking (far) method to tackle this problem. far forces a centrality-based model to select summary sentences from different facets by incorporating the relevance between the candidate summary and the document. however, this method faces two problems when the document is extremely long: 1) as the input length increases, the document will have more noise and insignificant facets. the relevance computation between the candidate summary and the document may cause the facet-aware ranking to be influenced by insignificant facets. 2) the running time of far will rise rapidly as the number of extracted sentences increases. due to far needs to compute the relevance score number of combinations c k m times, where k is the number of extracted summary sentences and m is the number of candidate salient sentences. to tackle these problems, in this paper, we propose a novel coarse-to-fine facet-aware ranking (c2f-far) framework based on semantic blocks, which consists of two stages with different granularities: semantic blocks and sentences. the semantic block means continuing sentences that describe the same facet. we use a simple example in fig. 1 to describe the motivation for building two stages. fig. 1 shows four facet-aware semantic blocks. each block contains continuous sentences describing the same facet, which is listed on the right. from the coarse-level view, we should first filter blocks with unimportant facets in the document, e.g. the block related to "additional goal of pilt" in fig. 1 . then, from the fine-level view, we should select proper sentences in each block, which are more relevant to the block facet. note that we only show the most relevant sentences with the facet of each semantic block and omit unrelated sentences due to the space limitation. finally, the highlighted sentences should be selected as the summary. following this intuitive process, we designed our framework with a coarse-level stage and a finelevel stage. the coarse-level stage aims to select several salient facet-aware semantic blocks for the fine-level stage. we first segment the document into facet-aware semantic blocks by our proposed new document segmentation algorithm, which is inspired by texttiling @xcite . then, we filter insignificant facets via a coarse-level centrality estimator to measure the salience of blocks. the fine-level stage aims to select final summary sentences from previously selected blocks. we first select candidate sentences in each block to represent its facet by simply computing relevance between sentences and the block. finally, we extract the final summary from candidate sentences by sentence-level centrality-based estimator. overall, the coarse-level stage can identify all facets of the document effectively and filter insignificant ones. the fine-level stage can reduce the influence of facets with many sentences by only selecting several related sentences for the final ranking. this framework with a hierarchical coarse-to-fine structure can guarantee effective and efficient long document summarization. we evaluate the effectiveness and efficiency of our c2f-far on four long-document summarization datasets with two different metrics. our method achieves new state-of-the-art performance on gov-report and billsum. it is comparable to strong baselines on arxiv and pubmed. besides, our method can achieve a speedup of 4-28 times more than two strong baselines. ## methodology we show the workflow of our proposed coarse-tofine facet-aware ranking (c2f-far) framework in fig. 2 . after encoding the document into sentence embeddings, the workflow contains two main stages and each stage contains two steps. we describe the details of each step in the following sections. ## analysis in this section, we first analyze the parameter sensitivity of our c2f-far and then discuss the reason why our method is inferior to the far on arxiv and pubmed via facets analysis of extracted sentences. ## conclusion in this paper, we focus on unsupervised long document summarization tasks, which is a vital and long-standing challenge in text summarization. to obtain summary sentences efficiently and effectively, we proposed a novel coarse-to-fine facetaware ranking framework. our method can achieve new state-of-the-art results on two datasets. experiments show that our approach is effective and efficient for the long document summarization task. in future work, we will investigate how to refactor this process into an end-to-end paradigm.