Unsupervised Text Summarization of Long Documents using Dependency-based Noun Phrases and Contextual Order Arrangement (2022)

Abstract: Unsupervised extractive summarization has recently gained importance since it does not require labeled data. Among unsupervised methods, graph-based approaches have achieved outstanding results. These methods represent each document by a graph, with sentences as nodes and word-level similarity among sentences as edges. Common words can easily lead to a strong connection between sentence nodes. Thus, sentences with many common words can be misinterpreted as salient sentences for a summary. This work addresses the common word issue with a phrase-level graph that (1) focuses on the noun phrases of a document based on grammar dependencies and (2) initializes edge weights by term-frequency within the target document and inverse document frequency over the entire corpus. The importance scores of noun phrases extracted from the graph are then used to select the most salient sentences. To preserve summary coherence, the order of the selected sentences is re-arranged by a flow-aware orderBERT. The results reveal that our unsupervised framework outperformed other extractive methods on ROUGE as well as two human evaluations for semantic similarity and summary coherence.

## introduction text summarization helps in preserving and compressing representative information from long documents. this work aims at the extractive summarization, which condenses a document by extracting a few salient sentences. it produces fluent sentences with less training data than abstractive methods. most extractive summarization research focuses on supervised learning methods @xcite @xcite @xcite to derive models for automatically observing salient sentences based on specified golden labels. nonetheless, it is impractical to expect the availability of such high-quality training datasets. on the rich unpaired data. in this method, researchers model textual content into sentence-level @xcite @xcite or hybrid @xcite graphs and and adopt pagerank-based algorithms algorithm @xcite to retrieve the salient sentences in a document. due to the characteristics of the graph, the extracted salient sentences are easily affected by common or function words that have high connectivities and are overestimated as key nodes. in addition, coherence is considered as an important attribute in summarization as it keeps the flow of concepts smooth and logical. however, few studies take coherence into consideration. to address the above issues, this work assumes the major concepts in a document are expressed by key noun phrases and constructs a phrase-level graph specific for noun phrases that leverage grammar dependencies. with salient sentences extracted by key noun phrases, a sentence re-ordering step is applied to ensure the flow of concepts is contextually correct for the reader's understanding. there are two major steps in the proposed framework: key noun phrase extraction and salient sentence extraction, as shown in figure 1 . our the 34th conference on computational linguistics and speech processing (rocling 2022) @xcite . the association for computational linguistics and chinese language processing contributions are summarized as: ## key noun phrase extraction keyphrases represent important information in sentences and documents but not all of them contribute the same amount of information. with noun phrases indicated as the most commonly occurring structures among different types of corpora @xcite , the proposed method assumes that they potentially provide coverage of the major conceptual points of the document. by focusing on extracting key noun phrases from documents, this work proposes a graph-based keyphrase extraction for noun phrases in an unsupervised manner. the key noun phrase extraction is separated into two steps: noun phrase hyper relation extraction and graph-based keyphrase scoring. the former is designed to extract the noun phrases along with the relations between them in a complete as possible manner according to the grammar dependency. to extract important noun phrases and avoid an undue influence of common words, further relations are adopted as a guide in constructing the dependency graph for specific noun phrases. ## salient sentence extraction consistent with our assumption that noun phrases potentially provide coverage of the major conceptual aspects of a document, the salient sentences are also selected based on these noun phrases with the corresponding importance scores derived by the weighted graph. in the following section, a contextaware bert is derived to perform sentence reordering for the top salient scores in order to maintain summary coherence. ## conclusion in this research, a fully unsupervised framework is proposed for extractive summarization. the proposed method addresses the common word domination issue from a graph-based approach by using a phrase-level graph that focuses on key noun phrases based on grammar dependencies. the extracted key nounphrases effectively capture the major concepts of a document and can be used to construct an extractive summary. experiments showed that the proposed method outperformed all the extractive baselines, even for supervised methods. a human evaluation also showed that the use of keyphrases and sentence reordering successfully benefited the coherence of the summaries. in the future, we aim to adapt the proposed key noun-phrases for unsupervised abstractive summarization.