A Spectral Method for Unsupervised Multi-Document Summarization (2020)

Abstract: Multi-document summarization (MDS) aims at producing a good-quality summary for several related documents. In this paper, we propose a spectral-based hypothesis, which states that the goodness of summary candidate is closely linked to its so-called spectral impact. Here spectral impact considers the perturbation to the dominant eigenvalue of affinity matrix when dropping the summary candidate from the document cluster. The hypothesis is validated by three theoretical perspectives: semantic scaling, propagation dynamics and matrix perturbation. According to the hypothesis, we formulate the MDS task as the combinatorial optimization of spectral impact and propose an accelerated greedy solution based on a surrogate of spectral impact. The evaluation results on various datasets demonstrate: (1) The performance of the summary candidate is positively correlated with its spectral impact, which accords with our hypothesis; (2) Our spectral-based method has a competitive result as compared to state-of-the-art MDS systems.

## introduction given a cluster of documents related to the same topic or event, the task of multi-document summarization (mds) centers on a brief summary of the cluster. as emphasized by @xcite , for this task, the labeled training data (i.e. clustersummary pairs) are scarce. hence dealing with it in an unsupervised paradigm becomes a reasonable choice. for the unsupervised mds task, the automatic summarizer is required to discover the main content of the document cluster without the guidance of golden summaries. to preserve the fluency and grammaticality of summary, we mainly focus on the extractive method in which summary sentences are extracted from the original document cluster. in this paper, we propose a novel spectral-based hypothesis for the unsupervised mds task. the hypothesis states that the goodness (or effectiveness) of any summary candidate is closely linked with its spectral impact on the document cluster. the spectral impact of a summary candidate quantifies the perturbation to the dominant eigenvalue (in modulus) of affinity matrix when dropping the candidate from the document cluster. in other words, the hypothesis points out the spectral impact as an indicator of the mds task, which is the first attempt to characterize mds from a spectral viewpoint explicitly. as a representation of the document cluster, the affinity matrix supports the definition of spectral impact. adjusting the building of the affinity matrix can bring out the best in the hypothesis. to validate the proposed hypothesis, we provide both theoretical explanations and empirical evidence. theoretically, the spectral impact caused by dropping a summary from the cluster can be characterized from three different perspectives (see §2.4). empirically, for any summary candidate, the real dataset witnesses a positive correlation between its performance and computed spectral impact. for a particular mds task, applying the hypothesis leads to a constrained optimization problem where the objective function is spectral impact. our summarizer utilizes an accelerated greedy algorithm based on a surrogate of spectral impact. the competitive results of our summarizer have been obtained on various datasets. the differences between prior works and our method are clarified for unsupervised mds: our main contributions are twofold: (1) a novel spectral-based hypothesis for unsupervised mds, which gains support from both theoretical and empirical sides; (2) an accelerated greedy algorithm for solving the hypothesis-driven optimization problem. the rest of the paper is organized as follows. §2 gives the details of our method, including the spectral-based mds hypothesis and the greedy algorithm to solve the spectral optimization problem. evaluation results, related work and conclusions are covered in §3, §4 and §5 respectively. ## spectral-based mds what role does the summary play in the process of mds? our proposed hypothesis offers a spectral insight and brings out a workable formulation of mds. ## related work unsupervised mds. there are a bunch of works working on different hypotheses and models in this field. pagerank alike algorithms @xcite @xcite utilize random walks with some redundancy avoiding measures. regarding the document cluster as a manifold structure, @xcite @xcite use the manifold ranking process on data. there are also quite a few neural architecture based models for a hidden semantic representation of sentences, documents or subtopics, such as @xcite @xcite . spectral optimization. optimizing eigen-related metrics often leads to a specific collective optimization problem, which is believed to be hard in nature unless @xmath0. some particular examples @xcite have been proved np-hard. the typical solvers adopt the heuristics based on either perturbation theory @xcite or qr decomposition @xcite . ## conclusion we propose a novel hypothesis-driven method for unsupervised mds, where the goodness of any summary candidate can be determined from a spectral perspective when dropping it from the document cluster. various mds tasks of different sizes and domains show a promising result of our method. extending our method to an abstractive setting is meaningful future work.