S g S um:Transforming Multi-document Summarization into Sub-graph Selection (2021)

Abstract: Most of existing extractive multi-document summarization (MDS) methods score each sentence individually and extract salient sentences one by one to compose a summary, which have two main drawbacks: (1) neglecting both the intra and cross-document relations between sentences; (2) neglecting the coherence and conciseness of the whole summary. In this paper, we propose a novel MDS framework (SgSum) to formulate the MDS task as a sub-graph selection problem, in which source documents are regarded as a relation graph of sentences (e.g., similarity graph or discourse graph) and the candidate summaries are its sub-graphs. Instead of selecting salient sentences, SgSum selects a salient sub-graph from the relation graph as the summary. Comparing with traditional methods, our method has two main advantages: (1) the relations between sentences are captured by modeling both the graph structure of the whole document set and the candidate sub-graphs; (2) directly outputs an integrate summary in the form of sub-graph which is more informative and coherent. Extensive experiments on MultiNews and DUC datasets show that our proposed method brings substantial improvements over several strong baselines. Human evaluation results also demonstrate that our model can produce significantly more coherent and informative summaries compared with traditional MDS methods. Moreover, the proposed architecture has strong transfer ability from single to multi-document input, which can reduce the resource bottleneck in MDS tasks.

## introduction currently, most extractive models treat summarization as a sequence labeling task. they score and select sentences one by one @xcite . these models (called sentence-level extractors) do not consider summary as a whole but a combination of independent sentences. this may cause incoherent and redundant problem, and result in a poor summary even if the summary consists of high score sentences. some works @xcite treat summary as a whole unit and try to solve the weakness of sentencelevel extractors by using a summary-level extractor. however, these models neglect the intra and cross-document relations between sentences which also have benefits for extracting salient sentences, detecting redundancy and generating overall coherent summaries. relations become more necessary when input source documents are much longer and more complex such as multi-document input. in this paper, we propose a novel mds framework called sgsum which formulates the mds task as a sub-graph selection problem. in our framework, source documents are regarded as a relation graph of sentences (e.g., similarity graph or discourse graph) and the candidate summaries are its sub-graphs. in this view, how to generate a good summary becomes how to select a proper subgraph. in our framework, the whole graph structure is modeled to help extract salient information from source documents and the sub-graph structures are also modeled to help reflect the quality of candidate summaries. moreover, the summary is considered as a whole unit, so sgsum directly outputs the final summary in the form of sub-graph. by capturing relations between sentences and evaluating summary as a sub-graph, our framework can generate more informative and coherent summaries compared with traditional extractive mds methods. we evaluate sgsum on two mds datasets with several types of graphs which all significantly improve the mds performance. besides, the human evaluation results demonstrate that sgsum can obtain more coherent and informative summaries compared with traditional mds methods. more- ## summarization as sub-graph selection the graph structure is effective to model relations between sentences which is an essential point to select interrelated summary-worthy sentences in extractive summarization. @xcite utilize a similarity graph to construct an unsupervised summarization methods called lexrank. g-flow @xcite and discobert @xcite both use discourse graphs to generate concise and informative summaries. @xcite and @xcite propose to utilize event relation graph to represent documents for mds. however, most existing graph-based summarization methods only consider the graph structure of source document. they neglect that summary is also a graph and its graph structure can reflect the quality of a summary. for example, in a similarity graph, if selected sentences are lexical similar, the summary is probably redundant. and in a discourse graph, if selected sentences have strong discourse connections, the summary tend to be coherent. we argue that the graph structure of summary is equally important as the source document. document graph helps to extract salient sentences, while summary graph helps to evluate the quality of summary. based on this thought, we propose a novel mds framework sgsum which transforms summarization into the problem of sub-graph selection. sgsum captures relation of sentences both in whole graph structure (source documents) and sub-graph structures (candidate summaries). moreover, in our framework, summary is viewed as a whole unit in the form of sub-graph. thus, sgsum can generate more coherent and informative results than traditional sentence-level extractors. figure 1 shows the overview of our framework. firstly, source documents are transformed into a relation graph by well-known graph construction methods such as similarity graph and discourse graph. sentences are the basic information units and represented as nodes in the graph. and relations between sentences are represented as edges. for example, a similarity graph can be built based on cosine similarities between tf-idf representations of sentences. let g denotes a graph representation matrix of the input documents, where g[i][j] indicates the tf-idf weights between sentence s i and s j . candidate summaries are modeled by its sub-graph structure in the sub-graph encoder, then scored in a ranking layer.