Tractable & Coherent Multi-Document Summarization: Discrete Optimization of Multiple Neural Modeling Streams via Integer Linear Programming (2022)

Abstract: One key challenge in multi-document summarization is the generated summary is often less coherent compared to single document summarization due to the larger heterogeneity of the input source content. In this work, we propose a generic framework to jointly consider coherence and informativeness in multi-document summarization and offers provisions to replace individual components based on the domain of source text. In particular, the framework characterizes coherence through verb transitions and entity mentions and takes advantage of syntactic parse trees and neural modeling for intra-sentential noise pruning. The framework cast the entire problem as an integer linear programming optimization problem with neural and non-neural models as linear components. We evaluate our method in the news and legal domains. The proposed approach consistently performs better than competitive baselines for both objective metrics and human evaluation.

## introduction multi-document summarization (mds) approaches generate the summary of a corpus of documents consisting of a set of related topics. extractive summarization techniques extract a subset of sentences which topically represent the input corpus in a stipulated summary space @xcite . on the other hand, abstractive summarization techniques constructs a semantic representation of the source text and constructs the summary in its own learnt writing style @xcite . despite the attempts for abstractive summarization techniques using neural methods, extractive summarization techniques reserve its space for formulating ready to use summarization approaches. however, a set of (selected) sentences put together without considering the ordering and coherence of the content may not make much sense to the summary reader @xcite . hence, the to make such text generation applications more accessible to users, it is essential to improve qualitative dimensions such as coherence. studies on human written summaries shows that generic information is more relevant for summary content while more specific information is considered to be irrelevant @xcite . in mds, the unit of extraction is sentences. long sentences could often contain irrelevant information that is not essential to the summary and thus regarded as noisy information @xcite . hence mds systems should ideally be equipped for pruning intra-sentential noise. most previous work for extractive mds gave less importance in improving summaries in qualitative dimensions such as coherence and provided an incoherent reading to the summary reader @xcite . a subset of previous work aimed at removing intra-sentential noise by sentence compression @xcite . such works were successful in removing intra-sentential noise, however the problem of incoherent reading can be more severe as the parts of the sentences pruned away can be important s1 s2 s3 nubj obj compelled called admod obj alberta one statues concerned dep which the accurate news and information act amod nmod nmod province each det nmod advcl called alberta press act reference( 1938) were obj conj advcl ensure question nmod det the bank taxation act act appos alberta regulation act publication obj nmod announced october reserving obl nsubj ccomp lieutenant governor bowen num num reference re alberta legislation news and information bills were obj conj advcl ensure the bank taxation act alberta regulation act publication obj nmod news and information bills obj compelled alberta one statues concerned dep amod alberta press act reference( 1938) the accurate news and information act nubj announced reserving nsubj ccomp lieutenant governor bowen assent bills obj nmod s2 s1 s3 input document sentences s1) alberta press act @xcite , also called reference re alberta legislation, concerned 4 alberta statutes, one of which, the accurate news and information act, would have compelled each newspaper in the province, when called upon to do so by a government official, to publish the government's rebuttal of criticism that had appeared in the newspaper. ## related work text summarization can be achieved using extractive @xcite @xcite and abstractive methods @xcite . extractive summarization has the advantage of output fluency due to direct use of human-written texts. however, because of the higher level of granularity exhibited by sentences, these approaches cannot ensure a noise free and coherent summary. a subset of previous extractive summarization approaches utilized parsed sentence structures to execute noise pruning while extracting content for summary @xcite . but these techniques can merely prune noise, and can-not ensure a coherent reading for the summary reader. the attempt to achieve coherence in mutidocument summarization was attempted by some of the extractive summarization system. @xcite . however the attempts to achieve coherence in an mds scenario often cope with intra-sentencial noise for coherence. the following subsections explain each one of these steps in detail. ## discussions a system summary generated for an input corpus in law domain is shown in the table in the next page. the table also contains the sequence of original sentences in the corpus with intra-sentencial noisy information and the corresponding reference summary. clearly our method were successful in removing intra-sentencial noise and organizing summary for a coherent ordering. the neighboring sentences contained similar entities and the order of main verbs ((were, compelled, was)) is the most likely one as per the transition probabilities computed using neural model for verb transition. the summary is also infomative as per the human written abstractive reference summary