Multi-doc Hybrid Summarization via Salient Representation Learning (2023)

Abstract: Multi-document summarization is gaining more and more attention recently and serves as an invaluable tool to obtain key facts among a large information pool. In this paper, we proposed a multi-document hybrid summarization approach, which simultaneously generates a human-readable summary and extracts corresponding key evidences based on multi-doc inputs. To fulfill that purpose, we crafted a salient representation learning method to induce latent salient features, which are effective for joint evidence extraction and summary generation. In order to train this model, we conducted multi-task learning to optimize a composited loss, constructed over extractive and abstractive sub-components in a hierarchical way. We implemented the system based on a ubiquiotously adopted transformer architecture and conducted experimental studies on multiple datasets across two domains, achieving superior performance over the baselines.

## introduction multi-document summarization (mds) aims to produce a concise summary of non-redundant salient facts based on multiple source documents under the same topic @xcite , which is prevailingly useful in many application domains such as news-wire article summarization @xcite @xcite , scientific literature comparison @xcite , civil rights lawsuits summarization @xcite , and many others @xcite . two main principled approaches are developed accordingly, multi-doc extractive methods @xcite @xcite and multi-doc abstractive methods @xcite @xcite @xcite . ## proposed approach let x be a multi-doc input with n documents of the same topic, @xmath0, and each document be a sequence of sentences, e.g., d @xmath1, . . . , s i j , . . . , where s i j is the j-th sentence of the i-th docu-ment d i . similarly, each sentence is a sequence of words/tokens, e.g., s i @xmath2,j 1 , . . . , w i,j k , . . . , where w i,j k indicates the k-th word/token of sentence s i j . in this work, we are developing a prediction machine m : @xmath3, {s j } q @xmath4, {z j } q @xmath5, of which y is an abstractive summary, {s j } q @xmath6, and z j is the corresponding salient score for s j . based on this hierarchical input, we first flatten the whole word sequences and insert a documentseparator token, <d>, utilized to capture global interactions @xcite , along with a sentence-separator token, 
, used to assist sentence salience extraction. we then feed that flat sequence to our enhanced transformer (figure 2 ), for joint salient representation learning and multitask training/prediction. ## experiments on the news domain in this section, we present empirical investigations for the proposed summarization system on news domain to justify its effectiveness. ## experimental results on the legal domain in this section, we conduct experimental studies on the legal domain. the multi-lexsum dataset foot_6 @xcite ) contains about 9k expertauthored summaries, distributed into three subsets (tiny, short, long), based on source documents of legal cases. similarly, we present certain important stats in ## conclusion in this paper, we proposed to simultaneously perform summary generation and salient evidence extraction in a consistent way, where extracted salient representations are directly employed to enhance abstractive summary paraphrasing. to validate the efficacy, we conducted thorough experimental studies over multiple datasets across different domains. both quantitative and qualitative results reveal the effectiveness of this novel summarization system.