Promoting Topic Coherence and Inter-Document Consorts in Multi-Document Summarization via Simplicial Complex and Sheaf Graph (2023)

Abstract: Multi-document Summarization (MDS) characterizes compressing information from multiple source documents to its succinct summary. An ideal summary should encompass all topics and accurately model cross-document relations expounded upon in the source documents. However, existing systems either impose constraints on the length of tokens during the encoding or falter in capturing the intricate cross-document relationships. These limitations impel the systems to produce summaries that are non-factual and unfaithful, thereby imparting an unfair comprehension of the topic to the readers. To counter these limitations and promote the information equivalence between the source document and generated summary, we propose FIBER, a novel encoder-decoder model that uses pre-trained BART to comprehensively analyze linguistic nuances, simplicial complex layer to apprehend inherent properties that transcend pairwise associations and sheaf graph attention to effectively capture the heterophilic properties. We benchmark FIBER with eleven baselines over four widely-used MDS datasets – Multinews, CQASumm, DUC and Opinosis, and show that FIBER achieves consistent performance improvement across all the evaluation metrics (syntactical, semantical and faithfulness). We corroborate these improvements further through qualitative human evaluation.

## introduction multi-document summarization (mds) aims to formulate a summary that captures the essence and main points of multiple documents on a specific topic. in contrast to single document summarization (sds) that focuses on generating summaries from a single source, mds faces additional challenges such as dealing with a larger search space, redundant documents, and conflicting opinions. these challenges pose difficulties for deep learning models, often resulting in the generation of summaries that are the results of hallucination and they often lack faithfulness @xcite . the development of large language models (llms) such as bert @xcite , bart @xcite , and t5 @xcite has significantly advanced the field of text summarization. however, generating factually accurate and faithful summaries remains a persistent challenge @xcite . recent studies have focused on enhancing the faithfulness and factuality of summarization models, which is broadly categorized into three types -(i) post editing models for correcting generated summaries @xcite , (ii) using multi-task problems like questionanswering @xcite , entailment @xcite etc., and (iii) using external knowledge @xcite @xcite @xcite to support the model during summary generation. in contrast to these methods, we propose a novel approach that promotes topic coherence and interdocument connections, all without the need for additional parameters, post-editing, external knowledge, or auxiliary tasks. additionally, capturing semantic connections and multi-level representation can further aid the model in discerning the redundant and pivotal information in multiple documents. graph neural networks @xcite are constrained in terms of subpar performance in heterophilic settings foot_0 and oversmoothing @xcite when utilized with multi-layer neural networks. to mitigate these concerns, we propose fabric foot_1 in which we introduce simplicial complex @xcite layer and sheaf graph @xcite ) attention for multi-document summarization. simplicial complexes are employed to apprehend the interconnections among diverse elements of the text, encompassing words, phrases, or sentences. by representing these associations as simplices (geometric shapes formed by combining vertices, edges, and higher-dimensional counterparts), a simplicial complex furnishes a structure to examine the connectivity and coherence within the text. on the other hand, sheaf graphs facilitate the assimilation of diverse node types and attributes within the graph structure. each node can symbolize a specific element, such as a word or a phrase, and convey its own attributes or features. by considering these heterogeneous characteristics, sheaf graphs can apprehend and model the relationships among distinct types of nodes. this empowers fabric to comprehend various elements and their relationships, enabling the generation of faithful and accurate summaries. moreover, when dealing with mds, a critical challenge that neural networks often encounter is processing large documents. many recent studies have attempted to concatenate multiple documents into a flat sequence and train sds models on them @xcite . however, this approach fails to consider the inter-document relationship and the presence of redundant long input vectors. even very large language models like long-t5 @xcite and bart-4096 @xcite , which are capable of handling input vectors beyond 2000 tokens, face practical limitations due to the quadratic growth of input memory space. furthermore, such extensive global attention may exhibit lower performance compared to alternative methods. to address this issue, we propose a novel approach, called topic-assisted document segmentation. it involves using a segment of the document as a context vector, compressing it into an oracle summary while covering major topics, and appending it to the next segment to generate the subsequent context vector. by employing this method, summarization models can learn the representation of the document from a compressed form, overcoming the challenges associated with processing large documents. in short, our contributions are as follows. ## proposed methodology we present fabric, a novel multi-encoder-decoder model for multi-document abstractive text summarization. our approach harnesses the power of topic assisted document segments and leverages higher order topological spaces to adeptly extract essential information. figure 1 shows a schematic diagram of fabric. this section explains each component of fabric in detail. ## datasets we benchmark our study on four widely popular datasets. (i) multinews @xcite comprises news articles as source documents and human-written summaries as target summaries. (ii) cqasumm (chowdhury and chakraborty, 2019) dataset consists of question-answer pairs, where the accepted answer is used as the target summary and the remaining answers form the source document. (iii) duc (duc, 2002) includes news articles as source documents, with target summaries manually annotated by editors. (iv) opinosis @xcite combines user reviews from various platforms, where individual opinions serve as the source document, while the target summaries are human-annotated. ## evaluation setup we benchmark fabric on the quantitative metrics -rouge-1 (r1), rouge-2 (r2) and rouge-l (rl) @xcite , to evaluate the lexical overlap, as well as on qualitative metrics -bartscore @xcite to compute the semantic overlap, and factcc @xcite , and summac @xcite for faithfulness assessment. ## experimental results we perform quantitative and qualitative evaluation on the four datasets. we benchmark the datasets over eleven baselines and present quantitative and qualitative results. ## error analysis as indicated in ## fabric vs chatgpt we conducted a comparison between the summaries generated by fabric and chatgpt as shown in ## conclusion in this study, we presented fabric, a encoderdecoder model designed to enhance topic coherence and inter-document relations for multidocument abstractive summarization. the key components of fabric include bart for capturing linguistic aspects and implicial complex layer and sheaf graph attention for capturing inter-document relationships. we compared fabric with eleven baselines on four widely popular mds datasets. the results consistently demonstrated that fabric surpasses existing systems and achieves significant improvements across both quantitative and qualitative measures. these findings were further backed by human evaluation, validating the effectiveness of fabric in generating more accurate and faithful summaries that better represent the content of the source document.