Beyond the Scientific Document: A Citation-Aware Multi-Granular Summarization Approach with Heterogeneous Graphs (2025)

Abstract: Scientific summarization remains a challenging task due to the complex characteristics of internal structure and its external relations to other documents. To address this, our proposed model constructs a heterogeneous graph to represent a document and its relevant external citations. This heterogeneous graph enables the model to exploit information across multiple granularities, ranging from fine-grained textual components to the global document structure, and from internal content to external citation context, which facilitates context-aware representations and effectively reduces redundancy. In addition, we develop an effective encoder based on a multi-granularity graph attention mechanism and the triplet loss objective to enhance representation learning performance. Experimental results across three different scenarios consistently demonstrate that our model outperforms existing approaches. Source code is available at: https://github.com/quocanuetcs/CiteHeteroSum.

## introduction automatic summarization aims to create an abridged version that contains the most critical information from the original text(s) @xcite . as scientific publications are growing at an exponential rate-doubling every nine years-the need for efficient summarization tools is crucial to enhance the productivity of researchers @xcite . unlike abstractive summarization, which generates new sentences, extractive methods select only important information to form a summary. this reduces the risk of hallucination and makes them more reliable for scientific document summarization @xcite . summarizing scientific texts is challenging due to their complex internal structure. early graphbased models demonstrated significant potential for adapting to structured data @xcite . however, these unsupervised models, while effective for structuring, lack the ability to capture deep semantic features without labeled data. the advancement of deep learning models has led to the emergence of sequence-based approaches aimed at capturing cross-sentence relations @xcite @xcite . recently, graph neural networks have been proposed to combine deep learning with graph architecture, offering a high-level representation of text spans for more effective summarization @xcite @xcite . researchers have explored various strategies to optimize graph structures for summarization, from manual section-based input @xcite to automatic hierarchical learning @xcite . however, they primarily focus on internal information while overlooking external relations. a scientific document is rarely an isolated work; it is part of a larger academic conversation, using references to prior research to discuss or reuse information. as a result, citations have recently been recognized as a valuable source for enhancing summary quality @xcite . one efficient approach to leveraging citation information is to incorporate short citing spans from external documents that reference the target document, in order to capture the community's perspective and gather concise, up-to-date insights into the paper's contributions @xcite . figure 1 shows an example of creating a summary from a document and relevant citations. in this paper, we propose citeheterosum, a summarization model that creates a summary from multi-granular information, ranging from finegrained textual components to the global document structure, and from internal content to external citation context. unlike most existing heterogeneous graph-based models that rely solely on the input document, our model incorporates both the document and relevant citations as input. this dualinput is formatted in a heterogeneous graph before encoding through a multi-granularity graph encoder. finally, we estimate the importance of each text unit and select the most significant ones to form the summary. our contributions are: ## methodology this section formalizes the extractive summarization problem and describes the proposed model. figure 2 shows the overall model architecture. problem formulation we choose the elementary discourse unit (edu) as the summary fundamental unit. edu is commonly used in discourse parsing and is well-suited for structured documents. ## experiments this section details the experimental design for evaluating the effectiveness of the proposed model. ## results we present the main evaluation results of all baseline and proposed models, followed by a performance analysis assessing the contribution of the proposed components. ## conclusion in this paper, we have introduced a heterogeneous graph-based summarization model for scientific documents. unlike most existing approaches that rely solely on the input document, our model incorporates both the input document and its citations. we proposed a heterogeneous graph architecture that enables the model to capture information across multiple granularities, ranging from finegrained textual units to the global document structure, and from internal content to external citation context. we also employ a multi-granularity graph encoder to align with the graph architecture and combine triplet loss with summary loss to enhance model performance. evaluation on three different scenarios shows that our model outperforms related models across all evaluation metrics. besides, analysis experiments indicate that citation information and proposed components of our model were shown to contribute to its performance. in the future, we would like to continuously research effective ways to leverage information from scientific documents for summarization.