Chain of Knowledge Graph: Information-Preserving Multi-Document Summarization for Noisy Documents (2025)

Abstract: With the advent of large language models, the complexity of multi-document summarization task has been substantially reduced. The summarization process must effectively handle noisy documents that are irrelevant to the main topic while preserving essential information. Recently, Chain-of-Density (COD) and Chain-of-Event (CoE) have proposed prompts to effectively handle the noisy documents by using entity-centric approaches for the summarization. However, CoD and CoE are prone to information loss during entity extraction due to their tendency to overly filter out entities perceived as less critical but that could still be important. In this paper, we propose a novel instruction prompt termed as Chain of Knowledge Graph (CoKG) for multi-document summarization. Our prompt extracts entities and constructs relationships between entities to form a Knowledge Graph (KG). Next, the prompt enriches these relationships to recognize potentially important entities and assess the strength of each relation. If the acquired KG meets a predefined quality level, the KG is used to summarize the given documents. This process helps alleviate the information loss in multi-document summarization. Experimental results demonstrate that our prompt effectively preserves key entities and is robust to noisy documents.

## introduction the rise of foundation large language models (llms) is redefining the landscape of various natural language processing (nlp) tasks. llmpowered approaches are surpassing conventional supervised learning methods in tasks such as reasoning, sentiment analysis, and others, often with just a single prompt. with this advancement, text summarization has entered a new phase @xcite . instead of relying on 'golden' answers, texts can now be summarized with greater flexibility using llms. furthermore, instructions enable precise tailoring of summary length and style. examining promptbased text summarization methods, we find that most of the recently proposed methods fall under the chain-of-thought (cot) category @xcite . among the cot approaches, cod is a representative method @xcite . cod starts with a sparse entity set and refines it iteratively to obtain a denser entity set while balancing detail and abstraction for summarization. in real life, multi-document summarization (mds) is needed across various fields for diverse objectives. however, when collecting various documents from the web, such as news articles and community posts related to a specific event, it is common to come across documents that are irrelevant to the main topic. we define these documents as noisy documents. to effectively deal with mds, coe is recently proposed @xcite . coe consists of four sequential steps: event extraction, event abstraction, statistical event analysis and final document summarization. although coe and cod prompts are designed to be robust to noisy documents, their prompts can lead to significant information loss in mds 3. in coe, relying solely on frequency to determine entity importance can result in the omission of contextually significant entities. meanwhile, cod can suffer from lack of entities due to stringent conditions to be entities. for noise-robust and information-preserving summarization, we propose a novel instruction prompt termed as cokg for mds. cokg aims to extract enriched entities to minimize information loss. since our approach is entity-centric summarization, it is also robust to noisy documents. our contributions are as follows. despite the effectiveness of supervised methods, scalability issue is still challenging. with the rise of llms, since llms can generate a summary with a few lines of instruction, prompt-based summarization has gained attention to address the issue @xcite @xcite . mds is similar to general text summarization but differs in that it integrates and deals with diverse perspectives on a single topic. for example, coe minimizes irrelevant information and focuses on key events for a concise summary @xcite . ## conclusion mds complexity is eased by advent of llm-based approaches. however, the previous approaches often suffer from information loss. in addition, since generic prompt tends to show inferior performance to severely noisy document set, entity-centric approach is necessary. thus, we propose cokg that is robust to noisy documents and has information-preserving property. cokg maximally extracts topic-related entities to minimize information loss. in noise test, we observe that our approach is resilient to noise. in addition, the results from the multi-news and peersum benchmarks demonstrate that cokg effectively preserves information and that its summaries closely align with human-generated ones. these findings suggest cokg produces a reliable summary for multi-document.