Efficient Dynamic Clustering-Based Document Compression for Retrieval-Augmented-Generation (2025)

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a widely adopted approach for knowledge injection during large language model (LLM) inference in recent years. However, due to their limited ability to exploit fine-grained inter-document relationships, current RAG implementations face challenges in effectively addressing the retrieved noise and redundancy content, which may cause error in the generation results. To address these limitations, we propose an **E**fficient **D**ynamic **C**lustering-based document **C**ompression framework (**EDC ^2 -RAG**) that utilizes latent inter-document relationships while simultaneously removing irrelevant information and redundant content. We validate our approach, built upon GPT-3.5-Turbo and GPT-4o-mini, on widely used knowledge-QA and Hallucination-Detection datasets. Experimental results show that our method achieves consistent performance improvements across various scenarios and experimental settings, demonstrating strong robustness and applicability. Our code and datasets are available at https://github.com/Tsinghua-dhy/EDC-2-RAG .

## introduction in recent years, large language models (llms) have advanced rapidly, excelling in natural language processing (nlp) tasks such as question answering, code generation, and even medical diagnosis @xcite @xcite @xcite . despite their success, llms face two key challenges: expensive knowledge updates due to the large number of learnable parameters, and hallucinations that lead to misleading content @xcite @xcite . these issues impact the availability, reliability and consistency of llms @xcite . retrievalaugmented generation (rag) @xcite @xcite addresses these problems by integrating retrieval with generation, allowing llms to access external knowledge without parameter updates, reducing hallucinations, and improving reliability. however, the implementation of rag methods in real-world settings presents significant challenges. from a structural perspective, the effectiveness of rag frameworks derives from the information augmentation of integrated databases @xcite . in practical applications, the databases are often of limited quality due to the scarcity of high-quality data and the high cost of data cleaning. therefore, the candidate documents faced by retrievers tend to exhibit the following frequentlyencountered quality flaws: these issues can significantly reduce the effectiveness of retrieval and compromise the quality of the final generated output. faced with these practical challenges, it is increasingly significant to build a reliable rag system. however, current rag frameworks predominantly rely on querydocument similarity for retrieval, without explicitly addressing prevalent issues such as noise and redundancy in real-world document corpora. to solve the problems, we propose an efficient dynamic clustering-based compression method for a reliable document retrieval. specifically, we first encode the documents to get a denser content representation, then perform clustering to aggregate semantically similar documents, mitigating content repetition. subsequently, we use prompt-based techniques to guide the llms figure 1 : comparison between our method and prior approaches. unlike vanilla rag, which misses key information, and chunk compression, which is redundant and incomplete, our method clusters and compresses documents to extract concise and accurate answers. in query-specific compression to improve information density and eliminate noise. finally, we concatenate the compressed content into the prompts for response generation. in summary, our method leverages the latent relationships between documents to reduce noise and redundant content. to validate the effectiveness of our approach, we selected two types of widely used datasets: kqa tasks and hallucination detection tasks. systematic experiments conducted on gpt-3.5-turbo demonstrate that our method achieves significant performance improvements across different settings. meanwhile, our method also exhibits strong robustness and generalization potential to other scenarios. these findings indicate that by deeply exploring and utilizing fine-grained relationships among documents, rag methods can reach new performance heights, providing a novel direction for addressing the hallucination problem and knowledge update challenges in llms. the main contributions of our work are: retrieval semantic relation modeling. beyond post-retrieval methods, some studies focus on refining relationships between documents, chunks or entities. recent approaches frame rag as a multi-agent collaboration, where each agent processes a subset of retrieved content. long agent @xcite supports large contexts through chunk-level conflict resolution, while madam-rag @xcite uses agents to address conflicting responses. multi-agent rag is also applied to data integration @xcite , but these methods increase inference costs and latency, limiting real-world applicability. knowledge graphs (kgs) structure document information by compute embedding: v j ← f (d j ) 6: end for 7: select initial cluster root: c.r 1 ← arg max @xmath0, v j ) 8: for all d @xmath1: compute similarity: s j ← sim(v c.r1 , v j ) 10: end for 11: form c 1 with top-τ documents from v sorted by s j 12: remove c 1 members from v phase 2: iterative subgraph formation determine cluster size: form c k with top-size documents from v sorted by s j 9: ## problem definition consider a set of retrieved documents @xmath2, d 2 , . . . , d n }, where each document d i is associated with a query q. these documents are retrieved based on their relevance to q, but their exact utility in answering q is initially unknown. furthermore, there may exist potential overlaps and redundancies among the documents in v , as some documents may share similar or identical information, while others may provide complementary or conflicting details. let @xmath3, where i, @xmath4, 2, . . . , n}. these relationships can be categorized as: additionally, let @xmath5, u 2 , . . . , u n } denote the utility scores of the documents, where u i represents the degree to which d i contributes to answering q. these scores are initially unknown and must be inferred based on the relationships e and the content of the documents. the goal is to effectively utilize the retrieved documents v , their relationships e, and their inferred utilities u to construct a comprehensive and accurate response to the query q. this involves addressing the challenges of redundancy, inconsistency, and varying utility among the documents, while ensuring that the final output maximizes relevance and minimizes noise. ## conclusion in this study, we design an efficient dynamic clustering algorithm and apply compression techniques to exploit fine-grained relationships between documents. our method edc 2 -rag enhances evidence quality by filtering noise and capturing detailed document relationships, achieving consistent performance improvements on three hallucination-detection datasets and three kqa datasets, thus demonstrating the strong robustness and broad applicability of our method. extensive evaluations show that our approach outperforms competitive baselines across multiple metrics and model backbones.