C omp A ct: Compressing Retrieved Documents Actively for Question Answering (2024)

Abstract: Retrieval-augmented generation supports language models to strengthen their factual groundings by providing external contexts. However, language models often face challenges when given extensive information, diminishing their effectiveness in solving questions. Context compression tackles this issue by filtering out irrelevant information, but current methods still struggle in realistic scenarios where crucial information cannot be captured with a single-step approach. To overcome this limitation, we introduce CompAct, a novel framework that employs an active strategy to condense extensive documents without losing key information. Our experiments demonstrate that CompAct brings significant improvements in both performance and compression rate on multi-hop question-answering benchmarks. CompAct flexibly operates as a cost-efficient plug-in module with various off-the-shelf retrievers or readers, achieving exceptionally high compression rates (47x).

## introduction retrieval-augmented generation empowers language models to solidify their factual grounding, presenting relevant contexts to answer questions @xcite @xcite . while these approaches extend the knowledge scope of language models beyond their inherent capabilities, they also introduce challenges when it comes to handling long contexts @xcite @xcite . first, models often struggle to find key information within extensive contexts, which diminishes their abilities to reference documents @xcite . also, they often fail to integrate information across multiple † this work was done while the author was at korea university. figure 1 : performance of hotpotqa with different topk documents, using llama3-8b @xcite as the reader. compact shows solid performance improvements that align with those of gold documents. this highlights compact's ability to effectively leverage the benefits of increased top-k, unlike other methods that struggle with noisy context. one way to address these challenges is by compressing contexts into concise forms @xcite . the main goal of compression is to reduce the amount of tokens from the original text without losing core information. however, simply compressing contexts can be suboptimal for qa tasks @xcite , where important details may be filtered out during the compression process @xcite . maintaining redundant information without compression can harm performance, as it may serve as a distractor that can induce models to generate incorrect responses. to handle these limitations, query-focused compression emerges as an effective approach, which aims to preserve information rele-vant to the question @xcite @xcite . however, existing query-focused compressors still struggle to take advantage of information located behind lengthy contexts, leaving out potential opportunities for reader models to improve their answers. in figure 1 , the increase in retrieval recall parallel to the number of documents indicates that useful information is still present even in the lower-ranked results. this demonstrates that these documents should also be covered to fully exploit given information. furthermore, existing methods lack the ability to integrate information across multiple documents, which is required in real-world scenarios @xcite . figure 2 depicts an example: the question is "what 'virtual choir'-noted conductor has created works for the austin-based ensemble conspirare?". to answer this, not only do we need to retrieve information implied within the question, but we should also holistically connect and synthesize information across multiple documents. in other words, the quality of answers hinges on the ability of models to dynamically integrate information across multiple documents, which is yet to be fully explored in the field of compression. to this end, we propose compact, a novel framework that can address these challenges by using an active strategy to compress extensive documents and retain crucial information. our framework has two key components: active compression and early termination. during compression, the model actively encapsulates input documents by jointly analyzing previously compressed contexts with newly provided segments. this ensures that only the most relevant information (here we refer to the compressed text) to the question is preserved at each step, creating a dense and compact context. at each step, the model then decides whether to terminate the compression process. this decision is made based on the relevance and completeness of the information gathered to answer the query. our approach offers two distinct advantages. first, it effectively captures essential context from long documents by incorporating segments with the previously compressed context. this is crucial for complex qa tasks that require in-depth reasoning and synthesis of information. second, it condenses large volumes of documents with a high compression rate, without missing essential contexts. we demonstrate that our framework brings significant improvement in compression rate and end perfor-mance in multi-document qa benchmarks. this highlights the effectiveness of our framework, as it preserves the necessary context without losing critical information. our contributions are as follows: @xcite we propose compact, a novel framework that employs an active strategy for compressing extensive documents. we address the challenge of handling long contexts by dynamically preserving query-related contexts, focusing on integrating information across documents. (2) our framework outperforms existing compressors by a significant margin, achieving a 7.0 (f1) improvement on hotpotqa @xcite with a higher compression rate (47x). also, it surpasses the performance of long-context large language models in multi-document qa benchmark datasets. (3) we demonstrate the compatibility of compact with various retrievers and readers, underscoring its effectiveness as a plug-in module between retrievers and readers. ## compact we introduce compact, a novel compression framework that actively compresses documents until all necessary evidence for answering a question. to condense a large amount of information from documents, we devise an iterative architecture where the compressed contexts are updated at each iteration. in this section, we provide a comprehensive explanation of our framework and detail the data construction process for training our model. ## analysis we investigate ways to facilitate the use of com-pact as a plug-in module that collaborates with diverse retrievers and readers (section 5.1). we conduct an ablation study to assess the impact of components on performance (section 5.2) and examine the cost-effectiveness of our framework using proprietary black-box models (section 5.3). finally, we discuss computational efficiency involved in our framework (section 5.4). ## conclusion we introduce compact, a novel framework that employs an active strategy to compress extensive retrieved documents. our framework effectively captures pivotal information from a large number of documents by dynamically retaining essential contexts and incorporating information. we demonstrate that compact significantly outperforms existing compressors, showing a large performance gap with a higher compression rate in multi-document question-answering benchmarks. furthermore, it serves as a convenient plug-in module that can seamlessly collaborate with various off-the-shelf retrievers and readers while providing cost-saving benefits.