L ong RAG : A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering (2024)

Abstract: Long-Context Question Answering (LCQA), a challenging task, aims to reason over long-context documents to yield accurate answers to questions. Existing long-context Large Language Models (LLMs) for LCQA often struggle with the “lost in the middle” issue. Retrieval-Augmented Generation (RAG) mitigates this issue by providing external factual evidence. However, its chunking strategy disrupts the global long-context information, and its low-quality retrieval in long contexts hinders LLMs from identifying effective factual details due to substantial noise. To this end, we propose LongRAG, a general, dual-perspective, and robust LLM-based RAG system paradigm for LCQA to enhance RAG’s understanding of complex long-context knowledge (i.e., global information and factual details). We design LongRAG as a plug-and-play paradigm, facilitating adaptation to various domains and LLMs. Extensive experiments on three multi-hop datasets demonstrate that LongRAG significantly outperforms long-context LLMs (up by 6.94%), advanced RAG (up by 6.16%), and Vanilla RAG (up by 17.25%). Furthermore, we conduct quantitative ablation studies and multi-dimensional analyses, highlighting the effectiveness of the system’s components and fine-tuning strategies.Data and code are available at [https://github.com/QingFei1/LongRAG](https://github.com/QingFei1/LongRAG).

## introduction large language models (llms), such as gpt @xcite , glm @xcite and llama @xcite , boost the realworld development of multiple scenarios. longcontext question answering (lcqa) @xcite , which has been recently advanced significantly by llms, is a complex task that requires reasoning over a long document or multiple documents to provide accurate answers to questions. recently, several long-context llms have been introduced, such as gemini @xcite and gpt-4-128k, capable of ingesting entire relevant documents and generating answers directly. however, as shown in figure 1 , they frequently encounter the "lost in the middle" issue @xcite , that is, when the relevant context is in the middle of the document (rather than the beginning and end), they are prone to sub-optimal or even incorrect responses. instead, the retrieval-augmented generation (rag) system @xcite ) offers an alternative approach, mitigating this issue by employing a fixed-length chunking strategy @xcite . this strategy ensures the input to the llm is concise and highly relevant to the question. nevertheless, vanilla rag remains insufficient for the lcqa task due to two major limitations. first, the chunking strategy disrupts the contextual structure and background information in long documents (global information). some chunks may contain incomplete information @xcite , thereby causing llms to draw upon irrelevant context or fall back on their internal parameterized knowledge, potentially leading to inaccurate responses. as depicted in figure 1 , vanilla rag only retrieves "griffin" as the performer of "i'll say it" but misses the university from which "griffin" graduated. although the "university" is mentioned in the same paragraph, the system ultimately produces an incorrect response. second, low evidence density in long-context documents can lead to low retrieval quality. considerable noise present in long-context documents impairs llms' capacity to accurately identify key information (factual details), resulting in the retrieval of low-quality chunks and ultimately leading to erroneous answers @xcite . recently, several advanced rag systems have attempted to mitigate the aforementioned issues. specifically, self-rag @xcite employs self-reflection tokens to facilitate the autonomous exploration of global information in a corpus. however, its reliance on the accuracy of reflection tokens may result in the potential deletion of valid retrieval chunks with factual details. crag @xcite evaluates the question relevance of each chunk individually to enhance the identification of factual details. nevertheless, it overlooks the connections between chunks, provoking low-quality evaluation when valid details span multiple chunks, potentially leading to the omission of crucial factual details. in our work, we propose longrag, a general, dual-perspective, and robust rag system paradigm that effectively addresses the above-mentioned issues for lcqa, comprising four plug-and-play components with multiple strategies: a hybrid retriever, an llm-augmented information extractor, a cot-guided filter, and an llm-augmented generator. longrag enhances the rag system's ability to mine global long-context information and identify factual details. specifically, the long-context extractor employs a mapping strategy to orderly extend the semantic space of retrieved chunks into a higher dimensional long-context semantic space, then refining global information and contextual structure among chunks. meanwhile, the cot-guided filter utilizes the chain of thought (cot) @xcite to provide global clues according to the knowledge of all retrieved chunks, instructing llms to carefully review factual details and precisely filter out irrelevant chunks. this improves evidence density and enhances rag's ability to understand complex and lengthy contexts. additionally, we have curated an automated instruction data pipeline for constructing a high-quality dataset for fine-tuning. this fine-tuning strategy significantly enhances the "instruction-following" capabilities of the system's core components. it is also convenient to transfer longrag to other domains by leveraging the pipeline and fine-tuning strategy. extensive performance comparisons and quantitative ablation studies conducted on three multihop datasets from longbench @xcite demonstrate the superiority and effectiveness of longrag. the results suggest that longrag significantly outperformed both long-context llms and advanced rag methods. we also discuss longrag's performance with different fine-tuned llms and confirm its strong robustness and transferability. to sum up, our contributions are summarized as follows: 1) we construct longrag, a general, dual-perspective, and robust rag system paradigm. it significantly surpasses long-context llm (up by 6.94%), mainstream advanced rag (up by 6.16%), and vanilla rag (up by 17.25%). ## conclusion we build an effective and robust rag system paradigm -longrag -which enhances rag's performance in lcqa tasks via a dual information perspective. longrag addresses two main issues faced by existing methods: 1) the incomplete collection of long-context information; and 2) the dif-ficulty in precisely identifying factual information amid substantial noise. we conduct extensive multidimensional experiments, which demonstrate the superiority of longrag and the effectiveness of our proposed components and fine-tuning strategy. longrag significantly outperforms long-context llms, advanced rag methods, and vanilla rag based on various llms. our plug-and-play components successfully use small parameter-size llms, replacing expensive online api resources with lowcost local deployment solutions, while better than gpt-3.5-turbo. additionally, we provide an automated pipeline for fine-tuning instruction data construction, which greatly facilitates the application of our system to other specific-domain data.