Searching for Best Practices in Retrieval-Augmented Generation (2024)

Abstract: Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, particularly in specialized domains. While many RAG approaches have been proposed to enhance large language models through query-dependent retrievals, these approaches still suffer from their complex implementation and prolonged response times. Typically, a RAG workflow involves multiple processing steps, each of which can be executed in various ways. Here, we investigate existing RAG approaches and their potential combinations to identify optimal RAG practices. Through extensive experiments, we suggest several strategies for deploying RAG that balance both performance and efficiency. Moreover, we demonstrate that multimodal retrieval techniques can significantly enhance question-answering capabilities about visual inputs and accelerate the generation of multimodal content using a “retrieval as generation” strategy.

## introduction generative large language models are prone to producing outdated information or fabricating facts, although they were aligned with human preferences by reinforcement learning @xcite or lightweight alternatives @xcite @xcite . retrieval-augmented generation (rag) techniques address these issues by combining the strengths of pretraining and retrieval-based models, thereby providing a robust framework for enhancing model performance @xcite . furthermore, rag enables rapid deployment of applications for specific organizations and domains without necessitating updates to the model parameters, as long as query-related documents are provided. many rag approaches have been proposed to enhance large language models (llms) through query-dependent retrievals @xcite @xcite . a typical rag workflow usually contains multiple intervening processing steps: query classification (determining whether retrieval is necessary for a given input query), retrieval (efficiently obtaining relevant documents for the query), reranking (refining the order of retrieved documents based on their relevance to the query), repacking (organizing the retrieved documents into a structured one for better generation), summarization (extracting key information for response generation from the repacked document and eliminating redundancies) modules. implementing rag also requires decisions on the ways to properly split documents into chunks, the types of embeddings to use for semantically representing these chunks, the choice of vector databases to efficiently store feature representations, and the methods for effectively fine-tuning llms (see figure 1 ). what adds complexity and challenge is the variability in implementing each processing step. for example, in retrieving relevant documents for an input query, various methods can be employed. one approach involves rewriting the query first and using the rewritten queries for retrieval @xcite . alternatively, pseudo-responses to the query can be generated first, and the similarity between these pseudo-responses and the backend documents can be compared for retrieval @xcite . another option is to directly employ embedding models, typically trained in a contrastive manner using positive and negative query-response pairs @xcite . the techniques chosen for each step and their combinations significantly impact both the effectiveness and efficiency of rag systems. to the best of our knowledge, there has been no systematic effort to pursue the optimal implementation of rag, particularly for the entire rag workflow. in this study, we aim to identify the best practices for rag through extensive experimentation. given the infeasibility of testing all possible combinations of these methods, we adopt a three-step approach to identify optimal rag practices. first, we compare representative methods for each rag step (or module) and select up to three of the best-performing methods. next, we evaluate the impact of each method on the overall rag performance by testing one method at a time for an individual step, while keeping the other rag modules unchanged. this allows us to determine the most effective method for each step based on its contribution and interaction with other modules during response generation. once the best method is chosen for a module, it is used in subsequent experiments. finally, we empirically explore a few promising combinations suitable for different application scenarios where efficiency might be prioritized over performance, or vice versa. based on these findings, we suggest several strategies for deploying rag that balance both performance and efficiency. the contributions of this study are three-fold: ## related work ensuring the accuracy of responses generated by large language models (llms) such as chat-gpt @xcite and llama @xcite is essential. however, simply enlarging model size does not fundamentally address the issue of hallucinations @xcite , especially in knowledgeintensive tasks and specialized domains. retrievalaugmented generation (rag) addresses these challenges by retrieving relevant documents from exter-nal knowledge bases, providing accurate, real-time, domain-specific context to llms @xcite . previous works have optimized the rag pipeline through query and retrieval transformations, enhancing retriever performance, and fine-tuning both the retriever and generator. these optimizations improve the interaction between input queries, retrieval mechanisms, and generation processes, ensuring the accuracy and relevance of responses. ## rag workflow in this section, we detail the components of the rag workflow. for each module, we review commonly used approaches and select the default and alternative methods for our final pipeline. section 4 will discuss best practices. figure 1 presents the workflow and methods for each module. detailed experimental setups, including datasets, hyperparameters, and results are provided in appendix a. ## searching for best rag practices in the following section, we investigate the optimal practices for implementing rag. to begin with, we used the default practice identified in section 3 for each module. following the workflow depicted in figure 1 , we sequentially optimized individual modules and selected the most effective option among alternatives. this iterative process continued until we determined the best method for implementing the final summarization module. based on section 3.8, we used the llama2-7b-chat model fine-tuned where each query was augmented by a few random-selected and relevant documents as the generator. we used milvus to build a vector database that includes 10 million text of english wikipedia and 4 million text of medical data. we also investigated the impact of removing the query classification, reranking, and summarization modules to assess their contributions. ## conclusion in this study, we aim to identify optimal practices for implementing retrieval-augmented generation in order to improve the quality and reliability of content produced by large language models. we systematically assessed a range of potential solutions for each module within the rag framework and recommended the most effective approach for each module. furthermore, we introduced a comprehensive evaluation benchmark for rag systems and conducted extensive experiments to determine the best practices among various alternatives. our findings not only contribute to a deeper understanding of retrieval-augmented generation systems but also establish a foundation for future research.