Retrieving, Rethinking and Revising: The Chain-of-Verification Can Improve Retrieval Augmented Generation (2024)

Abstract: Recent Retrieval Augmented Generation (RAG) aims to enhance Large Language Models (LLMs) by incorporating extensive knowledge retrieved from external sources. However, such approach encounters some challenges: Firstly, the original queries may not be suitable for precise retrieval, resulting in erroneous contextual knowledge; Secondly, the language model can easily generate inconsistent answer with external references due to their knowledge boundary limitation. To address these issues, we propose the chain-of-verification (CoV-RAG) to enhance the external retrieval correctness and internal generation consistency. Specifically, we integrate the verification module into the RAG, engaging in scoring, judgment, and rewriting. To correct external retrieval errors, CoV-RAG retrieves new knowledge using a revised query. To correct internal generation errors, we unify QA and verification tasks with a Chain-of-Thought (CoT) reasoning during training. Our comprehensive experiments across various LLMs demonstrate the effectiveness and adaptability compared with other strong baselines. Especially, our CoV-RAG can significantly surpass the state-of-the-art baselines using different LLM backbones.

## introduction recent advancements in large language models (llms) @xcite @xcite @xcite have significantly transformed the landscape of natural language understanding technology. these models, characterized by their massive parameter sizes and proficient pre-training on extensive datasets, have demonstrated remarkable success in various natural language generation tasks, especially question answering (qa) (berant figure 1 : description of the hallucinations in rag includes external retrieval and internal generation error. note pink means wrong, and blue means correct. @xcite @xcite @xcite . in practice, even the most advanced llms often face hallucination problems @xcite @xcite , generating answers with factual errors due to persistent inappropriate knowledge. as suggested by @xcite , this issue may arise from polarized optimization objectives and limited knowledge generation abilities. to address the hallucination problem, the retrieval augmented generation (rag) has emerged by introducing retrieval knowledge from external sources @xcite @xcite . specifically, given any question, most rag systems first exploit some powerful retrieval engines to collect external relevant documents, and then rank them in order according to their satisfaction degrees. after that, in our method, the retriever recalls the top-5 most relevant paragraphs as references. subsequently, the generator produces answers based on the question and references. additionally, the verification assesses the accuracy of the references and answer through scoring and judgment, and, if necessary, revises to improve retrieval, refining factuality in multiiteration rag. moreover, cov-rag model also enhances the quality and consistency of single-iteration rag. the rag systems construct corresponding prompts using top satisfied documents, and feed the prompts to llms for final answer generation. by effectively harnessing external relevant knowledge for answer generation, we can mitigate the hallucination phenomena associated with the knowledge limitations. nevertheless, previous rag methods still confront numerous factual issues, which may be attributed to the following two aspects (figure 1 ): to validate cov-rag, we conducted experiments across multiple qa datasets, using traditional accuracy for objective assessment and gpt-4's automatic evaluation to gauge finer-grained dimensions like citation accuracy, truthfulness, and correctness. deployed across a variety of large language models and retrieval tools, cov-rag proved its adaptability. our results demonstrate cov-rag's effectiveness in addressing errors in external contextual knowledge during the retrieval phase and resolving hallucination issues in the gen-eration process, ultimately enhancing the factuality of question answering. in summary, this paper contributes in following aspects: ## methods as depicted in figure 2 , model cov-rag, is composed of two foundational elements: the generator, and the chain-of-verification(cov). by integrating cov, we introduce a novel mechanism for enhancing the factuality and consistency in rag. ## related work numerous studies indicate that most large language models(llms) usually suffer from the hallucinations @xcite @xcite . some studies argue that the hallucinations mainly due to llms overfitting to their training data hallucination @xcite , while other works claim the hallucination usually happens when the llms reach their knowledge boundaries @xcite @xcite . currently, there are various methods proposed to address the hallucination problem, such as hallucination detection @xcite @xcite , data augmentation(dai et al., 2023), and retrieval-augmented generation (rag) @xcite @xcite . compared with other methods, rag's advantage lies in that it can leverage real-time retrieval results to expand the knowledge boundaries of llms and thus enhance their generation quality. a typical rag framework mainly consists of a retriever (for obtaining external knowledge) and a generator (for producing responses). as for the retriever, some studies adopt end-to-end training techniques @xcite and additional ranking modules @xcite to enhance the retriever's performance. other researches improve the knowledge acquisition performance via extra modules, such as rewriting @xcite , and filtering retrieved content @xcite to improve retrieval quality. as for the generator, some researches prompt llms using the chain of thought (cot) strategy @xcite @xcite for reasoning or verifying answers, while other studies directly fine-tune a verification model, such as kalmv @xcite , which introduced a training method for an answer verification model. the aforementioned works mainly focus on optimizing rag modules separately, whereas we-bglm @xcite and self-rag @xcite propose to improved the entire process through joint optimization. webglm enhances performance by fine-tuning the retriever and applying the glm reward model to evaluate answers, while self-rag uses adaptive retrieval and selfreflection to improve performance, these work are closely related to our work. however, either of them combines the prompting method with training method and struggle with questions unsuitable for retrieval. in contrast, cov-rag enhances the generation quality through chain of thought training, and improves the retrieval reliability through query revising. ## conclusion in this paper, we introduce a novel retrieval augmented generation method, cov-rag. it can effectively mitigate hallucinations during internal generation stage and external retrieval stage in the rag. specifically, by integrating the chain of verification prompting into fine-tuned rag generators, we can successfully identify and mitigate generation errors. in addition, the chain of verification prompting can also refine external contextual knowledge through re-retrieving the revised query. we conduct a various experiments to assess the effectiveness of cov-rag over different language model backbones. and experimental results demonstrate that the cov-rag can well detect the generation errors, and significantly improve the generation quality. looking ahead, cov-rag paves the way for further research in refining knowledge augmentation strategies, contributing to the improvement of reliability and accuracy of rag.