HIRAG : Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (2025)

Abstract: Retrieval-augmented generation (RAG) has become a fundamental paradigm for addressing the challenges faced by large language models in handling real-time information and domain-specific problems. Traditional RAG systems primarily rely on the in-context learning (ICL) capabilities of the large language model itself. Still, in-depth research on the specific capabilities needed by the RAG generation model is lacking, leading to challenges with inconsistent document quality and retrieval system imperfections. Even the limited studies that fine-tune RAG generative models often lack a granular focus on RAG tasks or a deeper utilization of chain-of-thought processes. To address this, we propose that RAG models should possess three progressively hierarchical abilities (1) Filtering: the ability to select relevant information; (2) Combination: the ability to combine semantic information across paragraphs; and (3) RAG-specific reasoning: the ability to further process external knowledge using internal knowledge. Thus, we introduce our new RAG instruction fine-tuning method, Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (HIRAG) incorporates a “think before answering” strategy. This method enhances the model’s open-book examination capability by utilizing multi-level progressive chain-of-thought. Experiments show that the HIRAG training strategy significantly improves the model’s performance on datasets such as RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA.

## introduction retrieval augmentation generation (hereafter referred to as rag) helps large language models (llms) @xcite reduce hallucinations @xcite and access real-time data 1 *equal contribution. by incorporating an information retrieval component. while llms often use in-context learning @xcite for generation, practical issues such as low-quality or poorly ranked retrieved documents can hinder rag's effectiveness. these challenges emphasize the need for instruction-tuning tailored to rag tasks. fine-tuning generative models specifically for rag improves their ability to integrate retrieved information @xcite ) @xcite , resulting in more accurate and contextually relevant responses compared to generalpurpose models. raft @xcite enhances model performance in domain-specific rag tasks by introducing distractor documents during training. ev-idencerag @xcite improves large language models in evidence-based question answering by incorporating an indexing task, enhancing their ability to accurately cite and reflect source information. rankrag @xcite employs a two-stage training process to simultaneously optimize the context ranking and answer generation capabilities of large language models (llms) in rag tasks. despite significant research efforts on ragspecific generative models, several issues remain. thus, we introduce a new rag instruction tuning method: hierarchical-thought instruction-tuning retrieval-augmented generation (hirag) adapting to complex rag scenarios and propose that when fine-tuning rag generation models, we focus on three progressively hierarchical abilities shown in figure 1 : filtering: the ability that llm filters out noise and selects the direct information. combination: the ability of llms to merge, integrate, and summarize multiple pieces of useful information. rag-specific reasoning: the capability refers to the ability to answer a question by making implicit or explicit inferences based on the information in the documents when the relevant information is not directly provided. to better achieve these three capabilities, a "think before answering" approach based on progressively hierarchical thought has been introduced. the contributions of this work are summarized as follows: ## hirag in this section, we introduce our rag-focused instruction tuning methods: hirag (hierarchical-thought instruction-tuning retrieval-augmented generation), which incorporates a "think before answering" strategy to enhance progressively rag abilities: filtering, combination, and rag-pecific reasoning. ## conclusion in this work, we present hirag, a novel instruction tuning method specifically designed for rag (retrieval-augmented generation) models. this method provides a more granular enhancement of rag's three core capabilities: filtering, combination, and rag-specific reasoning. this is accomplished by employing a hierarchical "chain of thought" (cot) approach to improve the model's performance in open-book examinations. this approach demonstrates that hirag exhibits strong performance across a variety of document-based question-answering benchmarks, achieving outcomes that are not only competitive with but in some instances, exceed those of much larger models. in the future, we will focus more on the reasoning aspect of the chain of thought. using stackbased thought processes or reinforcement learning, we aim to enhance the diversity and coherence of reasoning pathways to achieve better performance in rag scenarios.