U ni RAG : A Unified RAG Framework for Knowledge-Intensive Queries with Decomposition, Break-Down Reasoning, and Iterative Rewriting (2025)

Abstract: Knowledge-intensive queries require accurate answers that are explicitly grounded in retrieved evidence. However, existing retrieval-augmented generation (RAG) approaches often struggle with query complexity, suffer from propagated reasoning errors, or rely on incomplete or noisy retrieval, limiting their effectiveness. To address these limitations, we introduce UniRAG, a unified RAG framework that integrates entity-grounded query decomposition, break-down reasoning, and iterative query rewriting. Specifically, UniRAG decomposes queries into semantically coherent sub-queries, explicitly verifies retrieved sub-facts through a dedicated reasoning module, and adaptively refines queries based on identified knowledge gaps, significantly improving answer completeness and reliability. Extensive benchmark evaluations on complex question-answering datasets, including multi-hop HotPotQA and 2WikiMultihopQA, biomedical MedMCQA and MedQA, and fact-verification FEVER and SciFact, demonstrate that UniRAG consistently achieves performance improvements across various state-of-the-art LLMs, such as LLaMA-3.1-8B, GPT-3.5-Turbo, and Gemini-1.5-Flash.

## introduction large language models (llms) have demonstrated remarkable capabilities across a wide range of natural language processing tasks @xcite @xcite ), yet their static training hinders access to up-to-date factual knowledge and causes hallucinations due to difficulty keeping pace with evolving world knowledge. to mitigate these limitations, retrieval-augmented generation (rag) emerged @xcite @xcite ; however, despite providing external grounding, these methods achieved only modest gains on complex queries, such as complex multi-hop and domain-specific, requiring nuanced external knowledge, highlighting the core challenge of effectively bridging the semantic gap between such complex questions and relevant information scattered across large knowledge bases. recent advancements in rag have shifted towards enabling llms to actively guide retrieval for improved relevance and quality. early active retrieval methods can be categorized into query decomposition, which breaks down complex questions into sub-queries @xcite @xcite @xcite , and query rewriting, where the llm refines the query @xcite . despite their promise, these strategies face distinct challenges, in which decomposition often requires substantial supervision and struggles with optimal sub-query granularity @xcite , while rewriting can suffer from llm hallucinations leading to inaccurate queries and often necessitates large training datasets @xcite . more recent efforts in active retrieval have focused on knowledge reasoning, where an llm processes retrieved evidence to generate an answer or determine whether additional retrieval is needed @xcite @xcite . reasoning-based approaches, such as chain-of-thought (cot) prompting and iterative retrieval, enable deeper interaction with retrieved content and typically outperform earlier methods based solely on query decomposition or query rewriting. despite these strengths, current knowledge reasoning methods remain limited by their susceptibility to propagating early errors, sensitivity to prompt variations, and the substantial computational overhead introduced by iterative retrieval-generation cycles @xcite @xcite . these limitations become particularly critical when addressing knowledge-intensive queries, whose accuracy heavily depends on the reliable grounding of evidence. typical examples include (i) multi-hop questions requiring reasoning across multiple documents, (ii) complex-factoid queries demanding simultaneous verification of several independent constraints, and (iii) domain-specific questions involving interpretation or synthesis of externally retrieved domain knowledge. existing knowledge reasoning-based methods frequently struggle to explicitly verify every required aspect or to adequately handle incomplete or noisy retrieval. consequently, their answers often suffer from incompleteness or insufficient evidential support, significantly reducing their reliability in practice. thus, in this paper, we introduce unirag, a novel unified rag framework specifically designed to enhance performance on knowledgeintensive queries. unirag synergistically integrates three complementary active retrieval strategies-query decomposition, break-down reasoning, and iterative query rewriting-to effectively address existing limitations. the main contributions of this work are summarized as follows: query decomposition previous research has extensively investigated the decomposition of complex questions into simpler sub-queries to enhance question answering capabilities, exploring methods ranging from supervised approaches for multi-hop reading comprehension @xcite to unsupervised techniques for question decomposition @xcite and multi-stage frameworks designed to make multi-hop questionanswering more interpretable @xcite . more recently, the advent of llms, it has spurred approaches that utilize prompting strategies, including cot @xcite , least-to-most, and iterative successive prompting, to guide the decomposition process and facilitate complex reasoning @xcite @xcite . however, these methods often rely on extremely large language models requiring significant computational resources @xcite , necessitate substantial amounts of costly or difficult-to-obtain intermediate supervision or synthetic data for complex reasoning types, introduce challenges in selecting the appropriate decomposition granularity @xcite , or depend on external models for factual correction that may lack interpretability or be limited in scope @xcite . query rewriting query rewriting transforms complex questions into more effective inputs for retrieval systems, evolving from early rule-based and statistical methods to recent advancements leveraging llms to generate improved queries through prompting or fine-tuning @xcite . more advanced techniques utilize llms to generate queries from multiple perspectives, such as simulating different user demographics, aiming to enhance retrieval robustness against query variations @xcite . some frameworks employ iterative refinement processes where an llm acts as a rewrite editor, often guided by reinforcement learning signals based on retrieval performance, emphasizing the generation of informative rewrites that provide rich context for the retriever @xcite . despite its promise, llm-based query rewriting faces significant challenges as llms can suffer from hallucination, leading to inaccurate or irrelevant generated queries @xcite . ensuring the generated rewrites are consistently correct, informative, and non-redundant is difficult @xcite . furthermore, these approaches often require substantial computational resources or large datasets of high-quality humanannotated rewrites for effective training and reliable performance @xcite . knowledge reasoning recent rag research has increasingly focused on enabling llms to perform sophisticated external-knowledge-based reasoning, moving beyond simple direct prompting @xcite . early work on cot prompting revealed its emergent reasoning ability is often limited to sufficiently large models, resulting in illogical outputs or failures at smaller scales and contributing to high computational costs, while also highlighting that cot does not guarantee correct reasoning paths, can suffer from factual hallucinations, and is sensitive to prompting details @xcite . approaches like react @xcite , which synergize reasoning and actions by allowing interaction with external tools, introduce structural constraints that can reduce generation flexibility and may lead to reasoning errors or repetitive outputs if retrieval is non-informative or the model struggles to recover @xcite . also, self-ask @xcite which decomposes questions into explicit follow-up steps, can face challenges in obtaining reliable intermediate answers or integrating external information sources effectively, sometimes being limited in scope or interpretability. iterative retrieval-generation methods, like iter-retgen @xcite , aim for improved relevance by guiding retrieval with previous generations, but risk using incorrect augmentation outputs and incurring iterative process overheads. prompting-based rag approaches generally face challenges in effectively balancing parametric versus non-parametric knowledge and suffer from unreliable automated evaluation metrics @xcite . blendfilter @xcite improves rag for complex questions and noisy retrieval by blending multi-source query generation and us-ing the llm for filtering, although it introduces a tunable hyperparameter k, may not eliminate all noise, and the fundamental challenge of capturing relevance for complex queries persists. the proposed unirag framework builds upon query decomposition, query rewriting, and knowledge reasoning, but differs from existing methods reviewed in this section in several important respects. our query decomposition method is entitygrounded, explicitly preserving semantic alignment with the original query and ensuring each sub-query remains focused and contextually precise. our knowledge reasoning employs a dedicated breakdown reasoning module to independently verify each sub-fact, jointly ensuring factual completeness and accuracy. finally, our query rewriting component actively leverages insights from this reasoning process, adaptively refining queries to address identified knowledge gaps and improve retrieval quality. ## methods in this section, we introduce unirag, a unified rag framework designed to improve performance on knowledge-intensive queries. our unirag framework consists of three main phases (figure 1 ): we note that unirag operates entirely through prompting and does not require any additional model training, enabling efficient and flexible deployment. in the following subsections, we describe each of these phases in detail. ## conclusion the unirag framework enhances llm efficacy for knowledge-intensive questions by integrating query decomposition and rewriting with our proposed reasoning strategy. our framework breaks down the complex information needs while improving query searchability in rag systems. unirag further utilizes a reranker and a "let's break it down" prompting strategy to ensure logical synthesis and reduce hallucinations. demonstrating superior performance over the previous sota baselines, unirag shows robust generalization across diverse llms like llama-3.1-8b, gpt-3.5-turbo, and gemini-1.5-flash.