RGAR : Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answering (2025)

Abstract: Medical question answering fundamentally relies on accurate clinical knowledge. The dominant paradigm, Retrieval-Augmented Generation (RAG), acquires expertise conceptual knowledge from large-scale medical corpus to guide general-purpose large language models (LLMs) in generating trustworthy answers. However, existing retrieval approaches often overlook the patient-specific factual knowledge embedded in Electronic Health Records (EHRs), which limits the contextual relevance of retrieved conceptual knowledge and hinders its effectiveness in vital clinical decision-making. This paper introduces RGAR, a recurrence generation-augmented retrieval framework that synergistically retrieves both factual and conceptual knowledge from dual sources (i.e., EHRs and the corpus), allowing mutual refinement through iterative interaction. Across three factual-aware medical QA benchmarks, RGAR establishes new state-of-the-art performance among medical RAG systems. Notably, RGAR enables the Llama-3.1-8B-Instruct model to surpass the considerably larger GPT-3.5 augmented with traditional RAG. Our findings demonstrate the benefit of explicitly mining patient-specific factual knowledge during retrieval, consistently improving generation quality and clinical relevance.

## introduction large language models (llms) have demonstrated remarkable capabilities in general question answering (qa) tasks, achieving impressive performance across diverse scenarios @xcite . however, when facing domain-specific questions that require specialized expertise, from medical diagnosis @xcite to legal charge prediction @xcite , these models face significant challenges, often generating unreliable ehr: clinical presentation includes fever, hypotension… physical examination reveals mucopurulent cervical discharge …laboratory studies show platelet count 14,200/mm3… question: when phenol is applied to a sample of the patient's blood…a blood culture is most likely to show which of the following? conclusions due to both hallucinations @xcite and potentially stale knowledge embedded in their parameters @xcite . retrieval-augmented generation (rag) @xcite has emerged as a promising approach to address these challenges by leveraging extensive, trustworthy knowledge bases to support llm reasoning. the effectiveness of this approach, however, heavily depends on the relevance of retrieved documents. in the medical domain, current rag approaches concatenate all available contextual information from a given example into a single basic query for retrieval, aiming to provide comprehensive context for model reasoning @xcite . while this method has demonstrated substantial improvements on early knowledge-intensive medical qa datasets such as pubmedqa @xcite , its limitations have become increasingly apparent with the emergence of ehr-integrated datasets that better reflect real-world clinical practices @xcite . as shown in figure 1 a), electronic health records (ehrs) typically contain extensive patient data, including diagnostic test results, medical histories, and other longitudinal information @xcite @xcite . however, for any specific medical query, only a small subset of this information is typically relevant @xcite . incorporating all available ehrs into retrieval queries often introduces substantial irrelevant information, which degrades the performance of llm-and rag-based qa systems @xcite . despite ongoing efforts to improve retrieval through query expansion and generation, such as generation-augmented retrieval (gar) @xcite , how to effectively extract and utilize query-relevant factual knowledge from noisy and large-scale ehrs remains an open problem @xcite . as shown in figure 1 b ), inspired by bloom's taxonomy @xcite , we categorize the knowledge required to address realworld medical qa problems into four types: factual knowledge, conceptual knowledge, procedural knowledge, and metacognitive knowledge. the latter two represent higher-order knowledge typically embedded within advanced rag systems, rather than retrieved. specifically, procedural knowledge refers to the processes and strategies required to solve problems, such as problem decomposition and retrieval @xcite , while metacognitive knowledge pertains to an llm's ability to assess whether it has sufficient knowledge or evidence to perform effective reasoning @xcite @xcite . factual knowledge, such as patient-specific information from ehrs, and conceptual knowledge, such as general medical understanding from corpora, together form the complete context inputs required for answering medical questions. processing both types of knowledge requires navigating long contexts filled with irrelevant information. unfortunately, current rag systems do not differentiate between these types of retrieval targets, overlooking the necessity of retrieval from ehrs. to overcome this limitation, we propose rgar, a system designed to simultaneously retrieve factual knowledge and conceptual knowledge through a recurrent query generation and interac-tion mechanism. this approach iteratively refines queries to enhance the relevance of retrieved professional and factual knowledge, thereby improving performance on knowledge-intensive and factualaware medical qa tasks. our key contributions are listed as follows: in recent years, numerous advanced rag systems have emerged. advanced rag systems focus on designing multi-round retrieval structures, including iterative retrieval @xcite , recursive retrieval @xcite , and adaptive retrieval @xcite . a notable work in medical qa is medrag @xcite , which analyzes retrievers, corpora, and llms, offering practical guidelines. follow-up work, i-medrag @xcite , improved performance through multi-round decomposition and iteration, albeit with significant computational costs. with the rise of the agentic paradigm, agentic rag frameworks have emerged, treating rag and cot as executable actions that can be flexibly invoked based on an llm's metacognitive capabilities @xcite . these approaches focus primarily on optimizing the retrieval process, overlooking the retrievability of factual knowledge. in contrast, rgar introduces a recurrent structure, enabling continuous query optimization through dual-end retrieval and extraction from ehrs and professional knowledge corpora, thereby enhancing access to both knowledge types. query optimization. query optimization, or prompt optimization, is crucial for improving ai system performance, especially for retrieval-based tasks. it is widely applied in fields like text-toimage @xcite and code generation @xcite . existing approaches for retrieval tasks largely follow two paths: query expansion and decomposition. query expansion methods range from augmenting queries with generated text @xcite , to using llm-generated reasoning chains as the query itself @xcite , or even replacing retrieved documents entirely with generated evidence @xcite . query decomposition breaks down complex questions into simpler, more targeted sub-queries @xcite . however, these methods face key limitations. first, replacing verifiable documents with generated content compromises the transparency and traceability essential for high-stakes domains like medicine. second, their effectiveness often relies on resourceintensive fine-tuning, limiting scalability. while some work has explored filtering retrieval outputs @xcite , the crucial step of filtering and optimizing the input query remains underexplored. to address this gap, our work proposes a finetuning-free approach to query optimization. we conceptualize it as contextual filtering: we first retrieve pertinent factual knowledge from a trusted, domain-specific source (e.g., ehrs). this knowledge is then used to filter and refine the original query, producing a more precise and factuallygrounded input for the main retrieval from a general corpus. ## methodology in this section, we introduce rgar framework, as illustrated in figure 2 . it begins by prompting a general-purpose llm to generate multiple queries from an initial basic query. these multiple queries are then used to retrieve conceptual knowledge from the corpus ( § 3.2). then retrieved conceptual knowledge is subsequently used to extract factual knowledge from the electronic health records (ehrs) and transform it into retrieval-optimized representations ( § 3.3). the recurrence pipeline continuously updates the basic query and iteratively executes the two aforementioned components. this process optimizes the retrieved results, ultimately improving the quality of responses.( § 3.4). ## experimental setup the core implementation of the rgar framework and the output json files can be accessed via our repository on dbsxfz/rgar. ## conclusion in this work, we propose rgar, a novel rag system that distinguishes two types of retrievable knowledge. through comprehensive evaluation across three factual-aware medical benchmarks, rgar demonstrates substantial improvements over existing methods, emphasizing the significant impact of in-depth factual knowledge extraction and its interaction with conceptual knowledge on enhancing retrieval performance. notably, our rgar enables the llama-3.1-8b-instruct model to outperform the considerably larger, rag-based proprietary gpt-3.5. from a broader perspective, rgar represents a promising approach for enhancing general llms in real-world clinical diagnostic scenarios that demand extensive factual knowledge processing. this framework shows potential for extension to other professional domains where factual awareness is crucial, offering a viable solution for specialized applications requiring precise factual knowledge management.