FRAG : A Flexible Modular Framework for Retrieval-Augmented Generation based on Knowledge Graphs (2025)

Abstract: To mitigate the hallucination and knowledge deficiency in large language models (LLMs), Knowledge Graph (KG)-based Retrieval-Augmented Generation (RAG) has shown promising potential by utilizing KGs as an external resource to enhance LLM reasoning.However, existing KG-RAG approaches struggle with a trade-off between flexibility and retrieval quality. Modular methods prioritize flexibility by avoiding the use of KG-fine-tuned models during retrieval, leading to fixed retrieval strategies and suboptimal retrieval quality. Conversely, coupled methods embed KG information within models to improve retrieval quality but at the expense of flexibility.In this paper, we propose a novel flexible modular KG-RAG framework, termed FRAG, which synergizes the advantages of both approaches. FRAG estimates the hop range of reasoning paths based solely on the query and classifies it as either simple or complex.To match the complexity of the query, tailored pipelines are applied to ensure efficient and accurate reasoning path retrieval, thus fostering the final reasoning process. By using the query text instead of the KG to infer the structural information of reasoning paths and employing adaptable retrieval strategies, FRAG improves retrieval quality while maintaining flexibility. Moreover, FRAG does not require extra LLM fine-tuning or calls, significantly boosting efficiency and conserving resources. Extensive experiments show that FRAG achieves state-of-the-art performance with high efficiency and low resource consumption. The code for our method is publicly available at https://github.com/gzy02/FRAG.

## introduction large language models (llms) excel in various nlp tasks but are prone to hallucinations (2023) and errors (2023a) when answering questions that require knowledge beyond training data. these limitations undermine the trustworthiness of llms and raise security concerns ( 2024 ). to mitigate these issues, retrieval-augmented generation (rag) (2020; 2020; 2023) has been developed, which dynamically retrieves information from external sources during the retrieval phase, while the generation phase leverages this retrieved data to improve generation quality. @xcite @xcite relying on unstructured data often struggle with capturing relevant knowledge and may introduce noise, hindering effective reasoning. in response to these challenges, knowledge graphs (kgs) are increasingly being integrated into rag (i.e., kg-rag) as external knowledge sources @xcite . kgs offer editable and explicit knowledge in a structured format, clarifying the context and multi-level interrelations among entities. kg-rag retrieves "reasoning paths" that are relevant to the query, providing concise and structured contextual information to enhance the reasoning ability of @xcite @xcite @xcite . there are two lines of research on kg-rag frameworks, as shown in figure 1 , depending on their ways of integration with llms during the retrieval phase, resulting in either modular or coupled kg-rag. the former separates the retrieval process from llms, prioritizing isolation, flexi-bility, and scalability, while the latter tightly integrates kgs, improving generation quality but at the expense of increased complexity and reduced flexibility and scalability. modular kg-rag frameworks (2023; 2023b; 2023; 2024), aligning with conventional rag principles, are developed to keep independence between the retrieval phase and kg-fine-tuned llms or specific models, while meeting essential criteria of isolation, flexibility, and scalability. this isolation prevents interference with llms' internal reasoning. flexibility is achieved through seamless plug-and-play integration with external kg sources, and scalability allows these frameworks to effectively handle large kgs and complex queries. this is accomplished by using traditional algorithms for reasoning path search (2024; 2024; 2024) and ranking (2024; 2024), without requiring any additional fine-tuning to incorporate kgs information prior to reasoning. despite the advantages, modular kg-rag frameworks face challenges, due to their lack of prior knowledge about kgs and correct reasoning paths. this limitation impedes effective adjustment of searching and ranking parameters, leading to inferior generation quality @xcite . for example, research like @xcite highlights that fixed search parameters can result in redundancy in simple tasks with short reasoning paths or the omission of critical details in more complex tasks with longer paths, ultimately weakening the reasoning capabilities of llms. in contrast, coupled kg-rag frameworks, while offering enhanced retrieval of reasoning paths, come with significant overhead due to the extensive fine-tuning of llms with embedded kg information (2024b; 2024; 2024a). for instance, rog (2024b) not only use kgs as an external knowledge base but also fine-tune llms or train the specific models with kgs information, enabling the generation of "relation paths" grounded in kgs as retrieval templates tailored to the query context. while this method enhances retrieval and generation quality, it compromises key advantages offered by modular kg-rag frameworks, such as the isolation of the retrieval process, flexibility in integrating various external sources, and scalability to kgs of different sizes (2023; 2023), not to mention the substantial expense involved in fine-tuning llms. this leads us to our research goal: can we develop a solution that combines the strengths of both frameworks? specifically, can we implement lightweight pre-computation to obtain sufficient information to guide the settings of ranking and searching parameters during the retrieval phase, thereby enhancing generation quality while maintaining isolation, flexibility, and scalability-without the extensive overhead associated with fine-tuning llms for specific kgs? in this work, we propose a novel and flexible modular kg-rag framework, called the frag. in a nutshell, frag dynamically adapts the retrieval process based on the complexity of the query context to improve reasoning accuracy, without requiring kgs information. as shown in figure 2 , we analyzed the information sources associated with the correct reasoning path p for a query context q on the kg. a path p comprises two types of information: semantic (i.e., entities and relations) and structural (i.e., number of hops). semantic information mainly originates from kgs and is hard to perceive and utilize in advance. however, structural information is related to both kgs and the q (2024a). generally, the more complex the query context q, the greater the number of hops in the path p (i.e., indicating a more difficult reasoning task) (2019; 2020). thus, within a tolerable margin of error, we can predict the number of hops in p based solely on the query context q. the predicted number of hops can then serve as a key factor in enhancing the non-specific retrieval process. starting from the above insights, the frag framework is primarily featured by two key modules: "reasoning-aware" and "flexible-retrieval". the reasoning-aware module considers three key aspects of structural information prediction. first, to reduce the impact of inherent prediction errors, it simplifies the prediction task by estimating the coarse-grained hop range, categorizing the reasoning complexity from the query context as either simple or complex based on a hop count threshold. second, it collects kgs and queries across vari-ous domains, extracting semantic and statistical features from the query context to train a flexible and generalizable cross-domain classifier. finally, an optimization strategy utilizing feedback from llms is employed to enhance classifier performance on specific kgs as needed. the flexible retrieval module refines the kg-rag retrieval process into a "preprocessing, retrieval, and postprocessing" pipeline, facilitating the tailored customization of its three components for both simple and complex reasoning tasks. each component is built on traditional algorithms and models, ensuring the generality of our framework. for simple reasoning tasks, which typically involve shorter reasoning paths, we employ breadthfirst search (bfs) and ranking as the core retrieval strategies, enabling efficient and accurate retrieval. in contrast, for complex reasoning tasks characterized by longer reasoning paths, we advocate for the use of shortest path retrieval and ranking to minimize computational overhead and reduce noise, thereby improving retrieval effectiveness. the remainder of this paper is organized as follows: in section 2, we review rag and kg-rag frameworks. section 3 covers basic kg-rag concepts. section 4 details the frag framework design. section 5 evaluates frag on benchmark datasets against sota methods. finally, section 6 summarizes our work and future directions. ## related work retrieval-augmented generation (rag). rag (2021) enhances llms by integrating retrieved knowledge during contextual learning, mitigating knowledge gaps and hallucination issues. the naiverag frameworks (2023; 2023c; 2023a) retrieve top-k relevant documents and incorporates them into the prompt for more accurate responses. later advancements (e.g.modular rag (2023d; 2023b), advanced rag (2022b; 2024)) improved retrieval accuracy with additional modules. however, document-based rag introduces noise and excessive context, impacting reasoning performance (2022a). recent studies (2024) focus on altering the storage format of external knowledge. recently, graphrag (2024) unifies various knowledge into (knowledge) graph format, transforming retrieval into a fine-grained knowledge path search, enhancing key information extraction. rag based on knowledge graphs (kg-rag). kgs, known for dynamic, explicit, and structured knowledge representation, are increasingly used as knowledge bases for rag. kg-rag (2024; 2024) retrieves the top-k reasoning paths relevant to the query, providing concise and accurate contextual information for llms reasoning. as outlined in section 1, kg-rag frameworks are categorized as modular or coupled, based on whether kgs information is fine-tuned into the llms. the latest modular approach, like tog (2024), enhances retrieval accuracy by replacing traditional ranking models with llms. however, tog still struggle due to a lack of prior kg knowledge and the need for frequent llm calls. for coupled framework, the most recent method, rog (2024b), fine-tunes llms with kg information, allowing them to generate "relation paths" as query templates that directly retrieve the correct reasoning paths from kgs. however, rog compromises interpretability, efficiency in knowledge updates, and the generality of the rag process across different domain kgs. general process of kg-rag. following existing works @xcite , kg-rag involves two main stages: retrieval and generation. given a query q, the first stage constructs a set of candidate "reasoning paths" by matching q with entities and relationships in the kgs. this is done by searching for relevant triples (s, r, e) from kg (g): retrieve(q, g) → {p i }. the retrieved reasoning paths set {p i } are then ranked by their relevance to q. a reasoning path p i can be formally defined as: simple and complex reasoning in kg-rag. following prior works on reasoning tasks in @xcite @xcite , kg-rag reasoning tasks can be categorized into simple and complex reasoning, based on the minimum number of hops required to find the correct answer from the kg. given a threshold δ foot_0 , if the minimum hop count h min of the reasoning path is less than or equal to δ, the problem is categorized as simple; otherwise, it is classified as complex: type(q) = simple if h min (q, g) ≤ δ else complex. for queries with multiple correct answers, the minimum hops across all correct reasoning paths are considered. this formal classification helps in distinguishing the complexity of reasoning tasks within kg-rag frameworks. ## conclusion in this paper, we propose frag, a modular kg-rag framework that addresses the challenge of enhancing reasoning accuracy in llms without compromising flexibility. by adapting the retrieval process based on the complexity of the query context, frag leverages structural information predictions to optimize retrieval strategies. thus, frag comprises two key modules: the reasoning-aware module predicts the complexity of the reasoning tasks (simple or complex) based solely on the query context, while the flexible-retrieval module customizes the retrieval process according to task complexity to enhance retrieval efficiency and effectiveness. extensive experiments show that frag improves retrieval quality while maintaining flexibility, outperforming existing kg-rag approaches. in the future, we aim to further enhance frag's adaptability to more diverse knowledge graph structures and complex reasoning scenarios.