G raph RAG : Leveraging Graph-Based Efficiency to Minimize Hallucinations in LLM -Driven RAG for Finance Data (2025)

Abstract: This study explores the integration of graph-based methods into Retrieval-Augmented Generation (RAG) systems to enhance efficiency, reduce hallucinations, and improve explainability, with a particular focus on financial and regulatory document retrieval. We propose two strategies—FactRAG and HybridRAG—which leverage knowledge graphs to improve RAG performance. Experiments conducted using Finance Bench, a benchmark for AI in finance, demonstrate that these approaches achieve a 6% reduction in hallucinations and an 80% decrease in token usage compared to conventional RAG methods. Furthermore, we evaluate HybridRAG by comparing the Digital Operational Resilience Act (DORA) from the European Union with the Federal Financial Institutions Examination Council (FFIEC) guidelines from the United States. The results reveal a significant improvement in computational efficiency, reducing contradiction detection complexity from O(n^2) to O(k \cdot n) —where n is the number of chunks—and a remarkable 734-fold decrease in token consumption. Graph-based retrieval methods can improve the efficiency and cost-effectiveness of large language model (LLM) applications, though their performance and token usage depend on the dataset, knowledge graph design, and retrieval task.

## introduction generative artificial intelligence (genai), exemplified by large language models (llms) such as openai's gpt series @xcite ), meta's llama models @xcite , and @xcite , has gained prominence in various fields, including healthcare, finance, and education. these models, while highly capable of producing coherent and contextually relevant responses, face challenges in generating factually accurate content-a phenomenon referred to as hallucination @xcite . hallucination arises from llms' reliance on potentially outdated or domain-general training data, leading to inaccuracies in real-world applications where precision is critical @xcite . to address the issue of hallucinations, retrieval-augmented generation (rag) has emerged as a promising approach. introduced by @xcite , rag combines a retriever that identifies relevant documents and a generator that creates coherent responses from this information. by combining llms with external knowledge bases, rag systems can enhance response accuracy and relevance by dynamically incorporating up-to-date and verifiable information into generated outputs @xcite . the original approach suggested dividing documents into 100-word disjoint chunks, but this can disrupt semantics and lead to hallucinations, as noted by @xcite . to mitigate these issues, enhancements like sliding window chunking, sentence-level splitting with surrounding context, and incorporating metadata such as document titles have been proposed to improve the quality and relevance of the generated outputs @xcite . while rag often improves the relevance of language model outputs, it faces notable limitations in real-world applications: these limitations highlight the need for advanced approaches, such as graphrag (figure 1 ), to incorporate structured relationships and provide richer, more contextually accurate information, while being more efficient in terms of token usage. to overcome these limitations, we explore several distinct approaches to enhancing rag systems with graphs, including: ## related work knowledge graphs (kgs) play a crucial role in enhancing the interpretability and factual accuracy of large language models (llms) by structuring information as entities and relationships @xcite . the integration of graph structures within retrieval-augmented generation (rag) frameworks has shown significant improvements in model performance. zhao et al. (2023) demonstrate that graph-based retrieval-augmented generation enhances contextual accuracy by allowing systems to retrieve relevant entities and relationships from kgs. similarly, yasunaga et al. (2022) highlights the benefits of graph-based retrieval-augmented language models for fact verification and knowledge enrichment, ensuring that generated outputs are relevant and accurate. liu et al. (2022) introduces the concept of graph retrieval augmentation, which enhances contextual and semantic understanding in llms, resulting in more coherent and pertinent responses. furthermore, the work of guu et al. @xcite illustrates how training in a language model with augmented retrieval with knowledge graphs can improve the accuracy and depth of the answer by using kg for the retrieval and structured data. graph rag stands out by retrieving graph elements from a pre-constructed knowledge graph, thereby enriching llm-generated responses with structured knowledge @xcite . this structure allows graph rag to capture semantic nuances, maintain contextual coherence, and reduce verbosity, making it particularly effective for applications in question-answering, recommendation systems, and complex information retrieval tasks relying on structured knowledge. ## problem statement large language models (llms) augmented by retrieval-augmented generation (rag) systems have advanced the ability to generate contextually relevant responses. however, despite rag's potential to reduce inaccuracies by integrating external knowledge, hallucinations -defined as the generation of factually incorrect or fabricated information -remain a significant issue. this paper explores the relevance of a graph-based approach to reduce hallucinations and optimize token consumption in language models. ## proposal: graph-based rag system we propose to enhance the classical rag system with knowledge from graph databases instead of raw texts. to this aim, we build a traditional text rag system to serve as a baseline, as well as two graph-flavored variants, which we call facts and kg-rag in the following. reproducibility code for both text and facts rag are available on github foot_0 . ## experiments and evaluation we conducted experiments to address the following two research questions on datasets related to the financial domain. ## conclusion this study demonstrates three significant contributions of graph-based approaches in enhancing classical rag systems: