Reflection on Knowledge Graph for Large Language Models Reasoning (2025)

Abstract: Recent research shows that supplementing Large Language Models (LLMs) with knowledge graphs can enhance their performance. However, existing methods often introduce noise in the retrieval and reasoning pipeline, hindering LLMs’ ability to effectively integrate external knowledge for complex multi-hop question answering. To address this, we propose RefKG, a novel framework designed to enhance the reasoning capabilities of LLMs through reflective engagement with knowledge graphs. RefKG autonomously conduct retrieval and reflection on knowledge graphs. It consists of three modules: Query Decoupling, LLM-Driven Knowledge Graph Exploration, and Inference with Knowledge Reconstruction. We also introduce a multi-task tuning strategy that not only integrates external knowledge into LLMs but also trains them to leverage this knowledge for answering questions. This significantly improves their performance on knowledge-intensive tasks. Experiments on fact verification and knowledge graph question answering demonstrate RefKG’s effectiveness.

## introduction large language models (llms) have demonstrated remarkable capabilities in solving various nlp tasks @xcite @xcite , such as machine translation @xcite and information extraction @xcite . however, given the ever-evolving nature of real-world knowledge @xcite @xcite , llms exhibit limitations in domain-specific expertise or in timely updating their knowledge bases. this shortfall often results in hallucinations within their responses, where the generated content deviates from factual accuracy @xcite . to alleviate the issue of hallucinations in llms on knowledge-intensive tasks such as knowledge graph question answering (kgqa) @xcite , a promising strategy involves augmenting llms with external knowledge sources @xcite , like knowledge graphs (kgs) @xcite @xcite . this approach retrieves relevant facts from knowledge bases to help llms generate more accurate responses. however, existing solutions still suffer from several shortcomings. first, due to the scale and complexity of knowledge graphs, retrieval and reasoning processes often introduce irrelevant or noisy information, complicating the model's ability to answer complex queries @xcite , as illustrated in figure 1 . second, recent investigations @xcite have predominantly performed blackbox testing on proprietary models such as chat-gpt. these studies often employ in-context learning techniques @xcite , where external knowledge is incorporated into the prompts to steer the model's response generation. although these training-free methods enable the integration of external knowledge, they do not enhance the interactive capabilities between llms and knowledge graphs, thereby limiting the potential of llms to efficiently acquire and deploy knowledge, especially when supervised signals are available. additionally, black-box models cannot be deployed privately, which significantly limits their flexibility and adaptability. in this paper, we introduce refkg, an innovative framework specifically crafted to enhance the reasoning capabilities of llms through reflective engagement with knowledge graphs. in particular, refkg is structured as a three-step framework: 1) a query decoupling module that decouples a complex query into multiple sub-queries that share a common knowledge background. 2) a llm-driven knowledge graph exploration module that iteratively and reflectively retrieves relevant evidence subgraphs from a knowledge base and refines the knowledge through an expert model. 3) an inference with knowledge reconstruction module that transforms structured knowledge into a natural language format that is more easily understood by the llm, and integrates it with the question to derive the answer. compared to approaches that directly use retrieved results in prompts @xcite , our approach maximizes the reflection capabilities of llms @xcite to critically assess and refine the evidence subgraph. furthermore, we have formulated a knowledge-driven multi-task tuning strategy that provides refkg with foundational expertise in knowledge-intensive reasoning. this is achieved by fine-tuning the model on a specially synthesized corpus, equipping it with the necessary skills for advanced reasoning tasks. together, the three-step process enables our approach to autonomously retrieve, reflect, and utilize knowledge in solving knowledge-intensive tasks. in summary, our main contributions are three-fold: the experimental results affirm that refkg outperforms previous kg-augmented methods across various open-source llms. llms reasoning for kgqa. recent work has focused on using llms for knowledge graph question answering (kgqa), with methods like kaping @xcite and kggpt @xcite prompting llms to generate answers by inserting retrieved triplets into predefined templates. however, these methods neglect the challenge that triplets from knowledge graphs are not always in natural language, complicating llm inference. additionally, they rely on black-box apis, limiting model training and deployment. retrieve-rewrite-answer @xcite addresses this by fine-tuning open-source llms on kg-to-text corpora, converting triplets into more readable text. however, none of these methods filter the extracted triplets, potentially introducing irrelevant information and leading to incorrect results. ## methodology as shown in figure 2 , our proposed refkg is structured with three modules: query decoupling, llm-driven knowledge graph exploration, inference with knowledge reconstruction. besides, we enhance the model's ability to utilize knowledge through knowledge-driven multi-task tuning. ## conclusion in this paper, we proposed the refkg framework, which engages with knowledge graphs in a reflective manner to identify the most likely relational paths and evidence, using this curated evidence to derive answers. to infuse the llm with the abilities to decouple, navigate, refine, reconstruct, and reason over knowledge, we developed a knowledgedriven multi-task tuning approach and built a corresponding training corpus. the experimental results prove its effectiveness on fact verification and knowledge graph question answering. our method can be deployed on any open-source llm, and the experimental results indicate that it achieves excellent performance in fact verification and knowledge graph question answering.