Memory-augmented Query Reconstruction for LLM -based Knowledge Graph Reasoning (2025)

Abstract: Large language models (LLMs) have achieved remarkable performance on knowledge graph question answering (KGQA) tasks by planning and interacting with knowledge graphs. However, existing methods often confuse tool utilization with knowledge reasoning, harming readability of model outputs and giving rise to hallucinatory tool invocations, which hinder the advancement of KGQA. To address this issue, we propose Memory-augmented Query Reconstruction for LLM-based Knowledge Graph Reasoning (MemQ) to decouple LLM from tool invocation tasks using LLM-built query memory. By establishing a memory module with explicit descriptions of query statements, the proposed MemQ facilitates the KGQA process with natural language reasoning and memory-augmented query reconstruction. Meanwhile, we design an effective and readable reasoning to enhance the LLM’s reasoning capability in KGQA. Experimental results that MemQ achieves state-of-the-art performance on widely used benchmarks WebQSP and CWQ.

## introduction large language models (llms) have demonstrated impressive reasoning capabilities in knowledge graph question answering (kgqa) task @xcite @xcite . using planning and interactive strategies, current llm-based kgqa methods conduct the reasoning process on the knowledge graph based on sparql tools and achieve remarkable performance across benchmarks @xcite @xcite . typically, part of these studies directly strengthen the reasoning ability of llms to plan tool-based paths and retrieve information from the knowledge graph @xcite . the others employ llms to construct knowledge reasoning agents that execute the reasoning process on the knowledge graph through continuous tool-based decision-making based on environmental observations @xcite . these methods have achieved impressive results in the kgqa task. however, existing methods often conflate tool invocations with the reasoning process, compromising readability and leading to tool invocation hallucinations. on one hand, current kgqa methods typically represent reasoning steps using sparql relations, which tend to be abstract and contain redundant information, significantly reducing readability. on the other hand, this complex representation places an unnecessary burden on the reasoning process and may introduce additional hallucination issues. for example, when answering the question "who is justin bieber's brother?", existing methods fail to generate readable reasoning steps (figure 1 , top right). instead, they rely on abstract sparql paths to represent the reasoning process (figure 1 , top left). in such cases, both poor reasoning and hallucinatory tool usage by llms can contribute to unreliable results, making it difficult for researchers to analyze the results (figure 1 , bottom left). to address the issue, we propose memoryaugmented query reconstruction for llm-based knowledge graph reasoning (memq) to decouple llm from the tool invocation task using an llm-built query memory. to establish the query memory, we employ a rule-based strategy to decompose queries into statements, which are then described using the icl capability of general-purpose llm, facilitating an independent reasoning process. we design an effective reasoning strategy based on natural language, enhancing readability and generating explicit reasoning steps. based on the developed steps, memq retrieves memory based on semantic similarity and reconstructs the final query to interact with the knowledge graph. by establishing this query memory, the memq approach enables the model to disengage from tool invocation and focus on generating readable knowledge reasoning steps. our main contributions are: knowledge graph question answering. early kgqa approaches focused on using networks like key-value memory and graph neural networks to represent inference paths @xcite @xcite , while other approaches teach models to build database queries such as sparql for direct answer retrieval @xcite . with the rise of large language models (llms), methods utilize llm's graph reasoning capability to enhance the reliability of reasoning process @xcite @xcite . certain approaches are developed to leverage scaled models to directly interact with knowledge graphs or for generating labels that assist smaller models in distilling reasoning abilities @xcite @xcite . other efforts focus on constructing decision datasets based on annotated data to perform a supervised fine-tuning process, which enhance llm's understanding of the knowledge reasoning process and their ability to interact with knowledge graphs @xcite . since llm-generated outputs are generally susceptible to hallucinatory behavior, some research has shifted to employing discriminative strategies instead of generative ones to reduce unfounded reasoning processes @xcite . however, the issue of confusing the tool invocation process with the knowledge reasoning process remains unresolved. the existing method often conducts reasoning based on sparqlformed edges like 'type.domain.property' or selfdesigned toolboxes, which diminishes the model's focus on the reasoning process and suffers from hallucinatory tool invocation behaviors. in this paper, we propose a memory-augmented kgqa reasoning method that effectively decouples the reasoning process from tool invocation. figure 2 : the overall framework of memq. during the memory construction stage, we describe the question with its query history using the llms to get the reasoning steps. in the inference stage, we reconstruct the query using the recalled query sentences based on the reasoning results. ## approach in this section, we introduce the overall approach of memq; the overall framework is illustrated in figure 2 . we propose to facilitate the kgqa process using three tasks including memory construction, knowledge reasoning and query reconstruction. we first formulate the task and then showcase our method. ## experiment in this section, we first introduce the datasets and evaluation methods used by memq. after presenting the main experimental results, we will follow up with reports on several analytical experiments to examine the characteristics of the memq method compared to previous methods from various perspectives. ## conclusion in this paper, we propose decoupling llm from tool invocation tasks using an llm-built query memory to alleviate hallucinatory tool invocation issues. by facilitating the kgqa process using three tasks, we established a memory module to augment the query reconstruction process in the kgqa task. based on the framework, we design an effective and readable reasoning strategy to enhance the llm's reasoning capability, which also alleviates hallucinatory behaviors in existing methods. experimental results show that our proposed memory-enhanced framework has achieved the state-of-the-art (sota) performance on two commonly used benchmarks.