S tep KE : Stepwise Knowledge Editing for Multi-Hop Question Answering (2025)

Abstract: Knowledge editing aims to update Large Language Models (LLMs) with new information without costly retraining. However, consistently reflecting these updates in complex multi-hop Question Answering (QA), which demands reasoning over interconnected facts, is challenging. Many existing methods overlook the interplay with pre-existing knowledge, leading to inconsistent edit propagation. To overcome this, we introduce StepKE (Stepwise Knowledge Editing for Multi-hop QA), a novel framework for robustly integrating edited and existing knowledge for coherent multi-hop reasoning. StepKE uniquely decomposes multi-hop questions into sequential single-hop sub-questions, retrieving relevant facts (both edited and pre-existing) from an external knowledge graph for each step. It employs context-aware prompting with prior reasoning history and fine-tuning for precise edit propagation. This systematic integration enables effective stepwise reasoning. Experiments show StepKE generates significantly more accurate and consistent responses than baselines, showcasing strong knowledge editing and integration in multi-hop QA.

## introduction large language models (llms) often struggle with outdated or new knowledge despite their advanced language capabilities. knowledge editing, updating llms without costly retraining, is thus a critical research area @xcite @xcite . while progress has been made, effectively ensuring modified knowledge is consistently reflected in complex, multi-hop question answering (qa) remains challenging. multi-hop qa requires reasoning over interconnected knowledge, making seamless integration of new edits with pre-existing knowledge essential @xcite @xcite . existing research on knowledge editing for multi-hop qa often assumes that only edited knowledge is used in responses. this overlooks interactions with the model's vast inherent background knowledge, potentially leading to inconsistent application of learned modifications during multi-step reasoning and resulting in incomplete or contradictory answers @xcite @xcite . figure 1 shows errors in multi-hop qa when edited knowledge is not reflected or when necessary existing knowledge is not adequately incorporated. for instance, methods like rae @xcite , despite considering both knowledge types, can struggle to simultaneously retrieve and integrate multiple items for complex multi-hop questions, limiting accurate and consistent reasoning chains. others like pokemqa @xcite and mello @xcite have explored question decomposition or external knowledge graphs, but robustly connecting edited with existing knowledge in the multi-hop reasoning flow remains a challenge, risking information distortion or omission during propagation. to address these limitations, we propose stepke (stepwise knowledge editing for multihop qa), a novel framework enhancing the integration and consistent application of edited knowledge in multi-hop reasoning. stepke's contribution lies in its unique, systematic approach synergistically combining key components. our framework meticulously decomposes complex multi-hop questions into simpler, single-hop sub-questions. for each, stepke retrieves relevant knowledge-both preexisting and newly edited facts-from an external knowledge graph, ensuring access to current and pertinent information at each step. a key aspect of stepke is its context-aware prompting and an lte-based fine-tuning approach @xcite . context-aware prompting ensures coherent reasoning by integrating prior knowledge and reasoning outcomes into subsequent steps. lte-based fine-tuning then enables accurate reflection of edits while preserving existing knowledge, ensuring precise edit propagation. this synergy of decomposition, knowledge graph augmentation, context-aware prompting, and targeted fine-tuning allows stepke to sequentially apply linked knowledge, maximizing reasoning consistency and accuracy after edits. our experiments show stepke significantly surpasses existing baselines in accuracy and efficiency. ablation studies confirm each component's meaningful contribution, underscoring our structured, stepwise approach for reliable multi-hop qa in dynamic knowledge settings. our main contributions are: our work on stepke builds upon research in knowledge editing for llms and multi-hop qa. 2.1 knowledge editing in llms efficiently updating llm knowledge without full retraining is crucial. approaches range from direct weight modification (e.g., mend (mitchell et al., 2022), rome (meng et al., 2023a)) to methods employing compensatory layers or external memory. lte (jiang et al., 2024), for instance, finetunes a hypernetwork for single-hop factual edits. while promising for atomic edits, ensuring consistent propagation in multi-step reasoning is challenging. stepke uses an lte-based fine-tuning for precise edits within a broader multi-hop consistency framework. 2.2 multi-hop question answering multi-hop qa requires reasoning over multiple pieces of evidence. traditional methods involve passage retrieval for a reader model. recent approaches leverage llms' reasoning via techniques like chain-of-thought (cot) (wei et al., 2022) or question decomposition. keldar (li et al., 2024b), for example, decomposes questions and retrieves facts from knowledge graphs. while sharing question decomposition, stepke distinctively focuses on integrating and propagating edited knowledge cohesively with existing facts at each step. 2.3 knowledge editing for multi-hop qa this intersection presents challenges in harmonizing updates with existing knowledge throughout long reasoning chains. rae (shi et al., 2024) retrieves a relevant subgraph and seeks a fact chain maximizing mutual information. however, its beam search can find local optima, and subgraph extraction may struggle with precise sequential integration for complex queries, potentially including irrelevant or omitting essential facts. stepke employs a structured, sequential approach, decomposing questions into single-hop queries and retrieving targeted knowledge, mitigating subgraph search complexities. pokemqa (gu et al., 2024) and mello (zhong et al., 2024) (provider of the mquake benchmark) also address this. while decomposing questions or using external knowledge, these methods can be limited in robustly connecting new edits with existing knowledge during sequential multi-hop reasoning. stepke retrieves relevant edited and existing facts and explicitly incorporates reasoning history and retrieved knowledge into subsequent prompts via its context-aware mechanism, enabling controlled information propagation. gmello (chen et al., 2024) relies on sparql for knowledge graph access, which can be limiting. stepke's semantic similarity-based retrieval offers more flexibility. a key characteristic of stepke is the systematic integration of knowledge editing with the demands of multi-hop reasoning. unlike prior works focusing on isolated aspects or struggling with edited information flow, stepke orchestrates decomposition, retrieval of existing/edited facts, contextaware prompting, and lte-based editing. this ensures edited knowledge is actively and accurately used throughout the reasoning chain. ablation studies and baseline comparisons confirm this integrated strategy's crucial role in robust multi-hop knowledge editing. ## preliminary in this section, we formalize knowledge editing for multi-hop qa, emphasizing a realistic setting that accounts for the alignment between existing and edited knowledge using a knowledge graph. ## stepwise knowledge editing as illustrated in figure 2 , we propose stepke, a framework designed to effectively incorporate edited knowledge into multi-hop qa while maintaining alignment with existing knowledge. stepke is not merely a linear sequence of steps, but rather a synergistic integration of four key components: 1) question decomposition for structuring the reasoning process, 2) knowledge augmented generation for grounding each reasoning step with relevant facts, 3) fine-tuning for knowledge editing to ensure accurate application of edits, and 4) context-aware prompting to maintain coherence across multiple reasoning hops. the first two components define the primary inference pipeline for each hop, while the latter two are crucial strategies for enabling reliable knowledge editing within this pipeline. ## experiments in this section, we present comprehensive experiments to evaluate our proposed methodology, stepke. we aim to demonstrate its effectiveness in answering multi-hop qa in scenarios where both existing and edited knowledge are provided, a setting that reflects more realistic knowledge editing challenges. ## conclusion in this paper, we introduce stepke, a framework that enhances multi-hop knowledge editing by integrating both existing and updated facts. stepke ensures stepwise reasoning consistency through question decomposition and context-aware prompting, leading to more accurate updates. on mquake benchmarks, stepke outperforms prior methods, effectively addressing knowledge misalignment and reasoning inconsistencies. moreover, stepke achieves higher accuracy with a lower average execution time, making it both efficient and scalable. overall, stepke provides a robust foundation for multi-hop knowledge editing, offering a more structured and computationally efficient way to integrate new knowledge into large language models. limitations while stepke has been extensively evaluated across multiple benchmark datasets and knowledge editing scenarios, several limitations remain. first, stepke primarily targets multi-hop reasoning where single-hop inferences are sequentially connected. however, real-world multi-hop questions often involve parallel reasoning or conditional dependencies, which require further generalization of the question decomposition and alignment techniques. second, stepke's stepwise inference method introduces efficiency trade-offs. while it enhances reasoning consistency, it may also increase inference latency compared to endto-end methods. optimizing the decoding strategy or incorporating beam search techniques may improve efficiency. finally, stepke's adaptability to domain-specific knowledge remains limited. domains like law or medicine require precise fact verification and terminology handling, which stepke does not explicitly incorporate. domain-specific extensions and fact-checking mechanisms could further enhance its applicability. despite these limitations, stepke presents a strong foundation for multi-hop knowledge editing and can be further extended to handle more complex reasoning scenarios.