Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition (2025)

Abstract: In a rapidly evolving world where information updates swiftly, knowledge in large language models (LLMs) becomes outdated quickly. Retraining LLMs is not a cost-effective option, making knowledge editing (KE) without modifying parameters particularly necessary. We find that although existing retrieval-augmented generation (RAG)-based KE methods excel at editing simple knowledge, they struggle with KE in multi-hop question answering due to the issue of ”edit skipping”, which refers to skipping the relevant edited fact in inference. In addition to the diversity of natural language expressions of knowledge, edit skipping also arises from the mismatch between the granularity of LLMs in problem-solving and the facts in the edited memory. To address this issue, we propose a novel Iterative Retrieval-Augmented Knowledge Editing method with guided decomposition (IRAKE) through the guidance from single edited facts and entire edited cases. Experimental results demonstrate that IRAKE mitigates the failure of editing caused by edit skipping and outperforms state-of-the-art methods for KE in multi-hop question answering.

## introduction contemporary large language models (llms) have achieved impressive performance comparable to humans in many tasks such as question answering @xcite , writing assistance @xcite , and code generation @xcite . as a world model @xcite , the temporal dimension of data is of great significance to llms in their processes of understanding, memorization, and reasoning. however, current llms struggle to adapt more flexibly and cost-effectively to the vast and ever-changing data generated in a rapidly evolving world. the cost of retraining llms for minor * corresponding author updates of data is prohibitively high, which leads to the need for knowledge editing (ke) without modifying model parameters. a series of methods based on retrieval-augmented generation (rag) @xcite have been developed. they guide llms to generate outputs that meet editing requirements by constructing an edited memory containing factual knowledge of edits and show effectiveness in solving editing tasks for single knowledge pieces. however, a more challenging task for ke is whether the edited model can correctly solve complex questions whose results should change as the impact of ke. this task is referred to as multihop question answering for knowledge editing (mquake) @xcite . mquake requires a sequence of interconnected knowledge facts to reach the final answer. most of the existing methods adopt a framework of decomposing the complex question first and retrieving edited facts that may be involved from edited memory based on the decomposed subquestions @xcite @xcite . the ke methods based on edited memory and rag rely on the textual representation of natural language (as opposed to directly modifying model parameters). however, diversity prevails in the expression of knowledge in natural language. for example, the wife of the president of the united states can be expressed as "first lady of the united states", or "the spouse of the president of the united states". moreover, when there is insufficient context, the perspectives for understanding a question and the methods for solving it are highly flexible and uncertain. for example, for the question "who is the successor of tarja turunen?", it can be understood in terms of the successor in her musical genre, or the successor to her position as the lead singer in the band. under the combined influence of the above circumstances, llms may suffer from the issue of edit skipping, which means skipping the impact of edited knowledge facts when answering some forms of multi-hop questions, thus leading to ineffective editing. the issue of edit skipping is rooted in the mismatch between the granularity of decomposing complex questions and the granularity of relevant edited knowledge facts in the edited memory. as shown in figure 1 (a), when solving the question "where was the current first lady of the united states of america born?", the granularity of the subquestion "who is the first lady of the united states" is coarser than that of the edited fact "the president of the usa is donald trump", making it challenging to directly align the subquestion with the edited fact. if an llm directly answers this subquestion, it may skip the influence of the edited fact. similarly, in figure 1 (b), the decomposition granularity of the subquestion "who wrote danse macabre?" is finer than that of the edited fact "danse macabre was created in france", which also increases the difficulty of matching the subquestion with the edited knowledge fact and may lead llms to skip the relevant edit. to mitigate the edit skipping issues, we propose an iterative retrieval-augmented ke method, irake, based on edit-guided question decomposition, where the guidance lies at both the edited fact level and the edited case level. at the edited fact level, irake selects the edited fact that is most helpful for the decomposition of the current complex question through pre-retrieval and judgment. it uses the corresponding atomic question to guide the question decomposition. at the edited case level, irake searches for the edited case with the most similar question as the current question and uses its solution to construct the dynamic guidance prompt and guide the question decomposition. we also design a state backtracking mechanism to alleviate the impact of failed guidance. extensive experiments demonstrate that irake outperforms state-of-the-art ke methods. our main contributions are outlined as follows: knowledge editing in multi-hop question answering is challenging, as it requires reasoning over and linking multiple facts to derive the answer. previous methods @xcite @xcite @xcite address ke in multi-hop question answering by decomposing complex questions into subquestions and resolving them iteratively to obtain the final answer. however, most of them adopt a decomposition-thenretrieval paradigm, without considering the issue of mismatch between decomposition and target edits, which may fail to retrieve the necessary edits and result in the issue of edit skipping. to address this issue, irake performs a pre-retrieval step before decomposition, retrieving relevant candidate edits and past decomposition records to guide subquestion generation. ## preliminaries notations. following previous works @xcite @xcite , we denote each piece of fact as a triplet (s, r, o), where s, o, and r represent the subject, object, and relationship, respectively. one factual edit e is modeled as updating the object in the triplet from o to o * , denoted by @xmath0, r, @xmath1. meanwhile, the question corresponding to (s, r) in one factual edit e is called the atomic question, denoted by q e @xcite . in the task of ke, there are usually multiple factual edits, which are stored in the edited fact memory: @xmath2, e 1 , . . . , e n }. the relevant edited facts in e are retrieved and used to modify the knowledge in the llm. without loss of generality, the edited fact memory stores both factual edits and their corresponding atomic questions: mquake. given a multi-hop question q, answering q requires sequentially querying and retrieving multiple facts. according to the retrieval order of the queries and their corresponding answers, these factual answers can form a chain of facts: @xmath3, r 1 , o 1 ), . . . , (s n , r n , o n )], where o @xmath4. replacing any fact (s i , r i , o i ) on this chain with an edited fact (s i , r i , o * i ) may affect the entire subsequent fact chain: c * = [(s 1 , r 1 , o 1 ), . . . , (s i , r i , o * i ), . . . , (s * n , r n , o * n )], where o * n is the updated final answer of q. a multi-hop question and its corresponding edited facts constitute an edited case with one question. the task of mquake can be formalized as follows: given an edited fact memory e and an llm m , derive a conditionally edited language model m * . for each multi-hop question affected by e, m * should produce the correct answer (successfully completing the edited case). the reasoning path in this process needs to align with c * , where c * denotes the gold path for question q. ## conclusion in this paper, we introduce the challenge of edit skipping caused by the knowledge granularity mismatch in mquake. we propose an iterative retrieval-augmented ke method based on editguided question decomposition from the perspectives of edited facts and cases. we also design a state backtracking mechanism to alleviate the impact of failed guidance. experimental results demonstrate that our method alleviates the "edit skipping" issue in mquake and outperforms stateof-the-art baselines.