Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models (2024)

Abstract: Recent work has showcased the powerful capability of large language models (LLMs) in recalling knowledge and reasoning. However, the reliability of LLMs in combining these two capabilities into reasoning through multi-hop facts has not been widely explored. This paper systematically investigates the possibilities for LLMs to utilize shortcuts based on direct connections between the initial and terminal entities of multi-hop knowledge. We first explore the existence of factual shortcuts through Knowledge Neurons, revealing that: (i) the strength of factual shortcuts is highly correlated with the frequency of co-occurrence of initial and terminal entities in the pre-training corpora; (ii) few-shot prompting leverage more shortcuts in answering multi-hop questions compared to chain-of-thought prompting. Then, we analyze the risks posed by factual shortcuts from the perspective of multi-hop knowledge editing. Analysis shows that approximately 20% of the failures are attributed to shortcuts, and the initial and terminal entities in these failure instances usually have higher co-occurrences in the pre-training corpus. Finally, we propose erasing shortcut neurons to mitigate the associated risks and find that this approach significantly reduces failures in multiple-hop knowledge editing caused by shortcuts. Code is publicly available at https://github.com/Jometeorie/MultiHopShortcuts.

## introduction large language models (llms) such as chatgpt @xcite and llama-2 @xcite , have impressive world knowledge modeling and reasoning capabilities within their parameters @xcite . when leveraging these two capabilities, it is intuitively anticipated that llms should be capable of reliably * corresponding authors. which continent will host the next olympic games? asia. europe? asia? after knowledge editing: before knowledge editing: which continent will host the next olympic games? answering multi-hop knowledge questions without any difficulty @xcite . nonetheless, the underlying reasoning processes of llms in responding to multi-hop knowledge questions have not received thorough investigation. ideally, an llm would systematically derive each single-hop answer and culminate in the correct result. however, in reality, llms may leverage factual shortcuts learned from pre-training corpora to directly obtain the final answer without performing intermediate reasoning. for conventional multi-hop question answering, the consistency of the final endpoints of shortcuts and multi-hop reasoning results may not cause risks and could even remain unnoticed. however, with the constant evolution of world knowledge, knowledge editing techniques are garnering increased attention @xcite . after knowledge editing, factual shortcuts in multi-hop scenarios may cause significant inconsistency. figure 1 illustrates the potential pitfalls asso-ciated with factual shortcuts. during the pretraining phase, an llm may have forged a direct association between the next olympic games and asia. consequently, when queried with the prompt: "which continent will host the next olympic games", the llm might bypass the need for reasoning about the country and can directly furnish the correct answer. however, applying knowledge editing to the llm, e.g., updating the host country of the olympic games to france, can expose a vulnerability. the persistence of the established shortcut may lead the llm to consistently output "asia" as the host continent even after the change, instead of the correct "europe", thereby impeding the success of multi-hop knowledge editing. in this paper, we systematically investigate the possibilities for llms to utilize factual shortcuts based on direct connections between the initial and terminal entities of multi-hop knowledge. firstly, we rethink and formalize the process through which llms reason about multi-hop knowledge. we introduce the hypothesis that llms may leverage factual shortcuts from pre-training corpora to facilitate cross-step reasoning. then, we deeply explore the existence of factual shortcuts. we conduct a frequency analysis of co-occurrences between the initial subject and terminal object of multi-hop knowledge instances in pre-training corpora. additionally, we employ knowledge neurons @xcite to quantify the overlap between the activated neurons for multihop questions and all single-hop questions. a low degree of overlap suggests that the reasoning pattern of llms in response to multi-hop questions is inconsistent with that of single-hop questions, indicating the presence of shortcuts. our experiments on multi-hop knowledge reveal that: additionally, to provide insights into the potential risks associated with multi-hop factual shortcuts, we conduct a detailed analysis of the reasons behind the failures in multi-hop knowledge editing. we find that approximately 20% of the failure instances are attributed to multi-hop factual shortcuts. furthermore, shortcut failure instances often exhibit higher co-occurrence frequencies of the initial and terminal entities, providing compelling evidence that the presence of shortcuts may disrupt the multi-hop reasoning consistency of llms after knowledge editing. finally, we explore the feasibility of employing knowledge neurons to eliminate factual shortcuts. we erase crucial neurons associated with factual shortcuts that co-occurred more than 10 times in the pre-training corpus. results show that the failure rate of multiple-hop knowledge editing caused by shortcuts significantly decreased, leading to an overall improvement in the success rate after our erasing approach. we hope this work can facilitate increased interest in exploring the multihop reasoning capabilities of llms and constrain reasoning shortcuts during the pre-training stage. ## rethinking the multi-hop knowledge a basic fact can be formulated as a single-hop knowledge tuple @xmath0, r, o) with a subject (s), a relation (r), and an object (o). for each query, we ask the llm if the object is correct given the subject and the relation 1 {f (t (s, r)) = o}, where f and t denote the outputs of the llm and the prompt template for splicing s and o into a cloze-style form. in this paper, we mainly focus on the multi-hop knowledge, which comprises a chain of single-hop knowledge: this question can also be formulated as asking the llm of the knowledge tuple t @xmath1, r mul , o n ), which proves unproblematic in general multi-hop question-answering, as t mul and t share the same endpoint o n . however, t mul is in fact a shortcut, treating a chain of relations as a separate composite relation. if a knowledge-editing approach is employed to modify the intermediate entity o i to o * i , the final answer of t will be altered. since t mul overlooks the intermediate entity, its answer remains unaffected by knowledge editing. it is also reasonable from the cause-and-effect perspective. for a two-hop knowledge e 1 → e 2 → e 3 , it requires first deducing the intermediate entity e 2 to obtain the correct output e 3 . any other reasoning path, such as e 1 → e 3 and e 1 → e 4 → e 3 , does not conform to the causal relationship. taking the multi-hop question of "which continent will host the next olympic games" as an illustrative example, if we edit the knowledge of the "country" from japan to france, according to the chain-relation reasoning, the "continent" hosting the olympic games should be converted to europe. however, if a composite relation is employed, the "continent" would remain unchanged despite alterations in the "country". a causal llm probably encodes such composite knowledge during the pre-training phase. the initial subject s 1 and the terminal object o n are likely to have direct associations in the corpus. still taking the example above, an llm may have learned the knowledge (the next olympic games, continent of the country, asia) from the corpus directly, neglecting the causal relationship between the country and the continent to which it belongs. therefore, for multi-hop knowledge, llms may potentially arrive at the correct answer through step-wise reasoning, but it is more likely that they memorize the outdated answer by leveraging the cooccurrence relationships in the pre-training corpus. ## exploring the existence of factual shortcuts in this section, we explore the extent of shortcuts in multi-hop question-answering. concretely, we first validate the correlation between multi-hop shortcuts and the word frequency in the pre-training corpus. then, we locate crucial neurons in singlehop, few-shot, and chain-of-thought questionanswering tasks to further elucidate the degree of potential factual shortcuts. ## reducing multi-hop factual shortcuts the existence of multi-hop factual shortcuts reveals the unreliability of current llms' reasoning and increases the risk of failures in multi-hop knowledge editing. since these shortcuts represent knowledge hardcoded into llms during the pretraining phase, it is challenging to eliminate these factual shortcuts fundamentally unless there are substantial changes in the pre-training phase. methods. to reduce the risks of multi-hop factual shortcuts and further validate the hypotheses presented in this paper, we adopt a simple yet effective method inspired by @xcite to erase these shortcuts (figure 6 ). compared to figure 1 , we erase crucial neurons related to the red factual shortcuts, compelling the llm to answer the continent that will host the next olympic games using the correct path of reasoning after knowledge editing. specifically, we use the integral gradient algorithm to locate the crucial neurons associated with multi-hop knowledge questions and set them to zero. for each piece of multi-hop knowledge, we query with three questions to obtain the intersection of crucial neurons. based on the previous experiments (figure 3 ), we posit that multi-hop knowledge with a co-occurrence frequency exceeding 10 exhibits evident shortcuts. consequently, we proceeded to eliminate these multi-hop factual shortcuts from the dataset d @xmath2. we compute the percentage of editing success (s) and shortcut failure rate (f shortcut ) for multi-hop knowledge editing before and after the erase of factual shortcuts, respectively. europe. ## related work in this section, we discuss two lines of research that are key to our work: knowledge editing and multi-hop reasoning. ## conclusion in this paper, we systematically explore the latent factual shortcuts that llms may employ when answering multi-hop knowledge questions. we first demonstrate the strong correlation between the strength of factual shortcuts and the co-occurrence of the initial subject and the terminal object in pretraining corpora. then, we delve into the potential risks introduced by these shortcuts in the context of multi-hop knowledge editing. our exploration reveals that approximately 20% of failures can be attributed to factual shortcuts, particularly in instances characterized by high co-occurrences within pre-training corpora. finally, we propose a straightforward yet efficient approach to mitigate shortcut failures in multi-hop knowledge editing by selectively erasing shortcut neurons. we advocate for increased research efforts directed towards exploring the true boundaries of llms in the realm of multi-hop reasoning, emphasizing the need to better constrain shortcut generation during the pretraining phase.