Investigating Multi-Hop Factual Shortcuts in Knowledge Editing of Large Language Models (2024)

Abstract: Recent work has showcased the powerful capability of large language models (LLMs) in recalling knowledge and reasoning. However, the reliability of LLMs in combining these two capabilities into reasoning through multi-hop facts has not been widely explored. This paper systematically investigates the possibilities for LLMs to utilize shortcuts based on direct connections between the initial and terminal entities of multi-hop knowledge. We first explore the existence of factual shortcuts through Knowledge Neurons, revealing that: (i) the strength of factual shortcuts is highly correlated with the frequency of co-occurrence of initial and terminal entities in the pre-training corpora; (ii) few-shot prompting leverage more shortcuts in answering multi-hop questions compared to chain-of-thought prompting. Then, we analyze the risks posed by factual shortcuts from the perspective of multi-hop knowledge editing. Analysis shows that approximately 20% of the failures are attributed to shortcuts, and the initial and terminal entities in these failure instances usually have higher co-occurrences in the pre-training corpus. Finally, we propose erasing shortcut neurons to mitigate the associated risks and find that this approach significantly reduces failures in multiple-hop knowledge editing caused by shortcuts. Code is publicly available at https://github.com/Jometeorie/MultiHopShortcuts.

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8987–9001 August 11-16, 2024 ©2024 Association for Computational Linguistics
Recent work has showcased the powerful capability of large language models (LLMs) in recalling knowledge and reasoning. However, the reliability of LLMs in combining these two capabilities into reasoning through multi-hop facts has not been widely explored. This paper systematically investigates the possibilities for LLMs to utilize shortcuts based on direct connections between the initial and terminal entities of multi-hop knowledge. We first explore the existence of factual shortcuts through Knowledge Neurons, revealing that: (i) the strength of factual shortcuts is highly correlated with the frequency of co-occurrence of initial and terminal entities in the pre-training corpora; (ii) few-shot prompting leverage more shortcuts in answering multi-hop questions compared to chain-of-thought prompting. Then, we analyze the risks posed by factual shortcuts from the perspective of multi-hop knowledge editing. Analysis shows that approximately 20% of the failures are attributed to shortcuts, and the initial and terminal entities in these failure instances usually have higher co-occurrences in the pre-training corpus. Finally, we propose erasing shortcut neurons to mitigate the associated risks and find that this approach significantly reduces failures in multiple-hop knowledge editing caused by shortcuts. Code is publicly available at https://github.com/Jometeorie/MultiHopShortcuts.

heading: 1 Introduction
text: Large Language Models (LLMs) such as ChatGPT (OpenAI, 2022) and LLaMA-2 (Touvron et al., 2023), have impressive world knowledge modeling and reasoning capabilities within their parameters (Zhao et al., 2023; Hao et al., 2023). When leveraging these two capabilities, it is intuitively anticipated that LLMs should be capable of reliably
*Corresponding authors.
answering multi-hop knowledge questions without any difficulty (Press et al., 2023).
Nonetheless, the underlying reasoning processes of LLMs in responding to multi-hop knowledge questions have not received thorough investigation. Ideally, an LLM would systematically derive each single-hop answer and culminate in the correct result. However, in reality, LLMs may leverage factual shortcuts learned from pre-training corpora to directly obtain the final answer without performing intermediate reasoning.
For conventional multi-hop question answering, the consistency of the final endpoints of shortcuts and multi-hop reasoning results may not cause risks and could even remain unnoticed. However, with the constant evolution of world knowledge, knowledge editing techniques are garnering increased attention (Wang et al., 2023b). After knowledge editing, factual shortcuts in multi-hop scenarios may cause significant inconsistency.
Figure 1 illustrates the potential pitfalls asso-
8987
ciated with factual shortcuts. During the pretraining phase, an LLM may have forged a direct association between the next Olympic Games and Asia. Consequently, when queried with the prompt:
“Which continent will host the next Olympic Games”, the LLM might bypass the need for reasoning about the country and can directly furnish the correct answer. However, applying knowledge editing to the LLM, e.g., updating the host country of the Olympic Games to France, can expose a vulnerability. The persistence of the established shortcut may lead the LLM to consistently output “Asia” as the host continent even after the change, instead of the correct “Europe”, thereby impeding the success of multi-hop knowledge editing.
In this paper, we systematically investigate the possibilities for LLMs to utilize factual shortcuts based on direct connections between the initial and terminal entities of multi-hop knowledge. Firstly, we rethink and formalize the process through which LLMs reason about multi-hop knowledge. We introduce the hypothesis that LLMs may leverage factual shortcuts from pre-training corpora to facilitate cross-step reasoning.
Then, we deeply explore the existence of factual shortcuts. We conduct a frequency analysis of co-occurrences between the initial subject and terminal object of multi-hop knowledge instances in pre-training corpora. Additionally, we employ Knowledge Neurons (Dai et al., 2022) to quantify the overlap between the activated neurons for multihop questions and all single-hop questions. A low degree of overlap suggests that the reasoning pattern of LLMs in response to multi-hop questions is inconsistent with that of single-hop questions, indicating the presence of shortcuts. Our experiments on multi-hop knowledge reveal that:
(i) Few-shot questions exhibit more shortcuts in comparison to chain-of-thought questions, suggesting that LLMs often arrive at multi-hop knowledge answers using unexpected cross-step reasoning patterns.
(ii) Knowledge instances with a higher cooccurrence frequency between initial subjects and terminal objects tend to have more shortcuts, indicating a strong correlation between the existence of multi-hop factual shortcuts and the word frequencies learned by LLMs during pretraining phase.
Additionally, to provide insights into the potential risks associated with multi-hop factual shortcuts, we conduct a detailed analysis of the
reasons behind the failures in multi-hop knowledge editing. We find that approximately 20% of the failure instances are attributed to multi-hop factual shortcuts. Furthermore, shortcut failure instances often exhibit higher co-occurrence frequencies of the initial and terminal entities, providing compelling evidence that the presence of shortcuts may disrupt the multi-hop reasoning consistency of LLMs after knowledge editing.
Finally, we explore the feasibility of employing Knowledge Neurons to eliminate factual shortcuts. We erase crucial neurons associated with factual shortcuts that co-occurred more than 10 times in the pre-training corpus. Results show that the failure rate of multiple-hop knowledge editing caused by shortcuts significantly decreased, leading to an overall improvement in the success rate after our erasing approach. We hope this work can facilitate increased interest in exploring the multihop reasoning capabilities of LLMs and constrain reasoning shortcuts during the pre-training stage.

heading: 2 Rethinking the Multi-Hop Knowledge
text: A basic fact can be formulated as a single-hop knowledge tuple t = (s, r, o) with a subject (s), a relation (r), and an object (o). For each query, we ask the LLM if the object is correct given the subject and the relation 1 {f (T (s, r)) = o}, where f and T denote the outputs of the LLM and the prompt template for splicing s and o into a cloze-style form.
In this paper, we mainly focus on the multi-hop knowledge, which comprises a chain of single-hop knowledge:
T = ⟨(s1, r1, o1) , ..., (sn, rn, on)⟩ , (1) where si = oi−1. For each query, we directly ask the LLM if the terminal object is correct given the initial subject and the chain relation 1 {f (T (s1, rmul)) = on}, where rmul = r1 → ... → rn. This question can also be formulated as asking the LLM of the knowledge tuple tmul = (s1, rmul, on), which proves unproblematic in general multi-hop question-answering, as tmul and T share the same endpoint on.
However, tmul is in fact a shortcut, treating a chain of relations as a separate composite relation. If a knowledge-editing approach is employed to modify the intermediate entity oi to o∗i , the final answer of T will be altered. Since tmul overlooks the intermediate entity, its answer remains unaffected by knowledge editing.
It is also reasonable from the cause-and-effect perspective. For a two-hop knowledge e1 → e2 → e3, it requires first deducing the intermediate entity e2 to obtain the correct output e3. Any other reasoning path, such as e1 → e3 and e1 → e4 → e3, does not conform to the causal relationship.
Taking the multi-hop question of “Which continent will host the next Olympic Games” as an illustrative example, if we edit the knowledge of the “country” from Japan to France, according to the chain-relation reasoning, the “continent” hosting the Olympic Games should be converted to Europe. However, if a composite relation is employed, the “continent” would remain unchanged despite alterations in the “country”.
A causal LLM probably encodes such composite knowledge during the pre-training phase. The initial subject s1 and the terminal object on are likely to have direct associations in the corpus. Still taking the example above, an LLM may have learned the knowledge (the next Olympic Games, continent of the country, Asia) from the corpus directly, neglecting the causal relationship between the country and the continent to which it belongs. Therefore, for multi-hop knowledge, LLMs may potentially arrive at the correct answer through step-wise reasoning, but it is more likely that they memorize the outdated answer by leveraging the cooccurrence relationships in the pre-training corpus.

heading: 3 Exploring the Existence of Factual Shortcuts
text: In this section, we explore the extent of shortcuts in multi-hop question-answering. Concretely, we first validate the correlation between multi-hop shortcuts and the word frequency in the pre-training corpus. Then, we locate crucial neurons in singlehop, few-shot, and chain-of-thought questionanswering tasks to further elucidate the degree of potential factual shortcuts.

heading: 3.1 Probing Shortcuts in Pre-training Corpus
text: Our analysis centers specifically on the MQUAKECF-3K dataset released by Zhong et al. (2023). It comprises 1,000 two-hop, 1,000 three-hop, and 1,000 four-hop instances of multi-hop questionanswering for knowledge editing extracted from Wikidata (Vrandecic and Krötzsch, 2014). Essential information for one sample from the dataset is shown in Appendix A. we compute the crucial neurons of the first question within the ’questions’
key alongside the entirety of questions within the ’single_hops’ key. It can also be adopted in subsequent sections for further investigating potential risks introduced by these multi-hop factual shortcuts.
Considering that the existence of factual shortcuts may drive from pre-training corpora, we first compute the frequency of co-occurrence of the initial subject s1 and the terminal object on among these 3,000 items of knowledge on Wikipedia (20231101-en) and Dolma corpus(v1_6sample) respectively. The Wikipedia dataset contains approximately 6.41M rows of text, while the Dolma dataset contains roughly 10 billion tokens. We chose these two corpora due to their comprehensive coverage of global knowledge and their frequent utilization as a significant component in the pre-training corpora for most LLMs. If s1 and on co-occur within the same paragraph, it is highly plausible that the LLM establishes a direct connection between them during the pre-training phase.
We first conduct a frequency analysis of the occurrences of these multi-hop knowledge shortcuts in the Wikipedia corpus (Figure 2). It can be observed that more than 2/3 of instances exhibited various degrees of shortcuts, with some even appearing over 10,000 times. This indicates that certain pieces of knowledge exhibit significant multi-hop shortcuts, which could potentially influence the reasoning processes of LLMs.
We’ve also conducted the same frequency analysis on the Dolma corpus (Figure 2). It can be observed that the co-occurrence rate distribution of vocabulary in the Dolma dataset is similar
to that in the Wikipedia dataset. We calculate the Pearson Coefficient of the co-occurrence rate between the two datasets, and the result is 0.74. The similarity in the co-occurrence rates between these two datasets indicates the strong correlation between the two corpora, and therefore, Wikipedia can be chosen as an approximate corpus.
Moreover, we select several examples with high and low frequencies for illustration (Table 1) in the Wikipedia corpus. It can be observed that instances with high frequency exhibit a clear, direct connection between s1 and on. For instance, “Twitter” is inherently strongly associated with “the United States”, obviating the need to think about the country of citizenship of “Twitter’s CEO”. In contrast, there is no apparent connection between “Jerry Rivers” and “Donald Trump”, necessitating the prior derivation of the nationality of “Jerry Rivers” to arrive at the correct answer. Since “Jerry Rivers” and “Donald Trump” rarely co-occur in the pre-training corpus, LLMs may not contain factual shortcuts related to such multi-hop knowledge.

heading: 3.2 Quantifying Shortcuts Using Knowledge Neurons
text: Methods. The presence of multi-hop factual shortcuts may result in a divergence in the reasoning mechanisms employed by the LLM when responding to multi-hop questions as opposed to directly answering individual single-hop questions. To quantify the disparities, we employ Knowledge Neurons (KN) proposed by Dai et al. (2022) to
locate crucial neurons activated by the LLMs when responding to various questions. Specifically, it gradually changes each neuron w(l)i stored in FFN from 0 to its original value w(l)i and meanwhile integrates the gradients. We use the Riemann approximation as a substitution for continuous integrals:
Ãttr(w(l)i ) = w
(l) i
m
m∑
k=1
∂P ( kmw (l) i )
∂w (l) i
, (2)
where P (w(l)i ) = p(y|x,w (l) i = ŵ (l) i ) is the probability of the correct answer predicted by the LLM when changing the value of neuron w(l)i to ŵ
(l) i , and m is the number of the approximation steps. We choose neurons with attribution values larger than v as crucial neurons reflecting LLM decision-making patterns:
N = { w
(l) i |Attr(w (l) i ) > v
} . (3)
In this paper, we set m to 20 and the attribution threshold v to 0.2. In the scenario of a multi-hop question devoid of any shortcuts, it should ideally encompass a broader array of crucial neurons inherent to single-hop questions, except those specifically dedicated to lower-level components such as lexical and syntactic neurons. Hence, we define O as the degree of overlap between the reasoning patterns of multi-hop knowledge answers and all single-hop knowledge answers:
O = |NT ∩Ntmul |
|NT | , (4)
where NT denotes the intersection of crucial neurons for all single-hop questions, Ntmul denotes the set of crucial neurons for multi-hop
questions. A higher degree of overlap indicates that LLM’s reasoning patterns for answering multihop questions are more closely aligned with those for answering single-hop questions.
It is noteworthy to emphasize that our objective does not entail the precise localization of neurons storing knowledge; rather, we aim to discern the decision-making processes of the LLMs across various questions. Despite Anonymous (2024)’s skepticism regarding whether neurons uncovered by KN in the FFN truly constitute “knowledge”, these neurons may store intricate “token expression patterns” that can still elucidate the LLM’s decision-making processes.
We separately evaluate the degree of shortcuts in few-shot and chain-of-thought multi-hop questions. All single-hop questions and few-shot multihop questions utilize the same demonstrations, while chain-of-thought multi-hop questions employ prompts with similar semantics.
For all single-hop questions, we adopt the fewshot prompt shown in Table 6. Subsequently, we locate crucial neurons based on the probability of correct answer output by the LLM following the "A:" prefix.
For multi-hop questions, we adopt both the fewshot and chain-of-thought prompts. The few-shot prompt mirrors that of single-hop questions, while the chain-of-thought prompt is constructed with semantically approximate expressions. We require the LLM to articulate its reasoning process upon receiving the question. Then we locate crucial neurons based on the probability of correct answer output by the LLM following the "Answer:" prefix (see Table 7). Both prompts are provided in Appendix B).
Besides, we partition the original dataset Do into two subsets Dcount≤τ and Dcount>τ based on word frequencies, where τ represents the threshold for word frequencies. We compute the degree of shortcuts for GPT-J (Figure 3).
Main Results. It can be observed that the LLM adheres to a greater extent to reasoning patterns overlapping with those for single-hop questions under the chain-of-thought prompt. This observation suggests that the chain-of-thought prompt indeed serves to induce the LLM to engage in stepwise reasoning. It also aligns with our hypothesis that LLMs tend to prioritize the utilization of latent multi-hop factual shortcuts, relinquishing them only when explicitly prompted to engage in step-wise reasoning. Furthermore, the instances within Dcount>τ exhibit lower degrees of reasoning overlap, suggesting that LLMs indeed learn the shortcut associations between s1 and on, with word frequencies significantly influencing the strength of these shortcuts.
Interestingly, although the overlap rates vary across different scenarios, their values remain low. We analyze the distribution of the number of activated knowledge neurons for different instances (Figure 4). Since single-hop knowledge typically involves 2-4 questions, the number of activated neurons is an order of magnitude higher than that for multi-hop questions. Moreover, activated neurons, in addition to reflecting the inherent knowledge, may also be influenced by factors such as the lexical and syntactic aspects of sentences. Hence, the reasoning overlap rates tend to be maintained at a low value.
For the hyper-parameter v, We randomly select 500 instances at v = 0.1 and v = 0.3 for experiments (Table 3). The results show the same trend in the figure with τ as the X-axis. Although the magnitude of the values varies, this is because as v decreases, more neurons will be considered crucial neurons, which will significantly increase the size of the denominator |NT |, leading to a decrease in the overlap rate. Nevertheless, our conclusions are correct under different parameters: (i) the strength of factual shortcuts is highly correlated with the frequency of co-occurrence of initial and terminal entities in
the pre-training corpora; (ii) few-shot prompting leverage more shortcuts in answering multi-hop questions compared to chain-of-thought prompting.

heading: 4 Exploring the Potential Risks of Factual Shortcuts
text: While these shortcuts may not have a significant impact on the results in general multi-hop question answering, their potential risks can be magnified in the context of knowledge editing. Zhong et al. (2023) have observed poor performance of LLMs in multi-hop knowledge editing. In this section, we will specifically analyze the reasons for the failure of multi-hop knowledge editing, particularly under the influence of shortcuts.
Concretely, we employ various knowledge editing methods to modify single-hop knowledge instances in MQUAKE-CF-3K and pose three different multi-hop questions about the edited knowledge. Subsequently, we quantify the effects of various knowledge editing methods and categorize error instances into three categories.
Failure Categories. We consider three key categories of failures. The first category of failure stems from the unsuccessful editing of singlehop knowledge. We designate the set of failures in this category as Fsingle. The second and third categories are built upon the assumption of successfully editing all single-hop knowledge instances, yet the LLM still fails to answer multihop questions correctly. The second category
signifies cases where the answer to multi-hop knowledge questions remains the same as the original unedited answer. We denote this set as Fshortcut. Given that all single-hop questions can be answered correctly, the persistence of the original result in multi-hop questions indicates the existence of shortcuts. The third category involves the LLM providing alternative incorrect answers, potentially arising from hallucinations or other reasons. We denote this set as Fother.
For each multi-hop edited knowledge, we interrogate the LLM with three distinct multi-hop questions. All multi-hop questions are prefixed with the same few-shot template comprising 16 demonstrations, which is consistent with the setup of Zhong et al. (2023). We calculate the percentage of editing successes (S) and failures (F) within three questions. Detailed experimental settings can be seen in the Appendix C.
Main Results. Table 2 presents the analysis results. Consistent with the findings of Zhong et al. (2023), knowledge editing algorithms exhibit catastrophic failures when addressing multi-hop factual questions, with only approximately 10%- 20% of instances avoiding complete errors across three queries. Fsingle stems from the editing failure of LLMs in addressing single-hop questions. Since multi-hop questions may necessitate more than one edit, it may be slightly higher than the editwise failure rate. Fother may originate from the insufficient reasoning capabilities of LLMs or the hallucinations generated during editing. While we utilize few-shot prompts instead of chain-ofthought prompts to expose factual shortcuts, it may also increase Fother.
It is noteworthy that Fshortcut also constitutes a significant proportion. This type of failure implies that LLMs respond with old ground truth for multihop questions while capable of correctly answering
single-hop questions after all edits. In other words, shortcuts enable LLMs to conveniently utilize the rmul hard-coded during the pre-training phase to directly obtain results, without genuinely engaging in multi-hop knowledge reasoning. Experiments indicate that these factual shortcuts are prevalent across various knowledge types in LLMs.
To further investigate the connection between shortcut failures and falsely learned relations in the pre-training corpus, we analyze the relationship between the average co-occurrence frequency of entities and the occurrence frequency of shortcut failures (Figure 5). We observe that instances with higher occurrences of shortcut failures, particularly those with three failures, exhibit higher word co-occurrence frequencies between s1 and on. This suggests that LLMs are highly likely to leverage the multi-hop knowledge hardcoded during the pre-training phase as reasoning shortcuts. The presence of these factual shortcuts significantly diminishes the reliability and plausibility of LLMs’ reasoning. In the context of multi-hop knowledge editing, the LLMs are easily entangled in the confusion between old shortcut knowledge and new multi-hop knowledge.

heading: 5 Reducing Multi-Hop Factual Shortcuts
text: The existence of multi-hop factual shortcuts reveals the unreliability of current LLMs’ reasoning and increases the risk of failures in multi-hop knowledge editing. Since these shortcuts represent knowledge hardcoded into LLMs during the pretraining phase, it is challenging to eliminate these factual shortcuts fundamentally unless there are
substantial changes in the pre-training phase.
Methods. To reduce the risks of multi-hop factual shortcuts and further validate the hypotheses presented in this paper, we adopt a simple yet effective method inspired by Dai et al. (2022) to erase these shortcuts (Figure 6). Compared to Figure 1, we erase crucial neurons related to the red factual shortcuts, compelling the LLM to answer the continent that will host the next Olympic Games using the correct path of reasoning after knowledge editing.
Specifically, we use the integral gradient algorithm to locate the crucial neurons associated with multi-hop knowledge questions and set them to zero. For each piece of multi-hop knowledge, we query with three questions to obtain the intersection of crucial neurons. Based on the previous experiments (Figure 3), we posit that multi-hop knowledge with a co-occurrence frequency exceeding 10 exhibits evident shortcuts. Consequently, we proceeded to eliminate these multi-hop factual shortcuts from the dataset Dcount>10. We compute the percentage of editing success (S) and shortcut failure rate (Fshortcut) for multi-hop knowledge editing before and after the erase of factual shortcuts, respectively.
Main Results. Table 4 presents the success rate and shortcut failure rate of multi-hop knowledge editing before and after the erase of factual shortcuts on Dcount>10. Compared to Tabel 2, both the success rate and shortcut failure rate of multiple-hop knowledge editing have increased on Dcount>10. The result implies that instances with
factual shortcuts are inherently more amenable to editing, yet the presence of factual shortcuts also entails a higher level of risk for these instances. Thus, these latent factual shortcuts are far more harmful than we realize.
Furthermore, the erasing of shortcuts can significantly reduce the risks associated with shortcut failures, leading to an appreciable improvement in the success rate of multi-hop knowledge editing. Due to the incapacity of knowledge editing methods to address shortcut knowledge tmul, inconsistencies arise in LLMs’ reasoning results. By erasing neurons corresponding to tmul, we ensure that LLMs reason along the correct path, thereby enhancing the success rate.
However, despite the efficacy of this approach in mitigating the risk posed by factual shortcuts to specific knowledge, it cannot serve as a comprehensive solution to the problem. Due to the ubiquitous nature of such shortcuts, it is
impractical to review and erase crucial neurons for every multi-hop knowledge. Fundamentally, to attain a trustworthy LLM with genuine multihop reasoning capabilities, it is imperative to address the issue at the pre-training stage to explore improved pre-training methodologies.

heading: 6 Related Work
text: In this section, we discuss two lines of research that are key to our work: knowledge editing and multi-hop reasoning.

heading: 6.1 Knowledge Editing
text: Numerous studies have explored efficient knowledge editing methods for LLMs, seeking resolutions to challenges arising from outdated knowledge. One prevalent and intuitive approach involves employing external memorization, wherein new knowledge is incorporated through external context or parameters, without necessitating modifications to the LLM weights (Mitchell et al., 2022b; Dong et al., 2022; Huang et al., 2023; Zheng et al., 2023; Zhong et al., 2023). While these approaches are simple and effective in ensuring consistency, the substantial influx of supplementary knowledge may result in redundancy and low timeliness at a later stage (Wang et al., 2023b).
Another line of work focuses on directly updating the LLM parameters. Some investigations are dedicated to constrained fine-tuning (Chen et al., 2020; Lee et al., 2022) or meta-learning (Lee et al., 2022; Mitchell et al., 2022a), which update the full parameters of LLMs. The other investigations involve a preliminary stage of
knowledge localization before editing, premised on the assumption that knowledge is stored in the form of key-value memories within the twolayer Feedforward Neural Network (FFN) (Geva et al., 2021). Dai et al. (2022) located and refined knowledge neurons (KN) through integral gradients (Sundararajan et al., 2017). Meng et al. (2022) et al. proposed the Rank-One Model method (ROME) to insert new knowledge in a specific FFN layer, while MEMIT (Meng et al., 2023) further extended address scenarios of mass editing.
While the effectiveness of single-hop knowledge editing has been thoroughly investigated, there is a notable dearth of attention given to multi-hop knowledge editing. Zhong et al. (2023) systematically focused on this issue by introducing the multihop knowledge editing evaluation benchmarks MQUAKE-CF and MQUAKE-T. Their findings revealed catastrophic performance degradation of existing knowledge editing methods. In this paper, we further investigate and elucidate the repercussions stemming from the presence of reasoning shortcuts in multi-hop knowledge editing.

heading: 6.2 Multi-Hop Reasoning
text: Multi-hop reasoning is often seen as a weakness for LLMs (Huang and Chang, 2023). Early efforts commonly employed in-context prompting, which involves the provision of few input-output demonstrations to LLMs (Brown et al., 2020; Zhao et al., 2021; Chen et al., 2022). This approach enables LLMs to solve problems through reasoning implicitly. However, its effectiveness diminishes significantly when confronted with multi-hop questions (Valmeekam et al., 2022). To incentivize LLMs to engage in explicit multi-hop reasoning, the concept of chain-of-thought was introduced by Wei et al. (2022). It encourages the LLM to think step by step and output intermediate deductive steps (Chu et al., 2023). In this paper, we elucidate the process by which LLMs handle multi-hop question-answering from the perspective of factual shortcuts. We provide evidence that the chain-of-thought prompting compels LLMs to attend to the single-hop knowledge more faithfully.

heading: 7 Conclusion
text: In this paper, we systematically explore the latent factual shortcuts that LLMs may employ when answering multi-hop knowledge questions. We first demonstrate the strong correlation between the
strength of factual shortcuts and the co-occurrence of the initial subject and the terminal object in pretraining corpora. Then, we delve into the potential risks introduced by these shortcuts in the context of multi-hop knowledge editing. Our exploration reveals that approximately 20% of failures can be attributed to factual shortcuts, particularly in instances characterized by high co-occurrences within pre-training corpora. Finally, we propose a straightforward yet efficient approach to mitigate shortcut failures in multi-hop knowledge editing by selectively erasing shortcut neurons. We advocate for increased research efforts directed towards exploring the true boundaries of LLMs in the realm of multi-hop reasoning, emphasizing the need to better constrain shortcut generation during the pretraining phase.

heading: Acknowledgements
text: This work is partially supported by the National Key R&D Program of China under No. 2023YF3303800 and the Joint Funds of the National Natural Science Foundation of China under No.U21B2020.
Limitations
We posit that Wikipedia serves as a comprehensive repository of global knowledge, thus making it a suitable substitute for the entirety of the pretraining corpora. However, despite our exhaustive traversal of the Wikipedia dataset to calculate the co-occurrence frequencies of initial and terminal entities, it is noteworthy that the pre-training corpora for LLMs often extend beyond the confines of this dataset. This potential discrepancy may introduce inaccuracies in statistical outcomes. We advocate for future investigations to extend statistical analyses to more expansive corpora.
For the erasing of factual shortcuts, our primary objective is to further substantiate the potential risks associated with these shortcuts, and the observed improvement in editing success rates after erasing serves to support this assertion. However, it is imperative to recognize that this approach functions as a mitigative measure, as the complete eradication of factual shortcuts through post-hoc removal is unattainable. A genuine and thorough elimination of factual shortcuts must be initiated during the pre-training phase, involving the alignment of LLMs’ multi-hop reasoning capabilities with human-level proficiency.
Finally, due to space and resource constraints, we only conduct detailed experiments on GPT-J (6B) and LLaMA-2 (7B) and do not encompass all publicly accessible LLMs, such as PaLM (Chowdhery et al., 2022), OPT (Zhang et al., 2022), and Pythia (Biderman et al., 2023). We encourage future research to undertake comprehensive experiments on a broader spectrum of LLMs.
Ethical Statement
We conduct a reassessment of the multi-hop reasoning capabilities of LLMs and demonstrate that the presence of factual shortcuts may compromise the consistency of results in multi-hop knowledge editing. Since the approach itself is unbiased and all experiments are conducted on publicly available datasets, we believe that our work creates no potential ethical risk. Additionally, all use of existing artifacts is consistent with their intended use in this paper.
However, we have exposed the indiscriminate use of shortcuts by LLMs during multihop reasoning, raising concerns regarding their genuine reasoning capabilities. LLMs struggle to engage in step-wise reasoning akin to human cognitive processes, and the potential for parameter confusion may arise following the assimilation of new knowledge. These factors contribute to our perplexity concerning the black-box nature of LLMs and apprehensions regarding their application in security-sensitive domains. We advocate for more rigorous ethical scrutiny and improvements in LLMs to ensure alignment with the human reasoning process.

heading: A Dataset
text: We select the MQUAKE-CF-3K dataset as the primary focus for exploration in this paper. It comprises 3,000 multi-hop English knowledge questions extracted from Wikipedia along with a corresponding knowledge editing task. We present essential information for one sample from the dataset (Table 5). For Section 3, we compute the crucial neurons of the first question within the ’questions’ key, alongside the entirety of questions within the ’single_hops’ key. For Section 4, we adopt knowledge editing methods of all knowledge encapsulated within the ’requested_rewrite’ key. Furthermore, we augment the original dataset by introducing a new key, labeled as ’shortcut_frequency’, which denotes the frequency of co-occurrence in the pre-training corpus between the initial subject and the terminal object for each instance.

heading: B Prompts for Knowledge Neurons
text: We employ prompt templates similar to that utilized by Zhong et al. (2023) for finding crucial neurons. Given the substantial computational overhead associated with Knowledge Neurons, we adopt a 2-shot prompt, which is already sufficient for the LLM to comprehend the task and furnish accurate responses. The few-shot prompt and chain-ofthought prompt are shown in Table 6 and Table 7.

heading: C Experimental Details
text: C.1 Language Models
Our experiments are conducted on GPT-J (6B) (Wang and Komatsuzaki, 2021) and LLaMA-2 (7B) (Touvron et al., 2023). The selection of GPT-J is motivated by the alignment with the pre-existing work on knowledge editing (Meng et al., 2022, 2023; Zhong et al., 2023), while opting for LLaMA2 is motivated by its status as a recent, prominent open-source LLM representative, providing a robust reflection of the current capabilities of LLMs. We use the huggingface package (Wolf et al., 2020) for the specific implementation.
C.2 Knowledge Editing
We use the cloze-style statement templates for knowledge editing, which is consistent with the previous studies. We employ the EasyEdit package (Wang et al., 2023a) for the specific implementation. All licenses of these packages allow us for
normal research use. The detailed specifics of the three knowledge editing methods that are employed in our training are as follows.
MEND. MEND (Mitchell et al., 2022a) trains a lightweight model editor network to produce edits to the LLM’s weight when provided with the standard fine-tuning gradient. We train our editor network on the ZsRE dataset (Levy et al., 2017) with a maximum number of training steps of 100,000. We set the learning rate scale to be 1.0 during inference. All experiments edit the MLP weights in the last 3 Transformer blocks.
ROME. ROME (Meng et al., 2022) stands out as a popular method for knowledge localization and editing. It introduces a based on corruption and restoration to identify relevant layers storing knowledge. Subsequently, it inserts new knowledge by key selection and value optimization in the corresponding feed-forward network (FFN) layer. We perform the intervention at layer 5 for GPT-J (6B) and 6 for LLaMA-2 (7B). We compute the second-order momentum statistics using 100,000 examples of Wikitext in fp32. For the remaining hyperparameters, we adopt the default values specified in Meng et al. (2022).
MEMIT. MEMIT (Meng et al., 2023) is a subsequent work to ROME, designed to handle extensive knowledge edits. In this paper, we perform the intervention at layer {3, 4, 5, 6} for GPT-J (6B) and {4, 5, 6, 7} for LLaMA-2 (7B). We also compute the covariance statistics using 100,000 examples of Wikitext in fp32. For the remaining hyperparameters, we adopt the default values specified in Meng et al. (2023).
C.3 Computational Budget For all the experiments mentioned in this paper, we use one Nvidia A100-SXM4 GPU with 80GB memory. We spend about 100, 200, and 250 GPU hours exploring the existence of factual shortcuts (Section 3), exploring the potential risks of factual shortcuts (Section 4) and reducing multihop factual shortcuts (Section 5).

heading: Future Work
text: We advocate for future investigations to extend statistical analyses to more expansive corpora. For the erasing of factual shortcuts, our primary objective is to further substantiate the potential risks associated with these shortcuts, and the observed improvement in editing success rates after erasing serves to support this assertion. However, it is imperative to recognize that this approach functions as a mitigative measure, as the complete eradication of factual shortcuts through post-hoc removal is unattainable. A genuine and thorough elimination of factual shortcuts must be initiated during the pre-training phase, involving the alignment of LLMs’ multi-hop reasoning capabilities with human-level proficiency. Finally, due to space and resource constraints, we only conduct detailed experiments on GPT-J (6B) and LLaMA-2 (7B) and do not encompass all publicly accessible LLMs, such as PaLM (Chowdhery et al. , 2022), OPT (Zhang et al. , 2022), and Pythia (Biderman et al. , 2023). We encourage future research to undertake comprehensive experiments on a broader spectrum of LLMs