Do Large Language Models Latently Perform Multi-Hop Reasoning? (2024)

Abstract: We study whether Large Language Models (LLMs) latently perform multi-hop reasoning with complex prompts such as “The mother of the singer of ‘Superstition’ is”. We look for evidence of a latent reasoning pathway where an LLM (1) latently identifies “the singer of ‘Superstition’” as Stevie Wonder, the bridge entity, and (2) uses its knowledge of Stevie Wonder’s mother to complete the prompt. We analyze these two hops individually and consider their co-occurrence as indicative of latent multi-hop reasoning. For the first hop, we test if changing the prompt to indirectly mention the bridge entity instead of any other entity increases the LLM’s internal recall of the bridge entity. For the second hop, we test if increasing this recall causes the LLM to better utilize what it knows about the bridge entity. We find strong evidence of latent multi-hop reasoning for the prompts of certain relation types, with the reasoning pathway used in more than 80% of the prompts. However, the utilization is highly contextual, varying across different types of prompts. Also, on average, the evidence for the second hop and the full multi-hop traversal is rather moderate and only substantial for the first hop. Moreover, we find a clear scaling trend with increasing model size for the first hop of reasoning but not for the second hop. Our experimental findings suggest potential challenges and opportunities for future development and applications of LLMs.

## introduction recent works have shown that transformerbased @xcite large language models (llms) store and retrieve factual information in their parameters to complete simple prompts such as "the mother of stevie wonder is" @xcite @xcite for the first hop, we change the input prompt to refer to the bridge entity (stevie wonder) and check how often it increases the model's internal recall of the bridge entity. for the second hop, we check if increasing this recall causes the model output to be more consistent with respect to what it knows about the bridge entity's attribute (mother of stevie wonder). in this work, we limit ourselves to prompts that express a composition of two facts such as "the mother of the singer of 'superstition' is" that humans can complete with two hops by (1) inferring a bridge entity (e.g., stevie wonder) and (2) inferring an attribute of that entity (e.g., who his mother is). then, we study how often llms process the prompt using a similar latent two-hop reasoning pathway, although this pathway may not be the most salient pathway that largely determines the predicted output. to this end, we first study these hops individually, as shown in figure 1 . to study the first hop, we propose the entity recall score to approximate llm's internal recall of the bridge entity by projecting specific hidden representations to vocabulary space. we test how changes to the input prompt affect this score. to study the second hop, we propose to measure the consistency score between the distributions for completions of the two-hop prompt and an equivalent recall-based one-hop prompt (e.g., "the mother of stevie wonder is"). we check how often an intervention to increase the entity recall score increases consistency as an indication of second-hop utilization. finally, we investigate how frequently both steps coincide. to study latent two-hop reasoning with diverse types of fact composition, we introduce twohop-fact dataset, which is based on wikidata @xcite and consists of 45,595 two-hop prompts of 52 types of fact composition. we experiment with llama-2 (touvron et al., 2023) 7b, 13b, and 70b. our findings can be summarized as follows. across a wide range of fact composition types for the two-hop prompts, we find substantial evidence for the first hop of the multihop reasoning. in about 70% of the times where we change the prompt to indirectly mention the bridge entity, the later layers of the transformer show increased bridge entity recall. for the second hop and overall traversal, the evidence appears weaker: in 60% of the cases where we increase entity recall score, consistency goes up. likewise, in about 40% of the time, both hops work together (compared to a random 25% baseline); changing the descriptive mention increases the entity recall score, and increasing this recall score increases consistency. while the above aggregate statistics do not suggest a very prevalent use of the latent multi-hop reasoning pathway, it is worth pointing out that up to 23% of the fact composition types demonstrate strong evidence of latent multi-hop reasoning, occurring in more than 80% of the cases. this suggests that the pathway exists but is highly contextual. additionally, we focus on a very narrow interpretation of the pathway -in reality, we expect it to be more distributed across layers and tokens. hence, the effects we see might be a lower bound on the model's ability to perform latent twohop reasoning. we also find striking scaling behavior: while the first hop clearly improves substantially with parameter count, the second hop (and the round-trip performance) remains relatively constant. this might indicate a fundamental limitation in today's architecture or pretraining. our contributions can be summarized as follows: • we establish a framework for the investigation of latent multi-hop reasoning in llms and show its existential evidence. ## related works recent works have shown that llms demonstrate remarkable in-context reasoning ability via prompting, which scales with model size @xcite @xcite . on the contrary, when the information to reason from 10211 notation example description (e 1 , r 1 , e 2 ) (superstition, singer, stevie wonder) fact triplets of named entities where e i are named entities and r i is a (e 2 , r 2 , e 3 ) (stevie wonder, mother, lula) relation function that maps e i uniquely to e i+1 , such that r i (e i ) = e i+1 e 2 stevie wonder bridge entity that connects the two fact triplets τ 1h "the mother of stevie wonder is named" one-hop prompt (requires one-hop reasoning) τ 2h while there have been works to investigate latent reasoning of llms, the exploration has been mostly done with simple single-hop reasoning tasks @xcite @xcite and/or controlled lightweight training/finetuning @xcite @xcite . also, many of the works that aim to identify latent reasoning pathways or circuits, have focused on finding the most salient reasoning pathway for simple synthetic tasks and/or toy models @xcite @xcite @xcite @xcite . on the other hand, we study the existence of a latent multi-hop reasoning pathway, which may not be the most salient, in pretrained llms without further training, using diverse types of natural two-hop prompts. model editing examines ways to amend factual knowledge in lms @xcite @xcite . however, recent works have shown that the existing editing approaches, largely focusing on single fact edits, fail to propagate the edits to facts that depend on the edited fact @xcite @xcite . our work explores the possibilities that such propagation could work. moreover, our work investigates a pathway that affects the consistency at inference, whereas prior work in consistency has focused on quantifying inconsistency and improving consistency posthoc @xcite @xcite @xcite @xcite . @xcite aim to improve multi-hop reasoning accuracy with a hypothesis that the errors stem from failure to recall the latent hop, while we investigate the foundations of this hypothesis of whether the model actually performs such a latent multi-hop reasoning. @xcite is a concurrent work showing that a large portion of multi-hop reasoning failure cases can be attributed to incorrectly performing or utilizing the first hop of the latent multi-hop reasoning. ## twohopfact dataset to answer our questions with prompts of diverse fact composition types, we construct twohop-fact using well-known named entities in wikidata @xcite and manually selected relations (appendix a). twohopfact consists of 45,595 unique pairs of one-hop and two-hop prompts of 52 fact composition types constructed from the same number of fact triplet pairs ((e 1 , r 1 , e 2 ), (e 2 , r 2 , e 3 )) as in ## first hop of multi-hop reasoning in this section, we answer rq1 of how often an llm performs the first hop of reasoning while processing two-hop prompts. we first introduce en-trec as a metric to approximate the llm's internal recall of the bridge entity upon its descriptive mention in a prompt ( §5.1). next, we propose to measure how often this recall increases when changing the input prompt to indirectly mention the bridge entity ( §5.2). then, we evaluate this using twohopfact and answer rq1 ( §5.3). ## second hop of multi-hop reasoning in this section, we answer rq2 of how often an llm performs the second-hop reasoning while processing two-hop prompts. we view the second hop of reasoning as the llm's utilization of what it knows about the bridge entity's attribute (stevie wonder's mother) to answer the two-hop prompt about the same attribute of the entity referred to by the descriptive mention (the singer of 'superstition"s mother). therefore, when an llm performs the second hop, we expect to see a connection between its recall of the bridge entity (i.e. resolving the first hop) and its similarity in responding to a two-hop prompt and a corresponding one-hop prompt about the bridge entity's attribute, e.g., the two-hop prompt "the mother of the singer of 'superstition' is" and the one-hop prompt "the mother of stevie wonder is". namely, the more strongly the model recalls the bridge entity (e.g., stevie wonder) while processing the two-hop prompt, the more similar the completion of this prompt should be to the completion of the one-hop prompt. in the following, we describe our approach for testing how often such a causal connection exists between entity recall and the similarity in the prompt completions, which we refer to as consistency. ## latent multi-hop reasoning in this section, we measure how often llms perform latent multi-hop reasoning while processing the two-hop prompt by combining our answers to rq1 and rq2. for each two-hop prompt, we consider successful outcomes for rq1 (an entity recall increase with entity/relation substitution) and rq2 (a consistency increase with increased entity recall) as evidence of the first and second hops of reasoning, respectively. four possible outcomes arise: (ss) success in both rq1 and rq2 that we view as the multi-hop reasoning; (fs) failure in rq1 but success in rq2; (sf) success in rq1 but failure in rq2; (ff) failure in both rq1 and rq2. there is moderate evidence of the latent multi-hop reasoning, which sometimes becomes stronger with increasing model size. figure 5 shows the relative frequency of the four cases, where green, blue, yellow, and red represent each of the cases of ss, fs, sf, and ff, respectively. llama-2 7b exhibits a relative frequency for successful multi-hop reasoning (green) above random chance (0.25), peaking at 0.46 (entity substitution) and 0.38 (relation substitution). the likelihood of partial multi-hop reasoning (green + blue + yellow) exceeds 0.8 in later layers. while entity substitution results do not show increased multi-hop reasoning with model size (figure 3d ), relation substitution exhibits a scaling trend. from 7b to 70b, the maximum relative frequency increases from 0.38 to 0.43, suggesting that larger models may facilitate multi-hop reasoning with relational changes (figure 3e ). relatively strong evidence supports latent multihop reasoning in up to 23% of fact composition types. considering 0.8 2 = 0.64 as the threshold, with respect to llama-2 7b-13b-70b, 7/3/12 types exceed the threshold with entity substitution and 3/3/9 types do so with relation substitution. the maximum frequency of "anthem of capital's country" ("the national anthem of the country led by president lazarus chakwera is named") exceeds let the increase of the entity recall with the input substitution for the first hop reasoning be the success case of rq1, and the increase of the consistency score with the increased entity recall for the second hop reasoning be the success case of rq2. the green, blue, yellow, and red bars show the cases of ss (success-success), fs, sf, and ff for rq1 and rq2, respectively. we manually set the value of the last layer as 0.5 multiplied by the relative frequency for rq1 because the intervention does not affect the consistency at that layer. ## discussion and conclusion our work studies the latent multi-hop reasoning abilities of llms. we find strong evidence of latent multi-hop reasoning for certain fact composition types with the reasoning pathway utilized in more than 80% of the cases. however, the utilization is highly contextual; there are also fact composition types where we see weak or almost no evidence of reasoning. the evidence of second and multi-hop reasoning across the whole set of prompts is rather moderate and only substantial in the first hop. moreover, while we see a clear scaling trend with the first hop of the latent multi-hop reasoning pathway with increasing model size, we do not see such scaling evidence for the second-hop reasoning pathway. this could be the reason behind the observation of ofir @xcite that the compositionality gap (the ratio of how often models can correctly answer all sub-problems but not generate the overall solution) does not decrease with increasing model size. although our analysis is based on llama-2 family of models of up to 70b parameters, our findings suggest potential limitations in the current scaling paradigm for promoting latent multi-hop reasoning. thus, we may need to study the choice of pretraining data, loss functions that promote knowl-edge retrieval and utilization, or model architectures with a stronger inductive bias towards internal knowledge representation for llms' stronger latent reasoning abilities. however, analyzing the subset of prompts with strong evidence of multihop reasoning with respect to pretraining dynamics and data may give insights into the emergence of such abilities even in the context of the current pretraining and scaling paradigm. overall, our findings advance the understanding of llm capabilities and can guide future research aiming to promote and strengthen latent multi-hop reasoning which is relevant for parameter efficiency, generalization, and controllability.