Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? (2024)

Abstract: State-of-the-art Large Language Models (LLMs) are accredited with an increasing number of different capabilities, ranging from reading comprehension over advanced mathematical and reasoning skills to possessing scientific knowledge. In this paper we focus on multi-hop reasoning—the ability to identify and integrate information from multiple textual sources.Given the concerns with the presence of simplifying cues in existing multi-hop reasoning benchmarks, which allow models to circumvent the reasoning requirement, we set out to investigate whether LLMs are prone to exploiting such simplifying cues. We find evidence that they indeed circumvent the requirement to perform multi-hop reasoning, but they do so in more subtle ways than what was reported about their fine-tuned pre-trained language model (PLM) predecessors. We propose a challenging multi-hop reasoning benchmark by generating seemingly plausible multi-hop reasoning chains that ultimately lead to incorrect answers. We evaluate multiple open and proprietary state-of-the-art LLMs and show that their multi-hop reasoning performance is affected, as indicated by up to 45% relative decrease in F1 score when presented with such seemingly plausible alternatives. We also find that—while LLMs tend to ignore misleading lexical cues—misleading reasoning paths indeed present a significant challenge. The code and data are made available at https://github.com/zawedcvg/Are-Large-Language-Models-Attentive-Readers

## introduction recent developments in the field of language modelling and the introduction of open @xcite and proprietary (openai, 2023) large language models (llms) have undeniably advanced the state of the art in natural language processing (nlp). llms have been credited with vari-original q: who created the 2003 remake of the 1983 overhead view, vehicular combat game developed by bally midway? hotpotqa paragraph 1: highway pursuit is a computer game remake of spy hunter created by adam dawes ous understanding and reasoning capabilities, ranging from arithmetic @xcite , deductive @xcite and formal @xcite reasoning and possessing general @xcite , and domain-specific @xcite knowledge. due to their size and generalisation capabilities @xcite , their evaluation on benchmarks requiring such types of reasoning is typically performed in zero-or few-shot settings on many nlp tasks, without the need for fine-tuning datasets. these zero-and few-shot capabilities seem to alleviate one of the weaknesses identified with the previous generation of fine-tuning based nlp architectures such as transformer-based @xcite , and pre-trained language models @xcite -the reliance on data-set specific "artefacts" @xcite and, as a consequence, lack of generalisation beyond specific datasets. for example, in one of the popular reading comprehension and reasoning benchmarks @xcite , the majority of questions starting with "how many" can be answered correctly with "2". following standard finetuning practice and splitting data in train and test randomly, such a simple heuristic will be present in both training and evaluation data, so a fine-tuned model will learn it and obtain high scores, without necessarily performing reasoning. llms seemingly circumvent this issue, as they are not finetuned on benchmark data. as such, they are not exposed to simplifying dataset artefacts by design, and it is reasonable to assume that they do not learn to exploit them. however, while there is a growing body of work investigating the strengths and limitations of llms @xcite , little research has been carried out to validate this assumption, and to investigate whether and to what extent llms inherit the "dataset artefact" weaknesses of their fine-tuned predecessors. this is an important research question to pursue, motivated by recent findings on benchmark leakage into pre-training or instruction-tuning data @xcite , which invalidate the zero-shot setting and potentially allow llms to learn such dataset artefacts. another line of research suggests that llms tend to "overreason" @xcite , perhaps due to "sycophancy" @xcite , i.e., the tendency to generate the presumably preferred answer over the correct one, leading to complicated reasoning where none is required. in this paper, we turn our attention to the wellstudied capability to perform multi-hop reasoning and reading comprehension-that is, to integrate textual information from multiple different source documents. typically, this capability is evaluated by asking questions where the necessary information to arrive at the correct answer is spread across multiple documents @xcite @xcite . it is important to understand to what extent nlp methods possess this capability, as it is required for many real-world tasks, such as retrieval-augmented generation @xcite when summarising retrieved documents, and because it is a necessary prerequisite to human-level reading comprehension @xcite . previous work has shown that nlp architectures might possess inadequate capabilities to perform multi-hop reasoning @xcite . however, these findings were established before the advent of large language models. to have a clear understanding of the limitations of the capabilities of state-of-the-art research, it is crucial to re-investigate these claims with the current generation of llm-based approaches @xcite . while there is vivid research on (open-book) multi-hop reasoning capabilities of llms @xcite @xcite , how well they perform when presented with multiple, seemingly plausible multi-hop reasoning paths remains unclear. to address this gap, we focus on the capability of llms to perform multi-hop reasoning when multiple seemingly plausible answers are present, where only minor details invalidate the alternative. we show that existing methods-calibrated to evaluate pre-llm architectures-are inadequate to evaluate llms, and that llm reasoning failures are indeed distinct from their fine-tuned plm predecessors. we present a methodology to generate challenging examples with "plausible distractors" to evaluate llms' capabilities to perform multi-hop reasoning when presented with seemingly correct, but ultimately wrong and thus distracting evidence. our results show that the reasoning capabilities of a range of open and proprietary llms, including gpt-4, are affected by these "plausible distractors". ## related work it has been shown that basic pattern matching @xcite and one-hop @xcite ) models can solve a large proportion of questions in multi-hop question answering datasets, presumably because the answer sentence often contains keywords common with the question, thus negating the need to follow a reasoning path and attend to multiple documents. particularly hot-potqa @xcite , due to its multi-hop question design, was the subject of multiple studies. approaches architecturally incapable of multi-hop reasoning still achieved close to state-of-the-art performance @xcite , suggesting questions answerable in such a way do not necessitate multi-hop reasoning. in light of these results, several adversarial attacks have been proposed to check whether the dataset evaluates multi-hop reasoning without exhibiting "shortcuts", by ensuring that the correct answer can only be procured if the evaluated model can retrieve and combine information from distinct reasoning hops. @xcite elicited distracting paragraphs by using the titles of the gold paragraphs and the answer, which are subjected to phrase-level perturbations and word replacement, thus creating a distracting paragraph. others decomposed the multi-hop questions in multiple single questions @xcite @xcite (e.g. decomprc in figure 2 ) showed that the-typically bert-or other plmbased-fine-tuned sota models struggled to answer both sub-questions correctly when answering the complete question, or were distracted by their alterations, suggesting the presence of reasoning shortcuts @xcite . by design, these methods bear only negative predictive power @xcite : failing to see a performance drop does not imply that the model performs the evaluated capability well, but rather that the methodology might have limited suitability to evaluate the investigated phenomenon, i.e., multihop reasoning. as the methodologies presented above focus on fine-tuned models, they assume that multi-hop reasoning is circumvented through simple, lexical similarity-based methods like word matching. for example, @xcite do not consider that their generated paragraphs are isolated, as they contain no explicit reference to other paragraphs in the context, such as a shared named entity. meanwhile, @xcite * only add a single distracting sentence. thus, simple word matching, which ensures that the final answer is of the same entity type as in the question, can often lead to the correct answer. this might not be sufficient for llms, as they-due to their size and emergent capabilities-might circumvent multi-hop reasoning by exploiting more subtle textual cues. indeed, in our empirical study, we show that existing methods, due to these limitations, do not adequately test an llm's reasoning capabilities. therefore, to analyse an llm's ability to reason more adequately, we go beyond the state of the art and introduce a novel method to more effectively evaluate the multi-hop reasoning capabilities of llms. specifically, we ensure the continuity of seemingly plausible alternative reasoning paths, which lead to answers that are ultimately wrong. to succeed, the model is required to pay close attention to small yet important details in the questions and paragraphs. this ability is important practically, for example when an llm is prompted to evaluate/summarise the outcome of a debate, where both sides will * no public code/dataset was made available present plausible arguments with only one being ultimately correct @xcite . with llms increasingly used to judge and improve (other) llms' potentially similar outputs on the same topic @xcite , it is important to establish, if they possess the necessary prerequisites to do so. more broadly, similar to other works in this line of research, we look at linguistic competence rather than performance @xcite : if we accredit multi-hop reasoning capabilities to llms, then, similar to humans, we expect them to exhibit these capacities not only in the majority of cases but in edge case scenarios as well, such as when presented with seemingly plausible alternate reasoning paths. ## methodology in this section, we describe our approach to evaluating the multi-hop reasoning capabilities of llms. we do so by creating "distractor" paragraphs that present seemingly plausible yet incorrect alternative paths in the reasoning chain while ensuring that this process doesn't affect the final solution. first, the question is treated as a two-hop question and converted into two sub-questions. this is done to be able to branch out alternative reasoning paths from each of the sub-questions. the sub-questions are analyzed to identify modifiable portions, which are then manipulated to create "distractor" sub-questions that lead to a different answer and thus a different reasoning chain, which is ultimately wrong, as the models are presented with the original, unmodified question. the "distractor sub-questions" are finally used to generate "distractor paragraphs" containing "distractor answers" utilizing an llm. the method comprises three main steps: i. acquiring the main entity, ii. extracting its modifiable details, and iii. creating the distractor paragraphs. question: what year did guns n roses perform a promo for a movie starring arnold schwarzenegger as a former new york detective? sub question 1: which movie stars arnold schwarzenegger as a former new york police detective? sub question 2: what year did guns n roses perform a promo for end of days (answer of the previous question)? figure 2 : example of a decomposed multi-hop question. ## experiment setup first, we investigate llm's capabilities and limitations compared to previous plm-based state of the art. then, we evaluate the multi-hop reasoning capabilities of llms using our proposed methodology. finally, we conduct an in-depth analysis of what makes reasoning hard for llms on our benchmark and conclude by evaluating state-ofthe-art llms and prompting techniques. unless mentioned otherwise, we use the chat models for llama-2. do llms suffer from the same flaws as finetuned models? llama-2-13b @xcite is used as the baseline llm. we evaluate using few-shot prompts, as these allow the model to stick to the expected output format better than zero-shot. this setting is used throughout the paper unless mentioned otherwise. two styles of prompts were used, normal and chain of thought, as per the strategies discussed in @xcite . all reported metrics are measured at token level and averaged across all the instances, following standard evaluation practice @xcite . we test the llms' performance when attacked with adddoc @xcite , an adversarial attack on hotpotqa for bert-based models. this is intended to check an llm's ability to handle "distracting" paragraphs. subqa was used to determine if the models could answer the individual questions before answering the entire question. it is a sample of 1000 questions and their sub-questions from the dev set of hotpotqa, with the sub-questions being human-verified. this allows us to evaluate model consistency in answering both the multi-hop question as well as the individual sub-questions correctly. it also allows us to investigate the opposite: when the (more complex) composite question is answered correctly, but either of the (simpler) decomposed questions is answered wrongly, the model might rely on some reasoning shortcuts, discarding sub-question information. finally, we evaluate if llms can retrieve the correct answer when necessary information from one of the gold paragraphs is missing, using the dire test set @xcite . do llms get distracted by seemingly plausible alternate reasoning paths? as described in section 3, the attack aims to create paragraphs that provide irrelevant information that is closely related to the property/entity being questioned about. here, we evaluate a representative sample of open-source and proprietary llms, specifically, llama-2-13b, llama-2-70b, mixtral-8x7b-instruct-v0.1, gpt-3.5 and gpt-4. to contextualise the performance of llms to their fine-tuned plm counterparts, we also finetune a longformer model on the hotpotqa training set and evaluate it on our proposed benchmark (see appendix for details). based on the chatbot leaderboard @xcite at the time of writing, the best state-of-the-art model was gpt-4. thus we evaluate gpt-4 to investigate how our findings generalise to stronger models. what are the effects of the different parameters? experiments are conducted to check the impact of the method's parameters on the performance of llms. specifically, the different parameters we investigate are: 1) number of "distractor" paragraphs generated, i.e., two or four; 2) whether the distractor paragraphs are generated from the two sub-questions belonging to the same multi-hop question or if the sub-questions belong to two independent multi-hop questions; 3) the type of modifiable portion that is changed in the sub-question, i.e., named entity or not; 4) whether the paragraphs, if not generated from two distinct sub-questions, are both generated from the second sub-question. ## experiment results in this section, we present the results of our experiments, compare them against prior work, and discuss deeper insights. unless otherwise stated, all reported results of the adversarial attack are statistically significant at @xmath0.05, determined by conducting a one-sided student's t-test. ii. reasoning shortcuts using subqa in the controlled setting on the subqa dataset. llama-2 performs much better on the individual sub-questions than the question requiring multihops. this finding, in line with analyses focusing on fine-tuned models @xcite