A nchor C o T : Anchors Pave the Way for Multi-hop Reasoning (2025)

Abstract: Large Language Models (LLMs) have made substantial strides in a broad array of natural language tasks. Recently, LLMs have demonstrated potential reasoning capabilities through prompt design, such as the Chain of Thought (CoT). Despite their superiority in question answering, LLMs still face challenges in answering questions that require multi-hop reasoning, often generating unreliable reasoning chains during answer generation. To improve LLMs’ performance in multi-hop reasoning, we introduce a novel reasoning approach, AnchorCoT, designed to assist LLMs in answering questions involving complex logical reasoning steps. AnchorCoT first predicts key entities which work as important “anchors” to guide the reasoning process and then employs a novel ranking algorithm to ensure the logical sequence of the predicted answers.We implement AnchorCoT on Qwen2.5-7B/14B and GPT-4o and evaluate our method on widely used multi-hop reasoning datasets, including HotpotQA, 2WikiMultiHopQA, and MuSiQue-Ans. The experimental results show that AnchorCoT outperforms existing methods in multi-hop question reasoning and provides more accurate reasoning results in multi-hop question answering tasks.

## introduction large language models (llms) have demonstrated impressive in-context learning abilities across a range of natural language processing (nlp) tasks @xcite @xcite , such as information retrieval @xcite , relation extraction @xcite ) and question answering @xcite . recently, the reasoning ability of llms @xcite has garnered growing attention due to its critical role in solving problems with complex logic. despite the significant success of standard large language models (llms) in tackling questionanswering tasks as demonstrated in various stud-ies @xcite @xcite , their performance tends to falter on complex reasoning tasks that necessitate multiple logical reasoning steps @xcite . llms have a tendency to generate hallucinations during the answer generation process in multi-hop reasoning tasks, often yielding responses that seem plausible but lack factual substantiation @xcite . current llms in multi-hop reasoning task generally follows chain-of-thought strategy. common strategies include instruction prompting @xcite @xcite , reasoning path searching @xcite @xcite , and majority voting @xcite . however, the generated reasoning chains often still exhibit hallucinations due to insufficient or weak supervision during the generation process. instruction prompting offers indirect and uncontrollable guidance, which makes the guidelines unreliable and difficult to assess their correctness. both reasoning path searching and majority voting approaches generate reasoning steps sequentially, where an error in one step can propagate and amplify in subsequent steps, leading to a cascading effect that ultimately derails the entire reasoning chain. figure 1 displays an qa example which requires 3-hop reasoning. for human, they usually split the question into sub-questions: who is the performer, which record label does the performer belongs to and finally who is the ceo of the record label. by answering each sub-question in order, the final answer can be concluded; for llm enabled with chain-of-thought strategy, it does output the reasoning process. however, it gives wrong answer for who is the performer of the song "let me love you tonight". as a result, the subsequent reasoning process is incorrect; inspired by human reasoning process, we propose a novel chain-of-thought reasoning approach anchorcot, which provides direct step-wise guidance for multi-hop reasoning generation. our approach first predicts the key entities which are critical to guide the reasoning process. we define these entities as "anchors". as depicted in figure 1 , we mine three related key entities: "pure prairie league", "peter edge" and "pca records" from the background context and the question. then we employ a ranking algorithm to improve the logical order of these anchors. finally, the anchors will serve as step-wise direct guidance for chain-of-thought reasoning generation. we test our approach on commonly used multi-hop reasoning datasets, including hotpotqa @xcite , 2wikimultihopqa @xcite and musique-ans @xcite datasets. experiments are conducted on qwen2.5-7b/14b (qwen team, 2024) and gpt-4o (openai, 2024) model. results demonstrate that our approach show superiority over not only existing instruction prompting methods, but also reasoning path searching and domain-specific approaches. our contributions can be summarized as follows: our approach excels not only on gpt-4o, but also on opensource large model qwen2.5-7b/14b. results demonstrate that our approach surpasses existing cot approaches on both exact match and f1 score metrics. ## methodology as shown in figure 2 , we introduce anchorcot, a novel reasoning framework with anchors as stepwise reasoning guidance. the approach involves three steps: anchor prediction, anchor ranking and answer generation. the model will first predict the intermediate answers as anchors, which will serve as the guidance in reasoning generation. second, the anchor ranking algorithm will rerank the model in logic order. finally, the llms generates reasoning steps under the guidance of anchors generated in the first step using chain-of-thought strategy. please refer to appendix e for detailed case study. ## conclusion in this paper, we introduced anchorcot, a novel chain-of-thought designed for multi-hop question answering. our approach facilitates logical reasoning chain generation by providing intermediate answers as step-wise guidance. we conduct extensive experiments on musique, hotpotqa and 2wikimultihopqa datasets on both open-source models and gpt-4o. our approach exhibits su-perior performance over other existing multi-hop reasoning methods, including instruction prompting, reasoning path searching and domain-specific methods. we hope our method will be applied in question answering to improve llm performance in various multi-hop reasoning tasks.