Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning (2023)

Abstract: In open-domain question-answering (ODQA), most existing questions require single-hop reasoning on commonsense. To further extend this task, we officially introduce open-domain multi-hop reasoning (ODMR) by answering multi-hop questions with explicit reasoning steps in open-domain setting. Recently, large language models (LLMs) have found significant utility in facilitating ODQA without external corpus. Furthermore, chain-of-thought (CoT) prompting boosts the reasoning capability of LLMs to a greater extent with manual or automated paradigms. However, existing automated methods lack of quality assurance, while manual approaches suffer from limited scalability and poor diversity, hindering the capabilities of LLMs. In this paper, we propose Self-prompted Chain-of-Thought (SP-CoT), an automated framework to mass-produce high quality CoTs of LLMs, by LLMs and for LLMs. SP-CoT introduces an automated generation pipeline of high quality ODMR datasets, an adaptive sampler for in-context CoT selection and self-prompted inference via in-context learning. Extensive experiments on four multi-hop question-answering benchmarks show that our proposed SP-CoT not only significantly surpasses the previous SOTA methods on large-scale (175B) LLMs, but also nearly doubles the zero-shot performance of small-scale (13B) LLMs. Further analysis reveals the remarkable capability of SP-CoT to elicit direct and concise intermediate reasoning steps by recalling ~50% of intermediate answers on MuSiQue-Ans dataset.

## introduction open-domain question-answering (odqa) is a longstanding and challenging task which addresses factoid commonsense questions without specific contexts provided. while existing works in odqa primarily focus on resolving questions that mostly require single-hop reasoning, there is a burgeoning interest in multi-hop question-answering (mhqa), which aims to derive the correct answer through multi-step reasoning over a collection of candidate articles @xcite . yet, a significant disparity exists between such scenarios and real-world applications, since the latter often lacks an explicit set of candidate articles provided by users. in light of this, we officially introduce open-domain multihop reasoning (odmr) as a progression task of odqa, which requires mhqa with explicit rationales in open-domain setting. for odmr, an emerging approach is to leverage large language models (llms) due to the vast knowledge stored within their numerous parameters. in recent years, llms have shown powerful reasoning and instruction-following capabilities, such as @xcite , palm @xcite and instructgpt @xcite . after extensive training on vast corpora of textual data, llms prove to be zero-shot reasoners on complex reasoning tasks by breaking down multi-step questions into intermediate ones for step-by-step reasoning before producing the final answer @xcite . such series of intermediate reasoning steps is known as chain-ofthoughts (cots) @xcite . cots often serve as in-context demonstrations for in-context learning (icl) @xcite , which enables llms to generate outputs that are formally consistent with a target task via a few reference examples provided as prompt. manual-cot @xcite adopt manually designed cots as in-context demonstrations to improve the reasoning performance of llms. however, it demands delicate and meticulous design by humans, and the demonstrations are the same for each question, which may be sub-optimal. zero-shot-cot @xcite was proposed to trigger automated cots by certain specific prompting techniques, such as "let's think step by step:". @xcite proposed auto-cot, an automated framework to mass-produce cots and build in-context demonstrations. however, previous works have not fully leveraged the strong instruction-following and zero-shot reasoning capabilities of llms. in this paper, we propose self-prompted chainof-thought (sp-cot), an llm-only framework to mass-produce high-quality cots for odmr. in general, sp-cot introduces an automated generation pipeline of odmr datasets, an adaptive sampler for cot selection and self-prompted inference via icl. the automated odmr datasets are mhqa datasets without candidate contexts, yet including multi-hop questions with six types of complex reasoning chains and step-by-step decomposition. each intermediate qa step is equipped with a short explanation to justify the answer. by leveraging the icl ability of llms, our method is generally effective on llms of different scales. we evaluate our method on four mhqa datasets in an open-domain setting: complexwebquestions (cwebq) @xcite , hot-potqa @xcite , 2wikimultihopqa (2wiki) @xcite and musique-ans (msq) @xcite . extensive experiments show that our proposed sp-cot not only significantly surpasses the previous sota methods on largescale (175b) llms, but also nearly doubles the zero-shot performance on small-scale (13b) llms in odmr. further analysis reveals the outstanding capability of sp-cot to elicit direct and concise intermediate reasoning steps by recalling ∼50% of intermediate answers on msq dataset. our contributions can be summarized as follows: our code and datasets are publicly available at https://github.com/noewangjy/sp-cot . ## experiments our research questions (rqs) are: rq1: to what extent can sp-cot boost the llms on our four odmr benchmarks, compared with other llm-only methods? ## analysis in this section, we explore the choices of the sampling methods and the number of demonstrations. then we examine the quality of the intermediate reasoning steps elicited by sp-cot and the quality of self-generation data. unless otherwise specified, we use chatgpt (gpt-3.5-turbo-0301) to conduct analysis on the same subsets mentioned in rq2 settings.