Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning
Authors: Jinyuan Wang
Junlong Li
hai zhao
Venue: EMNLP 2023 Findings (2023)
Keywords: Chain-of-Thought
Large Language Models
In-context-Learning
Open-domain question-answering
Multi-hop question-answering
TL;DR: Self-prompted Chain-of-Thought as In-context demonstrations to boost Large Language Models' performance for open-domain multi-hop reasoning, which is generally effective on large-scale (175B) and small-scale (13B) LLMs.

Abstract: In open-domain question-answering (ODQA), most existing questions require single-hop reasoning  on commonsense. To further extend this task, we officially introduce open-domain multi-hop reasoning (ODMR) by answering multi-hop questions with explicit reasoning steps in open-domain setting. Recently, large language models (LLMs) have found significant utility in facilitating ODQA without external corpus. Furthermore, chain-of-thought (CoT) prompting boosts the reasoning capability of LLMs to a greater extent with manual or automated paradigms. However, existing automated methods lack of quality assurance, while manual approaches suffer from limited scalability and poor diversity, hindering the capabilities of LLMs. In this paper, we propose Self-prompted Chain-of-Thought (SP-CoT), an automated framework to mass-produce high quality CoTs of LLMs, by LLMs and for LLMs. SP-CoT introduces an automated generation pipeline of high quality ODMR datasets, an adaptive sampler for in-context CoT selection and self-prompted inference via in-context learning. Extensive experiments on four multi-hop question-answering benchmarks show that our proposed SP-CoT not only significantly surpasses the previous SOTA methods on large-scale (175B) LLMs, but also nearly doubles the zero-shot performance of small-scale (13B) LLMs. Further analysis reveals the remarkable capability of SP-CoT to elicit direct and concise intermediate reasoning steps by recalling $\sim$50\% of intermediate answers on MuSiQue-Ans dataset.

Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning
Jinyuan Wangl.3 and Junlong Li and Hai Zhao
'SITU-Paris Elite Institute of Technology, Shanghai Jiao Tong University
Department of Computer Science and Engineering, Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University (steve_wang, lockonn tu. edu. cn, zhaohai@cs. sjtu. edu. cn
Abstract
contexts provided. While existing works in ODQA primarily focus on resolving questions that mostly require single-hop reasoning, there is a burgeoning interest in multi-hop quusstio--asserrigg (MHQA), which aims to derive the correct answer through multi-step reasoning over collection of candidate articles (Mavi et al., 2022). Yet, significant dis- parity exists between such scenarios and real-world applications, since the latter often lacks an explicit of candidate articles provided by users. In light of this, we officially introduce open-domain multi- hop reasoning (ODMR) as a progression task ODQA, which requires MHQA with explicit ratio- nales in open-domain setting.
In open-domain prestion-answering (ODQA), most existing questions require single-hop rea- soning on commonsense. To further extend this task, we officially introduce open-domant multi-hop reasoning (ODMR) by answering multi-hop questions with explicit reasoning steps in open-domain setting. Recently, large language models (LLMs) have found signifi- cant utility in facilitating ODQA without ex- ternal corpus. Furthermore, (CoT) prompting boosts the reasoning capabil- ity of LLMs to a greater extent with manual or automated paradigms. However, existing automated methods lack of quality assurance, while manual approaches suffer from limited scalability and poor diversity, hindering ca- pabilities of LLMs. In this paper. we propose Self-prompted Chain-of- Chain-of-Thought (SP-CoT), an automated framework to mass-produce high quality CoTs of LLMs, by LLMs and for LLMs. SP-CC SP-CoT introduces an automated gener- ation pipeline of high quality ODMR datasets, an adaptive sampler for in contentext CoT selec- tion and prom-pro inference via in-context learning. Extensive experiments four multi- hop question-answeringg benchmarks show that our proposed SP-CoT not only significantly surpasses the previous SOTA methods on large- scale (175B) LLMs, but also nearly doubles the zero-shot performance of small-scale (13B) LLMs.. Further analysis reveals the remarkable capability of SP-CoT to clicit direct and con- cise intermediate reasoning steps by recalling ~50 of intermediate answers on MuSiQue- Ans dataset.
For ODMR, an emerging approach is to lever- age large language models (LLMs) due to the vast knowledge stored within their numerous parame- ters. In recent years, LLMs have shown powerful reasoning and cnrruction-follwiinn capabilities, such GPT-3 (Brown al., 2020), PaLM (Chowd- hery al., 2022) and InstructGPT (Ouyang et al., 2022). After extensive training on vast corpora of textual data, LLMs prove to be zero-shot rea- soners on complex reasoning tasks by breaking down multi-step questions into intermediate ones for step-by-step reasoning before producing the fi- nal answer (Kojima et al., 2023). Such series of intermediate reasoning steps is known chain-of- thoughts (CoTs) (Wei et al., 2023). CoTs often serve as in-context demonstrations for in-context learning (ICL) (Brown et al., 2020), which enables LLMs to generate outputs that are formally consis- tent with target task via few reference exam- ples provided as prompt. Manual-CoT (Wei et al., 2023) adopt manually designed CoTs as in-context demonstrations to improve the reasoning perfor- mance of LLMs. However, it demands delicate and meticulous design by humans, and the demon- strations are the same for each question, which may be sub-optimal. Zero-shot-CoTT (Kojima et al., 2023) was proposed to trigger automated CoTs
1 Introduction
Open-domaie question answering (ODQA) is longstanding and challenging task which addresses factoid commonsense questions without specific
Corresponding author. This paper was partially sup- ported by the Joint Research Project of Yangtze River Delta Science and Technology Innovation Community (No. 2022CSJGG1400)

---

by certain specific prompting techniques, such as "Let's think step by step:". Zhang et al. (2022) proposed Auto-CoT, an automated frame- work to mass- prasuce CoTs and build in-context demonstrations. However, previous works have not fully leveraged the strong cnnttructioo-oolowinn and zero-shot reasoning capabilities of LLMs.
2 Related Works
2.1 Multi-Hop Dataset Creation
Creating an annotated MHQA dataset manually requires significant human resources. Therefore, some researchers are dedicated automating the generation of MHQA datasets. Jiang et al. (2020) elaborated the creation of a multi-hop fact veri- fication dataset from existing HotpotQA dataset. Trivedi et al. (2022) introduced a bottom-up pro- cess to build challenging multi- reading compre- hension QA dataset through meticulous selection and composition of single-hop questions derived from existing datasets. Press et al. (2023) proposed an automatically generated dataset with composi- tional 2-hop questions about celebrities. Neverthe- less, existing approaches are either only partially automated, still requiring crowdsourcing, or they are limited to less complex 1-2 hop questions. In this work, our proposed SP-CoT is capable of au- tomatically generating 2-4 hop questions with six different types of reasoning chains (Figure 6 in Appendix).
In this paper, propose Self- prompted Chain- of of-Thought (SP-CoT), I.MM-only framework mass-produce high-quality CoTs for ODMR. In general, SP-CoT introduces an automated gen- eration pipeline of ODMR datasets, an adaptive sampler for CoT selection and self-prompted in- ference via ICL. The automated ODMR datasets are MHQA datasets without candidate contexts, yet including multi-hop questions with six types of complex reasoning chains and step-by-step decom- position. Each intermediate QA step is equipped with short explanation to justify the answer. By leveraging the ICL ability of LLMs, our method is generally effective on LLMs of different scales.
We evaluate our method on four MHQA datasets in an open-domain setting: Complex WebQues- tions (CWebQ) (Talmor and Berant, 2018), Hot- potQA (Yang et al., 2018), 2WikiMulliHopQA (2Wiki) (Ho et al., 2020) and MuSiQue-Ans (MSQ) (Trivedi al., 2022). Extensive experiments show that our proposed SP-CoT not only significantly surpasses the previous SOTA methods on large- scale (175B) LLMs, but also nearly doubles the zero-shot performance on small-scale (13B) LLMs in ODMR. Further analysis reveals the outstanding capability of SP-CoT to elicit direct and concise intermediate reasoning steps by recalling ~50 of intermediate answers MSQ dataset.
2.2 Chain-of-Thought Prompting
Recent works on CoT prompting can be divided into two research lines. The first is prompting LLMs step by step to leverage their comprehen- sion and reasoning abilities to answer questions. Zero-shot-CoT (Kojima et al., 2023) adopts a two- stage design, which requires LLMs to first generate intermediate rationale and then produce an answer. Wang et al. (2022) introduced iCAP, which iter- atively prompts fine-tuned small-scale LLM to generate CoTs and then combines the generated rationales to formulate answers. Least-to-Most (Zhou et al., 2023) requires LLMs to first decom- pose complex question into sub-questions and then sequentially solve them arrive at the final answer.
Our contributions can be summarized as follows:
1. We introduce automated pipeline gen- erate high-quality ODMR datasets by LLMs, which include 2-4 hop questions with six types of complex reasoning chains.
The second research line focuses on designing effective CoT as demonstrations for ICL to re- lease more powerful reasoning abilities of LLMss Manual-CoT (Wei et al., 2023) was introduced to leverage manually designed CoT as in-context demonstrations to solve arithmetic, commonsense, and symbolic problems. A recent work (Zelikman et al., 2022) shed light on the practicality auto- mate the generation of rationales by LLMs. Subse- quently, Self-Ask (Press et al., 2023) was proposed to construct step-by step demonstrations with ex- plicit decision process and intermediate answers
2. We propose SP-CoT, an automated framework to mass-produce CoTs while ensuring quality and diversity.
3. We conduct extensive experiments to confirm the effectiveness of SP-CoT on four ODMR benchmarks. In ODMR setting, our approach significantly boosts the performance by elicit- ing high-quality intermediate reasoning steps.
Our code and datasets are publicly available at https//gtthbb.commewaaaggy....

---

Generation
Adaptive Col Construction
Self-prompted Inference via In-context Learning
Self-Generated
Question: What is the birth year of the person who remixed the song "Take on Me"?
ODMR Dataset
Answer: Let's think step by step:
Step I: The song "Take on Me" was remixed by John Ratclift.
Eneading
Step John Ratclift's birthdate Octoher
Therefore, the final answer in just one entity is: October 1943
Clustering
Question: What year the person who originally grouped Brahms with Bach and
Beethoven the "Three Be" music make his debut pianist after studying
with
Demos
Auswer: Let's think by step:
Step 1: The commnent grouping Brakams with Bach and Beethoven as the "Three
Bs" of music was originally made by Hans von Billw:
Step 2: Hans von Bülow made debut as planist 1848 after studying with Franz Liszt.
Therefore, final
just catity 1848
Test Sample
Question: Who sing 'Home Alone Tonight" with the performer of "country Man""? Answer: Let's think step by step:
Step I: The performer of "Country Man' is Luke Bryan.
Outpot
Step Lake Bryan sings "Home Alone Tonight" with Karen Fairchild of ittlee Big
Town,
Therefore, the final answer in just one entity is: Karen Fairchild
Figure I: The overall framework of our proposed SP-CoT, including an automated generation of ODMR datasets, an adaptive sampler for CoT selection and self prompted inference via ICL. Texts highlighted in purple refer to questions, in red to previously generated CoT's, in orange answers, and in green newly generated contents.
as COT, Zhang et al. (2022) proposed Auto-CoT, which automatically constructs CoTs via LLMs and adopts clustering methods to dynamically build demonstrations for each question.
paca (Taori et al., 2023) and Vicuna (Chiang et al.. 2023) collect the content generated by GPT-4 (Ope- nAI, 2023) and the corresponding prompts to train smaller-scale LLMs to achieve comparable perfor- mance. Another research line aims to boost the performance of farge-scale LLMs to higher levels by leveraging the self self-generated content. Some works use the self-generation as contexts assist themselves in answering questions, such as elicit- ing intermediate rationales as CoT (Kojima et al.. 2023) or generating background articles for read- ing comprehension (Yu et al., 2023). While oth- ers instruct LLMs to generate demonstrations for ICL during inference (Zhang et al., 2022), such prompting LLMs to generate reliable QA pairs self-prompted in-context demonstrations (Li al., 2022). Our work dedicated extending the second research line to ODMR by leveraging the automated self-generated CoT as in-context demon- strations. Compared to previous works (Kojima et al., 2023; Zhang et al., 2022; Li et al., 2022), our work taps into the potential of self-prompting LLMs with more complicated framework design to solve most challenging task.
However, existing methods have two significant limitations: 1) Over-reliance on the reasoning abil- ities of LLMs. Most methods are reported effec- tive on large-scale LLMs like InstructGPT, while reproducing these methods on small-scale LLMS is quite challenging. 2) Over-confidence on the quality of intermediate results. When prompting LLMs step by step, defects in previous steps may limit the performance of subsequent steps. Simi- larly, while automatically constructing in-context demonstrations, the effectiveness of ICL might be compromised by the unstable quality of CoT's. Ad- mittedly, manually constructed CoTs can ensure quality, yet they face a trade-off between content diversity and costs. To overcome the above draw- backs, our proposed SP-CoT automates CoT gener- ation with quality ensured by leveraging the strong instraction-follooiing capability of LLMs.
2.3 Model Enhancement via LLM Generation
With the powerful capabilities of LLMs on con- tent generation and instruction-followiin one re- cent research direction extensively leverages the content generated by LLMs enhance smaller LLMs. Recent works such as GPTeacher,' Al-
3 Methods
In this section, we elaborate our proposed SP-CoT in three stages (Figure 1):
In the first stage, we prompt LLMs to iteratively generate 2-hop commonsense QA quadruplets with

---

Stage 1: 2-Hop QAs via Self-Generation
Example
Step I: First-Hop Passage Generation
Step.3: 3: Second-Hog Passage Generation
In: Name 10 scientists:
Out: Albert Einstein, Marie Curie, Isaac Newton,
In: Generate a detailed passage from Wikipedia about Princeton.
Out: Princeton is university town in Mercer County, The muscum located on the campus of Princeton University and free the public.
Passage
In: Generate a detailed passage from Wikipediaa ahout Albert Finstein.
Gemannbor theoretical physicist who cm. and died on April 18, 1955, in Princeton. New Jersey.
Out: Albert Einstein
Estract named entities from the pasage
Extract named entities from the passage....
Step 2: Forst-Hop QA Generation
Step 4: Second-Hop QA Generation
Question
Pussage about Albert Einstein: Albert Einstein was German-born New Jersey. Generate a question to which the anewer the entity 'Princeunn.
In: Passage about Princeton:
Princeton is university
town ... the public. Generate question to which auswer is the cntity 'Mereer County:
Forification
Verifieation
Out: What the city where Albert Flistein died?
Out: What is the county that Univensity?
In: Passage about Albert Einstein: Albert Einstein was a German-
In: Passage about Princeton: Princcton is university town to
Explanation
born New Jersey. Question: What is the city where Alhert
the public, Question: What is the county that is home to
Einstcin died? Answer: Princcton According to an evidence from the
Princeton University? Answer: Mereer County According to an evidence from the passage support the answer, rewrite it to make its meaning clear without passage.
passage to support the answer, rewrite it to make its meaning clear passage.
Out: Alhert Einstein was awarded the Nohel Prize... and his services
Out: Princeton is university town in Mercer County, New
to Theoretical Physics.
Jensey, United States
Figure 2: Generation steps for 2-hop QA quadruplets. Each QA quadruplet comprises a question q. its corresponding answer a, and a context passage p. The explanation includes the answer from the passage p to address the question Text highlighted in orange refers to previously generated content, while the response of the LLM is highlighted in green.
context, question, answer and explanation.
to be named entities, we use Spacy² and NLTK (Bird and Loper, 2004) libraries to extract the named entities in the passage p1 as candidate an- swers. For each candidate answer ai, we require the LLM to raise a question qı to which the an- swer is a1 based on the passage pr. To ensure the quality of q1. we employ a double-check process, where we demand the LLM to answer the gener- ated question Qı given the context p1 to check if the generated answer a1 is accordant with a₁- Once the generated QA pair passes the double-check, we prompt the LLM to write a short explanation ei for it. Note that the candidate answers must exclude the keyword (o, + kj) because the answer in the first hop will become the keyword for the second hop (k2 = a1 1 # k1). In addition to that, a valid explanation must contain the answer (a1 E e1).
In stage 2, we construct multi-hop reasoning chains by connecting 2-hop QA quadruplets and build an ODMR dataset via composition.
In the last stage, we adopt (lustering-based sam- pling approach to dynamically select and construct in-context demonstrations for inference.
3.1 2-Hop QAs via Self-Generation
In the first stage, we prompt LLMs to iteratively generate 2-hop QA quadruplets with context, ques- tion, answer and explanation, which is illustrated in Figure 2. Inspired by Li et al. (2022), we design a 2-hop commonsense QA generation pipeline, in- cluding the following 4 steps:
Step 1: First-Hop Passage Generation To guar- antee the comprehensive coverage of commonsense knowledge, we manually design 29 diverse topics based on the statistics of TriviaQA (Joshi et al.. 2017). For each topic, we require the LLM to name a certain number of keywords. For each col- lected keyword k,, we ask the LLM to generate a Wiki-style passage p1. Despite some factoid errors (Li et al., 2022), such generated passages contain sufficient factual information to serve as context for QA generation.
Step 3: Second-Hop Passage Generation Before the first-hop answers are used as key- words for second-hop passage generation, we use Spacy to filter out the answers with certain labels (QUANTITY, ORDINAL, CARDINAL, PERCENT, MONEY, DATE, TIME), which are infeasible for Wiki-style passage generation. Given a keyword k2, we repeat the same prompts as described in Step I to generate the passage p2-
Step 4: Second-Hop QA Generation We be-
Step 2: First-Hop QA Generation Given that the answers for commonsense questions are likely
hhtts:///ppcy.io/

---

Stage 2: Multi-Hop QAs via Composition
Generatted Z-Hap QAe
Step Binary Question Generation
O+O
O+ XN
Question: When the International Court of Justice begin its work?
Demov (Tes)
Answer: April 1946
Factoid Questions
Reform the question general interrogutive sentence that can be answered with
x4
Yes: Did the International Court of Justice begin its work in April 1946?
Step 1: Reasoning Chain Compusition
10%
Question: What body of water Venice located on?
Tist
Answer: Adriatic Sea
Sample
O++,,
Reform the question
that can be answered with Yes:
(Yes)
Binery Questions
2
++++
Out: Is Venice located on the Adriatic Sea!
3
Step 4: Multi-Hop Question Generation
4
Take type 3 as coouupee
In: Raw question: How many [What were new students once called by others?] live in [Where was the football lournameat held?]?
Demay
Replace the sentence within with relative clause and make the raw question into a natural question: How many people whose name ... live
x4
5
country ... tournamen??
Raw question: Is it the Mississippi River [What is the American Gothic about!] region in [What country did Nikola Tesla... in New York City?]?
Test Sample
6
Replace the sentence within [] with a relative clause and ... into natural question:
Step 2: Duplication
Own Is the Mississippi River that serves as the castera border American Gothie paiating by that Nikola Tesla emigrated to in 1884 to work for Thomas Edison in New York City
Control
Figure 3: Generation steps for MHQA groups. In step 3 and step 4, we use 4 manually designed demonstrations for ICI.. Each MHQA group includes a multi-hop question, the corresponding answer and decomposed QA quadruplets. Nodes and texts highlighted in red, blue and green successively refer to the last hops, intermediate hops and generated hops. Manually designed texts are highlighted in purple.
gin with extracting candidate answers in the gen- erated passage P2 while blocking the keyword kj and the answer aj (also known as k2) in the first- hop QA to avoid cyclic graphs. For each candidate answer d2 we require the LLM to generate a ques- tion 92 which contains the first-hop answer ai and can be answered by the candidate answer 02. We examine the quality of 42 with the same double- check in Step 2, and ensure the second-hop ques- tion Q2 contains the first-hop answer ai (a) q2) for connected reasoning. Then we repeat the same prompts in Step 2 to generate explanation e2- So far, we have instructed the LLM to generate a 2-hop commonsense QA quadruplet pair, which .1)
multi-hop question Q with 02 valid answer if aj is a named entity and it is mentioned in 22- Such criteria are already satisfied when our 2-hop QA pairs are generated, we use this criterion for connecting more questions. We adopt reasoning graphs with 2-4 hops to build 6 types of multi-hop reasoning chains (Figure 6 in Appendix), and we ensure that in each reasoning chain: 1) the answer aj to intermediate question qi will appear and ONLY appear in its next-hop question q++1 1 to avoid shortcuts; 2) the answer to the last question will NOT appear in any intermediate questions.
Duplication Control Built by rule- based composition, our new dataset has consid- erably similar reasoning chains that have duplicate intermediate questions. To ensure the diversity and simplicity of our dataset, we filter out the reason- ing chains by a preset duplication degree which is defined by the number of questions that co-existed in other chains within the same reasoning type.
Step 2:
is (P1,4), :1 ai, e1) + (p2, 92, a22 e2) with ai E 92. Detailed prompt templates are shown in Figure 2 and Appendix B.
3.2 Multi-Hop QAs via Composition
In stage 2, we construct multi-hop reasoning chains with the connected 2-hop QA quadruplets, which is illustrated in Figure 3. We propose an auto- mated dataset construction pipeline to build ODMR datasets with 2-4 hops, which has the following 4 steps:
Step 3: Binary Question Generation We notice that MHQA datasets also include general interroga- tive questions which should be answered by "Yes" or "No", rather than named entity. Therefore, we leverage the LLM reform the last QA (gm, an) of some reasoning chains binary question with 4 manually designed in-context demonstrations. For each reasoning type, we randomly sample 10% rea- soning chains for positive question generation and
Step 1: Reasoning Chain Composition To con- nect more questions, we follow the composability criteria (Trivedi et al., 2022), that is, two single-hop QA pairs (g), an) and (q2, a2) are composable into

---

Table Comparison of different approaches on four MHQA benchmarks. The fine-tuning methods are fine-tuned on the train split of NQ (Kwiatkowski et al., 2019) dataset. Among them, methods marked with "I" use the Wikipedia dump (Karpukhin et al., 2020) as extra corpus. For retrieval-basce methods, we use fine-tuned DPR (Karpukhin et al.. 2020) to retrieve top-5 documents from Wikipedia as context and employ LLM Reader answer the question based on the context. Methods based on ChatGPT are performed by gpt-3. 5-turbo turbu-bbbb000 version.
Methods
MSQ
HotpotQA EM
2Wiki EM
CWebQ F1
Average
EM
F1
Fine-tuning methods with extra corpus
DPR (Karpukhin et al., 2020)
7.5
16.6
14.7
23.0
6.5
14,6
21.3
30.2
12.5
21.1
RAG (Lewis al., 2020)
3.9
8.2
9.6
15.5
15.7
18.6
13.3
15.8
10.6
14.5
REALM (Guv et al., 2020)
3.7
8.5
15.3
22.0
5.1
9.6
21.1
27.3
11.3
16.9
T5-1IB-SSM (Roberts et al., 2020)
10.6
16.6
15.4
22.4
15.6
20.5
28.5
35.5
17.5
23.8
Retrieval-based methods with LLMSs
DPR' ChatGPT
1.7
4.1
15.8
21.4
10.9
18.1
13.7
18.7
10.5
15.6
DPR Alpaca-13B (Taori et al., 2023)
2.2
8.4
12.0
12.6
21.2
15.9
27.1
10.7
19.5
DPR Vicuna-138 (Chiang et al., 20223)
2.2
7.4
18.6
25.8
23.9
27.6
20.3
27.9
16.2
22.1
DPR PirardLMM133) (Xu et al., 2023)
3.5
10.0
19.9
28.4
22.8
27.5
24.2
32.2
17.6
24.5
DPR InstructGPT (Ouyang al., 2022)
11.6
26.3
23.3
27.1
34.4
41.6
22.2
28.8
LLM-onll methods on ChatGPT
Zero-Sha
3.1
7.3
22.4
30.0
18.7
21.7
31.6
37.5
19.0
24.1
Self- Promptioptn (Li et al., 2022)
2.9
6.2
23.8
31.2
18.9
23.5
26.8
32.6
18.1
23.4
GENREAD (Yu et al., 2023)
8.6
14.6
33.2
42.6
30.4
35.3
33.7
40.1
26.5
33.2
Zeno-shot-CoT (Kojima et al., 2023)
5.0
8.8
22.6
29.6
24.3
27.1
30.3
36.2
20.6
25.4
Auto-CoT (Zhang et al., 2022)
8.1
13.6
26.1
36.3
26.2
30.2
29.9
38.4
22.6
29.6
Manual-CoTT (random) (Wei et al., 2023)
12.3
19.2
32.4
43.7
27.7
34.6
36.6
43.0
27.3
35.1
SP-CoT (Ours)
14.5
22.6
33.2
42.9
30.1
34.7
37.5
43.6
28.8
36.0
10% for negative ones. Then reform a new reasoning chain by the generated binary question together with its previous question hops and add the dataset.
works (Zhang et al., 2022; Li et al., 2022) have shown that clustering based methods benefit from the diversity of demonstrations, We adopt a clustering- retrieval approach to adaptively sample in-context demonstrations for the input question. First, all the questions are projected to a high dimension hidden space by encoding with Sentence BERT (Reimers and Gurevych, 2019). Suppose need n in-context demonstrations. Given a test question Q, we use k-means to cluster the question embeddings into clusters and adap- tively retrieve the question with the highest cosine similarity to Q from each cluster.
Step 4: Multi-Hop Question Generation Now we need to generate multi-hop questions, to which the previously generated question chains serve as their intermediate reasoning steps. For each ques- tion chain, we iteratively replace the answer a, to intermediate question gi in the next-hop ques- tion 4i+1 by [4:] until the last question In is re- placed, which indicates relative clause. Then we leverage the LLM to reform it into a natural multi- hop question with manually designed in-context demonstrations.
Build Reasoning Chain For each sampled ex- ample, we sequentially concatenate the explanation from each hop, prefaced by "Step {i):", to con- struct reasoning chain.
After the pipeline above, we construct high quality ODMR dataset with 2-4 hops, including the overall multi-ho hop question, the decomposed reason- ing chains with detailed QA quadruplets. With the double-check in generation and the composability criteria, we automatically build a high quality new dataset. Detailed prompt templates are presented in Figure 3 and Appendix B.
Experiments
Our research questions (RQs) are:
RQI: To what extent can SP-CoT boost the LLMs on our four ODMR benchmarks, compared with other L.LM-only methods?
3.3 Adaptive In-context Demonstration
RQ2: Is SP-CoT generally effective on recent popular instruction-followii LLMs?
this stage, we sample multi-hop questions from our generated ODMR dataset as in-context demon- strations.
To this end, we conduct experiments on four MHQA datasets that require complex multi-step reasoning and compare different methods across
Clustering-basee Retrieval Some previous

---

Table 2: The performance (EM) our method on recent popular LLMs. We use text-davinci-003 for InstructGPT.
On small-scale LLMs, SP-CoT nearly doubles the average zero-shot performance four MHQA benchmarks.
Model
Size
Method
MSQ
HotpotQA
2Wiki
CWebQ
Mean
Boost
Alpaca (Taori ct al., 2023)
13B
Zero-shot
0,9
9.5
12.5
12.6
8.9
Alpaca (Taori et al., 2023)
13B
SP-CoT
5.3
19.8
17.8
26.5
17.4
8.51
Vicuna (Chiang et al., 2023)
13B
Zero-shot
2.3
11.5
18.1
18.3
12.6
Vicuna (Chiang al., 2023)
13B
SP-CoT
8,5
23.3
22.0
32.2
21.5
8.91
WizardLM (Xu et al., 2023)
13B
Zero-shot
2.0
10.3
13.6
17.5
10.9
Wizardl M (Xu et al., 2023)
13B
SP-CoT
7.3
24.0
27.2
31.3
22.5
11.61
InstructGPT (Ouyang et al., 2022)
175B
Zero-shot
4.4
25.6
22.8
38.6
22.9
InstructGPT (Ouyang et al., 2022)
1758
SP-CoT
14.8
37.4
33.7
46.6
33.1
10.21
different LLMs.
davinci-003 model, which performs at a similar capability level to text-davinci -003 for infer- ence. We use the whole development set of each dataset in our experiments.
4.1
Benchmarks and Evaluation Metrics
We choose the following four MHQA datasets: CWebQ, HotpotQA, 2Wiki and MSQ. We set them ODMR benchmarks by taking only the question and the answer in each example. Dataset introduc- tion and statistics are detailed in Appendix A.
For RQ2, not only test InstructGPT (text- davinci-003), but also employ three smaller-scale (13B) LLMs: Alpaca (Taori et al., 2023), Vi- cuna (Chiang et al., 2023) and WizardLM (Xu et al., 2023), which are LLaMA (Touvron et al., 2023) models fine-tuned on different large-scale instruction-followingg datasets. To save computa- tional cost, we conduct this experiment on subsets of the four datasets by randomly selecting 1000 samples from the test sets.
We adopt the exact match (EM) and FI scores as our evaluation metrics. Based on the evaluation script of Karpukhin et al. (2020), we add a prepro- cessing step which ignores the content within "()" and splits the answer strings by certain delimiters to extract multiple answers.
4.3 Experiment Results
4.2 Experiment Settings
The main results of RQI are shown in Table 1. Even with extra corpus, the models fine-tuned on NQ (Kwiatkowski et al., 2019) present poor perfor- mance due to the inherent challenges of MHQA. With the same Retriever model, the performance of retrieval-basse methods depends largely on the LLM Readers. Compared to previous L.M-only works, our SP-CoT significantly outperforms the Auto-CoT by 6.2 EM and +6.4 F1 scores on average and surpasses the previous SOTA method GENREAD (Yu et al., 2023) by +2.3 EM and +2.8 Fl scores average. On the most challenging benchmark MSQ, SP-CoT empowers ChatGPT to outperform other LLM-only methods by decent margin.
For reference, experiment with fine-tuning methods using an extra corpus, which fine- tumed on the training split of NQ (Kwiatkowski et al., 2019) dataset and most of them adopt the Wikipedia dump (Karpukhin et al., 2020) as extra corpus. We also test our implementation of the retrieval-based methods on most recent LLMs for reference. Specifically, we use fine-tuned DPR (Karpukhin et al., 2020) to retrieve top-5 docu- ments from Wikipedia as context and employ LLM Reader to answer the question based on the con- text. Detailed prompt templates and parameter set- tings are provided in the Appendix B.
Unless otherwise specified, we use Sentence- BERT (all-mpnet-base-V) for question encod- ing following previous works. The default number of in-context demonstrations is 8 and the demon- strations are sampled by the maximum cosine simi- larity of questions in each cluster.
We notice that SP-CoT significantly outperforms GENREAD on MSQ, confirming the effective- ness of providing high quality CoTs as in-context demonstrations for complex multi-hop questions. On the other three datasets, SP-CoT delivers com- parable performance with GENREAD. However, GENREAD relies heavily on the generation faith- fulness of L.Mss, which challenging for small- scale LLMs. By breaking down demanding instruc- tions into step-by-step simple ones, our method
For RQ1, we adopt ChatGPT (gpt-3.55 1ept-3.5-turbo 0301) as the LLM to conduct the following exper- iments. According to OpenA1,
0301 is an improvement the InstructGPT text-
Phupss//platoom.oppna.ccm

---

Table The performance (EM) of different methods of demonstration selection. The results from random selection represent the value and standard deviation obtained from runs, each with different seed.
Method
MSQ
HotpotQA
2Wiki
CWebQ
Average
Random
12.0
29.5+0.6
25,0+1.2
34.3=0.9
25.4-0.3
Retrieve
10.5
27.9
24.3
33.5
24.1
ClusterCenter
10.4
26.2
22.8
33.0
23.1
RetrieveIn MerriveenTTypCClie TypeCluster
11.6
28.7
23.5
35.3
24.8
RetrievelnCluster
11.5
30.9
27.8
34.0
26.1
FM
F1
Zero-shot-CoT
Auto-CoT
SP-CoT
35
100
Average Score (%)
30
80
60
25
Score (%)
20
20
15
0
0
2
8
10
Clear
Concise
Comp.
Direct IA Recall
Number of Demonstrations
Criteria
Figure 4: Average EM and F1 scores of different num- bers of in-context demonstrations. The experiments are tested on 1k subsets of four ODMR benchmarks with ChatGPT (gpt-3.55 5-turbo--3e1),
Figure 5: Evaluation results of the CoT generated by three methods. The first four scores are in terms of clearness, conciseness, comprehensibility (Comp.) and directness given by GPT-4 on 50 examples. The recall accuracy of intermediate answers (IA Recall) is reported on the questions that are correctly answered by all 3 methods.
is more applicable to small-scale LLMs, which is validated by Table 2.
Table 2 presents the results for RQ2. Our pro- posed SP-CoT proves be generally effective by significantly boosting the performance all these four LLMs on all four benchmarks. With SP-CoT, the performance of small-scale (13B) LLMs can be boosted to be on par with directly prompting LLMs that are over 10x larger, regardless of the elicited high quality intermediate reasoning steps.
the closest to the centroid in each cluster (Cluster- Center), sampling by the maximum cosine similar- ity in each cluster (RetrievelnnCuster) and sampling the most similar QAs in each cluster in certain reasoning type (RetrievenTYypeClusee) The rea- soning type of the input question is determined by the most frequent reasoning type of its k-nearest neighbors. As indicated Table 3, RetrieveInClus- ter (Li et al., 2022) the best-performing strategy, which is exactly the strategy we adopt in previous experiments.
5 Analysis
In this section, explore the choices of the sam- pling methods and the number of demonstrations. Then we examine the quality of the intermediate reasoning steps elicited by SP-CoT and the quality of self-generation data. Unless otherwise speci- fied, we use ChatGPT (gpt-3. 5-turro--03e -0301) to conduct analysis on the same subsets mentioned in RQ2 settings.
5.2 Impact Demonstration Amount
Providing more in-context demonstrations empiri- cally improves ICL performance; however, also causes increasing computational cost. To this end, we investigate the trade-offs of number of demon- strations and the resulting performance boost. We report the EM and FI scores over the four bench- marks for 2, 4, 6, 8, and 10 in-context demonstra- tions, well as the scores in zero-shot setting. As illustrated in Figure 4, the performance of SP- CoT increases with the number of demonstrations when the count between and 8; however, us- ing 10 demonstrations doesn't 't yield any further
5.1 Methods of Demonstration Sampling
The performance of ICL depends largely the quality of demonstration sampling. We test the effectiveness of the following five strategies: ran- domly sampling (Random), sampling globally by maximum cosine similarity (Retrieve), sampling

---

performance boost. In our main experiments, we opted for as the default number of demonstra- tions, striking a balance between performance and cost.
Acknowledgements
This paper was partially supported by the Joint Research Project of Yangtze River Delta Sci- ence and Technology Innovation Community (No. 2022CSIGG1400) and under the technical sup- port of National Key R&D Program of China (No. 2021 YFC3340700). Our appreciation also extends to Ms. Yangyang Ding, who has been generous with her time and knowledge, offering constructive criticism and enriching discussions.
5.3 Intermediate Reasoning Quality Analysis
Given the high-quality CoTs constructed by our proposed SP-CoT, we investigate the quality of intermediate reasoning steps generated during in- ference. For this analysis, we use the development set of MSQ, as it's the most challenging of the four datasets and offers decomposed step-by-step QAs. We compare the CoTs generated by Zero-shot-CoT, Auto-CoT and SP-CoT during inference. For fair- ness, wC seleet 50 out of total of 59 questions that all of the three methods answered correctly. First, we use GPT-4 to evaluate" the intermediate reasoning steps in terms of clearness, conciseness, comprchensibility and directness separately scale of to 10. Additionally, we compute the re- call accuracy of intermediate answers co-occurring in the reasoning steps of each method. For fair- ness, we only report the intermediate answer recall accuracy of correctly answered questions for each method. As depicted in Figure 5, GPT-4 highly favors our SP-CoT, which achieves nearly a 50% recall accuracy for intermediate answers. This sug- gests that SP-CoT clicits high-quality reasoning steps in terms of clearness, conciseness, compre- hensibility, and directness.
Limitations
Our proposed method (SP-CoT) leverages the strong instructionfoolooinng power of LLMss. Such capability is easy to acquire through instruction fine-tuning for small-scale LLMs (even 7B), how- ever, some LLMs proposed in early years may show poor capability in following human instruc- tions due to lack of corresponding training be- fore their release. Therefore, the performance such LLMs may not be boosted by our proposed SP-CoT. In fact, we did not succeed in boosting the performance of GPT-NeoX by any of Zero- shot-CoT, Auto-CoT and SP-CoT. GPT-NeoX is 20B LLMs released in early 2022, which shows poor instruction followong capability. Please note that neither GENREAD ( (Yu et al., 2023) nor Self- prompting (Li et al., 2022) boosts the performance of GPT-NeoX.
6 Conclusion
It is acknowledged that the efficacy of LLM-only approaches is predominantly reliant on the LLMs themselves. With smaller-scale LLMs, specifi- cally those of 13B scale, our SP-CoT together with other CoT methodologics, demonstrate comparable or similar performance enhancement across four ODMR benchmarks as presented Table 6. The consistent performance of handcrafted CoTs re- mains ambivalent across different LLMs and bench- marks; our empirical observations indicate that Manual-CoT occasionally outperforms SP-CoT, while at other instances, it does not.
In this work, we harness the capabilities of LLMS combined with self-promplee CoTs to tackle the intricate MHQA task within the open-domain con- text, termed ODMR. Our innovative SP-CoT not only sets new benchmark by surpassing preced- ing CoT prompting techniques but also outclasses the erstwhile SOTA LLM-only methodologies in open-domain question question-answering anwwering.. A distinguish- ing feature of SP-CoT is its proficiency in elicit- ing high-caliber intermediate reasoning steps, and its universal efficacy across both large and small- scale LLMs. We anticipate our innovative self- generation pipeline for ODMR to not just be foun- dational for SP-CoT, but also to pave the way for future research, catalyzing shift towards lever- aging self-generation in LLMs, by LLMs, and for LLMs.
Given the potential for LLMs to generate impre- cise information, the process by which our SP-CoT produces datasets might also result in the emer- gence of inaccurate QA pairs well erroneous explanations. Despite the incorporation of a double- check mechanism to ensure data integrity, certain errors and inaccuracies are inevitably present.
'Script from
and mod-
ified.

---

References
pages 6609-6624 6625, Barcelona, Spain (Online). Inter- national Committee on Computational Linguistics.
Steven Bird and Edward Loper. 2004, NLTK: The natu- ral language tooliit. In Proceedings of the ACL In- leractive Poster and Demonstration Sessions, pages 214-217, Barcelona, Spain. Association for Compu- tational Linguistics.
Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Maneesh Singh, and Mohit Bansal. 2020. Hover: A dataset for many-hop fact extraction and claim verification.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Ncelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. 7iegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehen- sion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers), pages 1601-1611, Vancouver, Canada. Association for Computational Linguisties.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1769-6781 Online. Association for Computational Linguistics.
Wei-Lin Chiang. Zhuohan Li. Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang. Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2023. Large lan- guage models are zero-shot reasoners.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung. Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaram, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontaek Lim, Barret Zoph. Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov. Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Heiie--eelttern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- cling with pathways.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Fistein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wi Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452-466.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis. Wen-tau Yih, Tim Rock- täschel, et al. 2020. aeurieeel-augmmented generation for knowledge-intensive nlp tasks. Advances in Neu- ral Information Processing Systems, 33:9459-9474.
Junlong Li, Zhuosheng Zhang, and Hai Zhao. 2022. Self-prompting large language models for open- domain qa. arXiv preprint arXiv:2212.08613
Yu Gu, Sue Kase, Michelle Vanni, Brian Sadler, Percy Liang, Xifeng Yan, and Yu Su. 2021. Beyond i.i.d.: Three levels of generalization for question answering on knowledge bases. In Proceedings of the Web Conference 2021. ACM.
Vaibhav Mavi, Anubhav Jangra, and Adam Jatowt. 2022. A survey on multi-hop question answering and gen- eration.
OpenAL. 2023. Gpt-4 technical report.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasu- pat, and Ming-We Wei Chang. 2020. Realm: Retrieval- augmented language model pre-training.
Long Ouyang, Jeff Wu, Xu Jiang. Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray. John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller. Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback.
Xanh Ho. Anh- AnhaKhoa Duong Nguyen. Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi- hop QA dataset for comprehensive evaluation of reasoning steps. Proceedings of the 28th Inter- national Conference on Computational Linguistics,

---

Ofir Press, Muru Zhang. Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models.
In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2369-2330, Brussels, Belgium. Association for Com- putational Linguistics.
Nils Reimers and Iryna Gurevych. 2019. Sentence- BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP- (EMNLP-IJCNLP), pages 3982-3992, Hong Kong, China, Association for Com- putational Linguistics.
Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023. Generate rather than retrieve: Large language models are strong context generators. International Confer- ence for Learning Representation (ICLR).
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning.
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the param- the
eters of language model? In Proceedings 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5418 5418-544 8-5426, Online. Association for Computational Linguistics.
Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models.
Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-bass for answering complex questions.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le. and Ed Chi. 2023. Least-to-most prompting enables complex rea- soning in large language models.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction following Ilama model. https:// gi thub. cww/
A Datasets
A.1 Introduction
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Aune Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models.
HotpotQA (Yang al., 2018) HotpoiQA is widely used dataset for multi-hop question- answering (MHQA), which contains 113k multi- hop questions in natural language. The questions are collected by crowdsourcing based on Wikipedia articles with human annotated supporting evidence and answers.
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multi-
hop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539-554.
2 2WikiMultiHopQAA (Ho et al., 2020) 2Wiki- MultiHopQA s a recently proposed large-scale MHQA dataset, which contains over 192k samples constructed jointly from Wikipedia and Wikidata. MuSiQue-Ans (Trivedi et al., 2022) MuSiQue- Ans (MSQ) is recent challenging MHQA dataset created via single-hop question composition. It includes 25k 2-4 hop questions with six differ- ent composition structures. Although MSQ is composed from existing datasets, poses 3x the human-machine gap with substantially lower dis- connected reasoning score.
Boshi Wang, Xiang Deng, and Huan Sun. 2022. Itera- tively prompt pre-trained language models for chain of thought.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Lee, Fdd Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian lehter, Fei Xia, Ed Chi, Quoe Le. and Denny Zhou. 2023. ofain-of-thhughle prompting elic- its reasoning in large language models.
Complex WeQQusxWesQuestionss (Talmor and Berant, 2018) Complex WebQuestions is manually gen- erated MHQA dataset of 35k QA pairs. CWebQ is generated by rephrasing questions generated by machine from existing dataset.
Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang, 2023. WizardIm: Empowering large language models to follow complex instructions.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotporQA: A dataset for diverse, explainable multi-hop question answering.
A.2 Statistics
The statistics of four datasets are shown in Table 4.

---

Table 4: Statistics development set of four MHQA benchmarks, average tokens for questions, answers and average number of reasoning steps. CWebQ does not provide supporting evidence or question decomposition.
B.3 Binary Question Generation
Question: (gn A'nAnwwer: (ap AnReform the question to general interrogative question that can be answered with yes:
MSQ
HotpotQA
CWebQ
2Wiki
len.
18.11
15.83
13.37
11.98
Question: (gn AnAnswer: (an \nReform the question to general interrogative question that can be answered with
A len.
2.8
2.46
2.42
2.41
Steps
2.65
2.68
2.47
Size
2417
7405
3519
12576
B.4 Multi-Hop Question Generation
B Prompt Templates
Raw question: (gn AnReplace the sentence within with relative clause and make the raw question into a natural question:
B.1 First-Hop QA Generation
We following the notations described in Section 3.
The templates are:
B.5 CoT Construction
1. Name (Number) [Topic):
Suppose q" the generated multi-hop question, ej denotes the explanation from intermediate hop (gi aj ej), a* the answer the last hop (a = an). The template is:
2. Generate Wikipedia passage about (kr
3. Passage about {kj JAn(P1 /nnnGenerate
question to which the answer is the entity (a1).
Question: (q" AnAnswer: Let's think step by step:'nStep 1: {er JnnStep 2: (e2/In.. There- fore, the answer in just one entity is: (a")
4. Passage about (k1 ( /n'nQuestiion:'n (4i /nnnnExtacct the answer directly from the passage in less words as possible.
B.6 Inference
B.6.1 Zero-shot
5. Passage about [k] ( Mannn Question:An (q) \n'nAnswer:ln/ [1) AnhnAccording to an evidence from the passage support the an- swer, rewrite it to make its meaning clear with- out passage.
Given a question Q, the inference template is:
Answer the following question with just one entity:lnQuestion: (Q)saAnwwr::
B.6.2 SP-CoT
B.2 Second-Hop QA Generation
Suppose we have the input question Q and 2 demonstrations (q1, ri, a1), (92, r2, 02), where Ii rss 0g denote the question, CoT and answer of the 7th demonstration. The inference template is:
1. Generate a Wikipedia passage about (k2).
2. Passage about (k2 ) /nnnGeneaate question that meets the following conditions:
1. contains the term '(kg)' in question, 2. the answer is [.2).3. 3. Avoid the following entities in the question: )
Question: {q1 Mafri MulnQuestion: (92 An (r2) InnnQuusiio:: {Q)nnAnswer Let's think step by step:An
3. Passage about (Aog (p \ninQuestion:'n (42) /nnnExtract the answer directly from the passage in less words possible.
Experiment Settings
C.1 Hyperparameters
4. Passage about [k2]:n (p2) An'n Question:An {q) Annnnnswwrrru (a \n'nAccording an evidence from the passage support the an- swer, rewrite it to make its meaning clear with- out passage.
This section is the experiment settings on ChatGPT (gpt-3.5- 3.-t-..botobbo001111 only, for more settings of other L.MMs used in our experiments, please see our code. Our code and datasets are publicly available at

---

Table 5: Performance (EM/FI) of additional CoT variants. In our experiment, Manual-CoT (Cherry-Pick) adopts 8 chemry-icckdd questions and their CoTs manually writen by the authors. The results of Manual-CoT (Random) report the mean EM scores of randomly selected questions and theirs manual CoTss for experiments across the 4 benchmarks. Self- cofssConistency (Wang et al., 2023) is based on SP-CoT with responses for each question.
Method
MSQ
HotpotQA
2Wiki
CWebQ
Average
Zero-shot
3.177.3
22,4/30.0
18.7/21.7
31.6/37.5
19.0/24.1
7ero-shot-CoT (Kojima al., 2023)
5.0/8.8
22.6/29.6
24.3/27.1
30.3/36.2
20.6/25.4
Manual-CoT (Wei al., 2023) (Random)
12.3/19.2
32.4/43.7
27.7/34.6
36.6/43.0
27.3/35.1
Manual-Con CoT (Wei al., 2023) (Cherry-Pick,
13.7/21.9
33.9/44.7
31.5/38.6
37.2/44.0
29.1/37.3
Auto-Cor CoT (Zhang al., 2022)
1/1.63
26.1/36.3
26.2/30.2
29.9/38.3
22.6/29.6
SP-CoT (Ours) SP-CoT Self-Consistency (Wang al., 2023)
14.5/22.6
33.2/42.9
30.1/34.7
37.5/43.6
28.8/36.0
18.3/28.3
-/-
+/-
47.1/54.0
+/-
Table 6: The performance (EM) of CoT methods with recent popular LLMs on subsets of test sets. We use 3. 5- 0000 for ChatGPT and dext-davinci-oo for InstructGPT. On smaller-scale (13B) LLMs, CoT methods achieve comparable performance boost on four MHQA benchmarks.
Method
ChatGPT
InstructGPT
Alpaca- 13B
Vieuna-
Wizard-13B
Zero-shot
20.8
22.9
8.9
12.6
10.9
Manual-CoT (Wei al., 2023) (Cherry-Pick)
28.7
33.1
18.0
23.7
24.7
Manual-CoT (Wei al., 2023) (Random)
26.3
32.4
17.7
22.0
24.0
Auto-CoT (Zhang al., 2022)
21.2
31.4
16.9
21.3
23.3
SP-CoT (Ours)
26.1
33.1
17.4
21.5
22.5
C.1.1 System message
our experiment Manual-CoT (Cherry-Pick) adopts 8 cherry-picken questions and their CoTs manually writen by the authors. The results of Manual-Cof (Random) report the mean EM scores of randomly selected questions and theirs manual CoTs for 2 experiments across the 4 benchmarks. Self-Consistence (Wang al., 2023) is based on SP-CoT with responses for each question.
"You should use your knowledge answer the question to the best of your ability, not refuse to answer, even though you know your knowledge is sometimes out of date. If some references are uncertain, answer all possible cases rather than requesting further information."
C.1.2 Temperature
In most cases, the default temperature is set to 0 for obtaining a most deterministic response. When we ask ChatGPT to name some terms or generation a question, the temperature is set to 1.0 for more diversity.
To the best of our knowledge, Colf-Consistency (Wang et al., 2023) orthogonal to existing CoT methods, including SP-CoT. Although Self- Consistency boosts the performance of SP-CoT to higher level (10%-30% increase for runs), it's worth noting that the cost of Self- Self-Consistency is also 5 times higher.
C.2 LLMs
The 13B LLMs used in our experi- ments are from Huggingface Hub, use chavinlo/gptt-x-alppccc for Alpaca-13B, for Vicuna- 13B and TheBloke/wizard... 1----------111----11111-1----111 138-1.0-fp16 for
In Table 6, report the performance (EM) of CoT methods with recent popular LLMs 1k sub- sets of the test sets. The scores are the average EM scores on 4 ODMR benchmarks. Although Manual- CoT (Wei et al., 2023) outperforms Automated methods, it requires high quality human-labeled CoTs, which is not always accessible in real world applications. Since the cherry-picked CoTs take the dataset features in to consideration. sider their results as the theoretical upper limit of automated approaches. Compared to previously au- tomatic SOTA method (Auto-CoT), our proposed SP- PP-CoT shows decent performance boost most cases.
TheBLokk//iizard--iium------
WizardLM-13B.
D Additional Experiments
D.1 Comparison of CoT Variants
To provide more comprehensive picture of cur- rent CoT methods on ODQA, we report hereby (Table 5) the performance of additional CoT vari- ants, including Manual-CoT (Wei al., 2023) and Auto-CoT (Zhang et al., 2022) on ChatGPT (gpt-3.5 1.tt-3.5tture t-ttuurbo0000

---

Table The scale and average hops of each ODMR dataset generated in this paper. Please note that the scale and the average hops are largely decided by the self-generation setting and the duplication control process.
ChatGPT
Alpaca-
Vicuna-13B
WizardI. M-133
Number of Samples
3550
1354
1562
1604
Avg. Number of Hops
3.04
2.82
2.92
3.16
Avg. Question tokens
34.61
25.35
31.05
34,30
Avg. Answer tokens
1,86
1.99
2.02
1.91
D.2 SP-CoT on GrailQA
games, animals, foods, enterprises, interna- tional organizations.
We report our experiment of CoT methods on Ik subset of test set provided by GrailQA (Gu et al., 2021). According to our ODMR setting, no ex- ternal knowledge is provided to LLMs. From the results below, we notice that our proposed SP-CoT is effective GrailQA, our results on InstructGPT (text soxttddaincii-033 presented in Table 8.
E.3 Quality Control
To ensure the self-generation quality, two mecha- nisms are included our proposed method.
Self-validation in self-generation: To ensure the quality of generated QA pairs, we employ a double-check process, where we demand the LLM to answer the generated question given the gen- erated context and double-check If the generated answer is accordant with the target answer.
Table 8: Performance CoT methods Ik subset of test set provided by GrailQA (Gu et al., 2021).
Method
EM
F1
Composability criteria in composition: Two single-hop QA pairs (q), a1) and (92, (2) are com- posable into a multi-hop question Q with a2 as a valid answer a1 is a named entity and it men- tioned in 42. Such criteria are already satisfied when our 2-hop QA pairs are generated, we use it for connecting more questions.
Zero-shot
12.9
24.3
Zero-shot-CofT (Kojima et al., 2023)
13.5
25.2
Manual-CoT (Wei et al., 2023) (Random)
17.7
30.7
SP-CoT (Ours)
16.0
28.2
E
Constructed ODMR Datasets
E.1 Overview and scale
E.4 Composition rather than Generation
For better understanding of the constructed ODMR datasets, offer well-designed figure (Figure to illustrate the six types of generated questions and their step-by-step decomposition. The scale of the generated ODMR datasets about 1-4k (Ta- ble 7), however, it's largely dependent by the self- generation setting (how many examples to gener- ate?) and the Duplication Control process in Stage 2 Step 2 (How many examples to keep?) To be more specific, the number of topic terms for self- generation decides the scale of generated 2-hop question pairs, and the level of duplication (how many existing question hops are allowed when con- structing a new reasoning chain) decides the scale the remaining examples after filtering.
Directly generating k-hop questions will produce many highly-duplicated reasoning chains, which is less effective than conposition with 2-hop QAs. Take a connected 3-hop QAs as example: + (Q3, Aa), where A
QQ,, A1) +
22. A2 in Q3-
Suppose there are 2 valid next-hop QAs (Q,,A1 and (Qs,As) for we have 2 generated 4-hpp reasoning chains: (Q1,Ai) (Q2, 12) + (Q3, A3) - (Q4, A4) (Q), (Q3,
Now
and (Q1, (Qs, As)
which are highly-duplicane When directly generating k-hop reasoning chains, the number of highly-duplicated chains will increase exponen- tially and such chains will be filtered out in the Duplication Control Process (Stage 2, Step 2). Furthermore, when there more that 2 ques- tion hops in one reasoning chain, more effort should be made ensure direct acyelic graphs (DAGs). An example of cyclic reasoning chain is Q1, A1) - (Q2, A2) + (Q1, A1), which should be avoided.
E.2 Topics
The 29 manually designed topics for generation politicians, athletes, sports teams, sports events, countries, cities, historical figures, histori- cal events, wars, religions, singers, songs, actors or actresses, movies or TV series, writers, books, painters, paintings, composers, classical music, tourist attractions, scientists, scientific terms, video

---

Graph
QA
Decomposition
Q: What another commun name for some members of the which daffodils belong, besides daffodil?
Q:
What the gemus to which daffodils belong?
Narcissus
O+0
What another common narne for members
Narcissus genus besides daffodil?
A: Jonquil
Jonquil
Q: Who held the titles of King of the French and The Duke Orleans Lindley
1. Q: What is the subfamily of the family Asparugncene that hyacinths belong to?
during year first described the subfamily the family Asparaaceacc that byacinths belong to?
A: Scilloideae
2. Q: In what year was the subfamily Scilloidene first described by Lindley?
++0+
1830
A: Louis Philippe
Who became King French 1830 and also held
title The Duke of Orleans?
Louis Philippe
Q: Which country, commonly visited by ships traveling from Europe and where species of roses native of the remote island where Napolcon Bonaparte was exiled and eventually died?
1. Q: Where are most species of roses native to??
which remote was Napoleon Bonaparte exiled where eventually die?
located south
Saint Helena
What country is located south of Saint Helena and
common destination for ships traveling from Asia and Europe?
A: South Africa
A: South Africa
Q:
What work did the person who shared credit with the father of the biblical figure referenced in the tifle of the song "Stairway to Heaven" defend optimism in, while also famously contributing to the devel- opmnnt infinitesimal calculus Newton?
Who the biblical figure that the title of the song "Stairway to leaven" references?
A: Jacob
2. Q: Who was the father of Jacob?
A: Isaac
++
3. Q: Who shares credit with Isaac Newton for developing the infinitesimal calculus?
Gottfried Wilhelm Leibniz
Theodicy
Q: What work Gottfried Wilhelm Leibniz famously defend optimism in? Theodicy
Q: What body of water separates the country that a key ally of the country that Iraq launched Scud missiles at during the Gulf War, from Canada? Atlantic
Q: What country did Iraq launch Scud missiles during the Gulf War, and why is this relevant to the conflict?
A: Israel
2. Q: What is the native region of Carrot?
the Middle East
3. Q: What is the name of the country that key ally of Israel in Middle East
United States
What body of sepurates the United States from Canada?
the Atlantic Ocean
Q: What is the of the town the country that is the largest in South America after the country where Lionel Messi represents in international football competitions where Tom Brady's wife was bom? Trés Maio
Which country does Lionel Messi represent international football competitions?
Argentina
What the largest country in South America after Argentina?
A: Brazil
3. Q: What is the name of the model that Tom Brady is married to?
A: Gisele Bündchen
4. Q: What is the name of the town in Brazil where Gisele
Bündchen was bom?
Três de
Figure 6: The illustration of six reasoning types our automated dataset construction pipeline. These selected examples are self-generated by ChatGPT (gpt-3.5- 5-ttubb--tt Ficuna-13B and WizardL 13B.


Reviews:
[{"review_id": "2BTQlbqatO", "reviewer": "Reviewer_4LDF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your feedback and thoughtful suggestions.\n\n> 1. **The paper does not provide a comparative evaluation with the state-of-the-art (SOTA) methods in the domain of open-domain multi-hop QA. For example, ACL 2023 work Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. And 2022 work DEMONSTRATE–SEARCH–PREDICT: Composing retrieval and language models for knowledge-intensive NLP. On those works, the performance on HotpotQA is around EM 49 F1 60, which is much higher than this work SP-CoT EM 33, F1 42.**\n\nWe afraid that this concern comes from misunderstanding of our work setting, actually, our research scope is subject to a challenging and quite different setting,  multi-step question-answering without quoting any external knowledge source. Different from previous knowledge-dependent works in ACL 2022 [2] and ACL 2023 [1], we explore the limit capability of LLMs for multi-step reasoning via self-enhancement. It’s worth noting that our method is “**LLM-only**” for open-domain multi-hop QA, which does not require external knowledge. Admittedly, such works [1,2] will be added in Table 1 as SOTA results of retrieval-based methods for reference, rather than comparison baselines.\n\nIn the following table, we report more reference results on our proposed ODMR task, including latest knowledge-dependent methods [1,2] and our implementation of Retrieval-based methods on most recent LLMs (We use DPR to retrieve top-5 documents from Wikipedia as context and employ LLM as Reader to answer the question based on the context).  The scores are EM/F1 on the full dev set of 4 ODMR benchmarks.\n\n|                            Method                            | MusiqueQA | HotpotQA  | WikiMHQA  |   CWebQ   |   Mean    |\n| :----------------------------------------------------------: | :-------: | :-------: | :-------: | :-------: | :-------: |\n|              DPR+ChatGPT (`gpt-3.5-turbo-0301`)              |  1.7/4.1  | 15.8/21.4 | 10.9/18.1 | 13.7/18.7 | 10.5/15.6 |\n|             DPR+InstructGPT (`text-davinci-003`)             | 4.8/11.6  | 26.3/34.8 | 23.3/27.1 | 34.4/41.6 | 22.2/28.8 |\n|                       DPR+Alpaca(13B)                        |  2.2/8.4  | 12.0/21.5 | 12.6/21.2 | 15.9/27.1 | 10.7/19.5 |\n|                       DPR+Vicuna(13B)                        |  2.2/7.4  | 18.6/25.8 | 23.9/27.6 | 20.3/27.9 | 16.2/22.1 |\n|                       DPR+Wizard(13B)                        | 3.5/10.0  | 19.9/28.4 | 22.8/27.5 | 24.2/32.2 | 17.6/24.5 |\n|                      IRCoT (GPT-3) [1]                       |  -/30.8   |  -/59.1   |  -/66.5   |    -/-    |    -/-    |\n| DEMONSTRATE–SEARCH–PREDICT (Codex(`code-davinci-002`)) [2] |    -/-    | 51.4/62.9 |    -/-    |    -/-    |    -/-    |\n|       SP-CoT (Ours, ChatGPT (`gpt-3.5-turbo-0301`))       | 14.5/22.6 | 33.2/42.9 | 30.1/34.7 | 37.5/43.6 | 28.8/36.0 |\n\n\n\n> 2. **In Table 2, the author only compared zero-shot and SP-CoT. I think the author should at least compare SP-CoT with CoT rather than zero-shot. CoT (Chain-of-Thought Prompting Elicits Reasoning in Large Language Models), where the author can human-label 8 demos.**\n\nTo provide a more comprehensive picture of current CoT methods on ODQA, we report hereby the performance (EM scores) of additional CoT variants, including Manual-CoT [3] and Auto-CoT [4] for all LLMs mentioned in this paper.  In our experiment, Manual-CoT (Cherry-Pick) adopts 8 cherry-picked questions and their CoTs manually writen by the authors. The results of Manual-CoT (Random) report the mean EM scores of randomly selected questions and theirs manual CoTs for 2 experiments across the 4 benchmarks.\n\nFollowing are additional results of more LLMs on 1k subsets of the test sets. The scores are the average EM scores on 4 ODMR benchmarks.\n\n|            Method            | ChatGPT(`gpt-3.5-turbo-0301`) | InstructGPT(`text-davinci-003`) | Alpaca-13B | Vicuna-13B | Wizard-13B |\n| :--------------------------: | :---------------------------: | :-----------------------------: | :--------: | :--------: | :--------: |\n|           Zeroshot           |             20.8              |              22.9               |    8.9     |    12.6    |    10.9    |\n| Manual-CoT [3] (Cherry-Pick) |             28.7              |              33.1               |    18.0    |    23.7    |    24.7    |\n|   Manual-CoT [3] (Random)    |             26.3              |              32.4               |    17.7    |    22.0    |    24.0    |\n|         Auto-CoT [4]         |             21.2              |              31.4               |    16.9    |    21.3    |    23.3    |\n|        SP-CoT (Ours)         |             26.1              |              33.1               |    17.4    |    21.5    |    22.5    |\n\nAlthough Manual-CoT outperformes Automated methods, it requires high quality human-labeled CoTs, which is not always accessible in real world applications. Since the cherry-picked CoTs  take the dataset features in to consideration, we consider their results as the **theoretical  upper limit** of automated approaches. Compared to previously automatic SOTA method (Auto-CoT), our proposed SP-CoT shows a decent performance boost in most cases.\n\n\n\n\n\n[1] Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. *arXiv preprint arXiv:2212.10509*.\n\n[2] Khattab, O., Santhanam, K., Li, X. L., Hall, D., Liang, P., Potts, C., & Zaharia, M. (2022). Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP. *arXiv preprint arXiv:2212.14024*.\n\n[3] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. *Advances in Neural Information Processing Systems*, *35*, 24824-24837.\n\n[4] Zhang, Z., Zhang, A., Li, M., & Smola, A. (2022). Automatic chain of thought prompting in large language models. *arXiv preprint arXiv:2210.03493*.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper provides well-designed figures that articulate each step of the methodology. These visuals enhance the clarity of the presentation, making it accessible and easy for the audience to grasp the nuances of the proposed work. This commendable effort in visualization aids in bridging the gap between complex concepts and their intuitive understanding.", "reasons_to_reject": "The paper does not provide a comparative evaluation with the state-of-the-art (SOTA) methods in the domain of open-domain multi-hop QA. For example, ACL 2023 work Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. And 2022 work DEMONSTRATE–SEARCH–PREDICT: Composing retrieval and language models for knowledge-intensive NLP. On those works, the performance on HotpotQA is around EM 49 F1 60, which is much higher than this work SP-CoT EM 33, F1 42.  \nIn Table 2, the author only compared zero-shot and SP-CoT. I think the author should at least compare SP-CoT with CoT rather than zero-shot. CoT (Chain-of-Thought Prompting Elicits Reasoning in Large Language Models), where the author can human-label 8 demos.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The author proposed an automated approach to generate the demonstration for the multi-hop reasoning QA for large language models (LLMs).", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models\nDEMONSTRATE–SEARCH–PREDICT: Composing retrieval and language models for knowledge-intensive NLP\nInterleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions"}}, {"review_id": "T8DlQ1hdYn", "reviewer": "Reviewer_K9QY", "summary": "", "questions": "1. In the Method section, in stage 1, the generated questions already include answers to intermediate questions. However, in stage 2, during step 4, why are the intermediate questions replaced with corresponding answers again?\n2. It would be preferable if the examples presented in Figure 2 and Figure 3 in the paper could correspond to each other.\n3. Recently, one of the commonly used datasets for Knowledge Base Question Answering (KBQA) is GrailQA, which also features multi-hop reasoning. I suggest to conduct an experiment on this dataset as well.\n", "limitations": "", "rating": 3, "confidence": 1, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your feedback and suggestions, they are helping us to make our paper better. Here we are trying to make our idea more clear and easy to understand.\n\n> 1. **The multi-hop question-answering (MHQA) methods in the baseline are only up to 2020, and some of the latest works on MHQA have not been mentioned, such as [3,4,5].**\n\nActually, our research scope is subject to a challenging and quite different setting,  multi-step question-answering without providing any external knowledge source. Different from existing MHQA works, our research focus on “**LLM-only**” for MHQA, which explores the capability of LLMs for multi-step reasoning via self-enhancement.\n\nThe performance of retrieval-based models (DPR, RAG, REALM) are provided as reference results, rather than comparison baselines, to present the challenge of our newly proposed ODMR, which requires multi-step reasoning without external knowledge. To the best of our knowledge, reported ODQA models in this paper (DPR, RAG, REALM) have been the latest open-sourced models in consideration of our assumed knowledge-free setting in terms of our ODMR task. Such reference results could have been more recent if latest works [3,4,5] released open-sourced models. \n\nIn the following table, we report more reference results on our proposed ODMR task, including latest knowledge-dependent methods [1,2] and our implementation of Retrieval-based methods on most recent LLMs (We use DPR to retrieve top-5 documents from Wikipedia as context and employ LLM as Reader to answer the question based on the context). The scores are EM/F1 on the full dev set of 4 ODMR benchmarks.\n\n|                            Method                            | MusiqueQA | HotpotQA  | WikiMHQA  |   CWebQ   |   Mean    |\n| :----------------------------------------------------------: | :-------: | :-------: | :-------: | :-------: | :-------: |\n|              DPR+ChatGPT (`gpt-3.5-turbo-0301`)              |  1.7/4.1  | 15.8/21.4 | 10.9/18.1 | 13.7/18.7 | 10.5/15.6 |\n|             DPR+InstructGPT (`text-davinci-003`)             | 4.8/11.6  | 26.3/34.8 | 23.3/27.1 | 34.4/41.6 | 22.2/28.8 |\n|                       DPR+Alpaca(13B)                        |  2.2/8.4  | 12.0/21.5 | 12.6/21.2 | 15.9/27.1 | 10.7/19.5 |\n|                       DPR+Vicuna(13B)                        |  2.2/7.4  | 18.6/25.8 | 23.9/27.6 | 20.3/27.9 | 16.2/22.1 |\n|                       DPR+Wizard(13B)                        | 3.5/10.0  | 19.9/28.4 | 22.8/27.5 | 24.2/32.2 | 17.6/24.5 |\n|                      IRCoT (GPT-3) [1]                       |  -/30.8   |  -/59.1   |  -/66.5   |    -/-    |    -/-    |\n| DEMONSTRATE–SEARCH–PREDICT (Codex(`code-davinci-002`)) [2] |    -/-    | 51.4/62.9 |    -/-    |    -/-    |    -/-    |\n|       SP-CoT (Ours, ChatGPT (`gpt-3.5-turbo-0301`))       | 14.5/22.6 | 33.2/42.9 | 30.1/34.7 | 37.5/43.6 | 28.8/36.0 |\n\n\n\n> 2. **More information of the constructed ODMR datasets should be introduced, which would be useful for readers to understand the quality of generated questions. For example, the scale of ODMR datasets and the process of determining the number of intermediate problems for each complex issue.**\n\nFor better understanding of the constructed ODMR datasets, we offer a well-designed figure (Figure-6 in the Appendix, page 13) to illustrate the six types of generated questions and their step-by-step decomposition. The scale of the generated ODMR datasets is about 1-4k, however, it’s largely dependent by the self-generation setting (how many examples to generate?) and the Duplication Control process in Stage 2 Step 2 (How many examples to keep?) To be more specific, the number of topic terms for self-generation decides the scale of generated 2-hop question pairs, and the level of duplication (how many existing question hops are allowed when constructing a new reasoning chain) decides the scale of the remaining examples after filtering. \n\nTo ensure the self-generation quality, two mechanisms are included in our proposed method. \n\n- **Self-validation in self-generation:** To ensure the quality of generated QA pairs, we employ a double-check process, where we demand the LLM to answer the generated question given the generated context and double-check if the generated answer is accordant with the target answer.\n- **Composability criteria in composition:** Two single-hop QA pairs $(q_1, a_1)$ and $(q_2, a_2)$ are composable into an multi-hop question $Q$ with $a_2$ as a valid answer if $a_1$ is a named entity and it is mentioned in $q_2$. Such criteria are already satisfied when our 2-hop QA pairs are generated, we use it for connecting more questions.\n\nThe scale and average hops of each ODMR dataset generated in this paper are listed in the following table. Please note that the scale and the average hops are largely decided by the self-generation setting and the duplication control process.\n\n|          Model          | ChatGPT(`gpt-3.5-turbo-0301`) | Alpaca-13B | Vicuna-13B | Wizard-13B |\n| :---------------------: | :---------------------------: | :--------: | :--------: | :--------: |\n|   Number of  Samples    |             3550              |    1354    |    1562    |    1604    |\n|      Average Hops       |             3.04              |    2.82    |    2.92    |    3.16    |\n| Average Question tokens |             34.61             |   25.35    |   31.05    |   34.30    |\n|  Average Answer tokens  |             1.86              |    1.99    |    2.02    |    1.91    |\n\nThe number of intermediate problems is determined during construction. For example, suppose we have 2 2-hop question pairs: `(Q1, A1)->(Q2, A2)` and `(Q3, A3) -> (Q4, A4)`. If `A2` appears in `Q3`, then it’s possible to construct a 4-hop reasoning chain (`Q1->Q2->Q3->Q4`, should also check other criteria). If `A2` appears in `Q4`, then a 3-hop reasoning chain (`Q1->Q2->Q4`) may be possible.\n\n\n\n> 3. **Generating complex questions by composing intermediate sub-questions has been explored in traditional methods [6]. Selecting demonstrations via clustering-based methods has also been investigated by existing prompting methods for other tasks [7]. This weakens the innovation of the proposed methods. I suggest the authors to shed more light on the elaborate design of the proposed method for the ODMR tasks.**\n\nTrivedi et al [6] propose to construct multi-hop questions by combining QAs from existing commonsense QA datasets. Different from their work, we propose to construct multi-step QAs with the self-generated QA pairs of LLMs, which requires neither human annotation nor existing labeled datasets. Zhang et al [7] proposed a clustering based sampling method to select CoTs for each question, which proves to be effective in our experiment settings (Table 3 in Appendix, page 12). Therefore, we adopt the same strategy to sample demonstrations for in-context learning.\n\nIn this paper, we focus on elaborating the design of our self-generation framework, while only briefly introducing the reasoning chain composition (inspired by Trivedi et al [6]) in Stage 2 Step 1. Our main innovation can be summarized as follows:\n\n- An automated generation pipeline for ODMR datasets by LLMs’ self-generation, totally from scratch.\n- SP-CoT, an automated framework for mass-producing CoTs while ensuring quality and diversity, built upon the generated ODMR datasets.\n- General effectiveness of SP-CoT on larger-scale (175B) and smaller-scale (13B) LLMs and high-quality intermediate reasoning steps.\n\n\n\n> 4. **In the Method section, in stage 1, the generated questions already include answers to intermediate questions. However, in stage 2, during step 4, why are the intermediate questions replaced with corresponding answers again?**\n\nIn stage 1, the generated questions indeed include answers to last-hop questions, which is to ensure the composability criteria [6]. In stage 2 step 4, the goal is to transform the reasoning chains (linked Sub-QAs) to the natural questions that require multi-step reasoning. To do this, the intermediate answers (Sub-answers) are replaced by the corresponding questions (Sub-questions) to generate the corresponding multi-step questions. Following is an example of such process:\n\n**Reasoning chain:** `Sub-Q1 -> Sub-Q3 <- Sub-Q2`\n\n- Sub-Q1: What is the American Gothic painting by Grant Wood about?\n\n- Sub-Q2: What country did Nikola Tesla emigrate to in 1884 to work for Thomas Edison in New York City?\n\n- Sub-Q3: Is it the Mississippi River that serves as the eastern border of the [*Sub-A1*] region in [*Sub-A2*]?\n\n **Raw question:** Is it the Mississippi River that serves as the eastern border of the [*What is the American Gothic painting by Grant Wood about?*] region in [*What country did Nikola Tesla emigrate to in 1884 to work for Thomas Edison in New York City?*]?\n\n**Instruction to LLM**: Replace the sentence within [] with a relative clause and make the raw question into a natural question:\n\n**Reformulated Natural Question requiring multi-step reasoning**: Is it the Mississippi River that serves as the eastern border of the American Gothic painting by Grant Wood region in the country that Nikola Tesla emigrated to in 1884 to work for Thomas Edison in New York City?\n\n\n\n> 5. **It would be preferable if the examples presented in Figure 2 and Figure 3 in the paper could correspond to each other.**\n\nThanks for the suggestion, the examples presented in Figure 2 and Figure 3 will be correspondant.\n\n\n\n> 6. **Recently, one of the commonly used datasets for Knowledge Base Question Answering (KBQA) is GrailQA [8], which also features multi-hop reasoning. I suggest to conduct an experiment on this dataset as well.**\n\nWe report our experiment of CoT methods on 1k subset of test set provided by GrailQA [8]. According to our ODMR setting, no external knowledge is provided to LLMs. From the results below, we notice that our proposed SP-CoT is effective on GrailQA, our results on InstructGPT(`text-davinci-003`) are:\n\n| Metric | Zeroshot | Zeroshot-CoT [10] | Manual-CoT [9] (Cherry-Picked) | Manual-CoT [9] (Random) | SP-CoT (Ours) |\n| :----: | :------: | :---------------: | :----------------------------: | :---------------------: | :-----------: |\n|   EM   |   12.9   |       13.5        |              18.2              |          17.7           |     16.0      |\n|   F1   |   24.3   |       25.2        |              31.0              |          30.7           |     28.2      |\n\n\n\n[1] Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. *arXiv preprint arXiv:2212.10509*.\n\n[2] Khattab, O., Santhanam, K., Li, X. L., Hall, D., Liang, P., Potts, C., & Zaharia, M. (2022). Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP. *arXiv preprint arXiv:2212.14024*.\n\n[3] Qu, Y., Ding, Y., Liu, J., Liu, K., Ren, R., Zhao, W. X., ... & Wang, H. (2020). RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. *arXiv preprint arXiv:2010.08191*.\n\n[4] Sachan, D. S., Patwary, M., Shoeybi, M., Kant, N., Ping, W., Hamilton, W. L., & Catanzaro, B. (2021). End-to-end training of neural retrievers for open-domain question answering. *arXiv preprint arXiv:2101.00408*.\n\n[5] Banerjee, P., & Baral, C. (2020). Knowledge fusion and semantic knowledge ranking for open domain question answering. *arXiv preprint arXiv:2004.03101*.\n\n[6] Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2022). ♫ MuSiQue: Multihop Questions via Single-hop Question Composition. *Transactions of the Association for Computational Linguistics*, *10*, 539-554.\n\n[7] Zhang, Z., Zhang, A., Li, M., & Smola, A. (2022). Automatic chain of thought prompting in large language models. *arXiv preprint arXiv:2210.03493*.\n\n[8] Gu, Y., Kase, S., Vanni, M., Sadler, B., Liang, P., Yan, X., & Su, Y. (2021, April). Beyond IID: three levels of generalization for question answering on knowledge bases. In *Proceedings of the Web Conference 2021* (pp. 3477-3488).\n\n[9] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. *Advances in Neural Information Processing Systems*, *35*, 24824-24837.\n\n[10] Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners. *Advances in neural information processing systems*, *35*, 22199-22213.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.\tThe prompting method proposed for generating intermediate ODMR datasets is reasonable. Various details are taken into consideration, such as addressing the repetitiveness of intermediate questions and considering binary questions.", "reasons_to_reject": "1.\tThe multi-hop question-answering (MHQA) methods in the baseline are only up to 2020, and some of the latest works on MHQA have not been mentioned, such as [3,4,5].\n2.\tMore information of the constructed ODMR datasets should be introduced, which would be useful for readers to understand the quality of generated questions. For example, the scale of ODMR datasets and the process of determining the number of intermediate problems for each complex issue.\n3.\tGenerating complex questions by composing intermediate sub-questions has been explored in traditional methods [1]. Selecting demonstrations via clustering-based methods has also been investigated by existing prompting methods for other tasks [2]. This weakens the innovation of the proposed methods. I suggest the authors to shed more light on the elaborate design of the proposed method for the ODMR tasks.\n\n[1] Harsh Trivedi and Niranjan Balasubramanian and Tushar Khot and Ashish Sabharwal. MuSiQue: Multihop Questions via Single-hop Question Composition.\n[2] Zhuosheng Zhang  and  Aston Zhang  and  Mu Li  and  Alex Smola. Automatic Chain of Thought Prompting in Large Language Models.\n[3] Yingqi Qu and Yuchen Ding and Jing Liu and Kai Liu and Ruiyang Ren and Wayne Xin Zhao and Daxiang Dong and Hua Wu and Haifeng Wang. RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering.\n[4] Devendra Singh Sachan and Mostofa Patwary and Mohammad Shoeybi and Neel Kant and Wei Ping and William L Hamilton and Bryan Catanzaro. End-to-End Training of Neural Retrievers for Open-Domain Question Answering.\n[5] Pratyay Banerjee and Chitta Baral. Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering.\n", "questions_to_authors": "1. In the Method section, in stage 1, the generated questions already include answers to intermediate questions. However, in stage 2, during step 4, why are the intermediate questions replaced with corresponding answers again?\n2. It would be preferable if the examples presented in Figure 2 and Figure 3 in the paper could correspond to each other.\n3. Recently, one of the commonly used datasets for Knowledge Base Question Answering (KBQA) is GrailQA, which also features multi-hop reasoning. I suggest to conduct an experiment on this dataset as well.\n", "paper_topic_and_main_contributions": "In this study, the authors utilize Large Language Models (LLMs) with self-prompted Chain-of-Thought (SP-CoT) to tackle the challenging tasks of Open-Domain Multi-Hop Reasoning (ODMR). Existing automated paradigm lacks of quality assurance while manual approaches suffer from limited scalability. The proposed process involves three stages. In stage 1, LLMs are prompted to iteratively generate 2-hop commonsense QA quadruplets containing context, question, answer, and explanation. In stage 2, these quadruplets are connected to form multi-hop reasoning chains, leading to the creation of an ODMR dataset through composition. In stage 3, a clustering-based sampling approach is adopted to dynamically select and construct in-context demonstrations for inference. Instead of extracting demonstrations from existing ODMR datasets, their method automatically generates demonstrations via LLMs. They evaluate the proposed method, which could consistently achieve SOTA results on MSQ, HotpotQA, 2Wiki and CWebQ datasets compared with the existing prompting methods.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YGQxs53uaa", "reviewer": "Reviewer_mWKK", "summary": "", "questions": "A. Is it possible to self-generate 4-hop or k-hop QAs directly rather than 2-hop QAs with composition? Some additional rationales together with potential empirical demonstrations of  the effectiveness of the proposed 2-hop QAs  with composition would be beneficial.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our paper and for your thoughtful feedback.\n\n\n\n> 1. **Reported performance of additional CoT variants [1,2,3] (Table 1 and 2) would be beneficial to observe the complete picture of current CoT methods on ODQA tasks.**\n\n\n\nTo provide a more comprehensive picture of current CoT methods on ODQA, we report hereby the performance (EM scores) of additional CoT variants, including Manual-CoT [4] and Auto-CoT [3] on ChatGPT(`gpt-3.5-turbo-0301`).  In our experiment, Manual-CoT (Cherry-Pick) adopts 8 cherry-picked questions and their CoTs manually writen by the authors. The results of Manual-CoT (Random) report the mean EM scores of randomly selected questions and theirs manual CoTs for 2 experiments across the 4 benchmarks. Self-Consistency [1] is based on SP-CoT with 5 responses for each question. \n\nWe report additional results for Table 1 to offer a more general overview of the performance of existing methods on our proposed ODMR task. The scores are EM/F1 on the full dev set of 4 ODMR benchmarks.\n\n|            Method            | MusiqueQA | HotpotQA  | WikiMHQA  |   CWebQ   |   Mean    |\n| :--------------------------: | :-------: | :-------: | :-------: | :-------: | :-------: |\n|           Zeroshot           |  3.1/7.3  | 22.4/30.0 | 18.7/21.7 | 31.6/37.5 | 19.0/24.1 |\n|       Zeroshot-CoT [5]       |  5.0/8.8  | 22.6/29.6 | 24.3/27.1 | 30.3/36.2 | 20.6/25.4 |\n|   Manual-CoT [4] (Random)    | 12.3/19.2 | 32.4/43.7 | 27.7/34.6 | 36.6/43.0 | 27.3/35.1 |\n| Manual-CoT [4] (Cherry-Pick) | 13.7/21.9 | 33.9/44.7 | 31.5/38.6 | 37.2/44.0 | 29.1/37.3 |\n|         Auto-CoT [3]         | 8.1/13.6  | 26.1/36.3 | 26.2/30.2 | 29.9/38.3 | 22.6/29.6 |\n|        SP-CoT (Ours)         | 14.5/22.6 | 33.2/42.9 | 30.1/34.7 | 37.5/43.6 | 28.8/36.0 |\n|     Self-Consistency [1]     | 18.3/28.3 |    -/-    |    -/-    | 47.1/54.0 |    -/-    |\n\nTo the best of our knowledge, Self-Consistency [1] is orthogonal to existing CoT methods, including SP-CoT.  Although Self-Consistency boosts the performance of SP-CoT to a higher level (10%-30% increase for 5 runs), it’s worth noting that the cost of Self-Consistency is also 5 times higher. \n\nIn Complexity-based prompting [2], the authors set up a complexity threshold for voting. While in our ODMR task, questions usually requires 2-4 hops, and 4-step reasoning is not better than 2-step reasoning for a 2-hop question. Therefore, reasoning complexity is independent of the performance in our cases, and  we cannot select a reasonable and practical complexity threshold.\n\nAlthough Manual-CoT [4] outperformes Automated methods, it requires high quality human-labeled CoTs, which is not always accessible in real world applications. Since the cherry-picked CoTs  take the dataset features in to consideration, we consider their results as the **theoretical  upper limit** of automated approaches. Compared to previously automatic SOTA method (Auto-CoT), our proposed SP-CoT shows a decent performance boost in most cases.\n\nFollowing are additional results of more LLMs on 1k subsets of the test sets. The scores are the average EM scores on 4 ODMR benchmarks.\n\n|            Method            | ChatGPT(`gpt-3.5-turbo-0301`) | InstructGPT(`text-davinci-003`) | Alpaca-13B | Vicuna-13B | Wizard-13B |\n| :--------------------------: | :---------------------------: | :-----------------------------: | :--------: | :--------: | :--------: |\n|           Zeroshot           |             20.8              |              22.9               |    8.9     |    12.6    |    10.9    |\n| Manual-CoT [4] (Cherry-Pick) |             28.7              |              33.1               |    18.0    |    23.7    |    24.7    |\n|   Manual-CoT [4] (Random)    |             26.3              |              32.4               |    17.7    |    22.0    |    24.0    |\n|         Auto-CoT [3]         |             21.2              |              31.4               |    16.9    |    21.3    |    23.3    |\n|        SP-CoT (Ours)         |             26.1              |              33.1               |    17.4    |    21.5    |    22.5    |\n\n\n\n> 2. **Is it possible to self-generate 4-hop or k-hop QAs directly rather than 2-hop QAs with composition? Some additional rationales together with potential empirical demonstrations of the effectiveness of the proposed 2-hop QAs with composition would be beneficial.**\n\nDirectly generating k-hop questions will produce many highly-duplicated reasoning chains, which is less effective than conposition with 2-hop QAs. \n\nLet’s take a connected 3-hop QAs as example: `(Q1, A1) -> (Q2, A2) -> (Q3, A3)`, where `A1` in `Q2`, `A2` in `Q3`. \n\nSuppose there are 2 valid next-hop QAs `(Q4, A4)` and `(Q5, A5)` for `(Q3, A3)`. \n\nNow we have 2 generated 4-hop reasoning chains: \n\n- `(Q1, A1) -> (Q2, A2) -> (Q3, A3) -> (Q4, A4)`\n- `(Q1, A1) -> (Q2, A2) -> (Q3, A3) -> (Q5, A5)`\n\nwhich are highly-duplicated. When directly generating k-hop reasoning chains, the number of highly-duplicated chains will increase exponentially and such chains will be filtered out in the Duplication Control Process (Stage 2, Step 2).\n\nFurthermore, when there are more that 2 question hops in one reasoning chain, more effort should be made to ensure direct acyclic graphs (DAGs). An example of cyclic reasoning chain is `(Q1, A1) -> (Q2, A2) -> (Q1, A1)`, which should be avoided.\n\n\n\n[1] Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023\n\n[2] Fu et al., Complexity-Based Prompting for Multi-step Reasoning. ICLR 2023\n\n[3] Zhang et al., Automatic Chain of Thought Prompting in Large Language Models. ICLR 2023.\n\n[4] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. *Advances in Neural Information Processing Systems*, *35*, 24824-24837.\n\n[5] Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners. *Advances in neural information processing systems*, *35*, 22199-22213.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The work is well-written and presents clear motivations.\n\nThe automated pipeline to generate ODQA datasets can be promising for more complex NLP tasks. \n\nStrong performance improvements over zero-shot inference across different benchmarks and datasets.", "reasons_to_reject": "Reported performance of additional CoT variants [1,2,3] (Table 1 and 2) would be beneficial to observe the complete picture of current CoT methods on ODQA tasks. \n\n\n[1] Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023\n\n[2] Fu et al., Complexity-Based Prompting for Multi-step Reasoning. ICLR 2023\n\n[3] Zhang et al., Automatic Chain of Thought Prompting in Large Language Models. ICLR 2023.", "questions_to_authors": "A. Is it possible to self-generate 4-hop or k-hop QAs directly rather than 2-hop QAs with composition? Some additional rationales together with potential empirical demonstrations of  the effectiveness of the proposed 2-hop QAs  with composition would be beneficial.", "paper_topic_and_main_contributions": "The authors propose Self-prompted Chain-of-Thought (SP-CoT) to automatically produce high quality CoT of LLMs for Open-domain Question-Answering (ODQA) task. Specifically, the process is divided into 2 stages (1) Generating 2-hop commonsense quadruplets, (2) Connecting 2-hop QA quadruplets to construct multi-hop reasoning chains. The proposed method are empirically demonstrated to yield promising improvements over 4 benchmarks across different QA datsets. The quality of generated CoT is also extensively evaluated.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 421: sampling -> sample\n\nLine 499: shows-> show\n\nReferring to specific sections of the Appendix is recommended (Line 308, 443, etc.)"}, "extra_text": {"Missing_References": "[1] Wang et al., Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023\n\n[2] Fu et al., Complexity-Based Prompting for Multi-step Reasoning. ICLR 2023\n\n[3] Zhang et al., Automatic Chain of Thought Prompting in Large Language Models. ICLR 2023."}}]

Author rebuttal:

Meta-review: The paper introduces a method to automatically generating questions via LLMs and augment it as demonstrations for open domain multi hop reasoning. Their pipeline includes reasoning chain composition, duplication control and other aspects. 

Strengths:
The generation pipeline for ODMR datasets by LLMs’ self-generation and their method (SP-CoT) built upon the generated ODMR datasets. This yields good results on larger-scale (175B) and smaller-scale (13B) LLMs and high-quality intermediate reasoning steps.

Weaknesses:
- (Main concern) Comparison with (a) non-LLM approaches, and (b) with HotpotQA leaderboard systems, and (c) conceptual comparison with existing approaches with similar pipeline including decompositions (incl. recent open-domain QA with or without retrieval) ; would add depth to the paper. 
- Missing ablation studies of the pipeline. This will also strengthen the paper.
Please include the newly reported results on additional CoT baseline empirical studies in the next version.
Decision: Accept-Findings