SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures
Authors: Pei Zhou
Jay Pujara
Xiang Ren
Xinyun Chen
Heng-Tze Cheng
Quoc V Le
Ed H. Chi
Denny Zhou
Swaroop Mishra
Steven Zheng
Venue: NeurIPS 2024 poster (2024)
Keywords: Large Language Models
Reasoning
Prompting
Self-Improve
Self-Discover
Primary area: natural_language_processing
TL;DR: We introduce SELF-DISCOVER, an efficient and performant framework for models to self-discover a reasoning structure for any task from a seed set of general problem-solving skills.

Abstract: We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a self-discovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER substantially improves GPT-4 and PaLM 2’s performance on challenging reasoning benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER outperforms inference-intensive methods such as CoT-Self-Consistency by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.

Self-Discover: Large Language Models Self-Compose Reasoning Structures
Pei Zhou Quoc V. Lei
Jay Pujara
Xiang Ren
Xinyun Chen Heng-Tze Cheng! Huaixiu Steven Zheng!
Ed H. Chit
Denny Zhou
Swaroop Mishra
Google DeepMind
+ University of Southern California
Abstract
We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a self- discovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER sub- stantially improves GPT-4 and PaLM 2's performance on challenging reason- ing benchmarks such as BigBench- grounded agent reasoning, and MATH, by much as 32% compared to Chain of Thought (CoT). Furthermore, SELF- DISCOVER outperforms inference-intensive methods such as CoT-Self-Consistencc by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.
Task
Answer
Chain-of-Thought (CoT)
Task
Rationale
Answwer
Self-Discover Ressoning
Task
Reasoning Structure
Paek-Specitic
Structured Ressoning
Answer
Structures (Ours)
Discover Direct
Seit-Oiscover Over Chain ofThcught
BRH: +11%
Avg. BBH: +7% T4D: + 29%
T4D: 39%
MATH: +3-5%
MATH: +8.5%
Figure 1: SELF-DISCOVER guides LLMs to self-discover and compose atomic reasoning modules into reasoning structure to solve challenging tasks. Through testing challenging reasoning benchmarks incuding Big Bench-Hard (BBH), agent reasoning (T4D), and MATH, we find that SELF-DIsCoVER outperforms Direct Answering 23/25 and CoT on 21/25 tasks in zero-shot setting using PaLM 2-L. Full BBH results are in Appendix Table
Introduction
Large Language Models (LLM) Brown et al. 2020 Chowdhery et al. 2022 OpenAl 2023b Anil
2023 powered by transformers Vaswani et al. 2017) have produced impressive breakthroughs
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

in generating coherent texts (OpenAl 2022). and following instructions Zhong et al. 2021 Mishra al. 2022c Wei et al. 2021 Chung et al. 2022 Ouyang et al. 2022). In pursuit of the goal to
enhance L.MS' capability to reason and solve complex problems, various prompting methods have
been proposed, drawing inspirations from cognitive theories of how humans reason. For example,
few-shot and zero-shot tlain-of-thhught (Co'T) (Nye et al. 2021; Wei et al.] 2022] Kojima et al. [0222
Yasunaga et al. 2023) resembles how humans solve problems step-by-step. decomposition -bbaddd
prompting (Zhou et al.) 2022a Drozdov et al. 2022 Patel et al. 2022 Hao et al. 2023 Khot et al.
2022) is inspired by how humans breakdown a complex problem into a series of smaller subproblems, and then solve those subproblems one by one (Polya 2004), and step-back prompting (Zheng et al., 2023) is motivated by how humans reflect on task nature to derive general principles. However, a fundamental limitation is that each technique itself serves as atomic reasoning module making an implicit prior assumption of the process on how to tackle a given task. Instead, we argue that each task has a unique intrinsic structure underlying the reasoning process involved in solving it efficiently. For instance, least-to-most prompting (Zhou et al. 2022a Drozdov et al. (022) has shown to be much more effective than CoT (Wei al.) 2022) solving tasks such symbolic manipulation and compositional generalization, due to the decomposition structure of the tasks.
This paper aims at self-discovering the underlying reasoning structure unique to each task, while being highly efficient in terms of computation. Our approach. SELF-DISCOVER is inspired by how humans internally devise reasoning program for problem-solving (Newell et al. [958 Rasmussen 1983), as illustrated in Figure 2 From a set of atomic reasoning modules described in natural language such as "breakdown into sub tasks" and "critical thinking", LLM, and task examples without labels, SELF-DISCOVER composes a coherent reasoning structure intrinsic to the task (Stage 1) and then solves instances of the task using the discovered structure (Stage 2). Stage I operates at the task-level and uses three actions to guide the LLM to generate a reasoning structure for the task. At Stage 2. during the final decoding, the LLM simply follows the self-discovered structure to arrive at the final answer.
brings several benefits compared to other methods for LLM reasoning. First, the discovered reasoning structure is grounded in atomic reasoning modules benefiting from the strengths of multiple reasoning modules in contrast to applying priori module such as COT. Second, steps on the task-level, while being more performant than inference-heavy ensemble approaches such as self-consistency Wang et al. al.] (2022). Lastly, the discovered reasoning structure is intrinsic to the task, and conveys LLMs' insights about the task in a more interpretable way than the optimized prompts (Zhou et al.
Solving problems using
is efficient in computation as only requires 3 more inference
2022b Yang et al. 2023).
We test gun et al. 2022). Thinking for Doing (T4D) (Zhou et al.] 2023) and MATH (Hendrycks et al. 2021). SELF-DISCOVER outperforms CoT on 21/25 task with performance gains up 42% (Fig- ure| D. highlighting the advantage of the self-discovered reasoning structure composed from the atomic reasoning modules against single priori CoT module. Furthermore, demonstrate that SELF-DISCOVER achieves superior performance against inference- methods such as CoT + Self-Consistency and majority voting of every module while requiring 10-40x fewer inference compute (Figure 5). Finally, we compare SEL.FDISCOVER with prompts optimized (OPRO) using a still performs on par or better than OPRO while the self-discovered reasoning structure are much more interpretable.
25 challenging reasoning tasks including Big Bench- (BBH) (Suz-
training (Yang et al., 2023) (Figure 8 We find that
We conduct a set of analysis to understand the effectiveness of SELF-DISCOV down BBH tasks into 4 different categories, we find that requiring world knowledge and has moderate performance boost on algorithmic tasks compared to CoT (Figure This is further confirmed by the error analysis on MATH, where 74.7% model failures comes from computation errors (e.g. math). We also take a closer look the self-discovered reasoning structures, and show the universality of them by transferability study from PaLM 2-L to GPT-4, and from GPT-4 to Llama-2-700 2-BB. We hope to encourage more future work structured reasoning for solving challenging problems using LLMs.
By breaking performs best on tasks
2

---

2 Self-Discovering Reasoning Structures for Problem-Solving
We take inspiration from how humans use prior knowledge and skills to devise a reasoning program to solve problems (Newell et al. 1958; Rasmussen 1983). When we face a new problem, we often first search internally what knowled legge and skills from our prior experience might be helpful solve Then we will attempt to apply relevant knowledge and skills to this task. And finally we will connect multiple individual skills and knowledge solve the problem. We design SELF- GELF-DISCOVER to enact these steps into two stages as illustrated in Figure 2
Given a task and set of reasoning module descriptions representing high-level problem-solving heuristics such as "Use critical thinking" and "Let's 's think step by step". Stage 1 of SELFF DISCOVER aims uncover the intrinsic reasoning structure for solving this task via meta-reasoning. Specifically, we uses three meta prompts to guide LLMs to select, adapt, and implement an action- able reasoning structure with no labels or training required. We format the structure in key-value pairs similar to JSON due to interpretability and findings on following JSON boosts reasoning and generation quality Zhou et al. 2023 OpenAl 2023a). The structure of the meta- meta-prompts and full prompts are shown in Figure Stage operates on task-level, meaning we only need to run SELF-DISCOVER once for each task. Then, in Stage 2, we can simply use the discovered reasoning structure to solve every instance of the given task by instructing models to follow the provided structure by filling each key and arrive at final answer.
Stage 1: Discover Reasoning Structure on Task-Leve!
Reasoning Structure
Key-Value paits
Language Model
Task: Reasoning
"Type and color of each ttes
colored objects
"Wamber itens of each colort:
Seff-Discover
"Manbez of iten of each type":
"Mamber of itens of each color and type": "Final ansvee":
Atomic Reasoning Modules
Stage 2: Solve Problems Using Discovered Structure on Instance-Level
Pill the Viahees hased Kays durring decoding
Task Instance
Reasoning Structure
Language
Answer
Model
Figure 2: Illustration of using SELF-DISCOVER for problem-solving. Given a generative LM, task, and seed reasoning module descriptions, we guide LMs generate reasoning structure key-value format solve the task. Finally, models can follow the self-discovered structures to solve the every instance from the task by filling in the values in ISON step-by-step.
2.1 Stage 1: Self Self-Discover Task-Specific Structures
The first stage consists of three actions: 1) SELECT, where relevant reasoning modules for task- solving are chosen from the set of reasoning module descriptions; 2) ADAPT, where descriptions of selected reasoning modules are rephrased to more specific the task at hand; and 3) IMPLEMENT where the adapted reasoning descriptions are implemented into a structured actionable plan so that the task can be solved by following the structure.
SELECT First, not every reasoning module is helpful for every task, so the first stage of SELF- DISCOVER guides model to select modules that are useful based on task examples. For example, "reflective thinking" might help search for first princcprin theories on science problems, while "creative thinking" helps on generating a novel continuation to a story. Given raw set of reasoning module descriptions D such as "critical thinking". and "break the problem into sub-problems" (full set in Appendix (A). and a few task examples without labels 1 E T, - first selects a subset of reasoning modules Ds that are useful for solving the tasks by using model M and meta-prompt
ps:
Ds = Mips || [1).
(I)
ADAPT Since each reasoning module provides general description of how to solve problems, the next step of SELF-DISCOVER aims at lailoring each selected module to the task at hand. For example, from "break the problem into pooopleooes" to "calculate each arithmetic operation in order" for arithmetic problems. Given selected reasoning module subset Dg from the previous step, ADAPT
3

---

Self- Self-Discover
All Seed Modules
Modules
by
Language Model
Step-by-step
SELECT
Break down Propese-verify
Break down
Adapted Modules
ADAPT
Selected Modules
Language Model
Scep-by-ste analyze Break down to type and color
Reasoning Structure
IMPLEMENT
Adapted Modules
Language Model
"type color itua":
ibeas colos"; itexs type":
Figure 3: Illustration of three actions of SELF-DISCOVER We use LMs to compose a coherent reasoning structure by selecting relevant modules, adapting to task-specific descriptions, and implement reasoning structure in JSON.
rephrases each of the selected module to be more specific to the task. Similarly to SELECT, this stage uses a meta-prompt PA and a generative model M generate the adapted reasoning module descriptions DA:
DA M(pa Ds 4:)
IMPLEMENT Finally, given the adapted reasoning module descriptions DA SELF-DISCOVER op- erationalizes the reasoning modules into an implemented reasoning structure D, with specified instruction on what to generate for each step. In addition to meta prompt Pı, IMPLEMENT also provides a demonstration of convert the natural language descriptions into reasoning structure:
reasoning structure Shuman on another task to better
D = M(pr || Skuman || DA (.).
2.2 Stage 2:
Tackle Tasks Using Discovered Structures
After the three stages. we have an implemented reasoning structure D, uniquely adapted for the task we need to solve T. Then can simply append the reasoning structure to all instances of the task and prompt models to follow the reasoning structure to generate an answer A:
M(D, V
Experiment Setup
3.1 Tasks
We focus on diverse reasoning benchmarks that are still challenging for LLMSs: BIG-Bench Hard (BBH) (Suzgun al. 2022) contains 23 carefully-selected challenging tasks from BIG-Bench (Sri-
vastava et al. 2023). BBH tasks cover diverse range of reasoning problems spanning the following 4 categories according their authors: 1) Algorithmic and Multi-Step Arithmetic Reasoning, 2) Natural Language Understanding, 3) Use of World Knowledge, and 4) Multilingual Knowledge and Reasoning. We also test grounded social agent reasoning task called Thinking for Doing (T4D) where models must leverage mental state reasoning to determine actions to perform (Zhou et al. 2023), where GPT-4 with CoT only reaches around 50%. Finally, we subsample 200 examples from the MATH (Hendryeks et al. 2021) test set, and generate instance-level reasoning structures via one-shot demonstration adapt to complexity MATH tasks. For evaluations, we use accuracy to measure the model performance BBH, T4D and MATH (details can be found in Appendix B B

---

3.2
Models
We use several ofate-of-the-ar LLMs: GPT-4 (gpp--t-pee- (OpenAl 2023b), GPT-3.5-turbe (ChatGPT) (OpenAl 2022 instruction-tuns tuned PaLM 2-L (Anil et al. 2023 and an open-source LLM Llama2-70B (Touvron et al. 2023
3.3
Baselines
We compare SELF-DISCOVER with other zero-shot prompting methods for LLM reasoning:
Direct Prompting, where model directly generates the answer without intermediate reason-
ing steps.
CoT (Wei et al. 2022 Kojima et al. 2022), where models are prompted to generate reasoning process leading the final answer.
Plan-and-Sole (Wang et 2023), where models are prompted first generate a plan and then solve the problem. SELF-DISCOVER differs by grounding the reasoning structure in atomic reasoning modules, and prompting the decoding follow the explicit key-value reasoning structure.
Next, we also consider other baselines that make use of the raw seed reasoning modules (RM) we pass to SEL.-DISCOVER. We compare with the following methods' performance and the inference call efficiency on a subset of tasks.
CoT-Self-Consistency Consistency Wang et al. (2022), sample multiple outputs from LLM with CoT and aggregate answers to get the final answer. We compare this method on a subset of tasks due to the cost of repetitive queries.
Majority voting of each RM: we prompt models to solve the tasks by appending each RM and use majority voting of all answers to get the final answer. We examine whether integrating multiple RMs into coherent reasoning structure is advantageous to applying each RM to solve the task and use majority voting to ensemble them post-hoc, which costs much more inference computation.
Best of each R.M this method assumes that we have access to oracle labels and uses the highest accuracy from applying each RMM. compare with this to examine whether SELF- DISCOVER competes with methods that depend on perfect prior knowledge of which RM to use on a new task.
Furthermore, for analysis universality of reasoning structures, compare with prompt- optimization method that require a training set to improve prompts: LLMs as optimizers (OPRO) (Yang al. 2023). We aim show that when apply structures prompts opti- mized from one model, the reasoning structures can retain more performance gains than the wordings of prompts.
Results
We answer the following questions through experimental results: 1) Does discovering reasoning structures improve LLM reasoning capabilities? (4.1) 2) Which categories of problems do SELF- DISCOVER perform the best? (4.2 and 3) Can SELF-DISCOVE: boost LLM performance efficiently? (4.3) Finally, we will show qualitative examples self-discovered structures, LLM output following the structures, and compare with LLM output following other prompting methods for reasoning (4.4
4.1 Does SELF-DISCOVER Improve LLM Reasoning?
Overall, SELF-DISCOVER improves PaLM 2-L and GPT-4's reasoning across diverse set of reasoning tasks. Table shows the overall results on complex reasoning tasks of BBH, T4D and
accessed December 2023
For MATH, we use PaLM 2-L model with stronger instruction tuning to enable better instruction following of more complex reasoning structures.
5

---

Table Self-Discover significantly improves LLM reasoning across diverse set of 25 complex tasks: BBH, T4D and MATH. CoT: zero-shot Chain of Thought (Kojima et al. (022). PS: plan-and-sole and-solve prompting (Wang et 2023).
Method
BBH
T4D
MATH
PaLM 2-L
56%
30%
45%
PaLM 2-L CoT
60%
40%
42%
PaLM 2-I, PS
61% %
42%
49%
PaL. 2-L Self-Discover
69 %
50.5
GPT-4
58%
51%
70.5%
GPT-4 CoT
75%
52%
71%
GPT-4 + PS
73%
53%
70%
GPT4 Self-Discover
81 %
85%
73%
MATH using PaL. 2-1. and GPT-4. We compare Self-Discover with baselines including direct prompting, CoT, and Plan-and-Solle (PS).
On aggregated 23 tasks of BBH, SELF-DISCOVER achieves 7% and 6% absolute improvement on PaLM 2-L over Chain-of-Thought and Plan-and-Solve, respectively. Similar gains (6% and 8%) are observed when SELF-DISCOVER is applied to GPT-4. Breakdown results of each task's improvement over direct answering and CoT of PaLM 2-L are shown Figure ] where find SELF-DISCOVER outperforms them on over 20/24 tasks. For a per-task performance for all 23 BBH tasks, please refer Appendix C
On the grounded social agent task T4D, SELF-DISCOVER reaches over 2 27% (32%) absolute improvement all baselines on PaLM 2-L (GPT-4). SELF-DISCOVER achieves 69% and 85% accuracy on PaL.M 2-L and GPT-4, significantly outperforming previous SoTA prompting method such Foresee and Reflect (FaR) which employs expert-designed reasoning structure. In contrast, SELF-DISCOVER generates the reasoning structure automatically from a set of atomic reasoning modules without human interventions.
For MATH, we observe a moderate gain of 1%-7% (2%-3%) on PaLM 2-L (GPT-4) from SELF- DISCOVER compared to the baselines. Upon еπтог analysis (see Appendix E for details), find that the reasoning structures generated by PaLM 2-L from SELL--ISCOVER are correct 87.5% of the time: human experts follow the reasoning structures to solve the tasks perfectly. The majority of the failures (74.7%) comes from errors in executing the computations, consistent with prior findings Zheng al. 2023).
4.2 Which Types of Problems Do SELF-DISCOvER Help the Most?
SELF-DISCOVER performs best on tasks that require diverse world knowledge. Figure presents presentss the av- erage improvement in terms of delta in accuracy of DISCOVRR over direct answer and CoT on 4 cate-
Self Geff-Discover Performance Improvement Across 4 Categories
Self-Discover Over Direct
Self-Discover Over CoT
Avg. Accuracy Delta (95]
20
14.2
34.3
gories of reasoning tasks we test. We
12.0
adopt the categorization from Suzgun
10
et al. (2022). We find that SELFF-
8.0
DISCOVER improves over these two baselines all categories, but pecially on tasks that require world knowledge such as sports understand- ing, movie recommendation, and ruin names.
4.0
Multilingual
Algorithmic
NLU
World Knowledge
Problem Categeries
Figure Breakdown SELF-DISCOVER performance im- provement on categories on PaLM 2-L. SELF-DISCOVER per- forms the best on tasks requiring world knowledge.
These tasks demand models to reason using fact and general commonsense knowledge. We interpret SELF-DISCOVER advantages on these tasks as strength from integrating multiple reasoning modules from various perspectives only applying CoT might miss key knowl-

---

BBH-Movie Recommendation
GBommeotetric Shapes
85
Average Accuracy
55
Self-Discover
Direct
75
45
Pan-and--So Solve
Majority vot each AM
40
65
0
10
20
30
10
20
30
Inference Calls Instance
Figure 5: Comparison of accuracy with number of inference calls required per instance. For CoT-Selff- Consistency, we sample times. Best each method requires gold labels (*). DELF-DISCOVER requires only inference call per instance, same as Direct and CoT while reaching better performance compared with 40x more call required methods on GPT-4, We acknowledge that SELF-DISCOVER input and output are longer than CoT and Direct prompting, increasing cost. However, the number of instances increases, the efficiency of SELF- DILL--DDSOOER in terms of inference per instance is highly desirable.
edge in the reasoning process. We observe that the gain on the Algorithmic category is moderate, consistent with the findings from Sec. 4.1 on MATH.
4.3 How Efficient SELI-DISCOVER?
SELF-DISCOVER achieves better performance while requiring 10-40x fewer inference computer compared to self-consistency or majority voting. Here we examine a subset of tasks from BBH and present a more thorough comparison of methods including those requiring many inference calls that are too costly to run on all 24 tasks. Figure shows average accuracy and number of inference calls required per instance for each method using GPT-4, Accuracy wise (y-axis), find that SELF-DISCOVER outperforms other baselines even those that require repeated inference calls such as axis). SELF-DISCOvED only requires one call per instance and three more inference calls on the task-level, DUT-self-consistens requires 10 times more since we have to sample 10 times for each instance, and methods using each RM requires 40 times more as we use 40 RMs.. In summary, SELF-DiscoVER presents itself strong reasoning boosting method that efficient to deploy on large-scale.
consistency and majority voting of applying each RM. Efficiency wise (x-
4.4 Qualitative Examples
We show examples of mosee-discovered structures for different reasoning tasks in Figure I0 from PaLM 2-L. We observe that each structure is uniquely adapted to the task, integrates multiple reasoning modules, and provides insights on how to solve the tasks. Furthermore, example of comparing reasoning processes from Co'T, Plan-and-Solve, and SELF-DISCOVER is shown in Figure We find that CoT and Plan-and-Solve makes incorrect assertions early and arrives at a wrong answer while following structure from SELF-DISCOVER leads the model to generate logical conclusions ("path is closed as the beginning and ending coordinates are the same") and arrive at the correct answer.
5
Deep Diving Into Self-Discovered Reasoning Structures
After experimental results showing the effectiveness and efficiency of SELF-DISCOVER on range of reasoning tasks, this section further analyzes are all actions of SELF-DISCOVER needed and what other benefits can self-discovered structures bring? Sec. 5.1 we show that critical to the model's performance to use the reasoning structures discovered through the three steps of SELECT, ADAPT and IMPLEMENT. In Sec. 5.2, we demonstrate the universality of self-discovered reasoning structures by (1) applying the structures discovered by PaL. 2-L to GPT-4, (2) applying the structures discovered by GPT-4 to Llama-2-70B. We further show the commonalities between the reasoning structures and human reasoning patterns in Appendix
7

---

This SVG path element <path d= 55.57,80 ww..57.8000000 7,80.99 57 38,88.88800000 38,66,80...... 48.90,57 48.90.57 46 45.58,47.78 45.58.47.78 L. 53.25,36.07 66.29,48.90 78.69,61.00 55.57,80.68 69 draws a:
(A) circle (B) heptagon hexagon kite line octagon (G) pentagon(H) rectangle sector (J) triangle
Let's think step-by-step (CoT)
Let's devise plan and solve problem (Pla-aadd-Solve)
Self- Discover Reasoning Structure (Ours)
This BVG path consists of mutiplc tines drawn
This SVG Path elament consists many "L" commendde which drew straight from point point. But doas not cicse the path, meaningg the shape not reg dar potygon like hexagon, replagon, pantagon.
"Bimplity svG Path":
"Breakdown Fatb Commands"
belween different points
(80 "Sets
points
command without drawing anything."
regular geometric shape such circle
new
"Lice to Command
steps":
("Stazt and and coozdinates of wach 1ime aegrentt.. "M (5.57,00.91 38.85.60 From point (55.57 60/ to (57.56,
pentagon, heptagon,
haxagon, or octagon.....
65 80) and finally closing the shape (85.57, 60.691")
They do, however, seam to form shape with four which considered a form kite..
'Critical thinking Analysia':
"Legical
"Analyaia path continuity" by: path by mutiill segments that connect distinct points.
instructinn in total but they are not straight and not form paralle ogram ractangle...
connecting back the starting point, indicating a closed shape.". "Identi fatioon of closed shapes : "The final line segment connects the list point back to the fistt point, which is cheracteristic of closed shape. ".
Thus the final answer
(D) kito. X
Dacision" "With distinct points connacted in closed path, shape formed is hoptagon." Triell Anavez": 'B) heptagon')
Reasoning
(D)
Figure Comparison of generated reasoning process from CoT, Plan-and-Solve, and DIIL--DER on BBH-geometric shape task. Both CoT and Plan-and-Solve incorrectly asserts that the path does not form a regular shape as it is not closed path (highlighted in red) and arrive at wrong answer. The reasoning structure (in blue Courier font) from SEL.F- DiscoODE first breaks down each line segment and analyze the coordinates carefully, then leverages logical reasoning to conclude that forms closed shape the path ends the same coordinate (highlighted in purple and orange), and selects the correct answer through final reasoning.
5.1 Importance of DILF-DISCOVER Actions
We conduct ablation study on the three actions: SELECT, ADAPT, and IMPLEMENT to analyze the effects of SELF-DISCOVER actions. Figure 7 show results using GPT-4 on 4 rea- soning tasks when we apply SELECT (-S) or apply SELECT and ADAPT (-SA) or apply all three actions. We find that with each stage, model's zero- shot reasoning capability improve con- sistently across tasks, indicating that all three actions are beneficial.
100
Ableton Studies on Self Discover Actions: SELECT. ADAPT, SEAPC,.ADAPT, IMPLEMENT (SAI)
33
94
89
86
Ours-S
80-
80
Ours-SA
Ours
73 73
Accuracy
70
65
61
43
45
40
Snarks
Movie
Geametry
Tasks
Figure 7: Ablation study three SELF-DISCOVER actions reasoning tasks: all three actions are beneficial for task-solving.
5.2 Towards Universality of Discovered Reasoning Structures
Applying PaLM 2-L Discovered Structures to GPT-4 We first use
Transferrability
Optimized Prompts/Structures GPT.4
100
94
94
OPRO*
PaLM 2-L model discover the reasoning structures of reasoning tasks. Then, apply the resulting reasoning structures to the decoding of GPT-4 as grounding. We compare our approach to OPRO (Yang et al. 2023) which discovered zero-shot- prompts through optimizations. We apply OPRO prompts optimized using PaLM 2-L on each task to GPT-4 on the same reasoning tasks. Figure g outper-
90
Self-Discover
80
79
74
Accuracy
70
60
57
52
50
40
30
20
Snarks
Movie
T4D
Geometry
Tasks
shows that SELF-DISCOVE forms OPRO on out of tasks de- spite that OPRO used 20% data to op-
Figure 8: Transferrability tests of optimized prompts (OPRO)
and composed structures -ISCOVE.))
8

---

timize the prompt. In contrast, efficiency of our method and universality of the discovered reasoning structures.
is done in a zero-shot manner, demonstrating the
Applying GPT-4 Discovered Structures to Llama2 and ChatGPT
Motivated by transferrability
performance across LLMs, we further investigate can self-discovered reasoning structures from LLMs boost reasoning for smaller LMs that are challenging to come up with structures themselve We use GPT-4 to discover the task-intrinsic reasoning structures, and then apply those structures to well GPT-3.5-
(ChatGPT) two subsets of tasks from BBH. We find that using desf-diic structures on Llama2 (52%) outperforms CoT (42%) on disambiguation QA zero-shot and GPTT-3.5-uure (56%) outperforms CoT (51%) on geometry with 3-shot demonstration from structured reasoning process.
the decoding of open-sourced Llama2-70B
6 Related Work
6.1 Prompting Methods
Recent advancements in the area of LLMs have given rise to plethora of few-shot (Brown et al. 2020)
and instruction (Mishra et al.] 2022c Wei et al. 2021 Ouyang et al. 2022) prompting techniques,
including Chain-of-Thouae
prompting (CoT) Nye et al. 2021] Wei et al. 2022), Least-to-most
prompting (Zhou et al.) 2022a Drozdov et al. 2022). Decomposed prompting (Khot et al. 2022),
Reframing (Mishra et al. 2022b), Help Me Think Prompting (Mishra & Nouri 2023), Stepback
Prompting (Zheng et al.] (023) and search- based approaches like Thouuht (ToT) (Yao et al.
2023a), Graph-of-Thoughh (Besta al.] [023 Yao et al., (023b),
solve-merge (Saha et al.
2023) and RAP (Hao et al. (023). Each of the prompting methods has some strengths and weaknesses in terms of their successful application domain. Our work SELF-DISCOVER presents the missing piece in the prompting literature, as RELF--DSERV provides a way to self-compose over various prompting methods via the proposed self-discovery mechanism. Composing over prompting methods in SELF-DISCOV is analogous to the programming literature where program is written using various basic building blocks such for loop, if/else condition etc.
6.2 Reasoning and Planning
With the development of various reasoning and planning benchmarks such as GSM8K Cobbe et al. 20021) Math [Hendrycks et al. BigBench Srivastava et al. (023) etc., various methods have been proposed to improve model performance. Often these methods induce specific reasoning structures mimicking the reasoning structure of the underlying task associated with the dataset. For example, chain of thought Wei et al. (2022) and scratchpad Nye et al. (202T) induce generation of explanations associated with a reasoning question. Similarly other methods induces specific reasoning structures such as question summarization Kuznia et al. (2022). question decomposition Patel et al. (0022). program generation Mishra et al. (2022a); Chen et al. (2022); :: Gao et al.] (2023b), etc. However, in real world user traffic, queries can be diverse covering various reasoning structures. Our work SELF-DISCOVER allows models to combine multiple reasoning approaches by self-composing into a structure without the need to access task labels. There have been some related work that explores LLM combining skills in-context such as SkiC (Chen et al.) 2023), devising a strategy (Gao et al. 2023a), and planning with iterative quering (Liu al. 2023). However, they require human annotating skills and reasoning plans while SELF-DIsCOVER leverages a scalable solution with the help of LLM's meta-task reasoning capabilities.
7 Conclusion
We introduce SELF-DISCOVER, efficient and performant framework for models to self-discover reasoning structure for any task from a seed set of general problem-solving skills. We observe drastic improvements challenging reasoning benchmarks from multiple LLMs up to 30%. Ablations study of SELF-DISCOVER demonstrates that the composed reasoning structures are universally transferable between L.MMs. Forward looking, are excited explore more LLM structured reasoning to push the boundary of problem-solving and discover potentials for Human-Al collaboration.
We tried shot meta prompting Llama2 but observed low oww-quality structure outputs.

---

Acknowledgement
We thank anonymous reviewers for their constructive feedback during the discussion period. We thank team members from Google DeepMind, INK Lab, and JAUNTS Lab for their insightful feedback on this paper. This work was funded in part by the Defense Advanced Research Projects Ageney with awards HR001 12220046, HR001 12390061, and 166001192400
References
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P. Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:2305. 10403, 2023.
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Gianinazzi, L., Gajda, J., Lehmann, T., Podstawski, M., Niewiadomski, H., Nyczyk, P., al. Graph of thoughts: Solving elaborate problems with large language models. arXiv preprini wxxv:2308.09667, 2023.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Chen, J., Pan, X., Yu, D., Song, K., Wang. X., Yu, D., and Chen, J. Skills-in-contex prompting: Unlocking compositionality in large language models. arXiv preprint arXiv:2308.0030/ 2023.
Chen, W. Ma, X., Wang, X., and Cohen, W. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.122 12588. 2022.
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.0231/ 2022.
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction finetunn-deddd language models. arXiv preprint arXiv:2210.1141 2022.
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint 10.1::1, 2021.
Drozdov, A., Schärli, N., Akyürek, E., Scales, N., Song, X., Chen, X., Bousquet, O., and Zhou, D. Compositional semantic parsing with large language models. arXiv preprint arXiv:2209.15000 2022.
Fernando, C., Banarse, D., Michalewski, H., Osindero, S., and Rocktäschel, T. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309. 16797, 2023.
Gao, C., Jiang, H., Cai, D., Shi, S., and Lam, W. Strategyllm: Large language models strategy gener- ators, executors, optimizers, and evaluators for problem solving. arXiv preprint 2023a.
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G. Pal: Program- aided language models. In International Conference on Machine Learning, pp. 10764-10799. PMLR, 2023b.
Hao, S., Gu, Y., Ma. H., Hong, J. J., Wang. Z., Wang, D. Z., and Hu, Z. Reasoning with language model planning with world model. arXiv preprint arXiv:2305. 2499. 2023.
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang. F.,, Song, D., and Steinhardt, J. Measuring mathematical problem solving with the math dataset. Sort, 2(4):0-6.
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardr, J.
Measuring mathematical problem solving with the math dataset, 2021.
10

---

Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P., and Sabharwal, A. Decomposed prompting: A modular approach for solving complex tasks. In The Eleventh International Conference on Learning Representations, 2022.
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22 09-22199-32 199922211,, 2022.
Kuznia, K., Mishra, S., Parmar, M., and Baral, C. Less is more: Summary of long instructions is better for program synthesis. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 4532-4552, 2022.
Liu, T., Guo, Q Yang, Y., Hu, X., Zhang, Y., Qiu, X., and Zhang, 7. Plan, verify and switch: Integrated reasoning with diverse x-of-thoughts. arXiv preprint arXiv:2310. /4628, 2023.
Mishra, S. and Nouri, E. HELP ME THINK: A simple prompting strategy for non-experts to create customized content with models. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 11834-1 1890, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 001111666/1122233 1 URL
antho ogy. wttg////.33.. -
Mishra, S., Finlayson, M., Lu, P., Tang, L.., Welleck, S., Baral, C., Rajpurohit, T., Tafjord, O., Sabharwal, A., Clark, P., et al. Lila: A unified benchmark for mathematical reasoning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 5807-5832, 2022a.
Mishra, S., Khashabi, D., Baral, C.,, Choi, Y., and Hajishirzi, H. Reframing instructional prompts to gptk's ' language. In Findings of the Association for Computational Linguistics: ACL 2022, pp- 589- 612, 2022b.
Mishra, S., Khashabi, D., Baral, C., and Hajishirzi, H. Cross-task generalization via natural language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 3470 3470-3487, 2022c.
Newell, A., Shaw, J. C., and Simon, H. A. Elements of theory of human problem solving. Psychological review, 65(3):151, 1958.
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114 2021.
OpenAl. Chatgpt: Optimizing language models for dialogue, 2022. URL https://openai
com/ Com/blog/chane chattttttttt//
OpenAl. Json generation mode, 2023a. URL http:://plattfommoopenn.t
gui.es/text--eeeeeeeeeeeeeeeeeee modde
OpenAl, R. Gpt-4 technical report. arXiv, pp. 2303-08774, 2023b.
Ouyang, L.., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C.,, Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
Patel, P., Mishra, S., Parmar, M., and Baral, C. Is question decomposition unit all need? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 4553 4569, 2022.
Polya, G. How to solve it: A new aspect of mathematical method, volume 85. Princeton university press, 2004.
Rasmussen, J. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE transactions systems, man, and cybernetics, (3):257-266. 1983.
Saha, S., Levy, O., Celikyilmaz, A., Bansal, M., Weston, J., and Li, X. Branch-solve-merge improves large language model evaluation and generation. arXiv preprint arXiv:2310.15123, 2023.
11

---

Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Aonso, A., et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, 2023.
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q V., Chi, E. H., Zhou, D., et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprini arXiv:22 10.00266,, 2022.
Touvron, H., Martin, L.,, Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.0922
2023.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. Attention is all you need. In Advances in Neural In- formation Processing Systems, volume 30. Curran Associates, Inc., 2017. URL ottp://aeeeeearrooe.ooo/eeeeeeee https: 3ffe244554766e1111bbcccccc...... Paper,
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W,, and Lim, E.-P. Plan-and-solve prompt- ing: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.0409, 2023.
Wang. X., Wei, J., Schuurmans, D., Le, Q. V., Chi, H., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2022.
Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. Finetuned language models zero-shot learners. In International Conference Learning Representations, 2021.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-244833 2022.
Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. arXiv preprint arXiv:2309.03400 2023.
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305. 10601, 2023a.
Yao, Y., Li, Z., and Zhao, H. Beyond of effective graph-of-thought reasoning in large language models. arXiv preprint arXiv:2305. /6582, 2023b.
Yasunaga, M., Chen, X., Li, Y., Pasupat, P., Leskovee, J., Liang, P., Chi, E. H., and Zhou, D. Large language models analogical reasoners. arXiv preprint 23 2023.
Zheng. H. S., Mishra, S., Chen, X., Cheng, H.-T., Chi, E. H., Le, Q. V., and Zhou, D. Take step back: Evoking reasoning via abstraction large language models. arXiv preprint 2rXiv:231000611,, 2023.
Zhong, R., Lee, K., Zhang, Z., and Klein, D. Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections. arXiv preprint vrXivv2104.000 0467000 2021.
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le. Q. V., et al. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning Representations, 2022a.
Zhou, P., Madaan, A., Potharaju. S. P., Gupta, A., McKee, K. R., Holtzman, A., Pujara, J., Ren, X., Mishra, S., Nematzadeh, A., al. How far are large language models from agents with 2023.
theory-of-mind? arXiv preprint arXiv:23
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations, 2022b.
12

---

A Self-Discover Prompt Details
Table 2 shows all 39 reasoning modules for SELF-DISCOVER, adopted from Fermando et al. (2023). that contain cognitive heuristics of wooblem-solving
Figure 9 contains the structure of the three actions of SEL.F-ISSOOER during Stage 1, where it discovers an intrinsic reasoning structure on the task-level.
For Stage 2, where we use the self-discovered structure to solve the task instances, we start with the prompt: "Follow the step-by-step reasoning plan in JSON to correctly solve the task. Fill in the values following the keys by reasoning specifically about the task given. Do not simply rephrase the keys.". ". followed by the reasoning structure, and finally the task instance.
SELECT
ADAPT
IMPLEMENT
Select several reasoning modules that are erucial to utilize in order solve the piven task:
Rephrase and specify each reasoning module so that it better helps solving the rask:
Operationalize the reasoning modules into step-by-step reasoning plan in ISON formati
Paired IMPLEMENT Step Demonstration
All reasoning module descriptions Critical thinking:... " Step-by-Stepr.... " Propose and verify:
SELECTED module descriptions:
Critical thinking:
Step-by-Stepe
Reasoning g description Example
Reasoning Plan Example
Task examples wio Example I:.. Example 2:
Task examples w/o answer: Example 1:
ADAPTED module description:
Example
Task examples w/o answer:
Adapt each rensoning module description to better solve the tasks:
Implement reasoning structure for solvers to follow step-by-step and arrive at correct answers:
Select several modules that crucial for solving the tasks above:
Figure 9: Meta-Prompts the three actions of SELF-DISCOVER. Each meta- prompt consists an instruction the beginning and end, reasoning module descriptions, task examples without labels. IMPLEMENT to show model an example of reasoning structure (plan), we present uuman-written structure in ISON for another task.
B Evaluation Details
We use accuracy and exact matching as with other methods tested on BBH, T4D and MATH. To properly evaluate the generated answers from LLMs, prompt the models end the answer with "Thus, the final answer is [X]", where X is either one answer option such as "A" or string such as "valid". During evaluation, we manually examine each task's 's outputs from LLMs and design heuristics to extract the final answers. For MATH dataset, we find that it is challenging to extract the answers accurately. As result, we subsample 200 test examples from MATH, and manually sanity check and annotate the extracted answers for all methods tested in our paper.
Here also provide more details token counts for Self-Discover and other baselines. Since Self-Discover only added 3 calls per task, on per instance level it only needs to run once. The average length of self-discovered structures is 224 tokens for BBH, 183 tokens for T4D, and 152 for MATH, which is similar to ToT/GoT prompts with around 234 tokens. We run the stage 2 (generating solution based on the structure) only inference pass where the model fills the values the self-discovered structures. Thus, Self- Seff-Discover still reduces much inference cost compared to CoT-Self Consistency, majority voting, etc. which requires 20s-40s fold of each CoT reasoning path token counts.
C BBH Per Task Performance
Per-task performance BBH (23 tasks in total) shown in Table 3
13

---

Table 2: All 39 reasoning modules consisting of high-level cognitive heuristics for problem-solving. We adopt them from Fernando et al. (2023).
Reasoning Modules
How could devise an experiment to help sulve problem?
2 Make list of ideas for solving this problem, and apply them one hy one the problem to see if any progress can he made.
3 How could measure progress this prublem?
4 How can I simplify the problem so that it is casier to solve?
5 are the key assumptions underlying this problem?
6 What are the potential risks and bucks of each solution?
7 are alternative perspectives or viewpoints on problem?
8 What are the long-term implications of this problem and its solutions?
9 How break down this
into smaller, more manageable parts?
10 Critical Thinking. This style involves analyzing the problem from different perspectives, questioning assumptions, and evaluating the evidence information available. focuses on logical reasoning. evidence-based decision-making. and identifying potential biases or flaws in thinking.
Try creative thinking. generate innovative and out-of-the-box ideas solve the problem. Explore unconventional solutions, thinking beyond traditional boundaries, and encouraging imagination and originality.
Sock input and collaboration from others to solve the problem. Emphasize teamwork, open communication. leveraging the diverse perspectives and expertise of a group to come up with effective solutions.
Use systems thinking: Cousider the problem part of a larger system and understanding the interconneetedness of various elements.
Focuses on identifying the underlying causes, feedback loops, and interdependencces that influence the problem, and developing holistic solutions that address the system whole.
14 Use Risk Analysis; Evalunte potential risks, uncertainties, and tradeoffs associated with different solutions or approaches to a problem. Emphasize assessing the potential consequences and likelihood success failure, and making informed decisions based on a balanced analysis of risks and henefits.
15 Use Reflective Thinking: Step back from the problem, take the time for introspcction and reflectit, Examine personal biases, assumptions, and mental models that may influence problem-solvingg and being open to learning from past experiences to improve future approcches.
16 What the core issue or problem that needs to addressed!
17 What are the underlying
factors contributing to the problem?
18 Are there any potential solutions or strulegies that have been tried before? If yes, what were the outcomes and lessons learned?
19 What are the potential obstacles challenges that might arise in solving this problem?
20 Are there any relevant data or information that can provide insights into the problem? If yes, what data sources are available, and how can they be analyzed?
Are there any stakeholders or vidivisuals who are directly affected by the problem? What are their perspectives and needs?
22 What resources (Anancial. human, technclogical. etc.) are needed to tackle the problem effectively?
How can progress or success in solving the problem be measured or evaluated?
24 What indicators metrics can be used?
Is the problem technical or practical one that requires specific expertise or skill set? Or more of a conceptual or theoretical problem?
Does the problem involve physical constraint, such as limited resources, infrastructure, or space?
27 is the problem related human behavior, such social, cultural. or psychological issuc?
Does the problem involve docision-making or planning, where choices need to be made under uncertainty or with competing objectives?
Is the problem an analytical one that requires data analysis, modeling, or optimization tochniques?
30 Is the problem design challenge that requires creative solutions and innovation?
Does the problem require addressing systemic or structural issues rather than just individual instances?
32 Is the problem time sensitive or urgent, requiring immediate attention and action?
problem specification? 34 Given the problem specification and the current hest solution, have a guess about other possihle solutions.
What kinds solution typically are produced fur
imagine the current solution totally wrong, what 36 What the best way to modify this current best solution, given what you know about these kinds of problem specification? Ignoring the current best solution, create an entirely new solution to the problem.
ways are there to think about problem specification?
38 Lat's think step by step.
make step by step plan and implement it with good notion and explanation.
Table 3: Big Hench-Hard (Suzgun et al. 2022 per-task performance of GPT-4 and PaLM 2-L with
Big Bench-Hord Task
Homan (Ang.)
Human (Max)
GP144 Direci
GPT4 CoT
GPT4 Seelf-Disower
PaL
Direct
PLLMEL
+ CoT
PALACLL -Sell-Disoover
bootcan, expressionss
79
100
73
B5
83
71
84
St.
causal jalgerment
70
100
67
75
80
46
59
date understnddng
77
100
74
73
78
78
disamiigaatio..
67
93
54
dyck languages
48
formal, Jullacies
91
100
geometric shapes
34
Toperbaton
75
100
68
80
75
logacal _ddductioooeoooooooo seven_obyects
40
89
(1)
70
70
45
39
50
mowie rocommnndation
61
90
70
70
86
83
54
66
beabsste__rihmmmticcog two
10
25
10
92
70
4
50
navigate
82
100
70
901
90
38
63
object couming
36
100
50
100
100
27
44
70
penguins lbbl
78
100
50
100
70
67
75
75
100
36
79
78
100
79
salient tramslation, erver detection
snarks
100
58
sports andesstandng
100
47
temporal sequences
91
100
96
100
99
97
trakingg stuttee wtw.t/tllooyeeettseeeoooooo bttooeeececcc_ooeooooeeett
65
100
24
22
58
30
wit of lies
S1
100
15
80
71
54
42
67
wond_ suring
63
100
65
90
85
12
4
15
14

---

reasoning about colored objects
causal judaement
Type and color of each item":
"Identify the chain of events the story":
"Number of items of each color's :
"Number items each type" :
"Identify the tonsequences
"Number items
color
event":
and type":
"Identify the cause-and-effect
"Final answer":
Break down
relationships between events":
sub-tusks
"Choose a final answer based on the reasoning".
Reflect task nature
dyck languages
Devise algorithm
"Parentheses that are not closed properly":
"Stack to store the closing parentheses",
closing parenthesis, pop the stack and check popped symbol matches the symbo!":
the next symbol
"If the stack is empty, add the next symbal to the stack":
Figure 10: Examples self-discovered structures on BBH tasks using PaLM 2-L. We observe traits of atomic reasoning modules such as "step-by-ste" thinking", "reflect on task nature", and an interesting creative thinking case where models devise an algorithm using stack to solve parenthesis parsing task.
Table 4: Additional baselines including Tree-of- The-uuhh (ToT) and Graph-of-Thought (GoT) [Added rows].
Method
BBH
T4D
MATH
PaLM 2-L
56%
30%
45%
PaLM 2-L CoT
60%
40%
42%
PaLM 2-L ToT
58%
41%
44.5%
PaLM 2-L GoT
60%
40%
40%
PaLM 2-I, PS
61 %
42%
49%
Pal. 2-L. Self-Discover
69 %
50.5 %
GPT-4
58%
51%
70.5%
GPT-4 CoT
75%
52%
71%
GPT-4 + ToT
76%
50%
69%
GPT4 GoT
75%
52%
70%
GPT4+ +
73%
53%
70%
81%
85%
73%
D Additional Experiments
We further include Tree-of- Thouught (Yao et al. 2023a) and Graph-of-Thought (Besta et al. 2023 (zero-shot versions) as baselines for comparison shown in Table 4
To show the effectiveness of Self-Discover on more general tasks, tested on subset of MMLU (10 subtasks, with 50 diverse questions each, all randomly sampled) and results are shown in Table 5 find that GPT-44Self-Disoovrr Dilio-Dieeverr wins GPT-4+CoT zero-shot on out of 10, ties on 2 out of 10, and loses on I out of 10 tasks. In addition, we tried Self-Discover on the instance-level, where for each question, we run stage to output the reasoning structure, then solve the task. We find that the instance-level Self-Discover performs even better MMLU, outperforming CoT by 7.2% average for all tasks. This result, combined with those in main content, shows that the strength of Self-Discover spans across two types of tasks: for well-defined hard reasoning tasks such as BBH, task-level Self- Discovis works well while being very efficient; for very open-domain tasks such as do instance-level Self-Discover, which significantly outperforms CoT while still fewer inference required than consisensst..cy.
MMLU,
We show additional examples of self-discovered structures in Figure 10 We observe traits of atomic reasoning modules such as "step-by-step thinking", "reflect on task nature". and an interesting creative thinking case where models devise algorithm using stack to solve parenthesis parsing task.
15

---

Table 5: MMLU (Suzgun al. 2022 per-task performance of GPT-4 and PaLM 2-L with SELF- DISCOVER. We sampled 10 tasks with 50 examples each. SD (instance) refers to that we run stage one on each question and use the generated structure during solving, to acount for the diversity of questions. [New Table]
GPI-4
GPT-4
GPT-4 +SD
GSD
P4LM
P-LM2-L
PaLM2-L
PaLM 2-L+SD
MMLU Tasks
Direct
CoT
(instance)
Direct
CaT
+ +SD
(instance)
celics
78
83
91
83
high school world_history
64
69
74
83
59
61
66
machine Jeaming
72
80
88
75
78
college_mediiingg medicine
45
52
50
54
44
45
45
49
high school staristics
75
interaatonal law
70
77
77
82
60
69
63
71
conceptual physics
marketing
71
75
76
82
67
69
71
74
jurisprudence
70
76
69
moral dispules
62
68
69
73
65
66
68
Task :asi-aavigttion: If you follow these instructions, do you return to the starting point? Always face forward. Take 1 step backward. Take 9 steps left. Take 2 steps backward. Take 6 steps forward. Take 4 steps forward. Take 4 steps backward. Take 3 steps right.
Human-Written Structure:
Midee-Discovered Structure:
Shared Structure: Step-wise
"Position after instruction 1
"Beek down instructions into individual movements
"Position after instruction 2":
Instruction 1
mental notes
"Effect on position after Instruction 1":
"Position after instruction n": "Is final position the starting
'nstruction ":
Effect on position after Instruction
position?":
Additional instructions present":
Simplify the sequence of movemnnts's 'Simplified representation of series":
Figure 11: Case study of human-written structure shares commonalities with L.M--dixoveered reasoning structure. We observe similar reasoning patterns-both both structures contain step-wise analysis each instruction.
E
Error Analysis
We perform error analysis of the failure modes. We manually annotate whether the generated reasoning structure correct or not together with whether the correctness of model prediction using SELF-DISCOVER, A reasoning structure defined as correct a human expert can solve the task by simply following the reasoning structure.
the MATH dataset of 200 samples to understand
Out of 200 examples, we find that 87.5% (175) examples have correct reasoning structures. 12.5% (25) examples have incorrect reasoning structures leading to prediction errors. Table 6 shows 4 such examples where the LLM misunderstands the task, or makes an error one of the steps or adds unnecessary steps in the reasoning structure.
Next, analyze the errors made by the model in SELF-DISCOVER model prediction is wrong, wrong reasoning structures account for only 25.3% the errors. The remaining 74.7% errors due Table shows 3 examples of such error.. This insight indicates that future improvements should aim improving the step-wise calculation accuracy of LLMs, such using tools or code generation.
of 99 examples where the
in the intermediate calculations such math computations.
Further Anaysis
Model-Discovered Reasoning Structures vs. Human Reasoning Patterns We investigate whether I. --dd-oeeeeee reasoning structures share some commonalities with human reasoning patterns. We give humans 3 task instances without labels and example reasoning structure (same SELF-
16

---

Table 6: Examples of wrong reasoning structures for MATH. The first emror in the reasoning structure highlighted in red.
Prompt
Reasoning Structure
Error
How many numbers between I and 2005 are integer multi-
1. Find the number of multiples of between I and
Need to sub- tract the num- ber of mul- tiples of 12 twice instead of once.
2005.
ples of 3 or but not 12?
2. Find the number of multiples of 4 between and 2005.
3. Find the number of multiples of 12 between and 2005.
4. Add the number of multiples of and 4.
5. Subtract the number of multiples of (to avoid double counting).
6. The result the number of integers between and 2005 that are multiples of 3 or 4 but not 12.
How many numbers are in the list 1,7,,,,,,,14,15... 94,95,98?
1. Find the number of pairs in the list.
LLM misun- derstands the task: the rea- soning struc- ture is inrele- vant.
2. Find the number of numbers in each pair.
3. Multiply the number of pairs by the number of numbers in each pair to find the total number of numbers in the list.
A restaurant offers three
1. Define the variables:
There are 365 nights in the year 2003.
desserts, and exactly twice
M: number of main courses
as many appetizers as main
A: number of appetizers (twice the number of
courses. dinner consists of
main courses)
an appetizer, main course,
D: number of desserts (fixed at 3)
and a dessert. What the
T: total number of possible dinner combinations
least number of main courses that the restaurant should of- fer that customer could have a different dinner each night in the year 2003?
2. Calculate the total number of possible dinner
combinations: T M DD
3. Set the total number of possible dinner combi- nations equal to the number of nights in the year
2003:
2003
4. Substitute the expression for A in terms of M:
T M * (2 * M) * 3
5. Solve the equation for M:
6M 2003
6. Find the smallest integer value of M that satis-
fies the equation.
Determine the number of ways to arrange the letters of the word NINE.
1. Count the total number of letters in the word NINE
Step 5 un- necessary and leads to error in the final re- sult.
2. Identify the number of times each letter appears
in the word.
3. Calculate factorial of the total number of letters.
4. Divide the factorial by the factorial of the num- ber of times each letter appears.
5. Multiply the results from step for each unique letter.
17

---

Table 7: Examples of wrong calculations for MATH. The first error in the intermediate computations highlighted in red.
Prompt
Self-Discover Prediction
Error
How many ways are there to arrange 6 people around a circular table with 7 seats? (Two seatings are considered the same if one rotation of the other.)
"I. Calculate the total number of ways to arrange 6 people in 7 seats without considering rotations as the same.": ":
There are tations instead of 6 that are considered the same.
4 There are 7 choices for the first person.
5040
There 6 choices for the second person.
There are 5 choices for the third person.
There are 4 choices for the fourth person.
There are 3 choices for the fifth person.
There are 2 choices for the sixth person. "2. Divide the result from step by the number of rotations that are considered the same.": ": 5040 / 6 840
There are rotations that are considered the same.
A customer ordered 15 pieces of gourmet chocolate. The order can be packaged small boxes that contain 1, 2 or 4 pieces of chocolate. Any box that is used must be full. How many different combinations of boxes can be used for the customer's 15 chocolate pieces? One such combination to be included use seven 2-piece boxes and one 1 --pecce box.
"1. Find the number of ways to package the choco- lates using only boxes of piece.": ": 1,
There is way to package 15 chocolates us- ing only boxes of 2 pieces.
"2. Find the number of ways to package the choco- lates using only boxes of pieces." ":
"3. Find the number of ways to package the choco- lates using only boxes of pieces.": 0,
"4. Find the number of ways to package the choco- lates using combination of boxes of and 2 pieces.": 7,
"5, Find the number ways to package the choco- lates using combination of boxes of 1 and 4 pieces.":
"6. Find the number of ways to package the choco-
lates using a combination boxes of 2 and 4 picces.": 2,
"7. Find the number ways to package the choco-
lates using combination of boxes of 1, and 4 pieces.":
"8. Sum up the total number of ways from the 4= 18
seven cases above.": + + 0 7
How many primes are in the row of Pascal's Triangle that starts with followed by 6?
"1. Identify the row of Pascal's Triangle that starts
6 is not
with a followed by 6.": "1, 15, 20, 15, 6, 1".
prime.
"2. List all the numbers in that row.": [1. 6, 15, 20,
15. 6. 1].
"3. Check each number in the list for primality.":
[False, True, False, False, False, True, False),
"4. Count the number of primes found in the list.":
18

---

DISCOVER meta meta-reasonin reasoning stage) and ask them to write a reasoning structure for task before solving it. T shows comparison of human and LLM-composed reasoning structures on the BBH-navigation task. We observe similar structures such as mental-noting after each movement. From promising findings of L.LM self-discovered structures boost and share traits of human meta- reasoning, we hope to encourage more future work to study humna-Al collaboration for complex problem-solving.
To show whether the chosen reasoning modules and discovered structures provide diverse coverage, we have included more details on frequency of selected reasoning modules Table Furthermore, we observe a very diverse set of self-discovered structures as the 12 examples shown in Figure 6, 10, 11. and Table 6 and 7. Math problem structures (Table tend be very different from BBH ones (Figure 10). Due to the non non-detemiiiistic nature of LLMs, the number of reasoning structures that can be discovered by Self-Discover infinite, because can use multiple reasoning modules in many different orders. In our prompts, we specifically do not restrict how the structures should use the modules and find examples where models use new modules not in the seed list (Figure 10 Dyck-Language example where model devises a stack algorithm, which is not in the seed list).
19

---

Table 8: Selected reasoning module frequency on 25 sub tasks (BBH, T4D and MATH), for full reasoning module descriptions, please refer to Table 2 in Appendix. Top 5 selected modules are in bold.
Short Module
Reasoning Module Index
Frequency Over 25 Tasks
Description
(For Full see Table 2)
Devise experiment
9/25
Make a list of ideas
4/25
3
Mcasure Progress Simplify the problem
6/25
17/25
5
Key assumptions
10/25
Risks of solution
3/25
7
Alternative perspectives
12/25
8
Long-term implications
2/25
Break down to smaller problems
14/25
10
Critical thinking
18/25
11
Creative thinking
12/25
12
Seek input from others
5/25
Systems thinking
11/25
14
Risk analysis
3/25
Reflective thinking
13/25
16
Core issues?
7/25
Underlying causes or factors
6/25
18
Solutions tried before
2/25
Obstacles or challenges?
10/25
20
Relevant data?
9/25
21
Stakeholder needs?
1/25
22
Resources needed?
8/25
23
Evaluate progress?
9/25
24
Metrics can be used?
7/25
25
Technical or practical problem
4/25
26
Physical constraint?
1/25
27
Human behavior?
3/25
28
Decision-making planning
15/25
29
Analytical or optimization
7/25
30
Design challenge?
2/25
31
Systemic or structural issues? Time-sensitive?
4/25
1/25
33
Typical solutions?
5/25
34
Other possible solutions
12/25
35
Imagine current solution is wrong
8/25
Modify current solution
7/25
37
Create entirely new solution
5/25
Think step-hy-step
17/25
39
Step-by-stey plan with explanation
21/25
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
We accurately report the results from our experiments in the abstract and introduction.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We present an extensive analysis of the limitation of Self-Discover in both Sections and with an extended error analysis on MATH in Appendix E.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
21

---

Answer:
[NA]
Justification:
We do not present theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We provided detailed prompts, explanation of how each inference procedure, and each model specification to reproduce the main experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
22

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: Due to legal constraints, not able fully release the code and data the current time. But we provide, with the best faith, details to reproduce our results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and data submission guidelines (https://nips.. cc/
public/ /oddee//ooeeeuunnnne ss onn ollic)) for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See NeurlPS code and data submission guidelines (https
/ //nips. httccc/pbbbt wtcp/pbblcc/gee wttp//uiieeeeeee// Sudde ss onPol icy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide details of inference procedures including initial prompts, frame- work of each data/test split, and model specifications.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
We include discussions of statistical significance and most improvements are significant.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We specify the LLM models we use. And since our experiments do not involve large-scale training, it is not computationally costly.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics /tth://htttss://eurri http://errrrs...ccooooooocccctcc
ines?
Answer: [Yes]
Justification:
We follow NeurlPS code of ethics carefully.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work is foundational study model's reasoning capabilities.
Guidelines:
The answer NA means that there is no societal impact of the work performed.

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cite all models, data, and code we use in our experiments.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
25

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paaperswi paperswithoooes com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not involve extensive crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
We do involve extensive crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
26

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
27


Reviews:
[{"review_id": "JPbxYA878J", "reviewer": "Reviewer_5mv1", "summary": "This paper proposes a prompt engineering scheme:\nGiven a task and 39 prompts for solving tasks, \nan LLM is prompted to select which of the 39 prompts are suitable for the task.\nThe selected prompts are then rephrased to be more specific to the task, \nand reformatted in json.\n\nThe paper uses GPT-4 Turbo, PaLM 2-L, and Llama2-70B\nand compares direct prompting, chain of thought, plan and solve, \nchain of though with self-consistency, majority vote of the 39 prompts, and the proposed method.", "questions": "When comparing prompting schemes, it is important to give equal compute time to different methods. \nWhen measuring efficiency did the work consider the lengths of the prompts and responses?", "limitations": "Yes", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This prompting scheme performs best on tasks that require diversity,\nand the work demonstrates the value of prompt diversity.\n\n2. Ablation experiments show the increased accuracy for each of the three steps of selecting prompts, rephrasing them, and reformatting (or implementation), over chain of thought.", "weaknesses": "1. As a prompt engineering method, it is unclear if the choice of the specific 39 prompts will withstand the test of time across LLMs and tasks. \n\n2. Recent prompt engineering tools, such as Anthropic's Claude 3.5 Prompt Engineer, receive a task as input and output a detailed prompt best suited for the task and LLM, which may supersede this approach. \n\n3. The claim of efficiency should be more rigorously validated and consider both the number of calls and the token lengths of the prompts and responses of each call.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your helpful feedback and suggestions. Our responses to weaknesses and questions are as follows:\n\n__[Specific 39 seed modules]__\n\nThanks for the very important point. The main contribution of the paper is the self-discover method and we demonstrate its effectiveness with the current 39 reasoning modules. Future work can certainly expand to more prompts to increase diversity and effectiveness of the method.\n\n__[Comparison to Anthropic’s PromptEngineer]__\n\nAs a piece of science work, we fully and openly discuss how our method discovers a structured prompt using LLM meta prompts with details. It is unclear how Prompt Engineer from Claude works as it is proprietary. We believe papers such as Self-Discover present an important step to open science.\n\n__[Efficiency: number of calls and token lengths]__\n\nWe compared the number of calls per instance in Figure 5 x-axis. Since Self-Discover only added 3 calls per task, on per instance level it only needs to run once. The average length of self-discovered structures is 224 tokens for BBH, 183 tokens for T4D, and 152 for MATH, which is similar to ToT/GoT prompts with around 234 tokens. We run the stage 2 (generating solution based on the structure) only in 1 inference pass where the model fills the values in the self-discovered structures. Thus, Self-Discover still reduces much inference cost compared to CoT-Self-Consistency, majority voting, etc. which requires 20s-40s fold of each CoT reasoning path token counts. We will add such details in the next version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4dYa7ZXkGn", "reviewer": "Reviewer_i8mj", "summary": "The paper introduces SELF-DISCOVER, a novel framework that enables large language models (LLMs) to self-discover and compose reasoning structures for tackling complex reasoning tasks. The core of SELF-DISCOVER is a self-discovery process where LLMs select multiple atomic reasoning modules, such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure to follow during decoding.", "questions": "See weaknesses.", "limitations": "See weaknesses.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1) SELF-DISCOVER significantly improves the performance of state-of-the-art LLMs on complex reasoning tasks, showcasing substantial gains over traditional prompting methods like Chain of Thought (CoT).\n2) The discovered reasoning structures are not only effective but also transferable across different model families, indicating a level of universality in the approach. SELF-DISCOVER provides a more interpretable way to understand the reasoning process of LLMs by generating explicit reasoning structures.\n3)  The paper includes a thorough empirical evaluation on a diverse set of reasoning tasks, demonstrating the framework's effectiveness across various domains.", "weaknesses": "1) The paper does not specify the exact number of reasoning structures that can be discovered by the SELF-DISCOVER framework. It's unclear whether the framework can generate a wide variety of structures or if it tends to converge on a limited set of common structures.\n2) It is not detailed whether different examples within the same task utilize the same reasoning structure or if the framework can adapt the structure to suit the nuances of individual examples.\n3) The paper's comparative analysis may be considered basic, as it does not delve into a comprehensive set of existing methods or include recent state-of-the-art approaches for comparison.\n4) Although the paper mentions a similarity to the \"Meta Reasoning for Large Language Models\" approach, it does not provide a direct comparative experiment or analysis to highlight differences and potential advantages of SELF-DISCOVER.\n5) The paper might not fully account for or cite the most recent literature, specifically works from 2024 and beyond, which could provide additional context and comparison points for the SELF-DISCOVER framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your helpful feedback and suggestions. Our responses to weaknesses and questions are as follows:\n\n__[Variety of structures]__\n\nWe have included more details on frequency of selected reasoning modules in the attached 1-page pdf. Furthermore, we observe a very diverse set of self-discovered structures as the 12 examples shown in Figure 6, 10, 11, and Table 6 and 7. Math problem structures (Table 6)  tend to be very different from BBH ones (Figure 10). Due to the non-deterministic nature of LLMs, the number of reasoning structures that can be discovered by Self-Discover is infinite, because it can use multiple reasoning modules in many different orders. In our prompts, we specifically do not restrict how the structures should use the modules and find examples where models use new modules not in the seed list (Figure 10 Dyck-Language example where model devises a stack algorithm, which is not in the seed list).\n\n__[Examples within the same task use the same structure]__\n\nSelf-Discover Stage 1 (generate a structure from the task) operates on task-level (Figure 2), thus each example receives the same structure. This ensures the efficiency of the Self-Discover approach because otherwise, we have to run multiple inferences per instance of task (shown in Figure 5). Even if we do not tailor to each example of each task, we observe significant performance increases compared to CoT, CoT, GoT, Plan-and-Solve (Table 4). Future work can investigate how we can tailor the reasoning structure to each example efficiently. \n\n__[Comparative analysis]__\n\nWe compared many SoTA methods including CoT, Plan-and-Solve, Self-Consistency, OPRO, ToT and GoT (Tables 1 and 4, Figures 5 and 8).\n\n__[Differences from other meta reasoning]__\n\nIn Section 6.2, we discussed differences with related work of meta reasoning. We didn’t directly compare with methods such as SkiC, strategy reasoning, and iterative planning because they require human efforts (annotating skills and reasoning plans) or training examples (to tune prompt or optimize strategy) while SELF-DISCOVER focuses on proposing a scalable solution with the help of LLM’s meta-task reasoning capabilities. We did include Plan-and-Solve, which is directly comparable, in our results and find Self-Discover significantly outperformed it (Table 1 and Figure 5)\n\n__[2024 and beyond work]__\n\nThank you for the note! We included related work to the best of our knowledge. We will definitely include the most recent work from 2024 and beyond if you can kindly provide us with pointers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RvJ0l7Ueux", "reviewer": "Reviewer_Kd8z", "summary": "This paper introduces SELF-DISCOVER, a framework designed to enhance the reasoning capabilities of Large Language Models (LLMs) such as GPT-4 and PaLM 2. The framework enables LLMs to self-discover and compose intrinsic reasoning structures tailored to specific tasks, improving performance on complex reasoning benchmarks.", "questions": "1. What is the additional cost of implementing this method, such as the extra tokens generated or the increase in inference time?\n\n2. Please refer to the correct section number in the Appendix in lines 83-84.\n\n3. The comparison in Figure 5 may not be fair. The self-discovery method requires fewer calls but could lead to more token generations.\n\n4. The design of the \"select, adapt, and implement\" steps seems ad-hoc. Why are these three steps proposed separately? For instance, why can't \"select\" and \"adapt\" be combined into a single step since their prompts are similar, as shown in Figure 9?\n\n5. What are the commonly selected reasoning structures? Are all 39 reasoning modules useful? It would be better to show the density of the selected reasoning modules to ensure that the selection of 39 modules is meaningful.\n\n6. Does the model perform consistently in the selection process?\n\n7. Do different models, such as GPT-4 and Palm-2, perform consistently? Do they discover similar reasoning structures?", "limitations": "This paper does not sufficiently illustrate the limitations of the proposed method. The checklist on lines 542-543 states, \"We present an extensive analysis of the limitations of Self-Discover in both Sections 4 and 5 with an extended error analysis on MATH in Appendix E.\" However, Sections 4 and 5 are primarily general evaluations of the proposed method and do not contain sufficient and clear illustrations of its limitations. This issue needs to be strictly addressed.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- By enabling LLMs to self-compose reasoning structures, this method significantly improves performance on complex reasoning tasks, demonstrating the ability to handle intricate problems more effectively than traditional methods.\n\n- SELF-DISCOVER achieves superior results while requiring substantially fewer inference steps (10-40x fewer) compared to inference-intensive approaches like CoT-Self-Consistency.\n\n- The reasoning structures discovered by the SELF-DISCOVER method show strong transferability across different LLM families, highlighting its broad applicability and robustness.", "weaknesses": "1. While atomic reasoning structures form the foundation of the SELF-DISCOVER method, their inherent simplicity can restrict the overall reasoning performance. These basic units, although useful, may not fully capture the complexity needed for certain advanced reasoning tasks, potentially limiting the depth and flexibility of the reasoning process.\n\n2. The proposed method is significantly reliant on the performance of the underlying LLM. Factors such as the model's ability to follow instructions accurately and the limitations imposed by context window sizes can pose significant challenges. For instance, if the base LLM struggles with instruction-following or is constrained by a limited context window, the effectiveness of the SELF-DISCOVER method may be compromised, limiting its potential for broader application and scalability.\n\n3. The iterative process of selecting, adapting, and implementing reasoning modules introduces extra computational costs. Each step in this process requires additional queries to the LLM, which can accumulate and result in increased computational demands. This overhead may offset some of the efficiency gains achieved through reduced inference steps, particularly in scenarios requiring frequent adaptation and customization of reasoning structures.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your helpful feedback and suggestions. Our responses to weaknesses and questions are as follows:\n\n__[Simplicity of atomic reasoning modules]__\n\nThank you for the great point and we agree that the 39 atomic reasoning modules can be improved. We would like to note two points: 1. This paper’s main message is to show the effectiveness of the self-discover method despite the inherent simplicity of the atomic reasoning structures used here. 2. Future work can grow and expand the atomic reasoning structure to increase their diversity, complexity to capture more advanced reasoning tasks.\n\n__[Reliant on underlying LLM]__\n\nWe believe that LLMs will become increasingly capable for instruction following and with longer context windows, even with smaller models. For example post-training for instruction-following is considered basic requirements for any LLMs to work. And the context window size is typical for the community standard of 2K tokens. Therefore, we think that the required capabilities in LLMs for Self-Discover to work are common for current LLMs and will become increasingly accessible (such as Llama 3).\n\n__[Computation overhead and Q1, 3]__\n\nGiven a task, Self-Discover only adds 3 more inference calls on the task level (select, adapt, and implement), then 1 inference call per instance of the task (lines 84-87). The average token # for the task-level 3-step prompting is around 1200 (select-input), 212 (select-output), 230 (adapt-input), 250 (adapt-output), 291 (implement-input), and 193 (implement-output) and for instance-level prompting is around 212 (input) and 265 (output). We would like to emphasize that the higher token cost is per-task and thus the amortized cost per-instance is less than baselines that use iterative inference methods such as CoT-Self-Consistency and fewer token length as Tree/Graph of Thought.\n\n__[Questions]__\n\nQ2: We will fix the section number in the next version.\n\nQ4: 3-step design: Initially we tried fewer steps in the first stage, but found that the final reasoning structure is often low quality (also supported in ablation studies Figure 7). Then we split the discovery phase into 3 steps (SELECT, ADAPT, IMPLEMENT) so the difficulty of generating a structure directly can be mitigated, and find that the quality much improved. Another reason we separate SELECT and ADAPT is that in the meta prompt: we give LLMs freedom to make changes of obviously wrong selected/adapted modules by instructing “Feel free to change or come up with new reasoning modules” and find this improved discovered structure quality.\n\nQ5: We have included more details on frequency of selected reasoning modules in the attached 1-page pdf. Common modules include critical thinking, step-by-step reasoning, simplifying, etc. Note that even though the same module might be selected in different tasks, the final reasoning structure might look very different and customized to the task itself.\n\nQ6 and 7: Yes, models perform consistently in the reasoning module selection process across a wide range of tasks. We observe that PaLM-2 and GPT-4 result in similar structures on BBH, but vary more on MATH tasks, indicating potentially different mathematical reasoning post-training procedures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7bZYiOji9Q", "reviewer": "Reviewer_nKMJ", "summary": "The paper introduces SELF-DISCOVER, a framework enabling large language models (LLMs) to autonomously identify and utilize intrinsic reasoning structures to tackle various reasoning tasks. The framework is applicable across different model families and aligns with human reasoning patterns.", "questions": "NA", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces a novel framework, SELF-DISCOVER, that allows LLMs to autonomously identify and use task-specific reasoning structures.\n2. The method is effective across various benchmarks.", "weaknesses": "1. The framework's effectiveness is heavily dependent on the quality and comprehensiveness of the atomic reasoning modules available, which may require significant human effort to define. Additionally, there is no guideline provided on how to develop these modules.\n2. The main results are reported using a very limited range of models, specifically GPT-4 and PaLM-2. A systematic evaluation across a broader range of models, such as GPT-3.5, Llama2, and Llama3, is necessary, as merely applying reasoning structures generated by GPT-4 to these models is insufficient.\n3. The compared baselines, such as zero-shot CoT and Plan-and-Solve, are not strong enough. Automatic prompting engineering methods, like LLM as Optimizers [1] and StrategyLLM [2], should be included in comparisons and discussions.\n\n[1] Large Language Models as Optimizers. https://arxiv.org/abs/2309.03409\n\n[2] StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving. https://arxiv.org/abs/2311.08803", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your helpful feedback and suggestions. Our responses to weaknesses and questions are as follows:\n\n__[Dependent on quality of atomic reasoning modules]__\n\nWe would like to point out that the main contribution of this paper is the Self-Discover process. To show its effectiveness despite simple and not human-crafted seed reasoning modules, we took modules available from the Promptbreeder paper [1] and showed that it worked. Future work can certainly expand to more prompts to increase even more on the diversity and effectiveness of the method.\n\n__[Range of models]__\n\nIn Section 5.2, we specifically tested GPT-3.5 and Llama2’s effectiveness on Self-Discover reasoning structures and found it outperformed CoT by a large margin. As footnote 3 suggested, we initially tried using Llama2 and GPT-3.5 to zero-shot self-discover reasoning structures (Llama3 was not made available at the time of submission) but observed low-quality structure outputs. Meta-reasoning capabilities seem to require stronger base LLMs. We believe that more LLMs will become increasingly capable of performing self-discover reasoning and more accessible.\n\n__[Compared baselines]__\n\nIn addition to CoT and Plan-and-Solve, we also compared Tree-of-Thought and Graph-of-Thought and observed similar improvements (Appendix Table 4). We will put these result highlights in the main content. Furthermore, we compared LLM as Optimizers (OPRO) on transferability in Figure 8. Note that OPRO and StrategyLLM is not directly comparable to Self-Discover because ORPO is a supervised prompting tuning method with 100+ examples whereas Self-Discover is zero-shot, similarly to StrategyLLM (cited in paper on line 314) which requires accuracy on a validation set for strategy optimization. \n\n[1] Fernando, Chrisantha, et al. \"Promptbreeder: Self-referential self-improvement via prompt evolution.\" arXiv preprint arXiv:2309.16797 (2023).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uwtO914sF2", "reviewer": "Reviewer_NVyo", "summary": "The paper introduces a new prompting method, Self-Discover, that enhances the complex reasoning abilities of Large Language Models. Self-Discover consists of two stages. At the first one, the method develops a task-adapted prompt by performing 3 steps: SELECT, when the LLM selects the reasoning modules (RMs) from a pre-defined list, ADAPT, when the model adapts the chosen reasoning modules to the task, and IMPLEMENT, when the model, conditioned on the adapted reasoning modules, generates the reasoning structure as a JSON. Each step is performed in a zero-shot manner and controlled by a dedicated meta-prompt. The resulting reasoning structure is used in the final second stage, as part of the prompt to solve each task and produce the output. For each task instance, the LLM first reasons by completing the JSON value fields, and then outputs the answer. Self-Discover is compared to several baselines: Direct Prompting, CoT (+Self-Consistency), Plan-and-Solve, Majority voting of RMs, and Best of each RM, as well as Graph-of-Thought (GoT) and Tree-of-Thought (ToT). Evaluation is performed on BIG-Bench Hard (BBH), Thinking for Doing (T4D), and (subsampled) MATH benchmarks. The results demonstrate the superiority of Self-Discover against the baselines above in terms of performance while maintaining comparatively low computational cost, expressed in the number of necessary inference calls. The ablation study highlights that each step of Stage 1 significantly contributes to the overall performance. The generated reasoning structures appear to be general and transferable between different LLMs, compared to Optimized Prompts (OPRO).", "questions": "- How robust is the method to changes in the list of reasoning modules? An ablation study on them, or at least the list of chosen modules for each task would reveal this information. Also, how robust is the method to the set of chosen task examples? What was their number in the experiments, and does varying their number or themselves affect the performance?\n- What were the parameters of ToT & GoT in the experiments? The value of the breadth parameter can largely affect the performance of these methods.\n- What modifications can be applied to the set of RM to increase the performance on the algorithmic tasks? I assume some specialized algorithmic prompting can be useful. What you also include the relationship between reasoning modules and the tasks in which they are useful, and assert that the model chooses the correct reasoning module?", "limitations": "As mentioned above, though the method is very effective, it is dependent on the set of chosen task examples, and so is task-adaptable, and not completely zero-shot. This limitation is not emphasized enough in the paper.\n\nHowever, overall, it is a nice paper with promising results, which once again underscores the power of optimal prompting and alignment.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper proposes a simple yet effective approach that greatly improves the reasoning abilities of LLMs. This is especially emphasized by the low computational costs of the method of only 3 extra inference calls per task. The method unifies multiple prompting-reasoning techniques and thus is scalable and adaptable. \n- The evaluations are purposeful, and every major claim of the paper is supported by empirical evidence, including all the necessary ablation studies justifying the steps of Stage 1. A thorough analysis of the methods' performance, showing tasks where the method excels more, is included in the paper.", "weaknesses": "- The paper doesn't emphasize enough that the method requires task adaptation. This is a crucial detail, as, unlike CoT or ToT, Self-Discover cannot be deployed as a general zero-shot agent, but requires some specialized prompting with task examples. This limitation is addressed barely in the paper. Though this is not a huge disadvantage, it places the method right between zero-shot methods (CoT / ToT / GoT / PS) and task-adaptive methods (OPRO / DSPy).  Thus, more comparisons with the prompt adaptation techniques should be present in the paper, and the efficiency of Self-Discover should be emphasized when comparing to them. \n- The authors claim that their method \"composes atomic reasoning models into a reasoning structure\", but this seems misleading. Figure 4 demonstrates that the method only slightly outperforms the best RM baseline, suggesting that no actual \"composition\" is happening, but only a selection and adaptation of the best fitting reasoning technique. This requires further investigation, for example, by analyzing which reasoning modules were picked for a task, and ablating the performance on these modules. This would reveal whether the model indeed uses multiple reasoning modules or only one in its prediction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for your helpful feedback and suggestions. Our responses to weaknesses and questions are as follows:\n\n__[Task adaptation]__\n\nThank you for the great point. We’d like to first clarify that though a few task examples are used in meta-prompting, Self-Discover does NOT require any answer to the examples. Thus, OPRO and other task-adaptation methods such as StrategyLLM are not directly comparable to Self-Discover because ORPO is a supervised prompting tuning method needing 100+ training examples whereas Self-Discover does not need any answer labels, similarly to StrategyLLM, which requires accuracy on a validation set for strategy optimization. We will make the task-adaptation clearer in the paper.\n\n__[Reasoning module analysis and Q1]__\n\nHere we provide more details on the reasoning module selection process. First, we observe that on harder reasoning tasks (MATH and some subtasks from BBH), best of a single RM lags behind Self-Discover by a larger margin.Second, we did find that several reasoning modules (critical thinking, step-by-step, simplifying) are picked up more frequently than others and we’ve attached the detailed frequency in the new 1-page pdf. Third, we conducted an ablation study on the chosen reasoning modules on BBH and observed an average of 4.7% drop in performance with an average minimum drop of 2.5%. This indicates that multiple reasoning modules work synergistically for structured reasoning. We plan to include more comprehensive ablation studies on all the tasks for every reasoning module using a newer cheaper version of GPT-4.\n\n__[Q1: Robustness to task examples]__\n\nWe randomly sampled 5 times of task examples and observed +-1% in performance difference, showing that the method is not impacted by task example choices.\n\n__[Q2: ToT and GoT parameters]__\n\nTo keep comparable with Self-Discover, we use the zero-shot version of ToT and GoT and prompt models to conduct reasoning with a breadth of 5. We tested between 3 to 7 and did not observe significant differences. \n\n__[Q3: Modifications on RM]__\n\nGenerally, we think first expanding the current set by prompting models and then pruning the generated set to remove redundancy would be a good starting point. Specifically, if we can use a specialized RM for say MATH and coding problems, we imagine it would drastically improve the reasoning performance!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their time and insightful comments. We are pleased to receive this positive feedback from reviewers, particularly:

- Significantly improves performance across diverse complex reasoning tasks and demonstrates the value of prompt diversity (Reviewer i8mj, nKMJ, and 5mv1)
- Simple yet effective approach that greatly improves the reasoning abilities of LLMs (Reviewer NVyo)
- The discovered reasoning structures indicate a level of transferability/universality, highlighting its broad applicability and robustness (Reviewer i8mj and Kd8z)
- Self-Discover provides an interpretable way to understand the reasoning process of LLMs (Reviewer i8mj)
- The evaluations are purposeful, and every major claim of the paper is supported by empirical evidence with ablations (Reviewer 5mv1 and NVyo)


In addition to the above comments, we received valuable feedback from the reviewers, and included a 1-page pdf containing detailed selected reasoning module statistics for our tasks addressing common reviewer questions.

We have addressed most of the feedback in the comments and will improve the paper in the final version. We thank you again for all the reviewers' efforts to make our paper better and please let us know if you have any more questions.

Meta-review: Among an already dense field of prompting approaches, the paper presents a compelling and novel approach to prompting based on self-discovery of a reasoning structure and then adaptation of this structure to the specific task at hand.  The idea is inspired by studies of human problem solving that have also motivated other prompting methodologies.  Extensive (and as reviewer NVyo points out -- purposeful) experimental results not only show the improvement of this method over numerous baselines, but also extensive ablations, quantitative, qualitative investigation of its performance.  The paper is well-written.  Rebuttal discussion led to analysis updates, clarifications, and agreement to add additional analysis in the Appendix, which the authors are requested to include in their final revision.
Decision: Accept (poster)