Title: SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures
Venue: NeurIPS 2024
Accepted: yes
Keywords: Large Language Models

Reasoning

Prompting

Self-Improve

Self-Discover
Abstract: We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a self-discovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER substantially improves GPT-4 and PaLM 2’s performance on challenging reasoning benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER outperforms inference-intensive methods such as CoT-Self-Consistency by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.

Self-Discover: Large Language Models Self-Compose Reasoning Structures

Abstract

We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a selfdiscovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER substantially improves GPT-4 and PaLM 2's performance on challenging reasoning benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as much as 32% compared to Chain of Thought (CoT). Furthermore, SELFDISCOVER outperforms inference-intensive methods such as CoT-Self-Consistency by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.

Task

Chain-of-Thought

(

CoT

)

Answer

Self-Discover

Reasoning

Structures (

Ours

)

Answer

Task

Task

Task-Specific

Rationale

Answer

Structured

Reasoning

Avg. BBH:

+11%

T4D:

+ 39%

MATH:

+5.5%

Avg. BBH:

+7%

T4D:

+ 29%

MATH:

+8.5%

Self-DiscoverOverDirectAnswer

Self-DiscoverOverChain-of-Thought

40

40

30

30

20

20

10

10

0

10

10

20

20

1 Introduction

Large Language Models (LLM) (Brown et al., 2020; Chowdhery et al., 2022; OpenAI, 2023b; Anil et al., 2023) powered by transformers (Vaswani et al., 2017) have produced impressive breakthroughs

in generating coherent texts (OpenAI, 2022), and following instructions (Zhong et al., 2021; Mishra et al., 2022c; Wei et al., 2021; Chung et al., 2022; Ouyang et al., 2022). In pursuit of the goal to enhance LLMs' capability to reason and solve complex problems, various prompting methods have been proposed, drawing inspirations from cognitive theories of how humans reason. For example, few-shot and zero-shot chain-of-thought (CoT) (Nye et al., 2021; Wei et al., 2022; Kojima et al., 2022; Yasunaga et al., 2023) resembles how humans solve problems step-by-step, decomposition-based prompting (Zhou et al., 2022a; Drozdov et al., 2022; Patel et al., 2022; Hao et al., 2023; Khot et al., 2022) is inspired by how humans breakdown a complex problem into a series of smaller subproblems, and then solve those subproblems one by one (Polya, 2004), and step-back prompting (Zheng et al., 2023) is motivated by how humans reflect on task nature to derive general principles. However, a fundamental limitation is that each technique itself serves as an atomic reasoning module making an implicit prior assumption of the process on how to tackle a given task. Instead, we argue that each task has a unique intrinsic structure underlying the reasoning process involved in solving it efficiently. For instance, least-to-most prompting (Zhou et al., 2022a; Drozdov et al., 2022) has shown to be much more effective than CoT (Wei et al., 2022) at solving tasks such as symbolic manipulation and compositional generalization, due to the decomposition structure of the tasks.

This paper aims at self-discovering the underlying reasoning structure unique to each task, while being highly efficient in terms of computation. Our approach, SELF-DISCOVER, is inspired by how humans internally devise a reasoning program for problem-solving (Newell et al., 1958; Rasmussen, 1983), as illustrated in Figure 2 . From a set of atomic reasoning modules described in natural language such as ' breakdown into sub tasks ' and ' critical thinking ', an LLM, and task examples without labels, SELF-DISCOVER composes a coherent reasoning structure intrinsic to the task (Stage 1) and then solves instances of the task using the discovered structure (Stage 2). Stage 1 operates at the task-level and uses three actions to guide the LLM to generate a reasoning structure for the task. At Stage 2, during the final decoding, the LLM simply follows the self-discovered structure to arrive at the final answer.

Solving problems using SELF-DISCOVER brings several benefits compared to other methods for LLM reasoning. First, the discovered reasoning structure is grounded in atomic reasoning modules benefiting from the strengths of multiple reasoning modules in contrast to applying a priori module such as CoT. Second, SELF-DISCOVER is efficient in computation as it only requires 3 more inference steps on the task-level , while being more performant than inference-heavy ensemble approaches such as self-consistency Wang et al. (2022). Lastly, the discovered reasoning structure is intrinsic to the task, and conveys LLMs' insights about the task in a more interpretable way than the optimized prompts (Zhou et al., 2022b; Yang et al., 2023).

We test SELF-DISCOVER on 25 challenging reasoning tasks including Big Bench-Hard (BBH) (Suzgun et al., 2022), Thinking for Doing (T4D) (Zhou et al., 2023) and MATH (Hendrycks et al., 2021). SELF-DISCOVER outperforms CoT on 21/25 task with performance gains up to 42% (Figure 1), highlighting the advantage of the self-discovered reasoning structure composed from the atomic reasoning modules against a single a priori CoT module. Furthermore, we demonstrate that SELF-DISCOVER achieves superior performance against inference-heavy methods such as CoT + Self-Consistency and majority voting of every module while requiring 10-40x fewer inference compute (Figure 5). Finally, we compare SELF-DISCOVER with prompts optimized (OPRO) using a training set (Yang et al., 2023) (Figure 8). We find that SELF-DISCOVER still performs on par or better than OPRO while the self-discovered reasoning structure are much more interpretable.

We conduct a set of analysis to understand the effectiveness of SELF-DISCOVER. By breaking down BBH tasks into 4 different categories, we find that SELF-DISCOVER performs best on tasks requiring world knowledge and has a moderate performance boost on algorithmic tasks compared to CoT (Figure 4). This is further confirmed by the error analysis on MATH, where 74.7% model failures comes from computation errors (e.g. math). We also take a closer look at the self-discovered reasoning structures, and show the universality of them by transferability study from PaLM 2-L to GPT-4, and from GPT-4 to Llama-2-70B. We hope to encourage more future work on structured reasoning for solving challenging problems using LLMs.

2 Self-Discovering Reasoning Structures for Problem-Solving

We take inspiration from how humans use prior knowledge and skills to devise a reasoning program to solve problems (Newell et al., 1958; Rasmussen, 1983). When we face a new problem, we often first search internally what knowledge and skills from our prior experience might be helpful to solve it. Then we will attempt to apply relevant knowledge and skills to this task. And finally we will connect multiple individual skills and knowledge to solve the problem. We design SELF-DISCOVER to enact these steps into two stages as illustrated in Figure 2.

Given a task and a set of reasoning module descriptions representing high-level problem-solving heuristics such as ' Use critical thinking ' and ' Let's think step by step ', Stage 1 of SELFDISCOVER aims to uncover the intrinsic reasoning structure for solving this task via meta-reasoning. Specifically, we uses three meta-prompts to guide LLMs to select, adapt, and implement an actionable reasoning structure with no labels or training required. We format the structure in key-value pairs similar to JSON due to interpretability and findings on following JSON boosts reasoning and generation quality (Zhou et al., 2023; OpenAI, 2023a). The structure of the meta-prompts and full prompts are shown in Figure 9. Stage 1 operates on task-level , meaning we only need to run SELF-DISCOVER once for each task. Then, in Stage 2, we can simply use the discovered reasoning structure to solve every instance of the given task by instructing models to follow the provided structure by filling each key and arrive at a final answer.

Language

Model

Reasoning Structure

{

"Type and color of each item": ""

"Number of items of each color": ""

"Number of items of each type": ""

"Number of items of each color and type":

"Final answer":

}

Task:

Reasoning

colored objects

Self-Discover

Language

Model

Answer

Reasoning Structure

Task Instance

Key-Value pairs

Fill in the

Values

based on

during decoding

2.1 Stage 1: Self-Discover Task-Specific Structures

The first stage consists of three actions: 1) SELECT, where relevant reasoning modules for tasksolving are chosen from the set of reasoning module descriptions; 2) ADAPT, where descriptions of selected reasoning modules are rephrased to be more specific to the task at hand; and 3) IMPLEMENT, where the adapted reasoning descriptions are implemented into a structured actionable plan so that the task can be solved by following the structure.

SELECT First, not every reasoning module is helpful for every task, so the first stage of SELFDISCOVER guides model to select modules that are useful based on task examples. For example, ' reflective thinking ' might help search for first-principle theories on science problems, while ' creative thinking ' helps on generating a novel continuation to a story. Given raw set of reasoning module descriptions D such as ' critical thinking ', and ' break the problem into sub-problems ' (full set in Appendix A), and a few task examples without labels t i ∈ T , SELF-DISCOVER first selects a subset of reasoning modules D S that are useful for solving the tasks by using a model M and a meta-prompt p S :

ADAPT Since each reasoning module provides a general description of how to solve problems, the next step of SELF-DISCOVER aims at tailoring each selected module to the task at hand. For example, from ' break the problem into sub-problems ' to ' calculate each arithmetic operation in order ' for arithmetic problems. Given selected reasoning module subset D S from the previous step, ADAPT

Self-Discover

SELECT

ADAPT

IMPLEMENT

❖

Step-by-step

❖

Break down

❖

Propose-verify

All

❖

Step-by-step

❖

Break down

❖

…

Selected

Modules

❖

Step-by-step analyze each item

❖

Break down to type and color …

Adapted

Modules

{   "Type and color of each item":

"Number of items of each color":

"Number of items of each type":

...}

Reasoning Structure

Language

Model

Selected

Modules

Language

Model

Adapted

Modules

Language

Model

rephrases each of the selected module to be more specific to the task. Similarly to SELECT, this stage uses a meta-prompt p A and a generative model M to generate the adapted reasoning module descriptions D A :

IMPLEMENT Finally, given the adapted reasoning module descriptions D A , SELF-DISCOVER operationalizes the reasoning modules into an implemented reasoning structure D I with specified instruction on what to generate for each step. In addition to a meta prompt p I , IMPLEMENT also provides a demonstration of a human-written reasoning structure S human on another task to better convert the natural language descriptions into a reasoning structure:

2.2 Stage 2: Tackle Tasks Using Discovered Structures

After the three stages, we have an implemented reasoning structure D I uniquely adapted for the task we need to solve T . Then we can simply append the reasoning structure to all instances of the task and prompt models to follow the reasoning structure to generate an answer A :

3 Experiment Setup

3.1 Tasks

We focus on diverse reasoning benchmarks that are still challenging for LLMs: BIG-Bench Hard (BBH) (Suzgun et al., 2022) contains 23 carefully-selected challenging tasks from BIG-Bench (Srivastava et al., 2023). BBH tasks cover a diverse range of reasoning problems spanning the following 4 categories according to their authors: 1) Algorithmic and Multi-Step Arithmetic Reasoning, 2) Natural Language Understanding, 3) Use of World Knowledge, and 4) Multilingual Knowledge and Reasoning. We also test on a grounded social agent reasoning task called Thinking for Doing (T4D) where models must leverage mental state reasoning to determine actions to perform (Zhou et al., 2023), where GPT-4 with CoT only reaches around 50%. Finally, we subsample 200 examples from the MATH (Hendrycks et al., 2021) test set, and generate instance-level reasoning structures via a one-shot demonstration to adapt to the complexity of MATH tasks. For evaluations, we use accuracy to measure the model performance on BBH, T4D and MATH (details can be found in Appendix B).

3.2 Models

We use several state-of-the-art LLMs: GPT-4 (gpt-4-turbo-preview) (OpenAI, 2023b), GPT-3.5-turbo (ChatGPT) (OpenAI, 2022) 1 , instruction-tuned PaLM 2-L (Anil et al., 2023) 2 , and an open-source LLM Llama2-70B (Touvron et al., 2023).

3.3 Baselines

We compare SELF-DISCOVER with other zero-shot prompting methods for LLM reasoning:

Direct Prompting , where model directly generates the answer without intermediate reasoning steps.

CoT (Wei et al., 2022; Kojima et al., 2022), where models are prompted to generate a reasoning process leading to the final answer.

Plan-and-Solve (Wang et al., 2023), where models are prompted to first generate a plan and then solve the problem. SELF-DISCOVER differs by grounding the reasoning structure in atomic reasoning modules, and prompting the decoding to follow the explicit key-value reasoning structure.

Next, we also consider other baselines that make use of the raw seed reasoning modules (RM) we pass to SELF-DISCOVER. We compare with the following methods' performance and the inference call efficiency on a subset of tasks.

CoT-Self-Consistency Wang et al. (2022), we sample multiple outputs from LLM with CoT and aggregate answers to get the final answer. We compare this method on a subset of tasks due to the cost of repetitive queries.

Majority voting of each RM : we prompt models to solve the tasks by appending each RM and use majority voting of all answers to get the final answer. We examine whether integrating multiple RMs into a coherent reasoning structure is advantageous to applying each RM to solve the task and use majority voting to ensemble them post-hoc, which costs much more inference computation.

Best of each RM : this method assumes that we have access to oracle labels and uses the highest accuracy from applying each RM. We compare with this to examine whether SELFDISCOVER competes with methods that depend on perfect prior knowledge of which RM to use on a new task.

Furthermore, for analysis on universality of reasoning structures, we compare with a promptoptimization method that require a training set to improve prompts: LLMs as optimizers ( OPRO ) (Yang et al., 2023). We aim to show that when we apply structures or prompts optimized from one model, the reasoning structures can retain more performance gains than the wordings of prompts.

4 Results

We answer the following questions through experimental results: 1) Does discovering reasoning structures improve LLM reasoning capabilities? (4.1) 2) Which categories of problems do SELFDISCOVER perform the best? (4.2) and 3) Can SELF-DISCOVER boost LLM performance efficiently? (4.3) Finally, we will show qualitative examples of self-discovered structures, LLM output following the structures, and compare with LLM output following other prompting methods for reasoning (4.4).

4.1 Does SELF-DISCOVER Improve LLM Reasoning?

Overall, SELF-DISCOVER improves PaLM 2-L and GPT-4's reasoning across diverse set of reasoning tasks. Table 1 shows the overall results on complex reasoning tasks of BBH, T4D and

MATH using PaLM 2-L and GPT-4. We compare Self-Discover with baselines including direct prompting, CoT, and Plan-and-Solve (PS).

On aggregated 23 tasks of BBH, SELF-DISCOVER achieves 7% and 6% absolute improvement on PaLM 2-L over Chain-of-Thought and Plan-and-Solve, respectively. Similar gains (6% and 8%) are observed when SELF-DISCOVER is applied to GPT-4. Breakdown results of each task's improvement over direct answering and CoT of PaLM 2-L are shown in Figure 1, where we find SELF-DISCOVER outperforms them on over 20/24 tasks. For a per-task performance for all 23 BBH tasks, please refer to Appendix C.

On the grounded social agent task T4D, SELF-DISCOVER reaches over ≥ 27% ( 32% ) absolute improvement over all baselines on PaLM 2-L (GPT-4). SELF-DISCOVER achieves 69% and 85% accuracy on PaLM 2-L and GPT-4, significantly outperforming previous SoTA prompting method such as Foresee and Reflect (FaR) which employs an expert-designed reasoning structure. In contrast, SELF-DISCOVER generates the reasoning structure automatically from a set of atomic reasoning modules without human interventions.

For MATH, we observe a moderate gain of 1%-7% (2%-3%) on PaLM 2-L (GPT-4) from SELFDISCOVER compared to the baselines. Upon error analysis (see Appendix E for details), we find that the reasoning structures generated by PaLM 2-L from SELF-DISCOVER are correct 87.5% of the time: human experts can follow the reasoning structures to solve the tasks perfectly. The majority of the failures (74.7%) comes from errors in executing the computations, consistent with prior findings Zheng et al. (2023).

4.2 Which Types of Problems Do SELF-DISCOVER Help the Most?

SELF-DISCOVER performs best on tasks that require diverse world knowledge . Figure 4 presents the average improvement in terms of delta in accuracy of SELF-DISCOVER over direct answer and CoT on 4 categories of reasoning tasks we test. We adopt the categorization from Suzgun et al. (2022). We find that SELFDISCOVER improves over these two baselines on all categories, but especially on tasks that require world knowledge such as sports understanding , movie recommendation , and ruin names .

These tasks demand models to reason using fact and general commonsense

Multilingual

Algorithmic

NLU

World Knowledge

Problem Categories

0

5

10

15

20

25

Avg. Accuracy Delta (%)

4.0

14.2

14.3

19.8

12.0

5.2

8.0

19.7

Self-Discover Performance Improvement Across 4 Categories

Self-Discover Over Direct

Self-Discover Over CoT

knowledge. We interpret SELF-DISCOVER's advantages on these tasks as strength from integrating multiple reasoning modules from various perspectives as only applying CoT might miss key knowl-

0

10

20

30

40

# of Inference Calls Per Instance

65

70

75

80

85

Average Accuracy

BBH-Movie Recommendation

0

10

20

30

40

35

40

45

50

55

60

BBH-Geometric Shapes

Self-Discover

Direct

CoT

CoT+Self-Consistency

Plan-and-Solve

Majority voting each RM

Best of each RM*

edge in the reasoning process. We observe that the gain on the Algorithmic category is moderate, consistent with the findings from Sec. 4.1 on MATH.

4.3 How Efficient is SELF-DISCOVER?

SELF-DISCOVER achieves better performance while requiring 10-40x fewer inference computer compared to self-consistency or majority voting. Here we examine a subset of 2 tasks from BBH and present a more thorough comparison of methods including those requiring many inference calls that are too costly to run on all 24 tasks. Figure 5 shows average accuracy and number of inference calls required per instance for each method using GPT-4. Accuracy wise (y-axis) , we find that SELF-DISCOVER outperforms other baselines even those that require repeated inference calls such as CoT-self-consistency and majority voting of applying each RM. Efficiency wise (xaxis) , SELF-DISCOVER only requires one call per instance and three more inference calls on the task-level, CoT-self-consistency requires 10 times more since we have to sample 10 times for each instance, and methods using each RM requires 40 times more as we use 40 RMs. In summary, SELF-DISCOVER presents itself a strong reasoning boosting method that is efficient to deploy on large-scale.

4.4 Qualitative Examples

We show examples of model-discovered structures for different reasoning tasks in Figure 10 from PaLM 2-L. We observe that each structure is uniquely adapted to the task, integrates multiple reasoning modules, and provides insights on how to solve the tasks. Furthermore, example of comparing reasoning processes from CoT, Plan-and-Solve, and SELF-DISCOVER is shown in Figure 6. We find that CoT and Plan-and-Solve makes incorrect assertions early and arrives at a wrong answer while following structure from SELF-DISCOVER leads the model to generate logical conclusions (' path is closed as the beginning and ending coordinates are the same ') and arrive at the correct answer.

5 Deep Diving Into Self-Discovered Reasoning Structures

After experimental results showing the effectiveness and efficiency of SELF-DISCOVER on a range of reasoning tasks, this section further analyzes are all actions of SELF-DISCOVER needed and what other benefits can self-discovered structures bring? In Sec. 5.1, we show that it is critical to the model's performance to use the reasoning structures discovered through the three steps of SELECT, ADAPT and IMPLEMENT. In Sec. 5.2, we demonstrate the universality of the self-discovered reasoning structures by (1) applying the structures discovered by PaLM 2-L to GPT-4, (2) applying the structures discovered by GPT-4 to Llama-2-70B. We further show the commonalities between the reasoning structures and human reasoning patterns in Appendix F.

This SVG path element <path d="M 55.57,80.69 L 57.38,65.80 M 57.38,65.80 L 48.90,57.46 M 48.90,57.46 L

45.58,47.78 M 45.58,47.78 L 53.25,36.07 L 66.29,48.90 L 78.69,61.09 L 55.57,80.69"/> draws a:

(A) circle

(B) heptagon

(C) hexagon (D) kite (E) line (F) octagon (G) pentagon(H) rectangle (I) sector (J) triangle

This SVG path consists of

multiple lines drawn

between different points.

These points do not form

a regular geometric shape

such as a circle,

pentagon, heptagon,

hexagon

, or octagon. …

They do, however, seem

to form a shape with four

sides, which could be

considered a form of a

kite…

Thus the final answer is

(D) kite.

Let's think

step-by-step (

CoT

)

Let's devise a plan and solve

the problem (

Plan-and-Solve

)

This SVG Path element

consists of many "L"

commands which draw a

straight line from the

current point to the new

point. But it does not

close the path, meaning

the shape is not a regular

polygon like a

hexagon

,

heptagon, octagon or

pentagon. The

instructions make 4 lines

in total but they are not

straight and do not form a

parallelogram or a

rectangle. …

Thus the final answer is

(D) kite.

❌

❌

Self-Discover Reasoning Structure (

Ours

)

{   "

Simplify SVG Path

": ...

"

Breakdown of Path Commands

": {

"

Move to Command (M)

": "Sets the starting point for the next

command without drawing anything.",

"

Line to Command (L) steps

":

{"

Start and end coordinates of each line segment

":

"M 55.57,80.69 L 57.38,65.80: From point (55.57, 80.69) to (57.38,

65.80)"}, … and finally closing the shape at (55.57, 80.69)"}

"

Critical Thinking Analysis

": {

"

Logical Reasoning

"

: {

"

Analysis of path continuity

": "The path includes

multiple line segments that connect distinct points. The path ends by

connecting back to the starting point, indicating a closed shape.",

"

Identification of closed shapes

": "The final line

segment connects the last point back to the first point, which is

characteristic of a closed shape."},

…

"

Final Reasoning and Decision

": "With 7 distinct points all

connected in a closed path, the shape formed is a heptagon.",

"

Final Answer

": "

B) heptagon

"}

✅

5.1 Importance of SELF-DISCOVER Actions

We conduct ablation study on the three actions: SELECT, ADAPT, and IMPLEMENT to analyze the effects of SELF-DISCOVER actions. Figure 7 show results using GPT-4 on 4 reasoning tasks when we apply SELECT (-S) or apply SELECT and ADAPT (-SA) or apply all three actions. We find that with each stage, model's zeroshot reasoning capability improve consistently across tasks, indicating that all three actions are beneficial.

Snarks

Movie

T4D

Geometry

Tasks

30

40

50

60

70

80

90

100

89

73

50

43

90

73

65

50

93

79

80

45

94

86

85

61

Ablaton Studies on 3 Self-Discover Actions: SELECT, ADAPT, IMPLEMENT (SAI)

CoT

Ours-S

Ours-SA

Ours-SAI

5.2 Towards Universality of Discovered Reasoning Structures

Applying PaLM 2-L Discovered Structures to GPT-4 We first use a PaLM 2-L model to discover the reasoning structures of 4 reasoning tasks. Then, we apply the resulting reasoning structures to the decoding of GPT-4 as grounding. We compare our approach to OPRO (Yang et al., 2023) which discovered zero-shotprompts through optimizations. We apply OPRO prompts optimized using PaLM 2-L on each task to GPT-4 on the same reasoning tasks. Figure 8 shows that SELF-DISCOVER outperforms OPRO on 3 out of 4 tasks despite that OPRO used 20% data to op-

Snarks

Movie

T4D

Geometry

20

30

40

50

60

70

80

90

100

94

74

52

43

94

79

79

57

Transferrability of PaLM 2-L Optimized Prompts/Structures on GPT-4

OPRO*

Self-Discover

Tasks

timize the prompt. In contrast, SELF-DISCOVER is done in a zero-shot manner, demonstrating the efficiency of our method and universality of the discovered reasoning structures.

Applying GPT-4 Discovered Structures to Llama2 and ChatGPT Motivated by transferrability performance across LLMs, we further investigate can self-discovered reasoning structures from LLMs boost reasoning for smaller LMs that are challenging to come up with structures themselves 3 . We use GPT-4 to discover the task-intrinsic reasoning structures, and then apply those structures to the decoding of open-sourced Llama2-70B as well as GPT-3.5-turbo (ChatGPT) on two subsets of tasks from BBH. We find that using self-discovered structures on Llama2 (52%) outperforms CoT (42%) on disambiguation QA zero-shot and on GPT-3.5-turbo (56%) outperforms CoT (51%) on geometry with 3-shot demonstration from structured reasoning process.

6 Related Work

6.1 Prompting Methods

Recent advancements in the area of LLMs have given rise to a plethora of few-shot (Brown et al., 2020) and instruction (Mishra et al., 2022c; Wei et al., 2021; Ouyang et al., 2022) prompting techniques, including Chain-of-Thought prompting (CoT) (Nye et al., 2021; Wei et al., 2022), Least-to-most prompting (Zhou et al., 2022a; Drozdov et al., 2022), Decomposed prompting (Khot et al., 2022), Reframing (Mishra et al., 2022b), Help Me Think Prompting (Mishra & Nouri, 2023), Stepback Prompting (Zheng et al., 2023) and search-based approaches like Tree-of-Thought (ToT) (Yao et al., 2023a), Graph-of-Thought (Besta et al., 2023; Yao et al., 2023b), Branch-solve-merge (Saha et al., 2023) and RAP (Hao et al., 2023). Each of the prompting methods has some strengths and weaknesses in terms of their successful application domain. Our work SELF-DISCOVER presents the missing piece in the prompting literature, as SELF-DISCOVER provides a way to self-compose over various prompting methods via the proposed self-discovery mechanism. Composing over prompting methods in SELF-DISCOVER is analogous to the programming literature where a program is written using various basic building blocks such as for loop, if/else condition etc.

6.2 Reasoning and Planning

With the development of various reasoning and planning benchmarks such as GSM8K Cobbe et al. (2021), Math Hendrycks et al., BigBench Srivastava et al. (2023) etc., various methods have been proposed to improve model performance. Often these methods induce specific reasoning structures mimicking the reasoning structure of the underlying task associated with the dataset. For example, chain of thought Wei et al. (2022) and scratchpad Nye et al. (2021) induce generation of explanations associated with a reasoning question. Similarly other methods induces specific reasoning structures such as question summarization Kuznia et al. (2022), question decomposition Patel et al. (2022), program generation Mishra et al. (2022a); Chen et al. (2022); Gao et al. (2023b), etc. However, in a real world user traffic, queries can be diverse covering various reasoning structures. Our work SELF-DISCOVER allows models to combine multiple reasoning approaches by self-composing into a structure without the need to access task labels. There have been some related work that explores LLM combining skills in-context such as SkiC (Chen et al., 2023), devising a strategy (Gao et al., 2023a), and planning with iterative quering (Liu et al., 2023). However, they require human annotating skills and reasoning plans while SELF-DISCOVER leverages a scalable solution with the help of LLM's meta-task reasoning capabilities.

7 Conclusion

We introduce SELF-DISCOVER, an efficient and performant framework for models to self-discover a reasoning structure for any task from a seed set of general problem-solving skills. We observe drastic improvements on challenging reasoning benchmarks from multiple LLMs up to 30%. Ablations study of SELF-DISCOVER demonstrates that the composed reasoning structures are universally transferable between LLMs. Forward looking, we are excited to explore more on LLM structured reasoning to push the boundary of problem-solving and discover potentials for Human-AI collaboration.

References

Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403 , 2023.

Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Gianinazzi, L., Gajda, J., Lehmann, T., Podstawski, M., Niewiadomski, H., Nyczyk, P., et al. Graph of thoughts: Solving elaborate problems with large language models. arXiv preprint arXiv:2308.09687 , 2023.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901, 2020.

Chen, J., Pan, X., Yu, D., Song, K., Wang, X., Yu, D., and Chen, J. Skills-in-context prompting: Unlocking compositionality in large language models. arXiv preprint arXiv:2308.00304 , 2023.

Chen, W., Ma, X., Wang, X., and Cohen, W. W. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588 , 2022.

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 , 2022.

Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022.

Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021.

Drozdov, A., Schärli, N., Akyürek, E., Scales, N., Song, X., Chen, X., Bousquet, O., and Zhou, D. Compositional semantic parsing with large language models. arXiv preprint arXiv:2209.15003 , 2022.

Fernando, C., Banarse, D., Michalewski, H., Osindero, S., and Rocktäschel, T. Promptbreeder: Self-referential self-improvement via prompt evolution. arXiv preprint arXiv:2309.16797 , 2023.

Gao, C., Jiang, H., Cai, D., Shi, S., and Lam, W. Strategyllm: Large language models as strategy generators, executors, optimizers, and evaluators for problem solving. arXiv preprint arXiv:2311.08803 , 2023a.

Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G. Pal: Programaided language models. In International Conference on Machine Learning , pp. 10764-10799. PMLR, 2023b.

Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., and Hu, Z. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992 , 2023.

Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring mathematical problem solving with the math dataset. Sort , 2(4):0-6.

Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring mathematical problem solving with the math dataset, 2021.

Khot, T., Trivedi, H., Finlayson, M., Fu, Y ., Richardson, K., Clark, P., and Sabharwal, A. Decomposed prompting: A modular approach for solving complex tasks. In The Eleventh International Conference on Learning Representations , 2022.

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. Large language models are zero-shot reasoners. Advances in neural information processing systems , 35:22199-22213, 2022.

Kuznia, K., Mishra, S., Parmar, M., and Baral, C. Less is more: Summary of long instructions is better for program synthesis. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 4532-4552, 2022.

Liu, T., Guo, Q., Yang, Y., Hu, X., Zhang, Y., Qiu, X., and Zhang, Z. Plan, verify and switch: Integrated reasoning with diverse x-of-thoughts. arXiv preprint arXiv:2310.14628 , 2023.

Mishra, S. and Nouri, E. HELP ME THINK: A simple prompting strategy for non-experts to create customized content with models. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Findings of the Association for Computational Linguistics: ACL 2023 , pp. 11834-11890, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.findings-acl.751. URL https://aclanthology.org/2023.findings-acl.751 .

Mishra, S., Finlayson, M., Lu, P., Tang, L., Welleck, S., Baral, C., Rajpurohit, T., Tafjord, O., Sabharwal, A., Clark, P., et al. Lila: A unified benchmark for mathematical reasoning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 5807-5832, 2022a.

Mishra, S., Khashabi, D., Baral, C., Choi, Y., and Hajishirzi, H. Reframing instructional prompts to gptk's language. In Findings of the Association for Computational Linguistics: ACL 2022 , pp. 589-612, 2022b.

Mishra, S., Khashabi, D., Baral, C., and Hajishirzi, H. Cross-task generalization via natural language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 3470-3487, 2022c.

Newell, A., Shaw, J. C., and Simon, H. A. Elements of a theory of human problem solving. Psychological review , 65(3):151, 1958.

Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114 , 2021.

OpenAI. Chatgpt: Optimizing language models for dialogue, 2022. URL https://openai. com/blog/chatgpt/ .

OpenAI. Json generation mode, 2023a. URL https://platform.openai.com/docs/ guides/text-generation/json-mode .

OpenAI, R. Gpt-4 technical report. arXiv , pp. 2303-08774, 2023b.

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 35:27730-27744, 2022.

Patel, P., Mishra, S., Parmar, M., and Baral, C. Is a question decomposition unit all we need? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pp. 4553-4569, 2022.

Polya, G. How to solve it: A new aspect of mathematical method , volume 85. Princeton university press, 2004.

Rasmussen, J. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE transactions on systems, man, and cybernetics , (3):257-266, 1983.

Saha, S., Levy, O., Celikyilmaz, A., Bansal, M., Weston, J., and Li, X. Branch-solve-merge improves large language model evaluation and generation. arXiv preprint arXiv:2310.15123 , 2023.

Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research , 2023.

Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261 , 2022.

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/ 3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf .

Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., and Lim, E.-P. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091 , 2023.

Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Chi, E. H., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations , 2022.

Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. Finetuned language models are zero-shot learners. In International Conference on Learning Representations , 2021.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-ofthought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems , 35:24824-24837, 2022.

Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. Large language models as optimizers. arXiv preprint arXiv:2309.03409 , 2023.

Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y ., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601 , 2023a.

Yao, Y., Li, Z., and Zhao, H. Beyond chain-of-thought, effective graph-of-thought reasoning in large language models. arXiv preprint arXiv:2305.16582 , 2023b.

Yasunaga, M., Chen, X., Li, Y., Pasupat, P., Leskovec, J., Liang, P., Chi, E. H., and Zhou, D. Large language models as analogical reasoners. arXiv preprint arXiv:2310.01714 , 2023.

Zheng, H. S., Mishra, S., Chen, X., Cheng, H.-T., Chi, E. H., Le, Q. V ., and Zhou, D. Take a step back: Evoking reasoning via abstraction in large language models. arXiv preprint arXiv:2310.06117 , 2023.

Zhong, R., Lee, K., Zhang, Z., and Klein, D. Adapting language models for zero-shot learning by meta-tuning on dataset and prompt collections. arXiv preprint arXiv:2104.04670 , 2021.

Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q. V., et al. Least-to-most prompting enables complex reasoning in large language models. In The Eleventh International Conference on Learning Representations , 2022a.

Zhou, P., Madaan, A., Potharaju, S. P., Gupta, A., McKee, K. R., Holtzman, A., Pujara, J., Ren, X., Mishra, S., Nematzadeh, A., et al. How far are large language models from agents with theory-of-mind? arXiv preprint arXiv:2310.03051 , 2023.

Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations , 2022b.

A Self-Discover Prompt Details

Table 2 shows all 39 reasoning modules we use for SELF-DISCOVER, adopted from Fernando et al. (2023), that contain cognitive heuristics of problem-solving.

Figure 9 contains the structure of the three actions of SELF-DISCOVER during Stage 1, where it discovers an intrinsic reasoning structure on the task-level.

For Stage 2, where we use the self-discovered structure to solve the task instances, we start with the prompt: ' Follow the step-by-step reasoning plan in JSON to correctly solve the task. Fill in the values following the keys by reasoning specifically about the task given. Do not simply rephrase the keys. ', followed by the reasoning structure, and finally the task instance.

Rmd1m1p¦

GLYPH<c=23,font=/LAEWUQ+font000000002b1aa0ca>ŃĸŅĴŇļłŁĴĿļōĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŅĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ŀłķňĿĸņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ļŁŇłGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņŇĸŃ˭ĵŌ˭ņŇĸŃGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ŅĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŃĿĴŁGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ļŁGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=18,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=23,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=22,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĹłŅŀĴŇ˪

GLYPH<c=23,font=/LAEWUQ+font000000002b1aa0ca>ŃĸŅĴŇļłŁĴĿļōĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻļŁľļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

GLYPH<c=17,font=/LAEWUQ+font000000002b1aa0ca>ŀŃĿĸŀĸŁŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŅĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņŇŅňĶŇňŅĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĹłŅGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ņłĿŉĸŅņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇłGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĹłĿĿłŊGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņŇĸŃ˭ĵŌ˭ņŇĸŃGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŁķGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ĴŅŅļŉĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĶłŅŅĸĶŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŁņŊĸŅņ˪

1d1!¦

GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca>ĸĿĸĶŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņĸŉĸŅĴĿGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŅĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŀłķňĿĸņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ŇĻĴŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŅĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĶŅňĶļĴĿGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇłGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ňŇļĿļōĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ļŁGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>łŅķĸŅGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ņłĿŉĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĺļŉĸŁGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĴņľ˪

GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca> ĸĿĸĶŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņĸŉĸŅĴĿGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŀłķňĿĸņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻĴŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŅĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ĶŅňĶļĴĿGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĹłŅGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņłĿŉļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĴņľ ņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>Ĵĵłŉĸ˪

GLYPH<c=9,font=/LAEWUQ+font000000002b1aa0ca> ĿĿGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŅĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŀłķňĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ķĸņĶŅļŃŇļłŁņ

Ɣ

GLYPH<c=11,font=/LAEWUQ+font000000002b1aa0ca>ŅļŇļĶĴĿGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻļŁľ ļŁĺ˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

Ɣ

GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca> ŇĸŃ˭ĵŌ˭GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca>ŇĸŃ˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

Ɣ

GLYPH<c=24,font=/LAEWUQ+font000000002b1aa0ca> ŅłŃłņĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŁķGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŉĸŅļĹŌ˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

ˬ

GLYPH<c=28,font=/LAEWUQ+font000000002b1aa0ca> ĴņľGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĸŋĴŀŃĿĸņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>Ŋ̂łGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŁņŊĸŅ˪

GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>ŋĴŀŃĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>̻˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>ŋĴŀŃĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>̼˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

GLYPH<c=4,font=/NGILCC+font000000002b1aa0ca>'GLYPH<c=4,font=/NGILCC+font000000002b1aa0ca>¦

GLYPH<c=26,font=/LAEWUQ+font000000002b1aa0ca>ĸŃĻŅĴņĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŁķGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņŃĸĶļĹŌGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĸĴĶĻGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŅĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ŀłķňĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņłGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻĴŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ļŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĵĸŇŇĸŅGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĻĸĿŃņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņłĿŉļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ŇĻĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĴņľ˪

GLYPH<c=9,font=/LAEWUQ+font000000002b1aa0ca>ķĴŃŇGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĸĴĶĻGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŅĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŀłķňĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>

ķĸņĶŅļŃŇļłŁGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇłGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĵĸŇŇĸŅGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ņłĿŉĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĴņľņ˪

GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=20,font=/LAEWUQ+font000000002b1aa0ca> GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=11,font=/LAEWUQ+font000000002b1aa0ca> GLYPH<c=28,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=12,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŀłķňĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ķĸņĶŅļŃŇļłŁņ˪

Ɣ

GLYPH<c=11,font=/LAEWUQ+font000000002b1aa0ca>ŅļŇļĶĴĿGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŇĻļŁľ ļŁĺ˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

Ɣ

GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca> ŇĸŃ˭ĵŌ˭GLYPH<c=27,font=/LAEWUQ+font000000002b1aa0ca>ŇĸŃ˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

ˬ

GLYPH<c=9,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=12,font=/LAEWUQ+font000000002b1aa0ca> GLYPH<c=9,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=24,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=28,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=12,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ŀłķňĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ķĸņĶŅļŃŇļłŁ˪

GLYPH<c=24,font=/CDKBLX+font000000002b1aa0ca> ĴļŅĸķGLYPH<c=3,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=17,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=21,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=24,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=20,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=13,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=21,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=13,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=22,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=28,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=3,font=/CDKBLX+font000000002b1aa0ca>GLYPH<c=27,font=/CDKBLX+font000000002b1aa0ca>ŇĸŃGLYPH<c=3,font=/CDKBLX+font000000002b1aa0ca>

GLYPH<c=12,font=/CDKBLX+font000000002b1aa0ca>ĸŀłŁņŇŅĴŇļłŁ

GLYPH<c=26,font=/LAEWUQ+font000000002b1aa0ca>ĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ķĸņĶŅļŃŇļłŁGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>ŋĴŀŃĿĸ

GLYPH<c=26,font=/LAEWUQ+font000000002b1aa0ca>ĸĴņłŁļŁĺGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=24,font=/LAEWUQ+font000000002b1aa0ca>ĿĴŁGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>ŋĴŀŃĿĸ

GLYPH<c=28,font=/LAEWUQ+font000000002b1aa0ca> ĴņľGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĸŋĴŀŃĿĸņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>Ŋ̂łGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŁņŊĸŅ˪

GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>ŋĴŀŃĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>̻˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

GLYPH<c=13,font=/LAEWUQ+font000000002b1aa0ca>ŋĴŀŃĿĸGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>̼˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ˬ

GLYPH<c=28,font=/LAEWUQ+font000000002b1aa0ca> ĴņľGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĸŋĴŀŃĿĸņGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>Ŋ̂łGLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>ĴŁņŊĸŅ˪GLYPH<c=3,font=/LAEWUQ+font000000002b1aa0ca>˫˫˫

B Evaluation Details

We use accuracy and exact matching as with other methods tested on BBH, T4D and MATH. To properly evaluate the generated answers from LLMs, we prompt the models to end the answer with ' Thus, the final answer is [X] ', where X is either one answer option such as ' A ' or a string such as ' valid '. During evaluation, we manually examine each task's outputs from LLMs and design heuristics to extract the final answers. For MATH dataset, we find that it is challenging to extract the answers accurately. As a result, we subsample 200 test examples from MATH, and manually sanity check and annotate the extracted answers for all methods tested in our paper.

Here we also provide more details on token counts for Self-Discover and other baselines. Since Self-Discover only added 3 calls per task, on per instance level it only needs to run once. The average length of self-discovered structures is 224 tokens for BBH, 183 tokens for T4D, and 152 for MATH, which is similar to ToT/GoT prompts with around 234 tokens. We run the stage 2 (generating solution based on the structure) only in 1 inference pass where the model fills the values in the self-discovered structures. Thus, Self-Discover still reduces much inference cost compared to CoT-Self-Consistency, majority voting, etc. which requires 20s-40s fold of each CoT reasoning path token counts.

C BBH Per Task Performance

Per-task performance on BBH (23 tasks in total) are shown in Table 3.

Reasoning Modules

2 Make a list of ideas for solving this problem, and apply them one by one to the problem to see if any progress can be made.

5 What are the key assumptions underlying this problem?

14 Use Risk Analysis: Evaluate potential risks, uncertainties, and tradeoffs associated with different solutions or approaches to a problem. Emphasize assessing the potential consequences and likelihood of success or failure, and making informed decisions based on a balanced analysis of risks and benefits.

15 Use Reflective Thinking: Step back from the problem, take the time for introspection and self-reflection. Examine personal biases, assumptions, and mental models that may influence problem-solving, and being open to learning from past experiences to improve future approaches.

17 What are the underlying causes or factors contributing to the problem?

18 Are there any potential solutions or strategies that have been tried before? If yes, what were the outcomes and lessons learned?

19 What are the potential obstacles or challenges that might arise in solving this problem?

20 Are there any relevant data or information that can provide insights into the problem? If yes, what data sources are available, and how can they be analyzed?

21 Are there any stakeholders or individuals who are directly affected by the problem? What are their perspectives and needs?

22 What resources (financial, human, technological, etc.) are needed to tackle the problem effectively?

23 How can progress or success in solving the problem be measured or evaluated?

24 What indicators or metrics can be used?

25 Is the problem a technical or practical one that requires a specific expertise or skill set? Or is it more of a conceptual or theoretical problem?

26 Does the problem involve a physical constraint, such as limited resources, infrastructure, or space?

27 Is the problem related to human behavior, such as a social, cultural, or psychological issue?

28 Does the problem involve decision-making or planning, where choices need to be made under uncertainty or with competing objectives?

29 Is the problem an analytical one that requires data analysis, modeling, or optimization techniques?

30 Is the problem a design challenge that requires creative solutions and innovation?

31 Does the problem require addressing systemic or structural issues rather than just individual instances?

32 Is the problem time-sensitive or urgent, requiring immediate attention and action?

33 What kinds of solution typically are produced for this kind of problem specification?

34 Given the problem specification and the current best solution, have a guess about other possible solutions.

35 Let's imagine the current best solution is totally wrong, what other ways are there to think about the problem specification?

36 What is the best way to modify this current best solution, given what you know about these kinds of problem specification?

37 Ignoring the current best solution, create an entirely new solution to the problem.

38 Let's think step by step.

39 Let's make a step by step plan and implement it with good notion and explanation.

D Additional Experiments

We further include Tree-of-Thought (Yao et al., 2023a) and Graph-of-Thought (Besta et al., 2023) (zero-shot versions) as baselines for comparison shown in Table 4.

To show the effectiveness of Self-Discover on more general tasks, we tested on a subset of MMLU (10 subtasks, with 50 diverse questions each, all randomly sampled) and results are shown in Table 5. We find that GPT-4+Self-Discover wins GPT-4+CoT in zero-shot on 7 out of 10, ties on 2 out of 10, and loses on 1 out of 10 tasks. In addition, we tried Self-Discover on the instance-level, where for each question, we run stage 1 to output the reasoning structure, then solve the task. We find that the instance-level Self-Discover performs even better on MMLU, outperforming CoT by 7.2% on average for all tasks. This result, combined with those in main content, shows that the strength of Self-Discover spans across two types of tasks: for well-defined hard reasoning tasks such as BBH, task-level Self-Discover works well while being very efficient; for very open-domain tasks such as MMLU, we can do instance-level Self-Discover, which significantly outperforms CoT while still fewer inference required than self-consistency.

We show additional examples of self-discovered structures in Figure 10. We observe traits of atomic reasoning modules such as ' step-by-step thinking ', ' reflect on task nature ', and an interesting creative thinking case where models devise an algorithm using stack to solve parenthesis parsing task.

TaskNavigation : If you follow these instructions, do you return to the starting point? Always face forward. Take 1 step backward. Take 9 steps left. Take 2 steps backward. Take 6 steps forward. Take 4 steps forward. Take 4 steps backward. Take 3 steps right.

E Error Analysis

We perform an error analysis of SELF-DISCOVER on the MATH dataset of 200 samples to understand the failure modes. We manually annotate whether the generated reasoning structure is correct or not together with whether the correctness of model prediction using SELF-DISCOVER. A reasoning structure is defined as correct if a human expert can solve the task by simply following the reasoning structure.

Out of 200 examples, we find that 87.5% (175) examples have correct reasoning structures. 12.5% (25) examples have incorrect reasoning structures leading to prediction errors. Table 6 shows 4 such examples where the LLM misunderstands the task, or makes an error in one of the steps or adds unnecessary steps in the reasoning structure.

Next, we analyze the errors made by the model in SELF-DISCOVER: out of 99 examples where the model prediction is wrong, wrong reasoning structures account for only 25.3% of the errors. The remaining 74.7% errors are due to errors in the intermediate calculations such as math computations. Table 7 shows 3 examples of such errors. This insight indicates that future improvements should aim at improving the step-wise calculation accuracy of LLMs, such as using tools or code generation.

F Further Anaysis

Model-Discovered Reasoning Structures vs. Human Reasoning Patterns Weinvestigate whether LLM-discovered reasoning structures share some commonalities with human reasoning patterns. We give humans 3 task instances without labels and an example reasoning structure (same as SELF-

DISCOVER meta-reasoning stage) and ask them to write a reasoning structure for a task before solving it. Figure 11 shows comparison of human and LLM-composed reasoning structures on the BBH-navigation task. We observe similar structures such as mental-noting after each movement. From promising findings of LLM self-discovered structures boost and share traits of human metareasoning, we hope to encourage more future work to study humna-AI collaboration for complex problem-solving.

To show whether the chosen reasoning modules and discovered structures provide a diverse coverage, we have included more details on frequency of selected reasoning modules in Table 8. Furthermore, we observe a very diverse set of self-discovered structures as the 12 examples shown in Figure 6, 10, 11, and Table 6 and 7. Math problem structures (Table 6) tend to be very different from BBH ones (Figure 10). Due to the non-deterministic nature of LLMs, the number of reasoning structures that can be discovered by Self-Discover is infinite, because it can use multiple reasoning modules in many different orders. In our prompts, we specifically do not restrict how the structures should use the modules and find examples where models use new modules not in the seed list (Figure 10 Dyck-Language example where model devises a stack algorithm, which is not in the seed list).

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We accurately report the results from our experiments in the abstract and introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We present an extensive analysis of the limitation of Self-Discover in both Sections 4 and 5 with an extended error analysis on MATH in Appendix E.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: We do not present theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provided detailed prompts, explanation of how each inference procedure, and each model specification to reproduce the main experimental results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: Due to legal constraints, we are not able to fully release the code and data at the current time. But we provide, with the best faith, details to reproduce our results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We provide details of inference procedures including initial prompts, framework of each data/test split, and model specifications.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We include discussions of statistical significance and most improvements are significant.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We specify the LLM models we use. And since our experiments do not involve large-scale training, it is not computationally costly.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We follow NeurIPS code of ethics carefully.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our work is a foundational study on model's reasoning capabilities.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite all models, data, and code we use in our experiments.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We do not involve extensive crowdsourcing.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: We do not involve extensive crowdsourcing.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a new prompting method, Self-Discover, that enhances the complex reasoning abilities of Large Language Models. Self-Discover consists of two stages. At the first one, the method develops a task-adapted prompt by performing 3 steps: SELECT, when the LLM selects the reasoning modules (RMs) from a pre-defined list, ADAPT, when the model adapts the chosen reasoning modules to the task, and IMPLEMENT, when the model, conditioned on the adapted reasoning modules, generates the reasoning structure as a JSON. Each step is performed in a zero-shot manner and controlled by a dedicated meta-prompt. The resulting reasoning structure is used in the final second stage, as part of the prompt to solve each task and produce the output. For each task instance, the LLM first reasons by completing the JSON value fields, and then outputs the answer. Self-Discover is compared to several baselines: Direct Prompting, CoT (+Self-Consistency), Plan-and-Solve, Majority voting of RMs, and Best of each RM, as well as Graph-of-Thought (GoT) and Tree-of-Thought (ToT). Evaluation is performed on BIG-Bench Hard (BBH), Thinking for Doing (T4D), and (subsampled) MATH benchmarks. The results demonstrate the superiority of Self-Discover against the baselines above in terms of performance while maintaining comparatively low computational cost, expressed in the number of necessary inference calls. The ablation study highlights that each step of Stage 1 significantly contributes to the overall performance. The generated reasoning structures appear to be general and transferable between different LLMs, compared to Optimized Prompts (OPRO).\n\nSTRENGTHS:\n- The paper proposes a simple yet effective approach that greatly improves the reasoning abilities of LLMs. This is especially emphasized by the low computational costs of the method of only 3 extra inference calls per task. The method unifies multiple prompting-reasoning techniques and thus is scalable and adaptable. \n- The evaluations are purposeful, and every major claim of the paper is supported by empirical evidence, including all the necessary ablation studies justifying the steps of Stage 1. A thorough analysis of the methods' performance, showing tasks where the method excels more, is included in the paper.\n\nWEAKNESSES:\n- The paper doesn't emphasize enough that the method requires task adaptation. This is a crucial detail, as, unlike CoT or ToT, Self-Discover cannot be deployed as a general zero-shot agent, but requires some specialized prompting with task examples. This limitation is addressed barely in the paper. Though this is not a huge disadvantage, it places the method right between zero-shot methods (CoT / ToT / GoT / PS) and task-adaptive methods (OPRO / DSPy).  Thus, more comparisons with the prompt adaptation techniques should be present in the paper, and the efficiency of Self-Discover should be emphasized when comparing to them. \n- The authors claim that their method \"composes atomic reasoning models into a reasoning structure\", but this seems misleading. Figure 4 demonstrates that the method only slightly outperforms the best RM baseline, suggesting that no actual \"composition\" is happening, but only a selection and adaptation of the best fitting reasoning technique. This requires further investigation, for example, by analyzing which reasoning modules were picked for a task, and ablating the performance on these modules. This would reveal whether the model indeed uses multiple reasoning modules or only one in its prediction.\n\nQUESTIONS:\n- How robust is the method to changes in the list of reasoning modules? An ablation study on them, or at least the list of chosen modules for each task would reveal this information. Also, how robust is the method to the set of chosen task examples? What was their number in the experiments, and does varying their number or themselves affect the performance?\n- What were the parameters of ToT & GoT in the experiments? The value of the breadth parameter can largely affect the performance of these methods.\n- What modifications can be applied to the set of RM to increase the performance on the algorithmic tasks? I assume some specialized algorithmic prompting can be useful. What you also include the relationship between reasoning modules and the tasks in which they are useful, and assert that the model chooses the correct reasoning module?\n\nLIMITATIONS:\nAs mentioned above, though the method is very effective, it is dependent on the set of chosen task examples, and so is task-adaptable, and not completely zero-shot. This limitation is not emphasized enough in the paper.\n\nHowever, overall, it is a nice paper with promising results, which once again underscores the power of optimal prompting and alignment.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-26"}

comments: SUMMARY:
The paper introduces SELF-DISCOVER, a framework enabling large language models (LLMs) to autonomously identify and utilize intrinsic reasoning structures to tackle various reasoning tasks. The framework is applicable across different model families and aligns with human reasoning patterns.

STRENGTHS:
1. The paper introduces a novel framework, SELF-DISCOVER, that allows LLMs to autonomously identify and use task-specific reasoning structures.
2. The method is effective across various benchmarks.

WEAKNESSES:
1. The framework's effectiveness is heavily dependent on the quality and comprehensiveness of the atomic reasoning modules available, which may require significant human effort to define. Additionally, there is no guideline provided on how to develop these modules.
2. The main results are reported using a very limited range of models, specifically GPT-4 and PaLM-2. A systematic evaluation across a broader range of models, such as GPT-3.5, Llama2, and Llama3, is necessary, as merely applying reasoning structures generated by GPT-4 to these models is insufficient.
3. The compared baselines, such as zero-shot CoT and Plan-and-Solve, are not strong enough. Automatic prompting engineering methods, like LLM as Optimizers [1] and StrategyLLM [2], should be included in comparisons and discussions.

[1] Large Language Models as Optimizers. https://arxiv.org/abs/2309.03409

[2] StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving. https://arxiv.org/abs/2311.08803

QUESTIONS:
NA

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces SELF-DISCOVER, a framework designed to enhance the reasoning capabilities of Large Language Models (LLMs) such as GPT-4 and PaLM 2. The framework enables LLMs to self-discover and compose intrinsic reasoning structures tailored to specific tasks, improving performance on complex reasoning benchmarks.\n\nSTRENGTHS:\n- By enabling LLMs to self-compose reasoning structures, this method significantly improves performance on complex reasoning tasks, demonstrating the ability to handle intricate problems more effectively than traditional methods.\n\n- SELF-DISCOVER achieves superior results while requiring substantially fewer inference steps (10-40x fewer) compared to inference-intensive approaches like CoT-Self-Consistency.\n\n- The reasoning structures discovered by the SELF-DISCOVER method show strong transferability across different LLM families, highlighting its broad applicability and robustness.\n\nWEAKNESSES:\n1. While atomic reasoning structures form the foundation of the SELF-DISCOVER method, their inherent simplicity can restrict the overall reasoning performance. These basic units, although useful, may not fully capture the complexity needed for certain advanced reasoning tasks, potentially limiting the depth and flexibility of the reasoning process.\n\n2. The proposed method is significantly reliant on the performance of the underlying LLM. Factors such as the model's ability to follow instructions accurately and the limitations imposed by context window sizes can pose significant challenges. For instance, if the base LLM struggles with instruction-following or is constrained by a limited context window, the effectiveness of the SELF-DISCOVER method may be compromised, limiting its potential for broader application and scalability.\n\n3. The iterative process of selecting, adapting, and implementing reasoning modules introduces extra computational costs. Each step in this process requires additional queries to the LLM, which can accumulate and result in increased computational demands. This overhead may offset some of the efficiency gains achieved through reduced inference steps, particularly in scenarios requiring frequent adaptation and customization of reasoning structures.\n\nQUESTIONS:\n1. What is the additional cost of implementing this method, such as the extra tokens generated or the increase in inference time?\n\n2. Please refer to the correct section number in the Appendix in lines 83-84.\n\n3. The comparison in Figure 5 may not be fair. The self-discovery method requires fewer calls but could lead to more token generations.\n\n4. The design of the \"select, adapt, and implement\" steps seems ad-hoc. Why are these three steps proposed separately? For instance, why can't \"select\" and \"adapt\" be combined into a single step since their prompts are similar, as shown in Figure 9?\n\n5. What are the commonly selected reasoning structures? Are all 39 reasoning modules useful? It would be better to show the density of the selected reasoning modules to ensure that the selection of 39 modules is meaningful.\n\n6. Does the model perform consistently in the selection process?\n\n7. Do different models, such as GPT-4 and Palm-2, perform consistently? Do they discover similar reasoning structures?\n\nLIMITATIONS:\nThis paper does not sufficiently illustrate the limitations of the proposed method. The checklist on lines 542-543 states, \"We present an extensive analysis of the limitations of Self-Discover in both Sections 4 and 5 with an extended error analysis on MATH in Appendix E.\" However, Sections 4 and 5 are primarily general evaluations of the proposed method and do not contain sufficient and clear illustrations of its limitations. This issue needs to be strictly addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces SELF-DISCOVER, a novel framework that enables large language models (LLMs) to self-discover and compose reasoning structures for tackling complex reasoning tasks. The core of SELF-DISCOVER is a self-discovery process where LLMs select multiple atomic reasoning modules, such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure to follow during decoding.\n\nSTRENGTHS:\n1) SELF-DISCOVER significantly improves the performance of state-of-the-art LLMs on complex reasoning tasks, showcasing substantial gains over traditional prompting methods like Chain of Thought (CoT).\n2) The discovered reasoning structures are not only effective but also transferable across different model families, indicating a level of universality in the approach. SELF-DISCOVER provides a more interpretable way to understand the reasoning process of LLMs by generating explicit reasoning structures.\n3)  The paper includes a thorough empirical evaluation on a diverse set of reasoning tasks, demonstrating the framework's effectiveness across various domains.\n\nWEAKNESSES:\n1) The paper does not specify the exact number of reasoning structures that can be discovered by the SELF-DISCOVER framework. It's unclear whether the framework can generate a wide variety of structures or if it tends to converge on a limited set of common structures.\n2) It is not detailed whether different examples within the same task utilize the same reasoning structure or if the framework can adapt the structure to suit the nuances of individual examples.\n3) The paper's comparative analysis may be considered basic, as it does not delve into a comprehensive set of existing methods or include recent state-of-the-art approaches for comparison.\n4) Although the paper mentions a similarity to the \"Meta Reasoning for Large Language Models\" approach, it does not provide a direct comparative experiment or analysis to highlight differences and potential advantages of SELF-DISCOVER.\n5) The paper might not fully account for or cite the most recent literature, specifically works from 2024 and beyond, which could provide additional context and comparison points for the SELF-DISCOVER framework.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nSee weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

comments: SUMMARY:
This paper proposes a prompt engineering scheme:
Given a task and 39 prompts for solving tasks, 
an LLM is prompted to select which of the 39 prompts are suitable for the task.
The selected prompts are then rephrased to be more specific to the task, 
and reformatted in json.

The paper uses GPT-4 Turbo, PaLM 2-L, and Llama2-70B
and compares direct prompting, chain of thought, plan and solve, 
chain of though with self-consistency, majority vote of the 39 prompts, and the proposed method.

STRENGTHS:
1. This prompting scheme performs best on tasks that require diversity,
and the work demonstrates the value of prompt diversity.

2. Ablation experiments show the increased accuracy for each of the three steps of selecting prompts, rephrasing them, and reformatting (or implementation), over chain of thought.

WEAKNESSES:
1. As a prompt engineering method, it is unclear if the choice of the specific 39 prompts will withstand the test of time across LLMs and tasks. 

2. Recent prompt engineering tools, such as Anthropic's Claude 3.5 Prompt Engineer, receive a task as input and output a detailed prompt best suited for the task and LLM, which may supersede this approach. 

3. The claim of efficiency should be more rigorously validated and consider both the number of calls and the token lengths of the prompts and responses of each call.

QUESTIONS:
When comparing prompting schemes, it is important to give equal compute time to different methods. 
When measuring efficiency did the work consider the lengths of the prompts and responses?

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-16