SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures

Authors: pei  zhou
jay  pujara
xiang  ren
xinyun  chen
heng-tze  cheng
quoc v le
ed h chi
denny  zhou
swaroop  mishra
†  huaixiu
steven  zheng
google  deepmind
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 16:16:08

Abstract: We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a self-discovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER substantially improves GPT-4 and PaLM 2’s performance on challenging reasoning benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER outperforms inference-intensive methods such as CoT-Self-Consistency by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.

Main paper: 
 claims: 
 Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: We accurately report the results from our experiments in the abstract and introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
self-discovering reasoning structures for problem-solving: 
 We take inspiration from how humans use prior knowledge and skills to devise a reasoning program to solve problems (Newell et al., 1958; Rasmussen, 1983) . When we face a new problem, we often first search internally what knowledge and skills from our prior experience might be helpful to solve it. Then we will attempt to apply relevant knowledge and skills to this task. And finally we will connect multiple individual skills and knowledge to solve the problem. We design SELF-DISCOVER to enact these steps into two stages as illustrated in Figure 2 .Given a task and a set of reasoning module descriptions representing high-level problem-solving heuristics such as "Use critical thinking" and "Let's think step by step", Stage 1 of SELF-DISCOVER aims to uncover the intrinsic reasoning structure for solving this task via meta-reasoning. Specifically, we uses three meta-prompts to guide LLMs to select, adapt, and implement an actionable reasoning structure with no labels or training required. We format the structure in key-value pairs similar to JSON due to interpretability and findings on following JSON boosts reasoning and generation quality (Zhou et al., 2023; OpenAI, 2023a) . The structure of the meta-prompts and full prompts are shown in Figure 9 . Stage 1 operates on task-level, meaning we only need to run SELF-DISCOVER once for each task. Then, in Stage 2, we can simply use the discovered reasoning structure to solve every instance of the given task by instructing models to follow the provided structure by filling each key and arrive at a final answer. The first stage consists of three actions: 1) SELECT, where relevant reasoning modules for tasksolving are chosen from the set of reasoning module descriptions; 2) ADAPT, where descriptions of selected reasoning modules are rephrased to be more specific to the task at hand; and 3) IMPLEMENT, where the adapted reasoning descriptions are implemented into a structured actionable plan so that the task can be solved by following the structure.SELECT First, not every reasoning module is helpful for every task, so the first stage of SELF-DISCOVER guides model to select modules that are useful based on task examples. For example, "reflective thinking" might help search for first-principle theories on science problems, while "creative thinking" helps on generating a novel continuation to a story. Given raw set of reasoning module descriptions D such as "critical thinking", and "break the problem into sub-problems" (full set in Appendix A), and a few task examples without labels t i ∈ T , SELF-DISCOVER first selects a subset of reasoning modules D S that are useful for solving the tasks by using a model M and a meta-prompt p S :EQUATIONADAPT Since each reasoning module provides a general description of how to solve problems, the next step of SELF-DISCOVER aims at tailoring each selected module to the task at hand. For example, from "break the problem into sub-problems" to "calculate each arithmetic operation in order" for arithmetic problems. Given selected reasoning module subset D S from the previous step, ADAPT rephrases each of the selected module to be more specific to the task. Similarly to SELECT, this stage uses a meta-prompt p A and a generative model M to generate the adapted reasoning module descriptions D A :D A = M(p A ∥ D S ∥ t i ).(2)IMPLEMENT Finally, given the adapted reasoning module descriptions D A , SELF-DISCOVER operationalizes the reasoning modules into an implemented reasoning structure D I with specified instruction on what to generate for each step. In addition to a meta prompt p I , IMPLEMENT also provides a demonstration of a human-written reasoning structure S human on another task to better convert the natural language descriptions into a reasoning structure:D I = M(p I ∥ S human ∥ D A ∥ t i ).(3)After the three stages, we have an implemented reasoning structure D I uniquely adapted for the task we need to solve T . Then we can simply append the reasoning structure to all instances of the task and prompt models to follow the reasoning structure to generate an answer A:EQUATION3 Experiment SetupQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We present an extensive analysis of the limitation of Self-Discover in both Sections 4 and 5 with an extended error analysis on MATH in Appendix E.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
tasks: 
 We focus on diverse reasoning benchmarks that are still challenging for LLMs: BIG-Bench Hard (BBH) (Suzgun et al., 2022) contains 23 carefully-selected challenging tasks from BIG-Bench (Srivastava et al., 2023) . BBH tasks cover a diverse range of reasoning problems spanning the following 4 categories according to their authors: 1) Algorithmic and Multi-Step Arithmetic Reasoning, 2) Natural Language Understanding, 3) Use of World Knowledge, and 4) Multilingual Knowledge and Reasoning. We also test on a grounded social agent reasoning task called Thinking for Doing (T4D) where models must leverage mental state reasoning to determine actions to perform (Zhou et al., 2023) , where GPT-4 with CoT only reaches around 50%. Finally, we subsample 200 examples from the MATH (Hendrycks et al., 2021) test set, and generate instance-level reasoning structures via a one-shot demonstration to adapt to the complexity of MATH tasks. For evaluations, we use accuracy to measure the model performance on BBH, T4D and MATH (details can be found in Appendix B).We use several state-of-the-art LLMs: GPT-4 (gpt-4-turbo-preview) (OpenAI, 2023b), GPT-3.5-turbo (ChatGPT) (OpenAI, 2022)1 , instruction-tuned PaLM 2-L (Anil et al., 2023) 2 , and an open-source LLM Llama2-70B (Touvron et al., 2023) .We compare SELF-DISCOVER with other zero-shot prompting methods for LLM reasoning:• Direct Prompting, where model directly generates the answer without intermediate reasoning steps.• CoT (Wei et al., 2022; Kojima et al., 2022) , where models are prompted to generate a reasoning process leading to the final answer.• Plan-and-Solve (Wang et al., 2023) , where models are prompted to first generate a plan and then solve the problem. SELF-DISCOVER differs by grounding the reasoning structure in atomic reasoning modules, and prompting the decoding to follow the explicit key-value reasoning structure.Next, we also consider other baselines that make use of the raw seed reasoning modules (RM) we pass to SELF-DISCOVER. We compare with the following methods' performance and the inference call efficiency on a subset of tasks.• CoT-Self-Consistency Wang et al. (2022) , we sample multiple outputs from LLM with CoT and aggregate answers to get the final answer. We compare this method on a subset of tasks due to the cost of repetitive queries.• Majority voting of each RM: we prompt models to solve the tasks by appending each RM and use majority voting of all answers to get the final answer. We examine whether integrating multiple RMs into a coherent reasoning structure is advantageous to applying each RM to solve the task and use majority voting to ensemble them post-hoc, which costs much more inference computation.• Best of each RM: this method assumes that we have access to oracle labels and uses the highest accuracy from applying each RM. We compare with this to examine whether SELF-DISCOVER competes with methods that depend on perfect prior knowledge of which RM to use on a new task.Furthermore, for analysis on universality of reasoning structures, we compare with a promptoptimization method that require a training set to improve prompts: LLMs as optimizers (OPRO) (Yang et al., 2023) . We aim to show that when we apply structures or prompts optimized from one model, the reasoning structures can retain more performance gains than the wordings of prompts.4 ResultsWe answer the following questions through experimental results: 1) Does discovering reasoning structures improve LLM reasoning capabilities? (4.1) 2) Which categories of problems do SELF-DISCOVER perform the best? (4.2) and 3) Can SELF-DISCOVER boost LLM performance efficiently? (4.3) Finally, we will show qualitative examples of self-discovered structures, LLM output following the structures, and compare with LLM output following other prompting methods for reasoning (4.4).4.1 Does SELF-DISCOVER Improve LLM Reasoning?Overall, SELF-DISCOVER improves PaLM 2-L and GPT-4's reasoning across diverse set of reasoning tasks. Table 1 shows the overall results on complex reasoning tasks of BBH, T4D and Table 1 : Self-Discover significantly improves LLM reasoning across a diverse set of 25 complex tasks: BBH, T4D and MATH. CoT: zero-shot Chain of Thought (Kojima et al., 2022) . PS: plan-and-solve prompting (Wang et al., 2023) . MATH using PaLM 2-L and GPT-4. We compare Self-Discover with baselines including direct prompting, CoT, and Plan-and-Solve (PS).On aggregated 23 tasks of BBH, SELF-DISCOVER achieves 7% and 6% absolute improvement on PaLM 2-L over Chain-of-Thought and Plan-and-Solve, respectively. Similar gains (6% and 8%) are observed when SELF-DISCOVER is applied to GPT-4. Breakdown results of each task's improvement over direct answering and CoT of PaLM 2-L are shown in Figure 1 , where we find SELF-DISCOVER outperforms them on over 20/24 tasks. For a per-task performance for all 23 BBH tasks, please refer to Appendix C.On the grounded social agent task T4D, SELF-DISCOVER reaches over ≥ 27% (32%) absolute improvement over all baselines on PaLM 2-L (GPT-4). SELF-DISCOVER achieves 69% and 85% accuracy on PaLM 2-L and GPT-4, significantly outperforming previous SoTA prompting method such as Foresee and Reflect (FaR) which employs an expert-designed reasoning structure. In contrast, SELF-DISCOVER generates the reasoning structure automatically from a set of atomic reasoning modules without human interventions.For MATH, we observe a moderate gain of 1%-7% (2%-3%) on PaLM 2-L (GPT-4) from SELF-DISCOVER compared to the baselines. Upon error analysis (see Appendix E for details), we find that the reasoning structures generated by PaLM 2-L from SELF-DISCOVER are correct 87.5% of the time: human experts can follow the reasoning structures to solve the tasks perfectly. The majority of the failures (74.7%) comes from errors in executing the computations, consistent with prior findings Zheng et al. (2023) . Self-Discover Performance Improvement Across 4 Categories Self-Discover Over Direct Self-Discover Over CoT SELF-DISCOVER performs best on tasks that require diverse world knowledge. Figure 4 presents the average improvement in terms of delta in accuracy of SELF-DISCOVER over direct answer and CoT on 4 categories of reasoning tasks we test. We adopt the categorization from Suzgun et al. (2022) . We find that SELF-DISCOVER improves over these two baselines on all categories, but especially on tasks that require world knowledge such as sports understanding, movie recommendation, and ruin names.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: Due to legal constraints, we are not able to fully release the code and data at the current time. But we provide, with the best faith, details to reproduce our results.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
which types of problems: 
 These tasks demand models to reason using fact and general commonsense knowledge. We interpret SELF-DISCOVER's advantages on these tasks as strength from integrating multiple reasoning modules from various perspectives as only applying CoT might miss key knowl- Consistency, we sample 10 times. Best of each RM method requires gold labels (*). SELF-DISCOVER requires only 1 inference call per instance, same as Direct and CoT while reaching better performance compared with 40x more call required methods on GPT-4. We acknowledge that SELF-DISCOVER input and output are longer than CoT and Direct prompting, increasing cost. However, as the number of instances increases, the efficiency of SELF-DISCOVER in terms of inference per instance is highly desirable.edge in the reasoning process. We observe that the gain on the Algorithmic category is moderate, consistent with the findings from Sec. 4.1 on MATH.SELF-DISCOVER achieves better performance while requiring 10-40x fewer inference computer compared to self-consistency or majority voting. Here we examine a subset of 2 tasks from BBH and present a more thorough comparison of methods including those requiring many inference calls that are too costly to run on all 24 tasks. Figure 5 shows average accuracy and number of inference calls required per instance for each method using GPT-4. Accuracy wise (y-axis), we find that SELF-DISCOVER outperforms other baselines even those that require repeated inference calls such as CoT-self-consistency and majority voting of applying each RM. Efficiency wise (xaxis), SELF-DISCOVER only requires one call per instance and three more inference calls on the task-level, CoT-self-consistency requires 10 times more since we have to sample 10 times for each instance, and methods using each RM requires 40 times more as we use 40 RMs. In summary, SELF-DISCOVER presents itself a strong reasoning boosting method that is efficient to deploy on large-scale.We show examples of model-discovered structures for different reasoning tasks in Figure 10 from PaLM 2-L. We observe that each structure is uniquely adapted to the task, integrates multiple reasoning modules, and provides insights on how to solve the tasks. Furthermore, example of comparing reasoning processes from CoT, Plan-and-Solve, and SELF-DISCOVER is shown in Figure 6 . We find that CoT and Plan-and-Solve makes incorrect assertions early and arrives at a wrong answer while following structure from SELF-DISCOVER leads the model to generate logical conclusions ("path is closed as the beginning and ending coordinates are the same") and arrive at the correct answer. 
deep diving into self-discovered reasoning structures: 
 After experimental results showing the effectiveness and efficiency of SELF-DISCOVER on a range of reasoning tasks, this section further analyzes are all actions of SELF-DISCOVER needed and what other benefits can self-discovered structures bring? In Sec. 5.1, we show that it is critical to the model's performance to use the reasoning structures discovered through the three steps of SELECT, ADAPT and IMPLEMENT. In Sec. 5.2, we demonstrate the universality of the self-discovered reasoning structures by (1) applying the structures discovered by PaLM 2-L to GPT-4, (2) applying the structures discovered by GPT-4 to Llama-2-70B. We further show the commonalities between the reasoning structures and human reasoning patterns in Appendix F.This SVG path element <path d="M 55. 57,80.69 L 57.38,65.80 M 57.38,65.80 L 48.90,57.46 M 48.90,57.46 L 45.58,47.78 M 45.58,47.78 L 53.25,36.07 L 66.29,48.90 L 78.69,61.09 L 55.57,80. 69"/> draws a: (A) circle (B) heptagon (C) hexagon (D) kite (E) line (F) octagon (G) pentagon(H) rectangle (I) sector (J) triangleThis SVG path consists of multiple lines drawn between different points. These points do not form a regular geometric shape such as a circle, pentagon, heptagon, hexagon, or octagon. … They do, however, seem to form a shape with four sides, which could be considered a form of a kite…Thus the final answer is (D) kite.Let's think step-by-step (CoT)Let's devise a plan and solve the problem (Plan-and-Solve)This SVG Path element consists of many "L" commands which draw a straight line from the current point to the new point. But it does not close the path, meaning the shape is not a regular polygon like a hexagon, heptagon, octagon or pentagon. The instructions make 4 lines in total but they are not straight and do not form a parallelogram or a rectangle. …Thus the final answer is (D) kite.Self-Discover Reasoning Structure (Ours){ "Simplify SVG Path": ... "Breakdown of Path Commands": { "Move to Command (M)": "Sets the starting point for the next command without drawing anything.", "Line to Command (L) steps": {"Start and end coordinates of each line segment": "M 55. 57, 80.69 L 57.38, 65.80: From point (55.57, 80.69) to (57.38, 65.80 )"}, … and finally closing the shape at (55.57, 80.69)"} "Critical Thinking Analysis": { "Logical Reasoning": { "Analysis of path continuity": "The path includes multiple line segments that connect distinct points. The path ends by connecting back to the starting point, indicating a closed shape.", "Identification of closed shapes": "The final line segment connects the last point back to the first point, which is characteristic of a closed shape."}, … "Final Reasoning and Decision": "With 7 distinct points all connected in a closed path, the shape formed is a heptagon.", "Final Answer": "B) heptagon"} ✅ Figure 6 : Comparison of generated reasoning process from CoT, Plan-and-Solve, and SELF-DISCOVER on BBH-geometric shape task. Both CoT and Plan-and-Solve incorrectly asserts that the path does not form a regular shape as it is not a closed path (highlighted in red) and arrive at a wrong answer. The reasoning structure (in blue Courier font) from SELF-DISCOVER first breaks down each line segment and analyze the coordinates carefully, then leverages logical reasoning to conclude that it forms a closed shape as the path ends at the same coordinate (highlighted in purple and orange), and selects the correct answer through final reasoning. We conduct ablation study on the three actions: SELECT, ADAPT, and IMPLEMENT to analyze the effects of SELF-DISCOVER actions. Figure 7 show results using GPT-4 on 4 reasoning tasks when we apply SELECT (-S) or apply SELECT and ADAPT (-SA) or apply all three actions. We find that with each stage, model's zeroshot reasoning capability improve consistently across tasks, indicating that all three actions are beneficial. Applying PaLM 2-L Discovered Structures to GPT-4 We first use a PaLM 2-L model to discover the reasoning structures of 4 reasoning tasks. Then, we apply the resulting reasoning structures to the decoding of GPT-4 as grounding. We compare our approach to OPRO (Yang et al., 2023) which discovered zero-shotprompts through optimizations. We apply OPRO prompts optimized using PaLM 2-L on each task to GPT-4 on the same reasoning tasks. Figure 8 shows that SELF-DISCOVER outperforms OPRO on 3 out of 4 tasks despite that OPRO used 20% data to op-timize the prompt. In contrast, SELF-DISCOVER is done in a zero-shot manner, demonstrating the efficiency of our method and universality of the discovered reasoning structures.Applying GPT-4 Discovered Structures to Llama2 and ChatGPT Motivated by transferrability performance across LLMs, we further investigate can self-discovered reasoning structures from LLMs boost reasoning for smaller LMs that are challenging to come up with structures themselves3 . We use GPT-4 to discover the task-intrinsic reasoning structures, and then apply those structures to the decoding of open-sourced Llama2-70B as well as GPT-3.5-turbo (ChatGPT) on two subsets of tasks from BBH. We find that using self-discovered structures on Llama2 (52%) outperforms CoT (42%) on disambiguation QA zero-shot and on GPT-3.5-turbo (56%) outperforms CoT (51%) on geometry with 3-shot demonstration from structured reasoning process.6 Related Work 
prompting methods: 
 Recent advancements in the area of LLMs have given rise to a plethora of few-shot (Brown et al., 2020) and instruction (Mishra et al., 2022c; Wei et al., 2021; Ouyang et al., 2022) prompting techniques, including Chain-of-Thought prompting (CoT) (Nye et al., 2021; Wei et al., 2022) , Least-to-most prompting (Zhou et al., 2022a; Drozdov et al., 2022) , Decomposed prompting (Khot et al., 2022) , Reframing (Mishra et al., 2022b) , Help Me Think Prompting (Mishra & Nouri, 2023) , Stepback Prompting (Zheng et al., 2023) and search-based approaches like Tree-of-Thought (ToT) (Yao et al., 2023a) , Graph-of-Thought (Besta et al., 2023; Yao et al., 2023b) , Branch-solve-merge (Saha et al., 2023) and RAP (Hao et al., 2023) . Each of the prompting methods has some strengths and weaknesses in terms of their successful application domain. Our work SELF-DISCOVER presents the missing piece in the prompting literature, as SELF-DISCOVER provides a way to self-compose over various prompting methods via the proposed self-discovery mechanism. Composing over prompting methods in SELF-DISCOVER is analogous to the programming literature where a program is written using various basic building blocks such as for loop, if/else condition etc.With 2023b), etc. However, in a real world user traffic, queries can be diverse covering various reasoning structures. Our work SELF-DISCOVER allows models to combine multiple reasoning approaches by self-composing into a structure without the need to access task labels. There have been some related work that explores LLM combining skills in-context such as SkiC (Chen et al., 2023) , devising a strategy (Gao et al., 2023a) , and planning with iterative quering (Liu et al., 2023) . However, they require human annotating skills and reasoning plans while SELF-DISCOVER leverages a scalable solution with the help of LLM's meta-task reasoning capabilities.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We provide details of inference procedures including initial prompts, framework of each data/test split, and model specifications.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion: 
 We introduce SELF-DISCOVER, an efficient and performant framework for models to self-discover a reasoning structure for any task from a seed set of general problem-solving skills. We observe drastic improvements on challenging reasoning benchmarks from multiple LLMs up to 30%. Ablations study of SELF-DISCOVER demonstrates that the composed reasoning structures are universally transferable between LLMs. Forward looking, we are excited to explore more on LLM structured reasoning to push the boundary of problem-solving and discover potentials for Human-AI collaboration.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We include discussions of statistical significance and most improvements are significant.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: Our work is a foundational study on model's reasoning capabilities.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite all models, data, and code we use in our experiments.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: We do not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: We do not involve extensive crowdsourcing.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: We do not involve extensive crowdsourcing.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
: 
 in generating coherent texts (OpenAI, 2022) , and following instructions (Zhong et al., 2021; Mishra et al., 2022c; Wei et al., 2021; Chung et al., 2022; Ouyang et al., 2022) . In pursuit of the goal to enhance LLMs' capability to reason and solve complex problems, various prompting methods have been proposed, drawing inspirations from cognitive theories of how humans reason. For example, few-shot and zero-shot chain-of-thought (CoT) (Nye et al., 2021; Wei et al., 2022; Kojima et al., 2022; Yasunaga et al., 2023) resembles how humans solve problems step-by-step, decomposition-based prompting (Zhou et al., 2022a; Drozdov et al., 2022; Patel et al., 2022; Hao et al., 2023; Khot et al., 2022) is inspired by how humans breakdown a complex problem into a series of smaller subproblems, and then solve those subproblems one by one (Polya, 2004) , and step-back prompting (Zheng et al., 2023) is motivated by how humans reflect on task nature to derive general principles. However, a fundamental limitation is that each technique itself serves as an atomic reasoning module making an implicit prior assumption of the process on how to tackle a given task. Instead, we argue that each task has a unique intrinsic structure underlying the reasoning process involved in solving it efficiently. For instance, least-to-most prompting (Zhou et al., 2022a; Drozdov et al., 2022) has shown to be much more effective than CoT (Wei et al., 2022) at solving tasks such as symbolic manipulation and compositional generalization, due to the decomposition structure of the tasks.This paper aims at self-discovering the underlying reasoning structure unique to each task, while being highly efficient in terms of computation. Our approach, SELF-DISCOVER, is inspired by how humans internally devise a reasoning program for problem-solving (Newell et al., 1958; Rasmussen, 1983) , as illustrated in Figure 2 . From a set of atomic reasoning modules described in natural language such as "breakdown into sub tasks" and "critical thinking", an LLM, and task examples without labels, SELF-DISCOVER composes a coherent reasoning structure intrinsic to the task (Stage 1) and then solves instances of the task using the discovered structure (Stage 2). Stage 1 operates at the task-level and uses three actions to guide the LLM to generate a reasoning structure for the task. At Stage 2, during the final decoding, the LLM simply follows the self-discovered structure to arrive at the final answer.Solving problems using SELF-DISCOVER brings several benefits compared to other methods for LLM reasoning. First, the discovered reasoning structure is grounded in atomic reasoning modules benefiting from the strengths of multiple reasoning modules in contrast to applying a priori module such as CoT. Second, SELF-DISCOVER is efficient in computation as it only requires 3 more inference steps on the task-level, while being more performant than inference-heavy ensemble approaches such as self-consistency Wang et al. (2022) . Lastly, the discovered reasoning structure is intrinsic to the task, and conveys LLMs' insights about the task in a more interpretable way than the optimized prompts (Zhou et al., 2022b; Yang et al., 2023) .We test SELF-DISCOVER on 25 challenging reasoning tasks including Big Bench-Hard (BBH) (Suzgun et al., 2022) , Thinking for Doing (T4D) (Zhou et al., 2023) and MATH (Hendrycks et al., 2021) . SELF-DISCOVER outperforms CoT on 21/25 task with performance gains up to 42% (Figure 1 ), highlighting the advantage of the self-discovered reasoning structure composed from the atomic reasoning modules against a single a priori CoT module. Furthermore, we demonstrate that SELF-DISCOVER achieves superior performance against inference-heavy methods such as CoT + Self-Consistency and majority voting of every module while requiring 10-40x fewer inference compute (Figure 5 ). Finally, we compare SELF-DISCOVER with prompts optimized (OPRO) using a training set (Yang et al., 2023) (Figure 8 ). We find that SELF-DISCOVER still performs on par or better than OPRO while the self-discovered reasoning structure are much more interpretable.We conduct a set of analysis to understand the effectiveness of SELF-DISCOVER. By breaking down BBH tasks into 4 different categories, we find that SELF-DISCOVER performs best on tasks requiring world knowledge and has a moderate performance boost on algorithmic tasks compared to CoT (Figure 4 ). This is further confirmed by the error analysis on MATH, where 74.7% model failures comes from computation errors (e.g. math). We also take a closer look at the self-discovered reasoning structures, and show the universality of them by transferability study from PaLM 2-L to GPT-4, and from GPT-4 to Llama-2-70B. We hope to encourage more future work on structured reasoning for solving challenging problems using LLMs. 
 
 Appendix: Table 2 shows all 39 reasoning modules we use for SELF-DISCOVER, adopted from Fernando et al. (2023) , that contain cognitive heuristics of problem-solving.Figure 9 contains the structure of the three actions of SELF-DISCOVER during Stage 1, where it discovers an intrinsic reasoning structure on the task-level.For Stage 2, where we use the self-discovered structure to solve the task instances, we start with the prompt: "Follow the step-by-step reasoning plan in JSON to correctly solve the task. Fill in the values following the keys by reasoning specifically about the task given. Do not simply rephrase the keys.", followed by the reasoning structure, and finally the task instance. We use accuracy and exact matching as with other methods tested on BBH, T4D and MATH. To properly evaluate the generated answers from LLMs, we prompt the models to end the answer with "Thus, the final answer is [X]", where X is either one answer option such as "A" or a string such as "valid". During evaluation, we manually examine each task's outputs from LLMs and design heuristics to extract the final answers. For MATH dataset, we find that it is challenging to extract the answers accurately. As a result, we subsample 200 test examples from MATH, and manually sanity check and annotate the extracted answers for all methods tested in our paper.Here we also provide more details on token counts for Self-Discover and other baselines. Since Self-Discover only added 3 calls per task, on per instance level it only needs to run once. The average length of self-discovered structures is 224 tokens for BBH, 183 tokens for T4D, and 152 for MATH, which is similar to ToT/GoT prompts with around 234 tokens. We run the stage 2 (generating solution based on the structure) only in 1 inference pass where the model fills the values in the self-discovered structures. Thus, Self-Discover still reduces much inference cost compared to CoT-Self-Consistency, majority voting, etc. which requires 20s-40s fold of each CoT reasoning path token counts.Per-task performance on BBH (23 tasks in total) are shown in Table 3 . "Identify the chain of events in the story":"Identify the consequences of each event":"Identify the cause-and-effect relationships between events": "Choose a final answer based on the reasoning": } dyck_languages { "Parentheses that are not closed properly": "Stack to store the closing parentheses": "If the next symbol is a closing parenthesis, pop the stack and check if the popped symbol matches the next symbol":"If the stack is empty, add the next symbol to the stack": }Figure 10 : Examples of self-discovered structures on BBH tasks using PaLM 2-L. We observe traits of atomic reasoning modules such as "step-by-step thinking", "reflect on task nature", and an interesting creative thinking case where models devise an algorithm using stack to solve parenthesis parsing task. We further include Tree-of-Thought (Yao et al., 2023a) and Graph-of-Thought (Besta et al., 2023) (zero-shot versions) as baselines for comparison shown in Table 4 .To show the effectiveness of Self-Discover on more general tasks, we tested on a subset of MMLU (10 subtasks, with 50 diverse questions each, all randomly sampled) and results are shown in Table 5 . We find that GPT-4+Self-Discover wins GPT-4+CoT in zero-shot on 7 out of 10, ties on 2 out of 10, and loses on 1 out of 10 tasks. In addition, we tried Self-Discover on the instance-level, where for each question, we run stage 1 to output the reasoning structure, then solve the task. We find that the instance-level Self-Discover performs even better on MMLU, outperforming CoT by 7.2% on average for all tasks. This result, combined with those in main content, shows that the strength of Self-Discover spans across two types of tasks: for well-defined hard reasoning tasks such as BBH, task-level Self-Discover works well while being very efficient; for very open-domain tasks such as MMLU, we can do instance-level Self-Discover, which significantly outperforms CoT while still fewer inference required than self-consistency.We show additional examples of self-discovered structures in Figure 10 . We observe traits of atomic reasoning modules such as "step-by-step thinking", "reflect on task nature", and an interesting creative thinking case where models devise an algorithm using stack to solve parenthesis parsing task. Step 5 is unnecessary and leads to error in the final result. -There are 6 choices for the second person.-There are 5 choices for the third person.-There are 4 choices for the fourth person.-There are 3 choices for the fifth person.-There are 2 choices for the sixth person. "2. Divide the result from step 1 by the number of rotations that are considered the same.": 5040 / 6 = 840 -There are 6 rotations that are considered the same.There are 7 rotations instead of 6 that are considered the same.A customer ordered 15 pieces of gourmet chocolate. The order can be packaged in small boxes that contain 1, 2 or 4 pieces of chocolate. Any box that is used must be full. How many different combinations of boxes can be used for the customer's 15 chocolate pieces? One such combination to be included is to use seven 2-piece boxes and one 1-piece box."1. Find the number of ways to package the chocolates using only boxes of 1 piece.": 1, "2. Find the number of ways to package the chocolates using only boxes of 2 pieces.": 1, "3. Find the number of ways to package the chocolates using only boxes of 4 pieces.": 0, "4. Find the number of ways to package the chocolates using a combination of boxes of 1 and 2 pieces.": 7, "5. Find the number of ways to package the chocolates using a combination of boxes of 1 and 4 pieces.": 3, "6. Find the number of ways to package the chocolates using a combination of boxes of 2 and 4 pieces.": 2, "7. Find the number of ways to package the chocolates using a combination of boxes of 1, 2, and 4 pieces.": 4, "8. Sum up the total number of ways from the seven cases above.": 1 + 1 + 0 + 7 + 3 + 2 + 4 = 18There is 0 way to package 15 chocolates using only boxes of 2 pieces.How many primes are in the row of Pascal's Triangle that starts with a 1 followed by a 6?"1. Identify the row of Pascal's Triangle that starts with a 1 followed by a 6.": "1, 6, 15, 20, 15, 6, 1", "2. List all the numbers in that row.": [1, 6, 15, 20, 15, 6 , 1], "3. Check each number in the list for primality.": [False, True, False, False, False, True, False], "4. Count the number of primes found in the list.": 2 6 is not a prime.DISCOVER meta-reasoning stage) and ask them to write a reasoning structure for a task before solving it. Figure 11 shows comparison of human and LLM-composed reasoning structures on the BBH-navigation task. We observe similar structures such as mental-noting after each movement.From promising findings of LLM self-discovered structures boost and share traits of human metareasoning, we hope to encourage more future work to study humna-AI collaboration for complex problem-solving.To show whether the chosen reasoning modules and discovered structures provide a diverse coverage, we have included more details on frequency of selected reasoning modules in Table 8 . Furthermore, we observe a very diverse set of self-discovered structures as the 12 examples shown in Figure 6 , 10, 11, and Table 6 and 7 . Math problem structures (Table 6 ) tend to be very different from BBH ones (Figure 10 ). Due to the non-deterministic nature of LLMs, the number of reasoning structures that can be discovered by Self-Discover is infinite, because it can use multiple reasoning modules in many different orders. In our prompts, we specifically do not restrict how the structures should use the modules and find examples where models use new modules not in the seed list (Figure 10 Dyck-Language example where model devises a stack algorithm, which is not in the seed list). Step-by-step plan with explanation 21/25NeurIPS Paper Checklistaccessed in October-December2 For MATH, we use a PaLM 2-L model with a stronger instruction tuning to enable better instruction following of more complex reasoning structures.We tried zero-shot meta prompting Llama2 but observed low-quality structure outputs. 


Decision: Among an already dense field of prompting approaches, the paper presents a compelling and novel approach to prompting based on self-discovery of a reasoning structure and then adaptation of this structure to the specific task at hand.  The idea is inspired by studies of human problem solving that have also motivated other prompting methodologies.  Extensive (and as reviewer NVyo points out -- purposeful) experimental results not only show the improvement of this method over numerous baselines, but also extensive ablations, quantitative, qualitative investigation of its performance.  The paper is well-written.  Rebuttal discussion led to analysis updates, clarifications, and agreement to add additional analysis in the Appendix, which the authors are requested to include in their final revision.

Reviews: review: paper_summary: paper_summary: This paper proposes a prompt engineering scheme:
Given a task and 39 prompts for solving tasks, 
an LLM is prompted to select which of the 39 prompts are suitable for the task.
The selected prompts are then rephrased to be more specific to the task, 
and reformatted in json.

The paper uses GPT-4 Turbo, PaLM 2-L, and Llama2-70B
and compares direct prompting, chain of thought, plan and solve, 
chain of though with self-consistency, majority vote of the 39 prompts, and the proposed method.
strength_weakness: strength_weakness: 1. As a prompt engineering method, it is unclear if the choice of the specific 39 prompts will withstand the test of time across LLMs and tasks. 

2. Recent prompt engineering tools, such as Anthropic's Claude 3.5 Prompt Engineer, receive a task as input and output a detailed prompt best suited for the task and LLM, which may supersede this approach. 

3. The claim of efficiency should be more rigorously validated and consider both the number of calls and the token lengths of the prompts and responses of each call.
questions: questions: When comparing prompting schemes, it is important to give equal compute time to different methods. 
When measuring efficiency did the work consider the lengths of the prompts and responses?
limitations: limitations: Yes
paper_summary: paper_summary: The paper introduces SELF-DISCOVER, a novel framework that enables large language models (LLMs) to self-discover and compose reasoning structures for tackling complex reasoning tasks. The core of SELF-DISCOVER is a self-discovery process where LLMs select multiple atomic reasoning modules, such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure to follow during decoding.
strength_weakness: strength_weakness: 1) The paper does not specify the exact number of reasoning structures that can be discovered by the SELF-DISCOVER framework. It's unclear whether the framework can generate a wide variety of structures or if it tends to converge on a limited set of common structures.
2) It is not detailed whether different examples within the same task utilize the same reasoning structure or if the framework can adapt the structure to suit the nuances of individual examples.
3) The paper's comparative analysis may be considered basic, as it does not delve into a comprehensive set of existing methods or include recent state-of-the-art approaches for comparison.
4) Although the paper mentions a similarity to the "Meta Reasoning for Large Language Models" approach, it does not provide a direct comparative experiment or analysis to highlight differences and potential advantages of SELF-DISCOVER.
5) The paper might not fully account for or cite the most recent literature, specifically works from 2024 and beyond, which could provide additional context and comparison points for the SELF-DISCOVER framework.
questions: questions: See weaknesses.
limitations: limitations: See weaknesses.
paper_summary: paper_summary: This paper introduces SELF-DISCOVER, a framework designed to enhance the reasoning capabilities of Large Language Models (LLMs) such as GPT-4 and PaLM 2. The framework enables LLMs to self-discover and compose intrinsic reasoning structures tailored to specific tasks, improving performance on complex reasoning benchmarks.
strength_weakness: strength_weakness: 1. While atomic reasoning structures form the foundation of the SELF-DISCOVER method, their inherent simplicity can restrict the overall reasoning performance. These basic units, although useful, may not fully capture the complexity needed for certain advanced reasoning tasks, potentially limiting the depth and flexibility of the reasoning process.

2. The proposed method is significantly reliant on the performance of the underlying LLM. Factors such as the model's ability to follow instructions accurately and the limitations imposed by context window sizes can pose significant challenges. For instance, if the base LLM struggles with instruction-following or is constrained by a limited context window, the effectiveness of the SELF-DISCOVER method may be compromised, limiting its potential for broader application and scalability.

3. The iterative process of selecting, adapting, and implementing reasoning modules introduces extra computational costs. Each step in this process requires additional queries to the LLM, which can accumulate and result in increased computational demands. This overhead may offset some of the efficiency gains achieved through reduced inference steps, particularly in scenarios requiring frequent adaptation and customization of reasoning structures.
questions: questions: 1. What is the additional cost of implementing this method, such as the extra tokens generated or the increase in inference time?

2. Please refer to the correct section number in the Appendix in lines 83-84.

3. The comparison in Figure 5 may not be fair. The self-discovery method requires fewer calls but could lead to more token generations.

4. The design of the "select, adapt, and implement" steps seems ad-hoc. Why are these three steps proposed separately? For instance, why can't "select" and "adapt" be combined into a single step since their prompts are similar, as shown in Figure 9?

5. What are the commonly selected reasoning structures? Are all 39 reasoning modules useful? It would be better to show the density of the selected reasoning modules to ensure that the selection of 39 modules is meaningful.

6. Does the model perform consistently in the selection process?

7. Do different models, such as GPT-4 and Palm-2, perform consistently? Do they discover similar reasoning structures?
limitations: limitations: This paper does not sufficiently illustrate the limitations of the proposed method. The checklist on lines 542-543 states, "We present an extensive analysis of the limitations of Self-Discover in both Sections 4 and 5 with an extended error analysis on MATH in Appendix E." However, Sections 4 and 5 are primarily general evaluations of the proposed method and do not contain sufficient and clear illustrations of its limitations. This issue needs to be strictly addressed.
paper_summary: paper_summary: The paper introduces SELF-DISCOVER, a framework enabling large language models (LLMs) to autonomously identify and utilize intrinsic reasoning structures to tackle various reasoning tasks. The framework is applicable across different model families and aligns with human reasoning patterns.
strength_weakness: strength_weakness: 1. The framework's effectiveness is heavily dependent on the quality and comprehensiveness of the atomic reasoning modules available, which may require significant human effort to define. Additionally, there is no guideline provided on how to develop these modules.
2. The main results are reported using a very limited range of models, specifically GPT-4 and PaLM-2. A systematic evaluation across a broader range of models, such as GPT-3.5, Llama2, and Llama3, is necessary, as merely applying reasoning structures generated by GPT-4 to these models is insufficient.
3. The compared baselines, such as zero-shot CoT and Plan-and-Solve, are not strong enough. Automatic prompting engineering methods, like LLM as Optimizers [1] and StrategyLLM [2], should be included in comparisons and discussions.

[1] Large Language Models as Optimizers. https://arxiv.org/abs/2309.03409

[2] StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving. https://arxiv.org/abs/2311.08803
questions: questions: NA
limitations: limitations: Yes
paper_summary: paper_summary: The paper introduces a new prompting method, Self-Discover, that enhances the complex reasoning abilities of Large Language Models. Self-Discover consists of two stages. At the first one, the method develops a task-adapted prompt by performing 3 steps: SELECT, when the LLM selects the reasoning modules (RMs) from a pre-defined list, ADAPT, when the model adapts the chosen reasoning modules to the task, and IMPLEMENT, when the model, conditioned on the adapted reasoning modules, generates the reasoning structure as a JSON. Each step is performed in a zero-shot manner and controlled by a dedicated meta-prompt. The resulting reasoning structure is used in the final second stage, as part of the prompt to solve each task and produce the output. For each task instance, the LLM first reasons by completing the JSON value fields, and then outputs the answer. Self-Discover is compared to several baselines: Direct Prompting, CoT (+Self-Consistency), Plan-and-Solve, Majority voting of RMs, and Best of each RM, as well as Graph-of-Thought (GoT) and Tree-of-Thought (ToT). Evaluation is performed on BIG-Bench Hard (BBH), Thinking for Doing (T4D), and (subsampled) MATH benchmarks. The results demonstrate the superiority of Self-Discover against the baselines above in terms of performance while maintaining comparatively low computational cost, expressed in the number of necessary inference calls. The ablation study highlights that each step of Stage 1 significantly contributes to the overall performance. The generated reasoning structures appear to be general and transferable between different LLMs, compared to Optimized Prompts (OPRO).
strength_weakness: strength_weakness: - The paper doesn't emphasize enough that the method requires task adaptation. This is a crucial detail, as, unlike CoT or ToT, Self-Discover cannot be deployed as a general zero-shot agent, but requires some specialized prompting with task examples. This limitation is addressed barely in the paper. Though this is not a huge disadvantage, it places the method right between zero-shot methods (CoT / ToT / GoT / PS) and task-adaptive methods (OPRO / DSPy).  Thus, more comparisons with the prompt adaptation techniques should be present in the paper, and the efficiency of Self-Discover should be emphasized when comparing to them. 
- The authors claim that their method "composes atomic reasoning models into a reasoning structure", but this seems misleading. Figure 4 demonstrates that the method only slightly outperforms the best RM baseline, suggesting that no actual "composition" is happening, but only a selection and adaptation of the best fitting reasoning technique. This requires further investigation, for example, by analyzing which reasoning modules were picked for a task, and ablating the performance on these modules. This would reveal whether the model indeed uses multiple reasoning modules or only one in its prediction.
questions: questions: - How robust is the method to changes in the list of reasoning modules? An ablation study on them, or at least the list of chosen modules for each task would reveal this information. Also, how robust is the method to the set of chosen task examples? What was their number in the experiments, and does varying their number or themselves affect the performance?
- What were the parameters of ToT & GoT in the experiments? The value of the breadth parameter can largely affect the performance of these methods.
- What modifications can be applied to the set of RM to increase the performance on the algorithmic tasks? I assume some specialized algorithmic prompting can be useful. What you also include the relationship between reasoning modules and the tasks in which they are useful, and assert that the model chooses the correct reasoning module?
limitations: limitations: As mentioned above, though the method is very effective, it is dependent on the set of chosen task examples, and so is task-adaptable, and not completely zero-shot. This limitation is not emphasized enough in the paper.

However, overall, it is a nice paper with promising results, which once again underscores the power of optimal prompting and alignment.

Comments: comment: We thank the reviewer for the feedback and we will include the additional details in the appendix in the next revision.
We are glad that your concerns were addressed and we will update in the appendix more ablations in the next revision. We thank the reviewer for helping us to improve our paper!
Thanks again for reviewing our work! Please let us know if we have addressed your questions and concerns.
Thanks again for reviewing our work! Please let us know if we have addressed your questions and concerns.
Thanks to the authors for their response.

My concerns regarding the synergy between the reasoning modules have been addressed. However, as the authors mentioned, I expect to see more ablation results in the appendix in the paper revision, which is why I intend to keep the positive score.
I thank authors for the response, please see my comments below.

> We have included more details on frequency of selected reasoning modules in the attached 1-page pdf. 

Please add this table in the appendix in the paper revision.

>  models perform consistently in the reasoning module selection process across a wide range of tasks. 

Please also include these details in the paper revision.

Based on the above discussions, I will keep my positive score.