Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (2023)

Abstract: Large language models (LLMs) have recently been shown to deliver impressive performance in various NLP tasks. To tackle multi-step reasoning tasks, Few-shot chain-of-thought (CoT) prompting includes a few manually crafted step-by-step reasoning demonstrations which enable LLMs to explicitly generate reasoning steps and improve their reasoning task accuracy. To eliminate the manual efforts, Zero-shot-CoT concatenates the target problem statement with “ Let’s think step by step ” as an input prompt to LLMs. Despite the success of Zero-shot-CoT, it still suffers from three pitfalls: calculation errors, missing-step errors, and semantic misunderstanding errors. To address the missing-step errors, we propose Plan-and-Solve (PS) Prompting. It consists of two components: first, devising a plan to divide the entire task into smaller subtasks, and then carrying out the subtasks according to the plan. To address the calculation errors and improve the quality of generated reasoning steps, we extend PS prompting with more detailed instructions and derive PS+ prompting. We evaluate our proposed prompting strategy on ten datasets across three reasoning problems. The experimental results over GPT-3 show that our proposed zero-shot prompting consistently outperforms Zero-shot-CoT across all datasets by a large margin, is comparable to or exceeds Zero-shot-Program-of-Thought Prompting, and has comparable performance with 8-shot CoT prompting on the math reasoning problem. The code can be found at https://github.com/AGI-Edgerunners/Plan-and-Solve-Prompting .

## introduction large language models (llms) @xcite @xcite have recently proven highly effective in various nlp tasks. unlike the previous pre-trained language models (ptms) @xcite ; liu * corresponding author. et al., 2019), these llms are typically provided as a service, with no access to model parameters due to commercial considerations and potential risks of misuse @xcite . thus, it is challenging to fine-tune llms for downstream tasks @xcite @xcite . instead, we leverage llms to solve complex reasoning problems by eliciting their strong reasoning abilities over their embedded knowledge using instructions (or trigger sentences). so far, llms have shown impressive abilities to solve new reasoning problems by simply conditioning them on a few illustrative examples (i.e., few-shot learning) or a prompt to solve new problems without illustrative examples (i.e., zero-shot learning). to tackle multi-step complex reasoning tasks using llms, @xcite proposes few-shot chain-of-thought (cot) prompting, which enables llms to explicitly generate the intermediate reasoning steps before predicting the final answer with a few manual step-by-step reasoning demonstration examples. in @xcite , zero-shot cot eliminates the need for manually crafted examples in prompts by appending "let's think step by step" to the target problem fed to llms such as gpt-3. this simple prompting strategy surprisingly enables llms to yield performance similar to few-shot cot prompting. despite the remarkable success of zero-shot-cot in solving multi-step reasoning tasks, its results on a sample of 100 arithmetic test examples still point to three pitfalls (as shown in figure 1 ): (i) calculation errors (in 7% of test examples): these are errors in the calculation leading to wrong answers; (ii) missing step errors (in 12% of test examples): these occur when some intermediate reasoning step(s) is missed-out especially when there are many steps involved; (iii) semantic misunderstanding (in 27% of test examples): there are other errors in semantic understanding of the problem and coherence of reasoning steps likely to be caused by the insufficient capability of llms. to address the issue of zero-shot-cot caused by missing reasoning steps, we propose plan-and-solve (ps) prompting. it consists of two components: first, devising a plan to divide the entire task into smaller subtasks, and then carrying out the subtasks according to the plan. in our experiments, we simply replace "let's think step by step" of zeroshot-cot with "let's first understand the problem and devise a plan to solve the problem. then, let's carry out the plan and solve the problem step by step" (see figure 2 (b) ). to address the calculation errors of zero-shot-cot and improve the quality of generated reasoning steps, we add more detailed instructions to ps prompting. specifically, we extend it with "extract relevant variables and their corresponding numerals" and "calculate intermediate results (pay attention to calculation and commonsense)" instructions. this prompting variant is called the ps+ prompting strategy (see figure 3 (b) ). despite its simplicity, ps+ strategy greatly improves the quality of the generated reasoning process. moreover, this prompting strategy can be easily customized to solve a variety of problems other than math reasoning, such as commonsense and symbolic reasoning problems. we evaluate our proposed prompting on six math reasoning datasets, including aqua @xcite , gsm8k @xcite , mul-tiarith, addsub, singleeq, and svamp @xcite , two commonsense reasoning datasets (commonsenseqa @xcite and strat-egyqa @xcite ), and two symbolic reasoning datasets (last letter and coin flip @xcite ). the results of our experiments with gpt-3 show that our proposed zero-shot-ps+ prompting consistently outperforms zero-shot-cot across all reasoning problems and datasets by a large margin, and is comparable to or exceeds zeroshot-program-of-thought (pot) prompting @xcite ). furthermore, although ps+ prompting does not require manual demonstration examples, it has a performance similar to an 8-shot cot prompting in arithmetic reasoning. overall, our results suggest that (a) zero-shot ps prompting is capable of generating a higher-quality reasoning process than zero-shot-cot prompting, as the ps prompts provide more detailed instructions guiding the llms to perform correct reasoning tasks; (b) zero-shot ps+ prompting outperforms few-shot manual-cot prompting on some datasets, indicating that in some instances it has the potential to outperform manual few-shot cot prompting, which hopefully will spark further development of new cot prompting approaches to elicit reasoning in llms. ## plan-and-solve prompting overview. we introduce ps prompting, a new zero-shot cot prompting method, which enables llms to explicitly devise a plan for solving a given problem and generate the intermediate reasoning process before predicting the final answer for the input problem. as opposed to prior few-shot cot approaches where step-by-step few-shot demonstration examples are included in the prompt, the zero-shot ps prompting method does not require demonstration examples, and its prompt covers the problem itself and a simple trigger sentence. similar to zero-shot-cot, zero-shot ps prompting consists of two steps. in step 1, the prompt first makes an inference using the proposed prompting template to generate the reasoning process and the answer to a problem. in step 2, it extracts the answer for evaluation by using the answer extraction prompting, such as "therefore, the answer (arabic numerals) is". ## conclusion in this paper, we find that zero-shot-cot still suffers from three pitfalls: calculation errors, missingreasoning-step errors, and semantic understanding errors. to address these issues, we introduce plan-and-solve prompting strategies (ps and ps+ prompting). they are new zero-shot prompting methods that guide llms to devise a plan that divides the entire task into smaller subtasks and then carries out the subtasks according to the plan. evaluation on ten datasets across three types of reasoning problems shows ps+ prompting outperforms the previous zero-shot baselines and performs on par with few-shot cot prompting on multiple arithmetic reasoning datasets. overall, our results suggest that (a) zero-shot ps+ prompting can generate a high-quality reasoning process than zero-shot-cot prompting since the ps prompts can provide more detailed instructions guiding the llms to perform correct reasoning; (b) zero-shot ps+ prompting has the potential to outperform manual fewshot cot prompting, which hopefully will spark further development of new cot prompting approaches to elicit reasoning in llms. moreover, ps(+) prompting is a general idea that can be used for non-reasoning tasks, and refining the plan is also an interesting idea. we leave them for future work.