Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Abstract
We explore how generating a chain of thought —a series of intermediate reasoning steps—signiﬁcantly improves the ability of large language models to perform complex reasoning.
In particular, we show how such reasoning abilities emerge naturally in sufﬁciently large language models via a simple method called chain-ofthought prompting , where a few chain of thought demonstrations are provided as exemplars in prompting.
Experiments on three large language models show that chain-of-thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks.
The empirical gains can be striking.
For instance, prompting a PaLM 540B with just eight chain-of-thought exemplars achieves state-of-the-art accuracy on the GSM8K benchmark of math word problems, surpassing even ﬁnetuned GPT-3 with a veriﬁer. A: The cafeteria had 23 apples originally.
They used 20 to make lunch.
So they had 23 - 20 = 3.
They bought 6 more apples, so they have 3 + 6 = 9.
The answer is 9. Chain of Thought Prompting Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
Each can has 3 tennis balls.
How many tennis balls does he have now?
A:
The answer is 11.
Q:
The cafeteria had 23 apples.
If they used 20 to make lunch and bought 6 more, how many apples do they have?
A:
The answer is 27.
Standard Prompting
Q: Roger has 5 tennis balls.
He buys 2 more cans of tennis balls.
Each can has 3 tennis balls.
How many tennis balls does he have now?
A: Roger started with 5 balls.
2 cans of 3 tennis balls each is 6 tennis balls.
5 + 6 = 11.
The answer is 11.
Q: The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have? [Figure]
Introduction
[Figure]
[Figure]
[Figure]
Finetuned GPT-3 and prior best are from Cobbe et al. The NLP landscape has recently been revolutionized by language models (Peters et al., 2018; Devlin et al., 2019;
Brown et al., 2020, inter alia ).
Scaling up the size of language models has been shown to confer a range of beneﬁts, such as improved performance and sample efﬁciency (Kaplan et al., 2020; Brown et al., 2020, inter alia ).
However, scaling up model size alone has not proved sufﬁcient for achieving high performance on challenging tasks such as arithmetic, commonsense, and symbolic reasoning (Rae et al., 2021).
This work explores how the reasoning ability of large language models can be unlocked by a simple method motivated by two ideas.
First, techniques for arithmetic reasoning can beneﬁt from generating natural language rationales that lead to the ﬁnal answer.
Prior work has given models the ability to generate natural language intermediate steps by training from scratch (Ling et al., 2017) or ﬁnetuning a pretrained model (Cobbe et al., 2021), in addition to neuro-symbolic methods that use formal languages instead of natural language (Roy and Roth, 2015; Chiang and Chen, 2019; Amini et al., 2019; Chen et al., 2019).
Second, large language models offer the exciting prospect of in-context few-shot learning via prompting .
That is, instead of ﬁnetuning a separate language model checkpoint for each new task, one can simply “prompt” the model with a few input–output exemplars demonstrating the task.
Remarkably, this has been successful for a range of simple question-answering tasks (Brown et al., 2020).
Both of the above ideas, however, have key limitations.
For rationale-augmented training and ﬁnetuning methods, it is costly to create a large set of high quality rationales, which is much more complicated than simple input–output pairs used in normal machine learning. For the traditional fewshot prompting method used in Brown et al. , it works poorly on tasks that require reasoning abilities, and often does not improve substantially with increasing language model scale (Rae et al., 2021).
In this paper, we combine the strengths of these two ideas in a way that avoids their limitations. Speciﬁcally, we explore the ability of language models to perform few-shot prompting for reasoning tasks, given a prompt that consists of triples: (cid:104) input, chain of thought , output (cid:105) . A chain of thought is a series of intermediate natural language reasoning steps that lead to the ﬁnal output, and we refer to this approach as chain-of-thought prompting .
An example prompt is shown in Figure 1.
We present empirical evaluations on arithmetic, commonsense, and symbolic reasoning benchmarks, showing that chain-of-thought prompting outperforms standard prompting, sometimes to a striking degree.
Figure 2 illustrates one such result—on the GSM8K benchmark of math word problems (Cobbe et al., 2021), chain-of-thought prompting with PaLM 540B outperforms standard prompting by a large margin and achieves new state-of-the-art performance.
A prompting only approach is important because it does not require a large training dataset and because a single model checkpoint can perform many tasks without loss of generality.
This work underscores how large language models can learn via a few examples with natural language data about the task (c.f. automatically learning the patterns underlying inputs and outputs via a large training dataset).
Chain-of-Thought Prompting
Consider one’s own thought process when solving a complicated reasoning task such as a multi-step math word problem.
It is typical to decompose the problem into intermediate steps and solve each before giving the ﬁnal answer: “After Jane gives 2 ﬂowers to her mom she has 10 . . .
then after she gives 3 to her dad she will have 7 . . .
so the answer is 7.”
The goal of this paper is to endow language models with the ability to generate a similar chain of thought —a coherent series of intermediate reasoning steps that lead to the ﬁnal answer for a problem.
We will show that sufﬁciently large language models can generate chains of thought if demonstrations of chain-of-thought reasoning are provided in the exemplars for few-shot prompting.
Figure 1 shows an example of a model producing a chain of thought to solve a math word problem that it would have otherwise gotten incorrect.
The chain of thought in this case resembles a solution and can interpreted as one, but we still opt to call it a chain of thought to better capture the idea that it mimics a step-by-step thought process for arriving at the answer (and also, solutions/explanations typically come after the ﬁnal answer (Narang et al., 2020; Wiegreffe et al., 2022; Lampinen et al., 2022, inter alia )).
Chain-of-thought prompting has several attractive properties as an approach for facilitating reasoning in language models.
First, chain of thought, in principle, allows models to decompose multi-step problems into intermediate steps, which means that additional computation can be allocated to problems that require more reasoning steps.
Second, a chain of thought provides an interpretable window into the behavior of the model, suggesting how it might have arrived at a particular answer and providing opportunities to debug where the reasoning path went wrong (although fully characterizing a model’s computations that support an answer remains an open question).
Third, chain-of-thought reasoning can be used for tasks such as math word problems, commonsense reasoning, and symbolic manipulation, and is potentially applicable (at least in principle) to any task that humans can solve via language.
Finally, chain-of-thought reasoning can be readily elicited in sufﬁciently large off-the-shelf language models simply by including examples of chain of thought sequences into the exemplars of few-shot prompting.
In empirical experiments, we will observe the utility of chain-of-thought prompting for arithmetic reasoning (Section 3), commonsense reasoning (Section 4), and symbolic reasoning (Section 5).
Arithmetic Reasoning
We begin by considering math word problems of the form in Figure 1, which measure the arithmetic reasoning ability of language models.
Though simple for humans, arithmetic reasoning is a task where language models often struggle (Hendrycks et al., 2021;
Patel et al., 2021, inter alia ).
Strikingly, chainof-thought prompting when used with the 540B parameter language model performs comparably with task-speciﬁc ﬁnetuned models on several tasks, even achieving new state of the art on the challenging
GSM8K benchmark (Cobbe et al., 2021).
3.1 Experimental Setup
We explore chain-of-thought prompting for various language models on multiple benchmarks.
Benchmarks.
We consider the following ﬁve math word problem benchmarks: (1) the GSM8K benchmark of math word problems (Cobbe et al., 2021), (2) the SVAMP dataset of math word problems with varying structures (Patel et al., 2021), (3) the ASDiv dataset of diverse math word problems (Miao et al., 2020), (4) the AQuA dataset of algebraic word problems, and (5) the MAWPS benchmark (Koncel-Kedziorski et al., 2016).
Example problems are given in Appendix Table 12.
Standard prompting.
For the baseline, we consider standard few-shot prompting, popularized by
Brown et al.
(2020), in which a language model is given in-context exemplars of input–output pairs before outputting a prediction for a test-time example.
Exemplars are formatted as questions and answers.
The model gives the answer directly, as shown in Figure 1 (left).
Chain-of-thought prompting.
Our proposed approach is to augment each exemplar in few-shot prompting with a chain of thought for an associated answer, as illustrated in Figure 1 (right).
As most of the datasets only have an evaluation split, we manually composed a set of eight few-shot exemplars with chains of thought for prompting—Figure 1 (right) shows one chain of thought exemplar, and the full set of exemplars is given in Appendix Table 20.
(These particular exemplars did not undergo prompt engineering; robustness is studied in Section 3.4 and Appendix A.2.)
To investigate whether chain-of-thought prompting in this form can successfully elicit successful reasoning across a range of [Table] Math Word Problems (free response) Math Word Problems (multiple choice)
CSQA [Table] commonsense [Table]
StrategyQA
Date [Table]
[Figure]
math word problems, we used this single set of eight chain of thought exemplars for all benchmarks except AQuA, which is multiple choice instead of free response.
For AQuA, we used four exemplars and solutions from the training set, as given in Appendix Table 21.
Language models.
We evaluate ﬁve large language models.
The ﬁrst is GPT-3 (Brown et al., 2020), for which we use text-ada-001, text-babbage-001, text-curie-001, and text-davinci-002, which presumably correspond to InstructGPT models of 350M, 1.3B, 6.7B, and 175B parameters (Ouyang et al.,
2022).The second is LaMDA (Thoppilan et al., 2022), which has models of 422M, 2B, 8B, 68B, and 137B parameters.
The third is PaLM , which has models of 8B, 62B, and 540B parameters.
The fourth is UL2 20B (Tay et al., 2022), and the ﬁfth is Codex (Chen et al., 2021, code-davinciin the OpenAI API).
We sample from the models via greedy decoding (though follow-up work shows chain-of-thought prompting can be improved by taking the majority ﬁnal answer over many sampled generations (Wang et al., 2022a)).
For LaMDA, we report averaged results over ﬁve random seeds, where each seed had a different randomly shufﬂed order of exemplars.
As LaMDA experiments did not show large variance among different seeds, to save compute we report results for a single exemplar order for all other models.
3.2 Results
The strongest results of chain-of-thought prompting are summarized in Figure 4, with all experimental outputs for each model collection, model size, and benchmark shown in Table 2 in the Appendix.
There are three key takeaways.
First, Figure 4 shows that chain-of-thought prompting is an emergent ability of model scale (Wei et al., 2022b).
That is, chain-of-thought prompting does not positively impact performance for small models, and only yields performance gains when used with models of ∼ 100B parameters.
We qualitatively found that models of smaller scale produced ﬂuent but illogical chains of thought, leading to lower performance than standard prompting.
[Figure]
Second, chain-of-thought prompting has larger performance gains for more-complicated problems.
For instance, for GSM8K (the dataset with the lowest baseline performance), performance more than doubled for the largest GPT and PaLM models.
On the other hand, for SingleOp, the easiest subset of MAWPS which only requires a single step to solve, performance improvements were either negative or very small (see Appendix Table 3).
Third, chain-of-thought prompting via GPT175B and PaLM 540B compares favorably to prior state of the art, which typically ﬁnetunes a task-speciﬁc model on a labeled training dataset.
Figure 4 [-showshow LM-
-]shows how PaLM 540B uses chain-ofthought prompting to achieve new state of the art on GSM8K, SVAMP, and MAWPS (though note that standard prompting already passed the prior best for SVAMP).
On the other two datasets,
AQuA and ASDiv, PaLM with chain-of-thought prompting reaches within 2% of the state of the art (Appendix Table 2).
To better understand why chain-of-thought prompting works, we manually examined modelgenerated chains of thought by LaMDA 137B for GSM8K. Of 50 random examples where the model returned the correct ﬁnal answer, all of the generated chains of thought were also logically and mathematically correct except two that coincidentally arrived at the correct answer (see Appendix D.1, and Table 8 for examples of correct model-generated chains of thought).
We also randomly examined 50 random samples for which the model gave the wrong answer.
The summary of this analysis is that 46% of the chains of thought were almost correct, barring minor mistakes (calculator error, symbol mapping error, or one reasoning step missing), and that the other 54% of the chains of thought had major errors in semantic understanding or coherence (see Appendix D.2).
To provide a small insight into why scaling improves chain-of-thought reasoning ability, we performed a similar analysis of errors made by PaLM 62B and whether those errors were ﬁxed by scaling to PaLM 540B. The summary is that scaling PaLM to 540B ﬁxes a large portion of one-step missing and semantic understanding errors in the 62B model (see Appendix A.1).
3.3 Ablation Study
The observed beneﬁts of using chain-of-thought prompting raises the natural question of whether the same performance improvements can be conferred via other types of prompting.
Figure 5 shows an ablation study with three variations of chain of thought described below.
Equation only.
One reason for why chain-of-thought prompting might help is that it produces the mathematical equation to be evaluated, and so we test a variation where the model is prompted to output only a mathematical equation before giving the answer.
Figure 5 shows that equation only prompting does not help much for GSM8K, which implies that the semantics of the questions in GSM8K are too challenging to directly translate into an equation without the natural language reasoning steps in chain of thought.
For datasets of one-step or two-step problems, however, we ﬁnd that equation only prompting does improve performance, since the equation can be easily derived from the question (see Appendix Table 6).
[Figure]
Results for other datasets are given in Appendix Table 6 and Table 7.
Variable compute only.
Another intuition is that chain of thought allows the model to spend more computation (i.e., intermediate tokens) on harder problems.
To isolate the effect of variable computation from chain-of-thought reasoning, we test a conﬁguration where the model is prompted to output a only sequence of dots ( . . . )
equal to the number of characters in the equation needed to solve the problem.
This variant performs about the same as the baseline, which suggests that variable computation by itself is not the reason for the success of chainof-thought prompting, and that there appears to be utility from expressing intermediate steps via natural language.
Chain of thought after answer.
Another potential beneﬁt of chain-of-thought prompting could simply be that such prompts allow the model to better access relevant knowledge acquired during pretraining.
Therefore, we test an alternative conﬁguration where the chain of thought prompt is only given after the answer, isolating whether the model actually depends on the produced chain of thought to give the ﬁnal answer.
This variant performs about the same as the baseline, which suggests that the sequential reasoning embodied in the chain of thought is useful for reasons beyond just activating knowledge.
3.4 Robustness of Chain of Thought
[Figure]
[Figure]
-thought prompting has variance for different prompt examples (as expected) but outperforms standard prompting for various annotators as well as for different exemplars.
Sensitivity to exemplars is a key consideration of prompting approaches—for instance, varying the permutation of few-shot exemplars can cause the accuracy of GPT-3 on
SST-2 to range from near chance (54.3%) to near state of the art (93.4%) (Zhao et al., 2021).
In this ﬁnal subsection, we evaluate robustness to chains of thought written by different annotators.
In addition to the results above, which used chains of thought written by an Annotator
A, two other co-authors of this paper (Annotators B and
C) independently wrote chains of thought for the same few-shot exemplars (shown in Appendix H).
Annotator A also wrote another chain of thought that was more concise than the original, following the style of solutions given in
Cobbe et al.
Figure 6 shows these results for LaMDA 137B on GSM8K and MAWPS (ablation results for other datasets are given in Appendix Table 6 / Table 7).
Although there is variance among different chain of thought annotations, as would be expected when using exemplar-based prompting (Le Scao and Rush, 2021; Reynolds and McDonell, 2021; Zhao et al., 2021), all sets of chain of thought prompts outperform the standard baseline by a large margin.
This result implies that successful use of chain of thought does not depend on a particular linguistic style.
To conﬁrm that successful chain-of-thought prompting works for other sets of exemplars, we also run experiments with three sets of eight exemplars randomly sampled from the GSM8K training set, an independent source (examples in this dataset already included reasoning steps like a chain of thought).
2 Figure 6 shows that these prompts performed comparably with our manually written exemplars, also substantially outperforming standard prompting.
In addition to robustness to annotators, independently-written chains of thought, different exemplars, and various language models, we also ﬁnd that chain-of-thought prompting for arithmetic reasoning is robust to different exemplar orders and varying numbers of exemplars (see Appendix A.2).
Commonsense Reasoning
Although chain of thought is particularly suitable for math word problems, the language-based nature of chain of thought actually makes it applicable to a broad class of commonsense reasoning problems, which involve reasoning about physical and human interactions under the presumption of general background knowledge.
Commonsense reasoning is key for interacting with the world and is still beyond the reach of current natural language understanding systems (Talmor et al., 2021).
Benchmarks.
We consider ﬁve datasets covering a diverse range of commonsense reasoning types.
The popular CSQA (Talmor et al., 2019) asks commonsense questions about the world involving complex semantics that often require prior knowledge.
StrategyQA
(Geva et al., 2021) requires models to infer a multi-hop strategy to answer questions.
We choose two specialized evaluation sets from the BIG-bench effort (BIG-bench collaboration, 2021): Date Understanding, which involves inferring a date from a given context, and Sports Understanding, which involves determining whether a sentence relating to sports is plausible or implausible.
Finally, the SayCan dataset (Ahn et al., 2022) involves mapping a natural language instruction to a sequence of robot actions from a discrete set.
Figure 3 shows examples with chain of thought annotations for all datasets.
Prompts.
We follow the same experimental setup as the prior section.
For CSQA and StrategyQA, we randomly selected examples from the training set and manually composed chains of thought for them to use as few-shot exemplars.
The two BIG-bench tasks do not have training sets, so we selected the ﬁrst ten examples as exemplars in the evaluation set as few-shot exemplars and report numbers on the rest of the evaluation set.
For SayCan, we use six examples from the training set used in Ahn et al. (2022) and also manually composed chains of thought.
Results.
Figure 7 highlights these results for PaLM (full results for LaMDA, GPT-3, and different model scales are shown in Table 4).
For all tasks, scaling up model size improved the performance of standard prompting; chain-of-thought prompting led to further gains, with improvements appearing to be largest for PaLM 540B. With chain-of-thought prompting, [-LM-
-]PaLM 540B achieved strong performance relative to baselines, outperforming the prior state of the art on StrategyQA (75.6% vs 69.4%) and outperforming an unaided sports enthusiast on sports understanding (95.4% vs 84%).
These results demonstrate that chain-of-thought prompting can also improve performance on tasks requiring a range of commonsense reasoning abilities (though note that gain was minimal on CSQA).
[Figure]
Symbolic Reasoning
[Figure]
Our ﬁnal experimental evaluation considers symbolic reasoning, which is simple for humans but potentially challenging for language models.
We show that chain-ofthought prompting not only enables language models to perform symbolic reasoning tasks that are challenging in the standard prompting setting, but also facilitates length generalization to inference-time inputs longer than those seen in the few-shot exemplars.
Tasks.
We use the following two toy tasks.
• Last letter concatenation.
This task asks the model to concatenate the last letters of words in a name (e.g., “Amy Brown” → “yn” ).
It is a more challenging version of ﬁrst letter concatenation, which language models can already perform without chain of thought.
3 We generate full names by randomly concatenating names from the top one-thousand ﬁrst and last names from name census data ( https://namecensus.com/ ).
• Coin ﬂip.
This task asks the model to answer whether a coin is still heads up after people either ﬂip or don’t ﬂip the coin (e.g., “A coin is heads up.
Phoebe ﬂips the coin.
Osvaldo does not ﬂip the coin.
Is the coin still heads up?”
→ “no” ).
As the construction of these symbolic reasoning tasks is well-deﬁned, for each task we consider an in-domain test set for which examples had the same number of steps as the training/few-shot exemplars, as well as an out-of-domain (OOD) test set, for which evaluation examples had more steps than those in the exemplars.
For last letter concatenation, the model only sees exemplars of names with two words, and then performs last letter concatenation on names withand 4 words.
4 We do the same for the number of potential ﬂips in the coin ﬂip task.
Our experimental setup uses the same methods and models as in the prior two sections.
We again manually compose chains of thought for the few-shot exemplars for each task, which are given in Figure 3.
Results.
The results of these in-domain and OOD evaluations are shown in Figure 8 for PaLM, with results for LaMDA shown in Appendix Table 5.
With PaLM 540B, chain-of-thought prompting leads to almost 100% solve rates (note that standard prompting already solves coin ﬂip with PaLM 540, though not for LaMDA 137B).
Note that these in-domain evaluations are “toy tasks” in the sense that perfect solution structures are already provided by the chains of thought in the few-shot exemplars; all the model has to do is repeat the same steps with the new symbols in the test-time example.
And yet, small models still fail—the ability to perform abstract manipulations on unseen symbols for these three tasks only arises at the scale of 100B model parameters.
As for the OOD evaluations, standard prompting fails for both tasks.
With chain-of-thought prompting, language models achieve upward scaling curves (though performance is lower than in the in-domain setting).
Hence, chain-of-thought prompting facilitates length generalization beyond seen chains of thought for language models of sufﬁcient scale.
Discussion
We have explored chain-of-thought prompting as a simple mechanism for eliciting multi-step reasoning behavior in large language models.
We ﬁrst saw that chain-of-thought prompting improves performance by a large margin on arithmetic reasoning, yielding improvements that are much stronger than ablations and robust to different annotators, exemplars, and language models (Section 3).
Next, experiments on commonsense reasoning underscored how the linguistic nature of chain-of-thought reasoning makes it generally applicable (Section 4).
Finally, we showed that for symbolic reasoning, chain-of-thought prompting facilitates OOD generalization to longer sequence lengths (Section 5).
In all experiments, chain-of-thought reasoning is elicited simply by prompting an off-the-shelf language model.
No language models were ﬁnetuned in the process of writing this paper.
The emergence of chain-of-thought reasoning as a result of model scale has been a prevailing theme (Wei et al., 2022b).
For many reasoning tasks where standard prompting has a ﬂat scaling curve, chainof-thought prompting leads to dramatically increasing scaling curves.
Chain-of-thought prompting appears to expand the set of tasks that large language models can perform successfully—in other words, our work underscores that standard prompting only provides a lower bound on the capabilities of large language models.
This observation likely raises more questions than it answers—for instance, how much more can we expect reasoning ability to improve with a further increase in model scale?
What other prompting methods might expand the range of tasks that language models can solve?
As for limitations, we ﬁrst qualify that although chain of thought emulates the thought processes of human reasoners, this does not answer whether the neural network is actually “reasoning,” which we leave as an open question.
Second, although the cost of manually augmenting exemplars with chains of thought is minimal in the few-shot setting, such annotation costs could be prohibitive for ﬁnetuning (though this could potentially be surmounted with synthetic data generation, or zero-shot generalization).
Third, there is no guarantee of correct reasoning paths, which can lead to both correct and incorrect answers; improving factual generations of language models is an open direction for future work (Rashkin et al., 2021; Ye and Durrett, 2022; Wiegreffe et al., 2022, inter alia ).
Finally, the emergence of chain-of-thought reasoning only at large model scales makes it costly to serve in real-world applications; further research could explore how to induce reasoning in smaller models.
Related Work
This work is inspired by many research areas, which we detail in an extended related work section (Appendix C).
Here we describe two directions and associated papers that are perhaps most relevant.
The ﬁrst relevant direction is using intermediate steps to solve reasoning problems.
Ling et al.
(2017) pioneer the idea of using natural language rationales to solve math word problems through a series of intermediate steps. Their work is a remarkable contrast to the literature using formal languages to reason (Roy et al., 2015; Chiang and Chen, 2019; Amini et al., 2019; Chen et al., 2019).
Cobbe et al. (2021) extend Ling et al.
by creating a larger dataset and using it to ﬁnetune a pretrained language model rather than training a model from scratch. In the domain of program synthesis,
Nye et al. (2021) leverage language models to predict the ﬁnal outputs of Python programs via ﬁrst line-to-line predicting the intermediate computational results, and show that their step-by-step prediction method performs better than directly predicting the ﬁnal outputs.
Naturally, this paper also relates closely to the large body of recent work on prompting.
Since the popularization of few-shot prompting as given by Brown et al.
(2020), several general approaches have improved the prompting ability of models, such as automatically learning prompts (Lester et al., 2021) or giving models instructions describing a task (Wei et al., 2022a; Sanh et al., 2022; Ouyang et al., 2022).
Whereas these approaches improve or augment the input part of the prompt (e.g., instructions that are prepended to inputs), our work takes the orthogonal direction of augmenting the outputs of language models with a chain of thought.
Conclusions
We have explored chain-of-thought prompting as a simple and broadly applicable method for enhancing reasoning in language models.
Through experiments on arithmetic, symbolic, and commonsense reasoning, we ﬁnd that chain-of-thought reasoning is an emergent property of model scale that allows sufﬁciently large language models to perform reasoning tasks that otherwise have ﬂat scaling curves.
Broadening the range of reasoning tasks that language models can perform will hopefully inspire further work on language-based approaches to reasoning. Acknowledgements
We thank Jacob Devlin, Claire Cui, Andrew Dai, and Ellie Pavlick for providing feedback on the paper.
We thank Jacob Austin, Yuhuai Wu, Henryk Michalewski, Aitor Lewkowycz, Charles Sutton, and Aakanksha Chowdhery for helpful discussions.
We thank Sid Maxwell for notifying us about a mistake in the manual error analysis in the original manuscript.
A Frequently Asked Questions A.1 Why does increasing model scale improve chain-of-thought prompting?
The ﬁnding that successful chain-of-thought reasoning predictably emerges only at certain model scales is intriguing.
Scaling up language models has been shown to confer beneﬁts such as improved performance and sample efﬁciency (Kaplan et al., 2020), but chain-of-thought reasoning is emergent in the sense that its success cannot be predicted only by extrapolating the performance of small scale models, as chain of thought actually hurts performance for most models smaller than 10B parameters.
The question of why model scale improves chain-of-thought prompting is certainly multi-faceted, and we made a preliminary attempt to shed insight into it via error analysis.
This small analysis involved manually reading 45 errors made by PaLM 62B and categorizing them into semantic understanding (20 errors), one step missing (18 errors), and other errors (7 errors).
The “other category” included hallucinations, repetitive outputs, and symbol mapping errors.
This categorization is a coarse one borrowed from the initial error analysis done on LaMDA in Appendix D.2, for which categories were conceived based on what improvements were needed to make the chain of thought correct.
As shown in Figure 9, scaling PaLM to 540B parameters ﬁxed a substantial portion of errors in all three categories.
Examples of semantic understanding and one-step missing errors that were ﬁxed by scaling PaLM to 540B are given in Figure 10.
This result appears consistent with a hypothesis that language models acquire a range of semantic understanding and logical reasoning skills as a function of model scale (though note that model scale is often conﬂated with other factors, such as amount of training compute).
[Figure]
repetitive outputs, and symbol mapping errors.
Scaling PaLM to 540B ﬁxed a substantial portion of errors in all categories.
There are also three notable points regarding why small language models fail.
The ﬁrst observation is that small language models fail at even relatively easy symbol mapping tasks.
As demonstrated in Section 5, for even symbolic reasoning tasks that only require generalization to new examples using the same chain of thought logical structure that was given in the few-shot exemplars, small language models still failed.
The second observation is that small language models seem to have inherently weaker arithmetic abilities, as shown by Brown et al.
, the ability to do simple arithmetic operations (without semantic understanding) requires sufﬁcient model scale.
Finally, we noticed qualitatively that small language models often did not generate a ﬁnal answer that could be parsed, due to either repetitions or logic that never arrived at a ﬁnal answer.
In summary, the success of chain-of-thought reasoning as a result of model scale is a complicated phenomena that likely involves a variety of emergent abilities (semantic understanding, symbol mapping, staying on topic, arithmetic ability, faithfulness, etc).
Future work could more thoroughly investigate what properties of pretraining data, model architecture, and optimization objective causally enable such reasoning capabilities.
[Table]
bill was $40.00.
The delivery vendor added 25% to the bill.
This means the bill was increased by 25% of $40.00.
This is 0.25 * $40.00 = $10.00.
The delivery fee was $3.00.
The tip was $4.00.
So the total bill was $40.00 + $10.00 + $3.00 + 540b model $4.00 = $57.00.
The answer is 57.
(Correct)
There are four schools competing at a basketball tournament.
Each school has sent a girls’ basketball team and a boys’ basketball team and each team hasplayers each.
Each school has also sent a coach for each team.
In total, how many people have all of the schools sent?
There are 4 schools.
Each school has sent a girls’ basketball team and a boys’ basketball team.
Each team has 5 players each.
That means there are 4 * 2 * 5 = 40 players.
Each team has a coach.
That means there are 40 + 4 = 44 coaches.
The answer is 44.
(Incorrect: one step missing error)
Each school has sent 2 teams, each with 5 players.
This means each school has sent 2 * 5 = 10 players.
Each school has also sentcoaches.
This means each school has sent 10 + 2 = 12 people.
There are 4 schools, so in total all of the schools have sent 4 * 12 =people.
The answer is 48.
(Correct) 62B
[Figure] Output 62B [Figure]
Output 62B Model Output 540B Model Output 540B Model Output 540B Model Output
Question
Question Question
A.2 What is the role of prompt engineering?
One of the key considerations of prompting is sensitivity to the exact prompt.
There is no shortage of work showing that prompts affect language models in unexpected ways (Min et al., 2022).
The general way that we created chain of thought annotations was by taking eight exemplars from the training set and decomposing the reasoning process into multiple steps leading to the ﬁnal answer.
Examples of chain of thought annotations are provided in Figure 3, with full prompts
given in Appendix G. To analyze how sensitive chain of thought is to prompt engineering, we performed robustness experiments with respect to various factors. • Different annotators.
We ﬁrst analyze robustness to three different annotators (Section 3.4 and Figure 6).
Although there is notable variance in performance (which we will discuss later), chain of thought performed better than the baseline by a large margin for all three annotators on eight datasets in arithmetic, commonsense, and symbolic reasoning (Table 6 and Table 7).
Similar to the annotation process in Cobbe et al.
(2021), annotators were not given speciﬁc instructions about
how to write the chain of thought annotations other than to simply write the step-by-step reasoning process that led to the ﬁnal answer.
Thus, the annotations were written in each annotator’s own linguistic “chain of thought” writing style.
• Annotators without machine learning background.
The GSM8K dataset (Cobbe et al., 2021) conveniently provides a training set with reasoning chains written by crowd compute workers, which enables us to investigate whether chain of thought still works with reasoning chains from an independent source without a background in machine learning.
So we randomly sampled three sets of eight exemplars with chains of thought from GSM8K. These chain of thought annotations also outperformed the baseline by a large margin for all four arithmetic datasets (Table 6), indicating that chain of thought is not dependent on a particular set of annotators.
• Different exemplars.
The different GSM8K exemplars experiment above (Table 6) also shows that chain-of-thought prompting works for different sets of exemplars.
Notably, we test every set of exemplars on all four arithmetic datasets (instead of picking exemplars from the training set for each dataset), which suggests that the exemplars do not necessarily have to come from the same dataset distribution as the test examples.
• Different order of exemplars.
Prior work has shown that in some cases (e.g., classiﬁcation) even the order of prompts matter—varying the permutation of few-shot exemplars can cause the accuracy of GPT-3 on SST-2 to range from near chance (54.3%) to near SOTA (93.4%)
(Zhao et al., 2021).
We show the standard deviation of performance from different exemplars in Tableand Table 7.
Standard deviations with respect to prompt order are relatively minimal in almost all cases.
The one exception is the coin ﬂip task, for which exemplar orders have high standard deviation, likely for the reason cited in Zhao et al.
(2021)—for classiﬁcation, many exemplars of the same category in a row biases the model outputs).
• Different number of exemplars.
We also found that gains from chain-of-thought prompting generally still held when there was a varying number of few-shot exemplars.
This is shown for ﬁve datasets in Figure 11 (we did not have the compute to run this for all datasets).
We also found in preliminary experiments that further increasing the number of exemplars in standard prompting did not lead to signiﬁcant gains (e.g., increasing from 8 to 16 exemplars did not improve the performance of standard prompting enough to catch up with chain-of-thought prompting).
• Different language models.
Another interesting question is whether certain prompts that work better for one model work better for other large language models.
We ﬁnd that with the same prompts, chain-of-thought prompting improves performance across all three models (LaMDA, GPT-3, and PaLM) for all datasets except CSQA and StrategyQA for GPT-
(Table 1, Table 4, Table 5).
The fact that gains from chain of thought did not transfer perfectly among models is a limitation; further work could investigate why how different pre-training datasets and model architectures affect the performance gain from chain-of-thought prompting.
Prompt engineering still matters, though.
Although the results are relatively robust to the prompt for arithmetic reasoning, we want to be clear that prompt engineering still does matter, and can improve performance signiﬁcantly in many cases.
Though most chain of thought annotations outperform standard prompting, there is large variation in many cases.
For instance, for the coin ﬂip task, the performance varied from 99.6% for Annotator A to 71.4% for Annotator C, though both were above standard prompting = 50.0%
(see Table 7).
There are even tasks where prompt engineering is a requirement for good performance.
In preliminary experiments, we tried using chain of thought to enable language models to reverse the order of a list of 5 items.
While two co-authors were not able to write chain of thought prompts that solved the task despite their best attempts, a third co-author was able to write a chain of thought that perfectly solved the task.
How to generate chain of thought annotations in a robust fashion could be an interesting direction for future work.
For instance, an idea here could be to use a large language model to automatically generate chains of thought via prompting (and potentially optimize this over a validation set)
. A.3 Will chain-of-thought prompting improve performance for my task of interest?
While chain-of-thought prompting is in principle applicable for any text-to-text task, it is more helpful for some tasks than others.
Based on the experiments in this paper, our intuition is that chain of thought helps the most when three conditions are met: (1) the task is challenging and requires multi-step reasoning, (2) a large language model is used, and (3) the scaling curve is relatively ﬂat.
Conversely, the beneﬁts are smaller when one or more of these conditions are not met.
These intuitions are perhaps supported by the arithmetic reasoning results.
The performance gain from chain-of-thought prompting is largest for PaLM 540B on GSM8K (challenging multi-step problems, ﬂat scaling curve), which meets these conditions.
The performance gain is small for the subsets of MAWPS that only require one or two steps (SingleOP, SingleEq, and AddSub), for which
[-LM-
-]PaLM 540B already achieves performance of 90% or higher (and it is also generally true that there is less headroom for improvement when performance is already strong).
Although in this paper we focused on multi-step reasoning tasks (arithmetic, commonsense, and symbolic), chain-of-thought prompting can potentially be applied to any task for which humans use a “chain of thought” to solve (at least in principle).
We leave the empirical evaluation of chain-of-thought prompting on such diverse tasks (e.g., machine translation, etc.) to future work.
A.4 Why is prompting with the equation only not enough for some arithmetic reasoning datasets?
Prompting with the equation only as an intermediate step does help on many datasets, especially when the datasets only require a few reasoning steps (SVAMP, ASDiv, MAWPS).
For GSM8K, however, using the equation only did not improve performance substantially.
Based on qualitative analysis, we believe that these questions are too semantically challenging for the model to directly translate them into a math equation.
Consider this example from LaMDA 137B:
Q UESTION : Mike plays ping pong for 40 minutes. In the ﬁrst 20 minutes, he scorespoints. In the second
minutes, he scores 25% more points. How many total points did
he score?
E QUATION ONLY ( WRONG ANSWER )
(4 + 20 * 0.25) = 6.
The answer is C HAIN OF THOUGHT ( CORRECT ): Mike played ping pong for 40 minutes.
In the ﬁrst 20 minutes, he scored 4 points. In the second 20 minutes, he scored more points
[Table]
It is hard for the model to directly translate all of the semantics into a single equation, but chain of thought allows it to better reason about each part of the question via intermediate steps in natural language.
B All Experimental Results
This section contains tables for experimental results for varying models and model sizes, on all benchmarks, for standard prompting vs. chain-of-thought prompting.
For the arithmetic reasoning benchmarks, some chains of thought (along with the equations produced) were correct, except the model performed an arithmetic operation incorrectly.
A similar observation was made in Cobbe et al.
Hence, we can further add a Python program as an external calculator (using the Python eval function) to all the equations in the generated chain of thought.
When there are multiple equations in a chain of thought
[Table]
we propagate the external calculator results from one equation to the following equations via string matching.
As shown in Table
[Table]
we see that adding a calculator signiﬁcantly boosts performance of chain-of-thought prompting on most tasks.
[Table]
[-(+7.8) 37.5 (+8.0) 46.
-][Table]
[Table] [Table] [Table]
[Table]
GPT-3 175B
Standard 15.
[Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
Codex
[-Standard 19. (code-davinci-002) Chain of thought 63.1 (+43.4) 76.4 (+6.5) 80.4 (+6.4) 45.3 (+15.8) 92.
-][Table]
[Table] [Table] [Table]
[-LM-
-]PaLM 540B
[Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table]
6.7B 13.
175B 90.9 88.
[Table]
[-LM-
-][Table]
8B 41.8 46.
62B 87.9 85.
540B 94.1 94.
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
62B
[Table]
[Table]
[Table] [Table]
540B
[Table]
[Table]
· equation only
[Table]
· variable compute only
[Table]
different annotator [Table] B)
[Table]
different annotator (C)
[Table]
intentionally concise style
[Table]
· exemplars from GSM8K ( α )
[Table]
Chain of thought generally outperforms ablations by a large amount.
Chain of thought prompting has variance (as expected) when used with prompts written by different annotators or when using other exemplars, but still outperforms standard prompting by a large margin.
Standard deviation shown is for different order of few-shot prompting exemplars, with ﬁve different random seeds.
Results here are shown for LaMDA 137B, as additional queries for GPT-3 and PaLM are both limited and expensive.
The exception is that we run SayCan using PaLM here, as the SayCan evaluation set is only 120 examples and therefore less expensive to run multiple times. [Table]
[Table]
C Extended Related Work
Chain-of-thought prompting is a general approach that is inspired by several prior directions: prompting, natural language explanations, program synthesis/execution, numeric and logical reasoning, and intermediate language steps.
C.1 Prompting
The recent success of large-scale language models has led to growing interest in improving their capability to perform tasks via prompting (Brown et al.
(2020), and see Liu et al.
(2021) for a survey).
This paper falls in the category of general prompting approaches, whereby input prompts are optimized to allow a single large language model to better perform a variety of tasks (Li and Liang, 2021; Lester et al., 2021; Reif et al., 2022, inter alia ).
One recent line of work aims to improve the ability of language models to perform a task by providing instructions that describe the task (Raffel et al., 2020;
Wei et al., 2022a; Ouyang et al., 2022; Sanh et al., 2022; Wang et al., 2022b).
This line of work is related because it also augments input–output pairs with meta-data.
But whereas an instruction augments the input to a task (instructions are typically prepended to the inputs), chain-of-thought prompting augments the outputs of language models.
Another related direction is sequentially combining the outputs of language models; human–computer interaction (HCI) work (Wu et al., 2022a,b) has shown that combining sequential generations of language models improves task outcomes in a 20-person user study.
C.2 Natural language explanations
Another closely related direction uses natural language explanations (NLEs), often with the goal of improving model interpretability (Zhou et al., 2020; Wiegreffe and Marasovi´c, 2021, inter alia ).
That line of work typically focuses on natural language inference (Camburu et al., 2018;
Yordanov et al., 2021; Bostrom et al., 2021), and produces explanations either simultaneously to or after the ﬁnal prediction (Narang et al., 2020; Majumder et al., 2021; Wiegreffe et al., 2021, 2022).
By contrast, the chain of thought processing considered in this paper occurs before the ﬁnal answer.
And while
NLE aims mostly to improve neural network interpretability (Rajagopal et al., 2021), the goal of chain-of-thought prompting is to allow models to decompose multi-hop reasoning tasks into multiple steps—interpretability is just a side effect.
Marasovi ´ c et al.
(2022) show that prompt-based ﬁnetuning with NLE improves NLI and classiﬁcation performance, though they largely focus on evaluating explanation plausibility.
In comparison, our work focuses on a range of arithmetic, commonsense, and symbolic tasks that require multi-hop reasoning.
C.3 Program synthesis and execution
Using intermediate reasoning steps has a long history in program synthesis and execution (Zaremba and Sutskever, 2014, inter alia ).
Recent work along in this direction has included a number of architectural innovations (Cai et al., 2017; Dong et al., 2019; Yan et al., 2020), as well as the use of large language models (Chen et al., 2021; Austin et al., 2021).
The program execution work closest to ours is perhaps Nye et al.
(2021), which show that large language models can perform up to 10-digit addition, evaluate polynomials, and execute python programs.
Whereas generating a program and then executing it can be viewed as a type of reasoning, our work generalizes such domain-speciﬁc primitives to natural language, which is open-domain and relevant to any text-to-text NLP task in principle.
C.4 Numeric and logical reasoning
Numeric and logical reasoning has been a long-studied task in machine learning and natural language processing (Lev et al., 2004, inter alia ).
Recent work has also aimed to inject numeric reasoning abilities in language models in various ways, such as augmenting BERT with a predeﬁned set of executable operations (Andor et al., 2019), including a graph neural network (Ran et al., 2019), and using specialized training procedures (Pi˛ekos et al., 2021).
Another line of work aims to enable language models to perform logical or formal reasoning, often by verablizing the rules in natural language formal rules using language (Clark et al., 2020; Saeed et al., 2021; Liang et al., 2021).
Perhaps the most-related work here is Recchia (2021), which shows that ﬁnetuning enables longhand module operations, which has previously been difﬁcult for performers.
Whereas work in this direction is often task-speciﬁc and uses ﬁnetuning, we show that chain-of-thought prompting works for a broad range of tasks without any ﬁnetuning.
C.5 Intermediate language steps
Extensive prior work has shown the beneﬁts of endowing neural networks with the ability to produce intermediate steps via training or ﬁnetuning confers various beneﬁts in a range of scenarios.
As examples, it has been shown that natural language intermediate steps can improve performance (Zaidan et al., 2007;
Yao et al., 2021; Hase and Bansal, 2022; Gu et al., 2022), improve robustness (Chen et al., 2022), speed up training (Hancock et al., 2018), mitigate bias (Dua et al., 2020), and even help in image and reinforcement learning settings (Andreas et al., 2018).
To endow models with the ability to produce intermediate steps, prior work typically ﬁnetunes models on either manually annotated training datasets (Camburu et al., 2018; Rajani et al., 2019, inter alia ) or generates synthetic datasets (Talmor et al., 2020; Zelikman et al., 2022).
Compared with these training or ﬁnetuning methods, our work shows that various natural language reasoning abilities can be elicited in off-theshelf language models of sufﬁcient scale simply via prompting.
This prompting setup is important because it allows for intermediate step reasoning without a large number of labeled annotations, and because a single model can perform a range of reasoning tasks without any gradient updates.
D Appendix: Additional Analysis D.1 Correct Chain of Thought Analysis
As mentioned in the main text, we analyze 50 chains of thought from LaMDA 137B that led to correct answers in the GSM8K dataset.
Of these 50, only one arrived at the correct answer through incorrect reasoning (shown in Table 9: “correct by chance”).
The other 49 had correct logic and math, with examples shown in Table 8.
Five had minor imperfections while maintaining coherent and understandable logic:
• One had underspeciﬁed statements (shown in Table 9: “correct but underspeciﬁed statements”).
• One made an unrelated but correct statement (shown in Table 9: “correct but unrelated statement”).
• Two had the correct logic but omitted speciﬁc steps in the equation, though it was still understandable (shown in Table 9: “correct but imperfect equation”). • One had the correct math but inverted the semantics (shown in Table 9: “correct but inverted semantics”)
[Table]
[Table] [Table]
Although we ﬁnd that chain-of-thought reasoning is mostly correct for math questions where the ﬁnal answer was correct, this is likely because it is hard to arrive at the correct ﬁnal answer by coincidence for free response questions.
For multiple choice or binary classiﬁcation problems, it was much more likely that models could arrive at the correct answer via an incorrect reasoning path (e
[Table]
g., all the commmonsense reasoning datasets we evaluate).
This is a limitation, and future work should perform an analysis of the factuality of such chains of thought.
[Table]
D.2 Incorrect Chain of Thought Analysis
We also manually analyze 50 randomly sampled outputs of the model that were incorrect on [-GSM8Kfor LM-
-]GSM8K for LaMDA
137B. There are many ways that a chain of thought can be incorrect, making the design of error categorization non-trivial.
We decided to categorize errors into what changes are needed to make the chain of thought correct, with the goal of elucidating how the model can be improved in the future.
We found that many chains of thought can be made correct with one of the following three classes of modiﬁcation.
• Calculator error only.
We found that 8% of the chains of thought were completely correct except for a calculator error—in other words, applying an external calculator to equations, as done in Cobbe et al.
(2021), would make the chain of thought correct.
An example of this type of error is shown in Table 10: “calculator error only”.
Indeed, the solve rate of chain-of-thought prompting on for LaMDA 137B GSM8K went up from 14.3% to 17.3% when we added a Python program as an external calculator, as shown in Table 2.
Also, 34% of the examples contained calculator errors in addition to other types of errors.
However, we perform the rest of the error categorization independently of calculator errors. • Symbol mapping error.
We next found that 16% percent of the chains of thought were correct except for what we call symbol mapping errors.
We deﬁne a symbol mapping error as when the chain of thought is correct except for the number symbols, and it could be made totally correct by modifying only the equations and not the words.
As one might argue that they could simply place the correct ﬁnal equation in any chain of thought, we constrain this category to chains of thought where the chain of thought can be modiﬁed to be a completely correct reasoning process (not just ﬁnal answer).
An example of this error category is shown in Table 10: “symbol mapping error”.
• One step missing error.
Our next category of error is chains of thought which were correct except that they were missing a single step.
In other words, these chains of thoughts could be rewritten to be correct by adding in an additional reasoning step that was missed by the model.
An example of this error category is shown in Table 10: “one step missing error”.
We found that 22% percent of the errors fell into this category.
[Table]
We found that the remaining chains of thought (27 of 50; 54%) would require substantial edits to make into a correct chain of thought.
Almost all cases here involved some error in semantic understanding (see Table 11: “semantic understanding error”), and 8 of the 27 also had incoherent chain of thoughts, meaning that some statements in the generated chain of thought did not follow from prior ones or violated basic world knowledge (see Table “incoherent chain of thought error”).
[Table]
Overall, there are no guarantees that the reasoning processes generated by large language models are coherent or factually correct, as underscored by the recent work evaluating the factuality of language model generations and explanations (Maynez et al., 2020; Rashkin et al., 2021; Ye and
Durrett, 2022; Marasovi ´ c et al., 2022; Wiegreffe et al., 2022).
Incorrect reasoning processes can lead to both incorrect ﬁnal answers as well as accidentally correct ﬁnal answers (with accidentally correct ﬁnal answers being more likely for tasks such as binary classiﬁcation as opposed to free response).
Improving the factuality of language model generations with respect to context and world knowledge is an important direction open problems in language model research and could also be expected to potentially improve multi-step reasoning abilities of language models.
One potential method for improving the quality of decoding could involve generating multiple reasoning paths and scoring each of them with a veriﬁer, though this requires training the veriﬁer (Cobbe et al., 2021;
Shen et al., 2021; Thoppilan et al., 2022).
D.3 Additional Robustness Analysis
As the experiments in the main paper use a ﬁxed number of few-shot exemplars (8; as constrained by the input length of 1024 tokens), we verify that the chain-of-thought prompting is robust to various numbers of few-shot exemplars.
We run experiments for LaMDA 137B, comparing chain-of-thought prompting with standard prompting for the ﬁve datasets where standard prompting had a mostly ﬂat scaling curve (the largest model did not achieve high performance).
As shown in Figure 11, the improvement of chain-of-thought prompting over standard prompting remains robust to varying the number of few-shot exemplars in the prompt.
[Figure]
Dataset
N
Example problem
GSM8K 1,319 Josh decides to try ﬂipping a house.
He buys a house for $80,000 and then puts in $50,000 in repairs.
This increased the value of the house by 150%.
How much proﬁt did he make?
SVAMP 1,000 Each pack of dvds costs 76 dollars.
If there is a discount of 25 dollars on each pack.
How much do you have to pay to buy each pack?
ASDiv 2,096 Ellen has six more balls than Marin.
Marin has nine balls.
How many balls does
Ellen have?
AQuA
A car is being driven, in a straight line and at a uniform speed, towards the base of a vertical tower.
The top of the tower is observed from the car and, in the process, it takes 10 minutes for the angle of elevation to change from 45 ◦ to 60 ◦ .
After how much more time will this car reach the base of the tower?
Answer
Choices: (a) 5 √ 3 + 1 (b) 6 √ 3 + √ 2 (c) 7 √ 3 - 1 (d) 8 √ 3 - 2 (e) None of these
MAWPS: SingleOp
If there are 7 bottle caps in a box and Linda puts 7 more bottle caps inside, how many bottle caps are in the box?
MAWPS: SingleEq
Benny bought a soft drink for
2 dollars and 5 candy bars.
He
spent
a total ofdollars.
How much did each candy bar cost?
MAWPS: AddSub
There were 6 roses in the vase.
Mary cut some roses from her ﬂower garden.
There are now 16 roses in the vase.
How many roses did she cut?
MAWPS: MultiArith
The school
cafeteria
ordered
red apples and 7 green apples for students lunches.
But, if only
students wanted fruit, how many extra
did
the cafeteria end up with?
E Additional Details Version Control
V4 → V5 .
Added Codex and UL2 results.
Small changes to writing and style of paper.
V3 → V4 .
Fixed typo in Figure 3 and added a couple citations.
V2 → V
Added GPT-3 results.
Added SVAMP and AQuA eval datasets for math.
Added SayCan eval for commonsense.
Added Extended Related Work section (Appendix C).
Added ablations for
Commonsense and Symbolic Reasoning (Table 7).
Added FAQ section (Appendix A).
Added raw results in Appendix B.
V1 → V2 .
Added PaLM results (V1 only had
LaMDA)
E.1 Reproducibility Statement
As our results make use of two sets of large language models that is not publicly available, we take the following actions to facilitate reproducibility.
First, we provide the exact input prompts for all tasks in Table 20–Table 27 in Appendix G (and emphasize that we do not perform any ﬁnetuning and only apply prompting to off-the-shelf language models).
Second, we conduct experiments using the publicly available GPT-3 API for four model scales text-ada-001, text-babbage-001, text-curie-001, text-davinci-002).
Finally, we make exact inputs, targets, and predictions for LaMDA 137B for each task available as a zip ﬁle in the supplementary material.
E.2 Computational Resources
For all three language models we evaluated, we did prompting-based inference only.
No ﬁnetuning was done for this paper.
For inference on LaMDA 137B we use TPU v3 (8x8 conﬁguration, 64 chips / 128 cores), and for inference on PaLM 540B we use TPU v4 (4x4x12 conﬁguration, 192 chips /cores).
GPT-3 experiments were done using the public API.
E.3 Dataset Details and Licenses
We list the details and licenses for all arithmetic and commonsense datasets used in this paper.
The symbolic reasoning datasets were created synthetically, as described in Section 4.
Arithmetic reasoning
• Math Word Problem Repository (Koncel-Kedziorski et al., 2016):
AddSub (Hosseini et al., 2014): https://www.cs.washington.edu/nlp/arithmetic ;
MultiArith (Roy and Roth, 2015), license: CC BY 4.0. • ASDiv (Miao et al., 2020): https://github.com/chaochun/nlu-asdiv-dataset .
• AQuA (Ling et al., 2017): https://github.com/deepmind/AQuA , license:
https:// github.com/deepmind/AQuA/blob/master/LICENSE . • GSM8K (Cobbe et al., 2021): https://github.com/openai/grade-school-math , MIT license: https://github.com/openai/grade-school-math/blob/master/ LICENSE .
• SVAMP (Patel et al., 2021): https://github.com/arkilpatel/SVAMP , MIT license: https://github.com/arkilpatel/SVAMP/blob/main/LICENSE .
Commonsense reasoning
• CSQA (Talmor et al., 2019): https://www.tau-nlp.org/commonsenseqa , https:// github.com/jonathanherzig/commonsenseqa .
• StrategyQA (Geva et al., 2021): we use the open-domain setting (question-only set) from BIG-bench collaboration (2021): https://github.com/google/BIG-bench/ tree/main/bigbench/benchmark_tasks/strategyqa .
The original dataset is from https://github.com/eladsegal/strategyqa , MIT license: https://github.com/ eladsegal/strategyqa/blob/main/LICENSE .
• Date understanding and sports understanding from BIG-Bench (BIG-bench collaboration, 2021): Apache License v.2: https://github.com/google/BIG-bench/blob/main/ LICENSE .
SayCan (Ahn et al., 2022): SayCan dataset can be accessed at https://say-can.github.
[Table] [Table]
[Table] [Table]
[Table] [Table]
So the answer is yes.
(cid:55) (Ground truth is “no”, being out at home is part of baseball.)
[Table]
P ROMPT FOR M ATH W ORD P ROBLEMS
[Table]
they
[Table]
[Table]
Shawn has ﬁve toys.
For Christmas, he got two toys each from his
[Table]
and dad.
[Table]
many toys does he
[Table]
now?
[Table]
started with 5 toys.
If he got 2 toys each from his mom and dad, then that is 4 more toys.
5 + [Table]
[Table]
: There were nine computers in the server room.
Five more computers were installed each day, from monday to thursday.
How many computers are now in the server room?
A:
There were originally 9 computers.
For each of 4 days, 5 more computers were added.
So 5 * 4 =computers were added.
9 + 20 is 29.
The answer is 29.
Q: Michael had 58 golf balls.
On tuesday, he lost 23 golf balls.
On wednesday, he lost 2 more.
How many golf balls did he have at the end of wednesday?
A: Michael started with 58 golf balls.
After losing 23 on tuesday, he had 58 - 23 = 35. After losing 2 more, he had 35 - 2 = 33 golf balls. The answer is 33. [Table]
: Olivia has $23.
She bought ﬁve bagels for $3 each.
How much money does she have left?
[Table]
: Olivia had 23 dollars.
5 bagels for 3 dollars each will be 5 x 3 = 15 dollars.
So she has 23 - 15 dollars left.
[Table]
15 is 8.
The answer is 8.
P ROMPT FOR AQ U A ALGEBRAIC WORD PROBLEMS
[Table]
P ROMPT FOR L AST L ETTER C ONCATENATION
[Table]
" and concatenate them.
A: The last letter of "Elon" is "n".
The last letter of "Musk" is "k".
Concatenating them is "nk".
The answer is nk.
Q: Take the last letters of the words in "Larry Page" and concatenate them.
A: The last letter of "Larry" is "y".
The last letter of "Page" is "e".
Concatenating them is "ye".
The answer is ye.
Q: Take the last letters of the words in "Sergey Brin" and concatenate them.
A: The last letter of "Sergey" is "y".
The last letter of "Brin" is "n".
Concatenating them is "yn".
The answer is yn.
Q: Take the last letters of the words in "Bill Gates" and concatenate them.
A: The last letter of "Bill" is "l".
The last letter of "Gates" is "s".
Concatenating them is "ls".
The answer is ls.
[Table]
does not ﬂip the coin.
Is the coin still heads up?
A: The coin was ﬂipped by Conception.
So the coin was ﬂipped 1 time, which is an odd number.
The coin started heads up, so after an odd number of ﬂips, it will be tails up.
So the answer is no.
Q: A coin is heads up.
Inga does not ﬂip the coin.
Elanor does not ﬂip the coin.
Is the coin still heads up?
A: The coin was ﬂipped by no one.
So the coin was ﬂipped 0 times.
The coin started heads up, and it was not ﬂipped, so it is still heads up.
So the answer is yes.
Q: A coin is heads up.
Ryan ﬂips the coin.
Shaunda ﬂips the coin.
Is the coin still heads up?
A: The coin was ﬂipped by Ryan and Shaunda.
So the coin was ﬂipped 2 times, which is an even number.
The coin started heads up, so after an even number of ﬂips, it will still be heads up.
So the answer is yes.
P ROMPT FOR CSQA
[Table]
P ROMPT FOR S TRATEGY QA
[Table]
Hydrogen
[Table]
can happen
[Table]
and June
[Table]
winter
[Table]
could
[Table]
Could a llama birth twice during War in
[Table]
War in Vietnam was 6 months.
The gestation period for
[Table]
llama is 11 months
[Table]
is more than
[Table]
Thus
[Table]
llama could not give birth twice during the War in Vietnam.
So the
[Table]
is no.
[Table]
Would a pear sink in water
[Table]
The density of a pear is about 0 .
6 g/cm 3 , which is less than water.
Objects less dense than water ﬂoat.
Thus
[Table]
So
[Table]
is
[Table]
P ROMPT FOR D ATE U NDERSTANDING
Q: 2015 is coming in 36 hours.
What
is
the date one week from
today
in MM/DD/YYYY?
A:
If
is
coming
in 36 hours,
then
it is
coming
in days days
before
/01/2015 is 12/30/
, so today is
So one week from today will be
[Table]
P ROMPT FOR S PORTS U NDERSTANDING
[Table] [Table]
[Table] [Table]
H Appendix: Alternate Annotators for MWP
[Table]
FOR M ATH W ORD
[Table]
P ROMPT FOR M ATH W ORD P ROBLEMS
[Table]