S ELF -C ONSISTENCY I MPROVES C HAIN OF T HOUGHT R EASONING IN L ANGUAGE M ODELS
A BSTRACT
Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks.
In this paper, we propose a new decoding strategy, self-consistency , to replace the naive greedy decoding used in chain-of-thought prompting.
It ﬁrst samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths.
Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer.
Our extensive empirical evaluation shows that self-consistency boosts the performance of chain-of-thought prompting with a striking margin on a range of popular arithmetic and commonsense reasoning benchmarks, including GSM8K (+17.9%), SVAMP (+11.0%), AQuA (+12.2%),
StrategyQA (+6.4%) and ARC-challenge (+3.9%)
I NTRODUCTION
Although language models have demonstrated remarkable success across a range of NLP tasks, their ability to demonstrate reasoning is often seen as a limitation, which cannot be overcome solely by increasing model scale (Rae et al., 2021; BIG-bench collaboration, 2021, inter alia ).
In an effort to address this shortcoming, Wei et al.
(2022) have proposed chain-of-thought prompting , where a language model is prompted to generate a series of short sentences that mimic the reasoning process a person might employ in solving a task.
For example, given the question “If there arecars in the parking lot and 2 more cars arrive, how many cars are in the parking lot?”
, instead of directly responding with “5” , a language model would be prompted to respond with the entire chain-of-thought: “There are 3 cars in the parking lot already.
2 more arrive.
Now there are 3 + 2 = 5 cars.
The answer is 5.” .
It has been observed that chain-of-thought prompting signiﬁcantly improves model performance across a variety of multi-step reasoning tasks (Wei et al., 2022).
In this paper, we introduce a novel decoding strategy called self-consistency to replace the greedy decoding strategy used in chain-of-thought prompting (Wei et al., 2022), that further improves language models’ reasoning performance by a signiﬁcant margin.
Self-consistency leverages the intuition that complex reasoning tasks typically admit multiple reasoning paths that reach a correct answer (Stanovich & West, 2000).
The more that deliberate thinking and analysis is required for a problem (Evans, 2010), the greater the diversity of reasoning paths that can recover the answer.
Figure 1 illustrates the self-consistency method with an example.
We ﬁrst prompt the language model with chain-of-thought prompting, then instead of greedily decoding the optimal reasoning path, we propose a “sample-and-marginalize” decoding procedure: we ﬁrst sample from the language model’s decoder to generate a diverse set of reasoning paths; each reasoning path might lead to a different ﬁnal answer, so we determine the optimal answer by marginalizing out the sampled reasoning paths to ﬁnd the most consistent answer in the ﬁnal answer set.
Such an approach is analogous to the human experience that if multiple different ways of thinking lead to the same answer, one has greater conﬁdence that the ﬁnal answer is correct.
Compared to other decoding methods, self-consistency avoids the repetitiveness and local-optimality that plague greedy decoding, while mitigating the stochasticity of a single sampled generation.
[Figure]
Self-consistency is far simpler than prior approaches that either train an additional veriﬁer (Cobbe et al., 2021) or train a re-ranker given additional human annotations to improve generation quality (Thoppilan et al., 2022).
Instead, self-consistency is entirely unsupervised , works off-the-shelf with pre-trained language models, requires no additional human annotation, and avoids any additional training, auxiliary models or ﬁne-tuning.
Self-consistency also differs from a typical ensemble approach where multiple models are trained and the outputs from each model are aggregated, it acts more like a “self-ensemble” that works on top of a single language model.
We evaluate self-consistency on a wide range of arithmetic and commonsense reasoning tasks over four language models with varying scales: the public UL2-20B (Tay et al., 2022) and GPT-3-175B (Brown et al., 2020), and two densely-activated decoder-only language models: LaMDA-137B (Thoppilan et al., 2022) and PaLM-540B (Chowdhery et al., 2022).
On all four language models, self-consistency improves over chain-of-thought prompting by a striking margin across all tasks.
In particular, when used with PaLM-540B or GPT-3, self-consistency achieves new state-of-the-art levels of performance across arithmetic reasoning tasks, including GSM8K (Cobbe et al., 2021)
(+17.9% absolute accuracy gains), SVAMP (Patel et al., 2021)
(+11.0%), AQuA (Ling et al., 2017)
(+12.2%), and across commonsense reasoning tasks such as StrategyQA (Geva et al., 2021)
(+6.4%) and ARCchallenge (Clark et al., 2018)
(+3.9%).
In additional experiments, we show self-consistency can robustly boost performance on NLP tasks where adding a chain-of-thought might hurt performance compared to standard prompting (Ye & Durrett, 2022).
We also show self-consistency signiﬁcantly outperforms sample-and-rank, beam search, ensemble-based approaches, and is robust to sampling strategies and imperfect prompts.
S ELF -C ONSISTENCY OVER D IVERSE R EASONING P ATHS
A salient aspect of humanity is that people think differently.
It is natural to suppose that in tasks requiring deliberate thinking, there are likely several ways to attack the problem.
We propose that such a process can be simulated in language models via sampling from the language model’s decoder.
For instance, as shown in Figure 1, a model can generate several plausible responses to a math question that all arrive at the same correct answer (Outputs 1 and 3).
Since language models are not perfect reasoners, the model might also produce an incorrect reasoning path or make a mistake in one of the reasoning steps (e.g., in Output 2), but such solutions are less likely to arrive at the same answer.
That is, we hypothesize that correct reasoning processes, even if they are diverse, tend to have greater agreement in their ﬁnal answer than incorrect processes.
We leverage this intuition by proposing the following self-consistency method.
First, a language model is prompted with a set of manually written chain-of-thought exemplars (Wei et al., 2022).
Next,
[Table]
we sample a set of candidate outputs from the language model’s decoder, generating a diverse set of candidate reasoning paths.
Self-consistency is compatible with most existing sampling algorithms, including temperature sampling (Ackley et al., 1985; Ficler & Goldberg, 2017), top- k sampling (Fan et al., 2018; Holtzman et al., 2018;
Radford et al., 2019)
[Table]
and nucleus sampling (Holtzman et al., 2020).
Finally, we aggregate the answers by marginalizing out the sampled reasoning paths and choosing the answer that is the most consistent among the generated answers.
In more detail, assume the generated answers a i are from a ﬁxed answer set, a i ∈ A , where i = 1 , . . .
, m indexes the m candidate outputs sampled from the decoder.
Given a prompt and a question, self-consistency introduces an additional latent variable r i , which is a sequence of tokens representing the reasoning path in the i -th output, then couples the generation of ( r i , a i ) where r i → a i , i.e., generating a reasoning path
r
i is optional and only used to reach the ﬁnal answer
a i .
As an example, consider Output 3 from Figure 1: the ﬁrst few sentences “ She eats 3 for breakfast ...
So she has 9 eggs * $2 = $18. ”
constitutes r i , while the answer 18 from the last sentence, “ The answer is $18 ”, is parsed as a i .
After sampling multiple ( r i , a i ) from the model’s decoder, self-consistency applies a marginalization over r i by taking a majority vote over a i , i.e., arg max a (cid:80) mi =1 1 ( a i = a ) , or as we deﬁned as the most “consistent” answer among the ﬁnal answer set.
In Table 1, we show the test accuracy over a set of reasoning tasks by using different answer aggregation strategies.
In addition to majority vote, one can also weight each ( r i , a i ) by P ( r i , a i | prompt , question ) when aggregating the answers.
Note to compute P ( r i , a i | prompt , question ) , we can either take the unnormalized probability of the model generating ( r i , a i ) given ( prompt , question ) , or we can normalize the conditional probability by the output length (Brown et al., 2020), i.e.,
[Equation]
where log P ( t k | prompt , question , t 1 , . .
, t k − 1 ) is the log probability of generating the k -th token
t k in ( r i , a i ) conditioned on the previous tokens, and K is the total number of tokens in ( r i , a i ) .
In Table 1, we show that taking the “unweighted sum”, i.e., taking a majority vote directly over a i yields a very similar accuracy as aggregating using the “normalized weighted sum”.
We took a closer look at the model’s output probabilities and found this is because for each ( r i , a i ) , the normalized conditional probabilities P ( r i , a i | prompt , question ) are quite close to each other, i.e., the language model regards those generations as “similarly likely”.
Additionally, when aggregating the answers, the results in Table 1 show that the “normalized” weighted sum (i.e., Equation 1) yields a much higher accuracy compared to its unnormalized counterpart.
For completeness, in Table 1 we also report the results by taking a “weighted average”, i.e., each a gets a score of its weighted sum divided by (cid:80) mi =1 1 ( a i = a ) , which results in a much worse performance.
Self-consistency explores an interesting space between open-ended text generation and optimal text generation with a ﬁxed answer.
Reasoning tasks typically have ﬁxed answers, which is why researchers have generally considered greedy decoding approaches (Radford et al., 2019; Wei et al., 2022; Chowdhery et al., 2022).
However, we have found that even when the desired answer is ﬁxed, introducing diversity in the reasoning processes can be highly beneﬁcial; therefore we leverage sampling, as commonly used for open-ended text generation (Radford et al., 2019; Brown et al., 2020;
Thoppilan et al., 2022), to achieve this goal.
One should note that self-consistency can be applied only to problems where the ﬁnal answer is from a ﬁxed answer set, but in principle this approach can be extended to open-text generation problems if a good metric of consistency can be deﬁned between multiple generations, e.g., whether two answers agree or contradict each other.
E XPERIMENTS
We conducted a series of experiments to compare the proposed self-consistency method with existing approaches on a range of reasoning benchmarks.
We ﬁnd that self-consistency robustly improves reasoning accuracy for every language model considered, spanning a wide range of model scales.
3. E XPERIMENT SETUP
Tasks and datasets.
We evaluate self-consistency on the following reasoning benchmarks.
• Arithmetic reasoning .
For these tasks, we used the Math Word Problem Repository (KoncelKedziorski et al., 2016), including AddSub (Hosseini et al., 2014), MultiArith (Roy & Roth, 2015), and ASDiv (Miao et al., 2020).
We also included AQUA-RAT (Ling et al., 2017), a recently published benchmark of grade-school-math problems (GSM8K; Cobbe et al., 2021), and a challenge dataset over math word problems (SVAMP; Patel et al., 2021).
• Commonsense reasoning .
For these tasks, we used CommonsenseQA (Talmor et al., 2019), StrategyQA (Geva et al., 2021), and the AI2 Reasoning Challenge (ARC) (Clark et al., 2018).
• Symbolic Reasoning .
We evaluate two symbolic reasoning tasks: last letter concatenation (e.g., the input is “Elon Musk” and the output should be “nk”), and Coinﬂip (e.g., a coin is heads-up, after a few ﬂips is the coin still heads-up?)
from Wei et al.
Language
models and prompts.
We evaluate self-consistency over four transformer-based language models with varying scales:
• UL2 (Tay et al., 2022) is an encoder-decoder model trained on a mixture of denoisers with 20billion parameters.
UL2 is completely open-sourced 4 and has similar or better performance than GPT-3 on zero-shot SuperGLUE, with only 20B parameters and thus is more compute-friendly; • GPT-3 (Brown et al., 2020) with 175-billion parameters.
We use two public engines code-davinci001 and code-davinci-002 from the Codex series (Chen et al., 2021) to aid reproducibility;• LaMDA-137B
(Thoppilan et al., 2022) is a dense left-to-right, decoder-only language model with 137-billion parameters, pre-trained on a mixture of web documents, dialog data and Wikipedia; • PaLM-540B (Chowdhery et al., 2022) is a dense left-to-right, decoder-only language model with 540-billion parameters, pre-trained on a high quality corpus of 780 billion tokens with ﬁltered webpages, books, Wikipedia, news articles, source code, and social media conversations
We perform all experiments in the few-shot setting, without training or ﬁne-tuning the language models.
For a fair comparison we use the same prompts as in Wei et al.
(2022): for all arithmetic reasoning tasks we use the same set of 8 manually written exemplars; for each commonsense reasoning task, 4-7 exemplars are randomly chosen from the training set with manually composed chain-of-thought prompts.
6 Full details on the prompts used are given in Appendix A.3.
Sampling scheme.
To sample diverse reasoning paths, we followed similar settings to those suggested in Radford et al.
; Holtzman et al.
(2020) for open-text generation.
In particular, for
UL2-20B and LaMDA-137B we applied temperature sampling with T = 0 .
5 and truncated at the top- k ( k = 40 ) tokens with the highest probability, for PaLM-540B we applied T = 0 .
7 , k = 40 , and for GPT-3 we use T = 0 .
7 without top- k truncation.
We provide an ablation study in Section 3.5 to show that self-consistency is generally robust to sampling strategies and parameters.
3. M AIN R ESULTS
We report the results of self-consistency averaged over 10 runs, where we sampled 40 outputs independently from the decoder in each run.
The baseline we compare to is chain-of-thought prompting with greedy decoding (Wei et al., 2022), referred to as CoT-prompting , which has been previously used for decoding in large language models (Chowdhery et al., 2022).
Arithmetic Reasoning
The results are shown in Table 2.
7 Self-consistency improves the arithmetic reasoning performance over all four language models signiﬁcantly over chain-of-thought prompting.
More surprisingly, the gains become more signiﬁcant when the language model’s scale increases, e.g., we see +3%-6% absolute accuracy improvement over UL2-20B but +9%-23% for LaMDA137B and GPT-3.
For larger models that already achieve high accuracy on most tasks (e.g., GPTand PaLM-540B), self-consistency still contributes signiﬁcant additional gains with +12%-18% absolute accuracy on tasks like AQuA and GSM8K, and +7%-11% on SVAMP and ASDiv.
With self-consistency, we achieve new state-of-the-art results on almost all tasks: despite the fact that selfconsistency is unsupervised and task-agnostic, these results compare favorably to existing approaches that require task-speciﬁc training, or ﬁne-tuning with thousands of examples (e.g., on GSM8K).
[Table]
[Table]
Commonsense and Symbolic Reasoning Table 3 shows the results on commonsense and symbolic reasoning tasks.
Similarly, self-consistency yields large gains across all four language models, and obtained SoTA results on 5 out of 6 tasks.
For symbolic reasoning, we test the out-of-distribution (OOD) setting where the input prompt contains examples of 2-letters or 2-ﬂips but we test examples of 4-letters and 4-ﬂips (this setting is more challenging as PaLM-540B or GPT-3 can already achieve perfect in-distribution accuracy).
In this challenging OOD setting, the gain of self-consistency is still quite signiﬁcant compared to CoT-prompting with sufﬁcient model sizes.
To show the effect of the number of sampled reasoning paths, we plot the accuracy (mean and standard deviation over 10 runs) with respect to varying numbers of sampled paths (1, 5, 10, 20, 40) in Figure 2.
The results show that sampling a higher number (e.g., 40) of reasoning paths leads to a consistently better performance, further emphasizing the importance of introducing diversity in the reasoning paths.
In Table 4, we show self-consistency yields a richer set of reasoning paths compared to greedy decoding with a few example questions from two tasks.
[Figure] [Figure]
[Table]
3. S ELF -C ONSISTENCY H ELPS W HEN C HAIN - OF -T HOUGHT H URTS P ERFORMANCE
Ye & Durrett (2022) show that sometimes chain-of-thought prompting could hurt performance compared to standard prompting in few-shot in-context learning.
Here we perform a study using self-consistency to see if it can help ﬁll in the gap, over a set of common NLP tasks, including (1)
Closed-Book Question Answering:
BoolQ (Clark et al., 2019), HotpotQA (Yang et al., 2018)
[Table]
and (2) Natural Language Inference: e-SNLI (Camburu et al., 2018), ANLI (Nie et al., 2020) and RTE (Dagan et al., 2005; Bar-Haim et al., 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009).
The results over PaLM-540B are shown in Table 5.
For some tasks (e.g., ANLI-R1, e-SNLI, RTE), adding chain-of-thought does hurt performance compared to standard prompting (Brown et al., 2020), but self-consistency is able to robustly boost the performance and outperform standard prompting, making it a reliable way to add rationales in few-shot in-context learning for common NLP tasks.
[Table]
3. C OMPARE TO OTHER EXISTING APPROACHES
We conduct a set of additional studies and show that self-consistency signiﬁcantly outperforms existing methods including sample-and-rank, beam search, and ensemble-based approaches.
Comparison to Sample-and-Rank One commonly used approach to improve generation quality is sample-and-rank, where multiple sequences are sampled from the decoder and then ranked according to each sequence’s log probability (Adiwardana et al., 2020).
We compare self-consistency with sample-and-rank on GPT-3 code-davinci-001 , by sampling the same number of sequences from the decoder as self-consistency and taking the ﬁnal answer from the top-ranked sequence.
The results are shown in Figure 3.
While sample-and-rank does improve the accuracy with additionally sampled sequences and ranking, the gain is much smaller compared to self-consistency.
[Figure]
Comparison to Beam Search In Table 6, we compare self-consistency with beam search decoding on the UL2-20B model.
For a fair comparison we report the accuracy under the same number of beams and reasoning paths.
On both tasks self-consistency outperforms beam search signiﬁcantly.
Note self-consistency can also adopt beam search to decode each reasoning path (results are shown as “Self-consistency using beam search”), but its performance is worse compared to self-consistency with sampling.
The reason is that beam search yields a lower diversity in the outputs (Li & Jurafsky, 2016), while in self-consistency the diversity of the reasoning paths is the key to a better performance.
[Table]
Comparison to Ensemble-based Approaches We further compare self-consistency to ensemblebased methods for few-shot learning.
In particular, we consider ensembling by: (1) prompt order permutation: we randomly permute the exemplars in the prompt 40 times to mitigate model’s sensitivity to prompt order (Zhao et al., 2021;
Lu et al., 2021); and (2) multiple sets of prompts (Gao et al., 2021): we manually write 3 different sets of prompts.
We took majority vote of the answers from greedy decoding in both approaches as an ensemble.
Table 7 shows that compared to self-consistency, existing ensemble-based approaches achieve a much smaller gain.
In addition, note that self-consistency is different from a typical model-ensemble approach, where multiple models are trained and their outputs are aggregated.
Self-consistency acts more like a “self-ensemble” on top of a single language model.
We additionally show the results of ensembling multiple models in
Appendix A.1.3 where the model-ensembles perform much worse compared to self-consistency
[Table]
3. A DDITIONAL S TUDIES
We conducted a number of additional experiments to analyze different aspects of the self-consistency method, including its robustness to sampling strategies and parameters, and how it works with imperfect prompts and non-natural-language reasoning paths.
Self-Consistency is Robust to Sampling Strategies and Scaling We show self-consistency is robust to sampling strategies and parameters, by varying T in temperature sampling (Ackley et al., 1985; Ficler & Goldberg, 2017), k in top- k sampling (Fan et al., 2018;
Holtzman et al., 2018;
Radford et al., 2019), and p in nucleus sampling (Holtzman et al., 2020), over PaLM-540B in Figure 4 (left).
Figure 4 (right) shows that self-consistency robustly improves performance across all scales for the
LaMDA-137B model series.
The gain is relatively lower for smaller models due to certain abilities (e.g., arithmetic) only emerge when the model reaches a sufﬁcient scale (Brown et al., 2020).
[Figure]
Self-Consistency Improves Robustness to Imperfect Prompts For few-shot learning with manually constructed prompts, human annotators sometimes make minor mistakes when creating the prompts.
We further study if self-consistency can help improve a language model’s robustness to imperfect prompts.
9 We show the results in Table 8: while imperfect prompts decrease accuracy with greedy decoding (17.1 → 14.9), self-consistency can ﬁll in the gaps and robustly improve the results.
Additionally, we found that the consistency (in terms of % of decodes agreeing with the ﬁnal aggregated answer) is highly correlated with accuracy (Figure 5, over GSM8K).
This suggests that one can use self-consistency to provide an uncertainty estimate of the model in its generated solutions.
In other words, one can use low consistency as an indicator that the model has low conﬁdence; i.e., self-consistency confers some ability for the model to “know when it doesn’t know”.
LaMDA
[Figure]
137B
[Figure] [Figure]
Self-Consistency Works for Non-Natural-Language Reasoning Paths and Zero-shot CoT
We also tested the generality of the self-consistency concept to alternative forms of intermediate reasoning like equations (e.g., from “ There are 3 cars in the parking lot already.
2 more arrive.
Now there are 3 + 2 = 5 cars. ”
to “ 3 + 2 = 5 ”).
The results are shown in Table 8 (“Prompt with equations”): self-consistency still improves accuracy by generating intermediate equations; however, compared to generating natural language reasoning paths, the gain is smaller since the equations are much shorter and less opportunity remains for generating diversity in the decoding process.
In addition, we tested self-consistency with zero-shot chain-of-thought (Kojima et al., 2022) and show that self-consistency works for zero-shot CoT as well and improves the results signiﬁcantly (+26.2%) in Table 8.
R ELATED WORK
Reasoning in language models.
Language models are known to struggle in Type 2 tasks, such as arithmetic, logical and commonsense reasoning (Evans, 2010).
Previous work has primarily focused on specialized approaches for improving reasoning (Andor et al., 2019; Ran et al., 2019; Geva et al., 2020; Pi˛ekos et al., 2021).
Compared to prior work, self-consistency is applicable to a wide range of reasoning tasks without any additional supervision or ﬁne-tuning, while still substantially improving the performance of the chain-of-thought prompting approach proposed in Wei et al.
Sampling and re-ranking in language models.
Multiple decoding strategies for language models have been proposed in the literature, e.g., temperature sampling (Ackley et al., 1985; Ficler &
Goldberg, 2017), top- k sampling (Fan et al., 2018; Holtzman et al., 2018; Radford et al., 2019), nucleus sampling (Holtzman et al., 2020), minimum Bayes risk decoding (Eikema & Aziz, 2020; Shi et al., 2022), and typical decoding (Meister et al., 2022).
Other work has sought to explicitly promote diversity in the decoding process (Batra et al., 2012;
Li et al., 2016; Vijayakumar et al., 2018).
Re-ranking is another common approach to improve generation quality in language models (Adiwardana et al., 2020; Shen et al., 2021).
Thoppilan et al.
(2022) collect additional human annotations to train a re-ranker for response ﬁltering.
Cobbe et al.
(2021) train a “veriﬁer” to re-rank generated solutions, which substantially improves the solve rate on math tasks compared to just ﬁne-tuning the language model.
Elazar et al.
(2021) improve the consistency of factual knowledge extraction by extending pre-training with an additional consistency loss.
All these methods require either training an additional re-ranker or collecting additional human annotation, while self-consistency requires no additional training, ﬁne-tuning, nor extra data collection.
Extract reasoning paths.
Some previous work has considered task-speciﬁc approaches for identifying reasoning paths, such as constructing semantic graphs (Xu et al., 2021a), learning an RNN to retrieve reasoning paths over the Wikipedia graph (Asai et al., 2020), ﬁne-tuning with human annotated reasoning paths on math problems (Cobbe et al., 2021), or training an extractor with heuristic-based pseudo reasoning paths (Chen et al., 2019).
More recently, the importance of diversity in the reasoning processes has been noticed, but only leveraged via task-speciﬁc training, either through an additional QA model over extracted reasoning paths (Chen et al., 2019), or by the introduction of latent variables in a commonsense knowledge graph (Yu et al., 2022).
Compared to these approaches, self-consistency is far simpler and requires no additional training.
The approach we propose simply couples the generation of reasoning paths and a ﬁnal answer by sampling from the decoder, using aggregation to recover the most consistent answer without additional modules.
Consistency in language models.
Some prior work has shown that language models can suffer from inconsistency in conversation (Adiwardana et al., 2020), explanation generation (Camburu et al., 2020), and factual knowledge extraction (Elazar et al., 2021).
Welleck et al.
(2020) use “consistency” to refer to generating an inﬁnite-length sequence in recurrent language models.
Nye et al.
(2021) improve the logical consistency of samples from a System 1 model by adding a System 2-inspired logical reasoning module.
In this paper we focus on a slightly different notion of “consistency”, i.e., utilizing answer consistency among diverse reasoning paths to improve accuracy.
C ONCLUSION AND D ISCUSSION
We introduced a simple yet effective method called self-consistency, and observed that it signiﬁcantly improves accuracy in a range of arithmetic and commonsense reasoning tasks, across four large language models with varying scales.
Beyond accuracy gains, self-consistency is also useful for collecting rationales when performing reasoning tasks with language models, and for providing uncertainty estimates and improved calibration of language model outputs.
One limitation of self-consistency is that it incurs more computation cost.
In practice people can try a small number of paths (e.g., 5 or 10) as a starting point to realize most of the gains while not incurring too much cost, as in most cases the performance saturates quickly (Figure 2).
As part of future work, one could use self-consistency to generate better supervised data to ﬁne-tune the model, such that the model can give more accurate predictions in a single inference run after ﬁne-tuning.
In addition, we observed that language models can sometimes generate incorrect or nonsensical reasoning paths (e.g., the StrategyQA example in Table 4, the two population numbers are not exactly correct), and further work is needed to better ground models’ rationale generations.
R EPRODUCIBILITY S TATEMENT
In experiments, we included four different language models with varying scales.
Two of them are public models: UL2 is a completely open-sourced model with model checkpoints available at https:// github.com/google-research/google-research/tree/master/ul2 ; GPT-3 is also a public model with public API available at https://openai.com/api/ .
For GPT-3, we have included two public engines (“code-davinci-001” and “code-davinci-002”) to further aid reproducibility, as Codex is currently free so anyone can reproduce the results.
In addition, as our results make use of LaMDA-137B and PaLM-540B that are not publicly available, we provide the exact input prompts for all tasks in Appendix A.3 (and note that we do not perform any ﬁnetuning and only apply prompting to off-the-shelf language models).
E THICS S TATEMENT
As we stated in the discussion, language models can sometimes generate nonsensical or non-factual reasoning paths, so one should use language models’ outputs with extra caution.
We deal with reasoning tasks mostly and the generated rationales are only used for inspecting how a model reaches its answer.
One could potentially use the generated rationales to further check why the model makes certain mistakes or whether the model contains any biases when performing a certain task.
For language model in real-world use, further work is needed to better ground models’ predictions and improve model’s factuality and safety, to ensure the models do not cause harms to users.
A A PPENDIX A. A DDITIONAL E XPERIMENT R ESULTS A.1. R OBUSTNESS TO S AMPLING S TRATEGIES AND P ARAMETERS
In Figure 6 we ablate the results with respect to different sampling strategies and parameters by varying T in temperature sampling and k in Top- k sampling, on LaMDA-137B.
We show that self-consistency is robust to various sampling strategies and parameters.
[Figure]
In Figure 7 and Figure
[Figure]
we show the results of self-consistency compared with greedy decoding a single path over LaMDA-137B and PaLM-540B
[Figure]
respectively.
Self-consistency improves over greedy decode by a quite signiﬁcant margin on both models, on top of high accuracy already achieved by scaling up model sizes
[Figure]
We further show additional sampled reasoning paths from the LaMDA-137B model in Table 12, and sampled reasoning paths from the PaLM-540B model in Table 13.
We see that the diversity in the additionally sampled reasoning paths indeed helps the model arrive at a more correct ﬁnal answer after aggregation.
A.1. R OBUSTNESS TO DIFFERENT SETS OF PROMPTS
In Table 9, we further show that self-consistency is quite robust to different sets of input prompts.
We manually wrote 3 different sets of chain-of-thought as prompts to the model.
Across all sets of prompts, self-consistency yields consistent gains over the original CoT approach.
A.1. C OMPARED TO MODEL ENSEMBLES
Additionally, we provide results of directly ensembling the outputs from multiple language models .
The results are shown in Table 10, by greedily decoding sequences from 3 language models and
[Figure] [Figure]
[Table]
taking the majority vote (averaged over 10 runs).
Note this is a typical ensemble approach (averaging over the predictions over multiple models) and it achieves a performance signiﬁcantly worse than self-consistency (self-consistency over PaLM-540B gets an accuracy of 74.4%), as lower-capacity models drag down the performance of higher-capacity models.
In addition, this approach is limited in two ways: 1) It requires multiple models for an ensemble which might not always be available, while self-consistency only requires one single model to “self-ensemble”; 2)
If one of the models is much weaker, it can actually hurt the ﬁnal performance
[Table]
A.1. C OMBINING SELF - CONSISTENCY WITH OTHER ENSEMBLING STRATEGIES
Self-consistency is completely compatible with other ensemble strategies, although the gains achieved by self-consistency are signiﬁcantly higher than other ensemble strategies (and can “override” the performance gains achieved by other ensemble strategies).
We further performed experiments and include the results in Table 11 (for a fair comparison, we use 40 sets of prompts, or 40 prompt permutations to compare with self-consistency with 40 paths
[Table]
A. D ETAILS ON R ESOURCES AND I NFERENCE
For all four language models we perform prompting-based inference only.
For UL2 we use TPU v (2x2 conﬁguration, 4 chips, 8 cores).
For GPT-3 models the experiments are done though the public
API.
For LaMDA-137B we use TPU v3 (8x8 conﬁguration, 64 chips, 128 cores).
For PaLM-540B we use TPU v4 (4x4x12 conﬁguration, 192 chips, 384 cores).
Most inference jobs take 1 to 4 hours (over about 1,000 examples) for each task on UL2 and LaMDA-137B, and about 2 to 12 hours on
PaLM-540B. Some tasks (e.g., commonsense reasoning) take longer but do not exceed 2 days for each task.
For GPT-3 models, we use 128 max tokens for all methods, without frequency penalty or presence penalty.
For all models, we take the generated outputs until the start of the next “Q:" to parse the ﬁnal answers, consistent with our prompting format.
[Table]
Theanswermustbeaplacewherethemanislayingdownandlook
[Table]
[Table]
helps repair the errors over greedy decode on
PaLM-540B. Two sampled reasoning paths that are consistent with the ground truth are shown.
A. F ULL S ETS OF P ROMPTS
We list the full details of the prompts used for two newly-introduced datasets, AQUA-RAT (Ling et al., 2017) and AI2 Reasoning Challenge (ARC) (Clark et al., 2018), where we manually composed the example chain-of-thought in this paper, in Table 14 and Table 15, respectively.
[Table] [Table]
[Table]
As additional information, we also list the exact set of prompts used for all arithmetic reasoning tasks in Table 17, since there are multiple sets of prompts introduced in Wei et al.
The prompts for
CommonsenseQA and StrategyQA are the same as used in Wei et al.
We provide the exact prompts used for common NLP tasks in the following tables as well, including
NLI (Table 18, Table 19, Table 20) and Closed-Book Question-Answering tasks (Table 16, Table 21).
[Table] [Table]
- yes
[Table]
no - it is not possible to tell
[Table]
"Vrenna and I both" contradicts "neither Vrenna nor myself".
The answer is no.
[Table]
is not possible to tell
[Table]
The children must be
[Table]
to see them smiling and waving
[Table]
[Table] [Table]
of
[Table]
Destruction Found in
[Table]
Yet."
Based on this premise, can we conclude the hypothesis "Weapons of Mass Destruction Found in Iraq." is true?
[Table]
: "No Weapons of Mass Destruction Found" contradicts "Weapons of Mass Destruction Found".
The answer is no.
Premise:"Aplace of sorrow, after Pope John Paul II died, became a place of celebration, as Roman Catholic faithful gathered in downtown Chicago to mark the installation of new Pope Benedict XVI.
" Based on this premise, can we conclude the hypothesis "Pope Benedict XVI is the new leader of the Roman
Catholic Church." is true?’
[Table]
." means "Pope Benedict XVI is the new leader".
The answer is yes.
[Table]
"A man is due in court later charged with the murder 26 years ago of a teenager whose case was the ﬁrst to be featured on BBC One’s Crimewatch.
Colette Aram, 16, was walking to her boyfriend’s house in Keyworth, Nottinghamshire, on 30 October 1983 when she disappeared.
Her body was later found in a ﬁeld close to her home.
Paul Stewart Hutchinson, 50, has been charged with murder and is due before Nottingham magistrates later."
Based on this premise, can we conclude the hypothesis "Paul Stewart Hutchinson is accused of having stabbed a
[Table] [Table]
Paul
[Table]
Hutchinson
[Table]
The answer
[Table] [Table]
Herceptin was already approved to treat the sickest breast cancer patients, and the company said, Monday, it will discuss with federal regulators the possibility of prescribing the drug for more breast cancer patients."
Based on this premise, can we conclude the hypothesis "Herceptin can be used to treat breast cancer.
" is true?
[Table]
The answer is yes.
[Table]
does system of a down have
[Table]
singers
[Table]
of
[Table]
Down
[Table]
Tankian
[Table]
do
[Table]
Q: do iran and afghanistan speak the same language?
A: Iran and Afghanistan both speak the Indo-European language Persian.
The answer is yes.
Q: is a cello and a bass the same thing?
A:
The cello is played sitting down with the instrument between the knees, whereas the double bass is played standing or sitting on a stool.
The answer is no.
Q: can you use oyster card at epsom station?
A: Epsom railway station serves the town of Epsom in Surrey and is not in the London Oyster card zone.
The answer is no.