Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose a two-stage template-based prompt method for general NLP tasks, named zero-shot-CoT. In this work, it proves that large pre-trained language models have a talent to extract reasoning chains without any few-shot examples for finetuning or imitating. They also investigate the inference ability of LMs with different scales on various datasets.
strengths_and_weaknesses: Strengths:
1. In this two-stage framework (called CoT), the first step extracts reasoning chains and the second can summary the final answers from extracted chains. 
2. The authors also employ post-processing strategies to acquire clean answers.

Weakness: 
1. There exists many zero-shot prompt methods by adding a short prompt like `Let’s think step by step`, which means this work is not novel enough nowadays. 
2. Large language models have been proven to have satisfying potential of zero-shot reasoning, especially GPT-3 and its variants.
3. Although there are some analyses about the robustness of different prompts in table 5 and 6, it is not enough to prove the proposed method can solve general tasks. The authors should provide detailed results on every tasks but not a score on a specific task.
questions: 1. The authors declare that larger LMs have better ability of zero-shot reasoning, but they only provide results of GPT-3 models with different scales except T0. Can this conclusion apply to models with other structures? Will there be a upheaval as the scale increasing?
2. Larger LMs can memorize more knowledge from pre-training corpus, therefore, these remarkable experimental results of large LMs may come from what they have learnt from pre-training. Will the proposed method be effective on more tasks of some specific domains like commonsense, news QA, or even bioscience?
3. A trivial error: In P3-L114, `Q:[X] A:[Z]` should be `Q:[X] A:[T]`?

limitations: The authors address the limitations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper explores the zero-shot reasoning abilities of large language models. Extending from recent work on few-shot chain-of-thought prompting, in this work the authors propose to elicit step-by-step reasoning by simply adding “let’s think step by step” in the beginning of the input, then adopt a two-stage prompting approach (Figure 2). Results on a collection of 12 reasoning tasks suggest that zero-shot chain-of-thought (this work) greatly outperforms the raw zero-shot baseline and outperforms 8-shot few-shot (in-context) baseline. Further analysis is conducted on model size, prompt selection etc. 
strengths_and_weaknesses: Strengths
* Comprehensive experiments to demonstrate that prompting with “let’s think step by step” greatly improves zero-shot capabilities of large language models on reasoning tasks. The method is simple yet effective.
* The authors also provide a detailed survey of relevant zero-shot/few-shot approaches in Sec 5 and Table 7, which is a useful roadmap for researchers interested in the topic.
* The examples in the appendix are enjoyable to read. Sample Study section is the hidden gem in the appendix.
* Clear writing and visualization.

Weaknesses/Limitations
* According to Table 3, the proposed method is only effective with 175B Original-GPT3 or Instruct-GPT3 model. The practical utility of the method is limited. It would also be great to have the cost of using GPT-3 listed in the paper for reference.
* As prompting is a new and rising field, I’m not certain whether the major finding in this paper (“let’s think step by step”) serve as solid contribution for NeurIPS publication. I hope the authors can further convince me on the impact, utility or significance of their findings.
questions: I think the paper is very informative already but I have several questions out of curiosity.
* Is it possible that the (pre-)training corpus contain some step-by-step reasoning examples that leads to the zero-shot reasoning behavior? Does any of the prompt appear in the training corpus already? Would like to hear the authors comments on this.
* In Table 5, would be helpful to try more diverse phrasing of the prompts and perhaps 1 or 2 “bad” prompts (similar to the analysis in Fig. 5 in https://arxiv.org/abs/2109.01247).

Minor suggestion:
* I believe there is still a little “prompt engineering” efforts needed for zero-shot CoT as described in Line 125. Perhaps the claim in Line 111 is a little strong.
limitations: Limitations are discussed in Section 5
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The papers proposes to use chain-of-thought prompting on a large language model (LLM) in a zero-shot fashion to work on tasks that require multi-step logical reasoning or arithmetic. The key idea is to add a "let's think step by step" at the end of the prompt to elicit a step-by-step approach from an LLM to do these tasks. The framework is two-stage, a first step a detail explanation is extracted out by the adding "let's think step by step" and in the second step, an output-specific answer prompt is used to get the answer in the required format for the task. The authors compare this prompt with normal manual zero-shot prompting, few-shot chain-of-thought prompting and manual few-shot prompting baselines on 12 datasets across 13 models in total. The zero-shot chain-of-thought prompting outperforms the zero-shot manual prompting in all experiments except the commonsense reasoning tasks, works better than few-shot manual prompting, but underperforms few-shot chain-of-thought prompting.
strengths_and_weaknesses: - The paper provides another verification in the debate of prompting that LLMs have learnt a vast amount of knowledge and certain prompts can elicit specific information from these LLMs.
- The method in itself is simple and it is interesting that it works so well across the tasks and models.
- The authors conduct various experiments to see what matters for the performance including testing out various other prompts that are similar in semantic sense.
- The method is effective when the model size is large, but can be detrimental when model size is small.
- The amount of difference that we are seeing in performance with different prompts worries me as we are basically moving more and more towards prompt-tuning where as before we were doing hyper-parameter tuning. The paper doesn't do any sort of leakage analysis which is important because in this we might just be distilling a specific part of the pretraining dataset. Overall, the findings in this paper are probably an artifact of the pretraining data and since this paper highlights that I believe it is an important contribution. 


questions: - I believe the title of the paper should be more specific. The experiments in the paper itself don't really end up in the claim of the title without proper data overlap and other analysis.
- For instructGPT, is it even clear what data it was fine-tuned on? If not, then it is not true zero-shot right?
limitations: Yes
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper shows that large language models can achieve better zero-shot performance on multiple reasoning tasks by simply attaching "Let's think step by step" at the end of prompts. 
strengths_and_weaknesses: Strengths:
1. The proposed method is very simple and effective. 
2. The paper is well written and presented.
3. Good analysis and interesting findings from the paper, such as: 1) model size matters for zero-shot learning, chain-of-thought works only when the model size is large enough; 2) zero-shot-cot generally works when the model is prompted with sentences encouraging reasoning, however, a few sentences significantly work better then the rest.

Weakness:
1.  The proposed method is more like a surprisingly effective hack/trick instead of a systematic technique. The whole paper boils down to one sentence "Let's think step by step". While the paper has well demonstrated the effectiveness of this trick, it can be strengthened by answering a few related questions: 1) how to find the optimal "template"? Is "let's think step by step" happen to work or the model really understands it? 2) What's the mechanism behind this? The template does not provide any extra information and why it can trigger the LLM?

2 The full pipeline in Figure 2 has two steps which I am not sure if it is necessary. I have tried a few examples using the Open AI play ground and it seems that by appending "Let's think step by step", the model naturally generates the final answer "375". 
questions: Discussed in previous sections.
limitations: Yes, the paper does discuss its limitations.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This is a paper that introduces a new zero-shot chain of thought prompting routine and examines its performance across various system2 tasks (e.g. coin flip, common sense, etc.) and on various models (e.g. gpt3, gpt-j, etc.). The prompt that works best in their experiments for zero-shot-cot is "Let's think step by step," which they use in most of their experiments. The new zero-shot-cot routine requires prompting the model twice, the second time just to extract the answer in the right structure. The experiment results show that zero-shot-cot (chain of thought) does better than zero-shot without chain of thought across most tasks. On MultiArith and GSM8K, zero-shot-cot does slightly worse than few-shot-cot methods but better than few-shot (without cot). The authors also show that zero-shot-cot performance scales well with model size and tests various types of zero-shot-cot prompts. (e.g. "Let's think...", "First")
strengths_and_weaknesses: 
Originality:
This paper is somewhat original. The zero-shot-cot concept isn't novel but it was the first time various templates were experimented with in various set-ups. Lots of good experimental results here for future reference.

Quality:
This is a somewhat high quality paper with lots of experiments run on various models of different sizes. The authors cite all relevant works, to the best of my knowledge. Table 7 in particular is very good and shows the authors engaging with the in-context-learning literature.

Clarity:
Very high clarity and easy to read. Mostly well written with few errors, but Section 3 ("zero-shot chain of thought") and Section 4 (before 4.1) felt like they could be combined into one section.

Significance:
Somewhat significant results. Zero-shot-cot performance was interesting and somewhat surprising. But that it does worse than few-shot-cot was expected. Also I would like to know why zero-shot-cot doesn't improve over zero-shot on tasks like common sense reasoning.

questions: Questions:
1. One of the strengths of zero-shot-cot, the authors claim, is that it's task agnostic and doesn't require manual labeling. However, the "2nd prompt" aspect of the two-stage prompting process seems to require a manually engineered "answer trigger". Sure the labels are more confined (multiple choice, yes/no, etc.) but doesn't this make this method equally "manual"?
2. Are there cases where in the chain of reasoning, a wrong answer is derived, but the final answer (after 2nd prompt) is correct?
3. Why don't the authors make a graph of Table 3? Also I disagree that zero-shot/few-shot performance stays flat with model scaling. I think it also improves, just not as significantly.  (maybe phrase differently)
4. The examples used in the error analysis Table 4 are odd? and actually make the case the zero-shot-cot is doing good reasoning. Example 2's question in particular, "What would be necessary for getting in shape?" seems to obviously elicit (common-sensically) multiple answers, but the gold-label only picks one. Maybe these question/answer pairs here are not optimal.
5. How are the table 5 templates derived? I would like to know if these are the result of trial-and-error experiments or thought-out/reasoned examples.

Typos/writing:
1. line 32, p1 (or automatically becomes...) should be ( or automatically has become)
2. line 59, p2 (Figure 1) should (Table 2)
3. lines 105-112, p3: this paragraph could be written better to make clearer
4. lines 114, p3: [T] is supposed to be [Z]?
5. line 125, p4 (trigger depends on the answer format) should be (trigger depending on the answer format)

limitations: I enjoyed reading this paper and it's a good overview and survey of in-context zero-shot/few-shot learning. I would have appreciated this paper more if it were less about experiment results and more about an explanation as to why some things work better than others. I would like, for instance, if the authors could explain table 5 in more detail. Why do some templates work better than others? Or why zero-shot-cot doesn't perform better than zero-shot in commonsense reasoning tasks. Perhaps some other designed experiments could help fill this void. An idea would be to run an experiment where a whole bunch of templates are generated (like 100s) and tested to see which do best. I think an experiment like this would make the results look more objective and systematic and less random. I appreciate the paper as a good comparison of performance, but it feels lacking in its explanatory power.
review_title: null

Meta-review: metareview: This paper shows that large language models, if prompted in a particular way, can drastically improve their performance on zero-shot reasoning tasks. Specifically, simply adding "Let's think step by step" to the prompt causes accuracy to improve by a huge margin. The paper provides a wide variety of experimental results corroborating the central claim, and situates itself well in the existing literature.

The reviewers are on the fence about this work, and as an area chair I find myself similarly torn: the contribution itself is unequivocally mundane -- and I mean this without any offense to the authors, but as a matter of principle I do not think that "prompt engineering", much like "tuning on the test set", is a scientific discipline that deserves much (if any) attention in a machine learning conference. That said, this paper has two main things speaking for it: the fact that **the improvement is so large** AND the fact that **it works increasingly well with scale**. Oftentimes simple ideas are the most impactful ones, and I think that this work can have an impact on the field as a sort of existence proof: the mere observation that a single prompt can lead to such large gains, and that these gains only come about in very large-scale models, either points to serious flaws in our evaluation capabilities (from data overlap to faulty benchmarks) or some as-yet-poorly-understood capability in these models. This work opens up the way to important follow-up work that examines these questions more closely and conclusively, and consequently I think the paper in its current form merits acceptance.

The main shortcoming of the initial submission was that it only used one model, but that has been addressed in the rebuttal with the addition of PaLM. I would strongly encourage the authors to heed Reviewer 1ywu's advice to try to add more models and look more closely at the training data. While it is true that open source models like Bloom (where the training data is publicly available) were not available yet by the time of submission, they are now, and the paper would be considerably strengthened by an analysis into the causes of this effect. In addition, I was a bit disappointed that the "instructive" prompts were not very diverse -- it is well known that large language models are particularly susceptible to minor variations in prompts, so it would have been interesting to try a more complete spectrum of prompts including things like "Let’s solve the problem bit by bit", "If we solve the problem stepwise/gradually/piecemeal", "Let’s be smart about this", "Let’s be careful/make sure we get the right answer" to try to get more clarity on what exactly is happening (and to show the audience they need to be careful with overly trusting their prompts will do the right thing). Lastly, the work almost invites anthropomorphization, along the lines of "humans also do better when prompted to solve a problem step by step", and I would have liked to see a more serious discussion of this aspect of the work (and hence why I think further analysis of training data/prompt variations should be included in the final version).
confidence: Certain
Decision: Accept