T EXT AND P ATTERNS : F OR E FFECTIVE C HAIN OF T HOUGHT , I T T AKES T WO TO T ANGO
Anonymous authors
Paper under double-blind review
A BSTRACT
The past decade has witnessed dramatic gains in natural language processing and an unprecedented scaling of large language models.
These developments have been accelerated by the advent of few-shot techniques such as chain of thought (C O T) prompting.
Speciﬁcally, C O T pushes the performance of large language models in a few-shot setup by augmenting the prompts with intermediate steps.
Despite impressive results across various tasks, the reasons behind their success have not been explored.
This work uses counterfactual prompting to develop a deeper understanding of C O T-based few-shot prompting mechanisms in large language models.
We ﬁrst systematically identify and deﬁne the key components of a prompt: symbols , patterns , and text .
Then, we devise and conduct an exhaustive set of deliberated experiments across four different tasks, by querying the model with counterfactual prompts where only one of these components is altered.
Our experiments across three models—PaLM, GPT-3, and C ODEX —reveal several surprising ﬁndings and brings into question the conventional wisdom around few-shot prompting.
First, the presence of factual patterns in a prompt is practically immaterial to the success of C O T. Second, our results conclude that the primary role of intermediate steps may not be to facilitate learning “how” to solve a task.
The intermediate steps are rather a beacon for the model to realize “what” symbols to replicate in the output to form a factual answer.
As such, the patterns are merely a channel to “trick” the model into forming sentences that resemble correct answers.
This pathway is facilitated by text, which imbues patterns with commonsense knowledge and meaning.
Our empirical and qualitative analysis reveals that a symbiotic relationship between text and patterns explains the success of fewshot prompting: text helps extract commonsense from the question to help patterns, and patterns enforce task understanding and direct text generation.
Such systematic understanding of C O T enables us to devise a concise chain of thought, dubbed as CC O T, where text and patterns are pruned by over 20 % , only retaining their key roles.
We achieve this reduction in the number of tokens while delivering on par or slightly higher solve rate 1 .
1. I NTRODUCTION
The ability to learn a previously unseen task by observing a few examples is one of the cornerstones of human intelligence (Lake et al., 2017).
This is in stark contrast with modern deep learning methods, which typically rely on a substantial labeled corpus of data.
Recently, large language models (LLMs) (Chowdhery et al., 2022; Brown et al., 2020; Chen et al., 2021a) have demonstrated remarkable performance in employing a prompt to perform a task, with no additional ﬁnetuning, commonly known as few-shot learning.
Few-shot learning has shown promising applications for a wide range of tasks (Gehrmann et al., 2021;
Wei et al., 2021; Sanh et al., 2021; Thoppilan et al., 2022; Liu et al., 2021a;
Reif et al., 2021; Wang et al., 2020; Chen et al., 2021b; Lewkowycz et al., 2022; Wu et al., 2022).
While beneﬁcial, this setting requires meticulous design of prompts (Le Scao & Rush, 2021;
Liu et al., 2021c; Mishra et al., 2021).
Ling et al.
(2017) pioneered the idea of using natural language rationales as the intermediate steps in prompts to help model performance for mathematical reasoning.
Recently, Wei et al.
(2022) proposed chain of thought (C O T) prompting, showing that the few-shot setting in LLMs similarly beneﬁts from intermediate natural language rationale across a range of complex reasoning tasks (Ling et al., 2017; Cobbe et al., 2021;
Patel et al., 2021; BIG-bench Collaboration, 2022).
Despite its wide-range usage, the rationale behind the success of C O T remains unclear.
Recent work draws (Ling et al., 2017; Wei et al., 2022) parallels to human thinking.
Humans often think about a problem before deducing a solution.
Akin to this process, it is argued that models should also be able to employ a similar mechanism.
While intuitive, such restrictive abstract explanations fall short in explaining why , when , and how these mechanisms operate.
Ultimately, LLMs are trained to estimate the next token distribution for a given context.
Therefore, there is presumably a systematic rationale behind their successes and failures.
In this work, we undertake initial steps towards understanding the mechanism behind C O T.
Contributions and ﬁndings.
We construct a series of tailored counterfactual prompts (Goyal et al., 2019), deliberately sketched as controlled studies.
First, we identify key components of an example in few-shot prompting as follows: Symbols , Patterns , and Text .
Next, we perform counterfactual prompting —keeping all but one component ﬁxed (e.g., replacing symbols (numbers) with Greek alphabets).
Finally, we elicit meaningful ﬁndings via conducting a systematic and qualitative analysis of the performance divergence between different prompt queries.
Our experiments on four diverse reasoning tasks and across three large language models—PaLM, GPT-3, and C ODEX , reveal several surprising ﬁndings: 1 We ﬁnd that the exact type of symbols in the prompt virtually does not affect the model performance.
In addition, our results and analysis demonstrate counterintuitive phenomena.
For example, we identify that the correctness of symbols and patterns is immaterial to the task solve rate.
We learn that patterns contribute chieﬂy as a venue to reinforce task understanding (Ouyang et al., 2022) and prompt the model to attain correct outputs.
3 Most importantly, we ﬁnd that text and patterns form a symbiotic relationship that plays a vital role in the success of C O T. Text helps generate useful patterns (e.g., by extracting commonsense knowledge), and patterns help reinforce task understanding, enabling the language model to generate text that helps solve the task.
Overall, we argue that one of the primary reasons behind the success of C O T is this interplay between text and patterns—C O T helps a language model in imitating the prompt and generating the right tokens for the task—and is conceivably less related to their reasoning abilities.
Finally, as indicated by applications such as PaLM-S AYCAN (Ahn et al., 2022), we posit that techniques like C O T will play a key role in enabling the success of LLMs on diverse use cases.
Thus, designing efﬁcient prompts informed by a set of key design principles is an important challenge.
To this end, we distill our ﬁndings to create concise prompting, dubbed CC O T. CC O T prunes the prompt (20 % Ó ) to only retain indispensable tokens without negative repercussions on the task solve rate.
2. C OUNTERFACTUAL P ROMPTING FOR C HAIN OF T HOUGHT
The primary objective of our study is to understand C O T through counterfactual prompting. Each counterfactual prompt alters C p p q only one particular aspect of the in-context examples in a p .
For example, consider a the math world problems in the GSM-8 K dataset (See Table 1).
We can modify the example by replacing all the numbers in the thoughts with symbols (e.g., α ).
Such analysis enables us to ask: “ what would the performance of the model have been, if all the numbers in the prompt were replaced with symbols? ”.
The performance disparity of a LLM on C symb _ abs p p q vs. p can thus indicate the role that using actual numbers plays in the success or failure of a task 2 .
[Table]
[Table]
[Table]
[Table]
across different tasks.
We select a diverse set of tasks that allow us to discern Symbols and Patterns
[Table]
[Table]
[Table]
Reasoning tasks.
We focus on reasoning tasks for which C O T presents ample improvements over D IRECT prompting (Wei et al., 2022), namely (cid:16)
M ATHEMATICAL (cid:17) (GSM-8 K Cobbe et al. (2021)), (cid:16)
C OMMONSENSE (cid:17) (date and sports understanding BIG-bench Collaboration (2022)), and (cid:16)
S YMBOLIC (cid:17) (S ORTING )—details in Appendix C and Appendix-Table 7.
2.1. S EMANTIC C OMPONENTS OF P ROMPTS
This work intends to tease apart the major semantic components of a prompt that play a critical role in the efﬁcacy of C O T. To achieve this, we identify and systematically construe three key semantic components of a prompt, listed as follows (See Appendix-Table 7 for the examples to which we refer in the deﬁnitions):
(cid:16) Symbols (cid:17)
are sequences of tokens in the prompt, about which the model reasons to solve a task.
For GSM-8 K and S ORTING , symbols are numerical quantities (e.g., 5, 4, 2, 13).
Similarly for the S PORTS dataset, we categorize players and activities as symbols.
We deﬁne the symbols in the D ATE dataset as date and time indicating expressions
(cid:16) Patterns (cid:17)
are either composition of symbols and operators or a structure of prompt that reinforces task understanding.
The isolation of patterns within a prompt is evident in datasets like GSM-8 K (equations), S PORTS ( person is a sport, activity is a sport ), and S ORTING (1 less than 2) .
However, for the D ATE dataset, the pattern is semantically implicit but consistent.
Each thought contains two parts: (a) x calculation y in which the information from the input is restated (e.g., ) and intermediate results are generated (e.g., “One day after 06/01/ is 06/02/1943”) through mathematical calculations and (b) x output y in which the ﬁnal requisite answer is generated using the intermediate results (e.g., “10 days before today is 05/23/1943”).
(cid:16) Text
(cid:17) are tokens that are neither symbols, nor part of patterns.
Speciﬁcally, text in prompts assists in either outlining the target task (e.g., is the sentence plausible ), connecting patterns to symbols (e.g., John is left with 4 - 2 = 2), or contextualizing symbols (4 toys ).
In a nutshell, text is the conceptual glue that binds different parts of a prompt.
Large language models. We restrict our analysis to models at the scale of PaLM, GPT-3, and C ODEX , that can successfully complete prompts p for complex reasoning tasks is typically present for LLM. To facilitate
[Table] conducting an exhaustive number of experiments, we center the main analysis of this paper around PaLM62 B 3 . For reproducibility, we also conduct the experiments on a diverse set of publicly available models such as GPT-3 and C ODEX , presented in Appendix F. Our ﬁndings concur across the studied LLMs.
Relying on counterfactual examples could be misleading (Laugel et al., 2019; Slack et al., 2021). However, unlike ﬁne-tuned methods, one can readily identify and collect a set of prompts that are critical for the model to generate particular outputs. This makes counterfactual prompting a tractable approach to gain insights into the behavior of the model.
Speciﬁcally, counterfactual examples that exhibit consistent and systematic performance divergence can help us interpret their failure modes and strengths. We neither rely on the results that do not exhibit such characteristics, nor reject prompts that pose contradictory observations. We discuss additional limitations in Appendix A.1.
Visually inspecting the examples in Table 1, it seems intuitive that symbols are important for steering the model towards comprehending (or presumably reasoning about) a target task.
We form a set of counterfactual prompts in which the symbols are deliberately altered in distinct ways (Table 2 and Appendix-Tablesummarize the results).
We conducted two sets of experiments using counterfactual prompts: replacing the symbols with abstract values, and replacing them with out-of-distribution symbols.
Abstract symbols v C symb_abs p p q w .
We ﬁrst experiment with C symb_abs p p q , a modiﬁed variant of prompt p in which some or all the symbols are replaced with an abstract placeholder (See Table 2).
The results in Table 2 illustrate that the performance has little to no impact when the symbols are replaced with abstract placeholders (statistical signiﬁcance tests in Appendix F).
Out of distribution symbols v C symb_ood p p q w .
To test the operational utility of symbols, we design counterfactual prompts C symb_ood p
p q , in which the symbols are sampled from a distinct distribution compared to [Figure] [Figure] the symbols in the questions.
The operations include replacing integers in GSM-8 K prompt with fractions, sportsperson in S PORTS prompt with random names, and changing dates in D ATE to dates after 3000 AD.
The results (Table 2) reinforce our initial ﬁnding that the type of symbols is primarily immaterial to the model performance.
A notable exception is S PORTS , where including artiﬁcial names and activities closes the gap between D IRECT and C O T.
However, surprisingly, even with entirely artiﬁcial names and activities in the S PORTS dataset, the model performance is marginally better than direct.
Another interesting exception occurs in the S ORTING dataset.
Compared to vanilla C O T, using larger integers ( ě 10) considerably improves the task solve rate (60.6 % Ñ 80.0 % ).
We postulate that in this scenario, the modiﬁed thoughts more effectively inform the model about the underlying task of sorting numbers.
These results indicate that placeholders and abstract values can do merely as well at eliciting effective thoughts.
However, we ﬁnd that completely deleting the symbols (e.g., numbers and dates) nulliﬁes the gains carried over by C O
T. We show additional results in Appendix-Table 12.
Nature of generated answers.
We observe that the task solve rates are relatively unaffected for both [Equation] q and C symb_ood p p q .
To check if there is a systematic difference in the generated answers, we compute the Cohen’s κ agreement score (Cohen, 1960) between predictions generated by p (D IRECT ) and various counterfactual prompts.
The results (Appendix F.2) show that there is a moderate (>0.4) to substantial (>0.61) agreement between C O T and symbolic counterfactual prompts.
In contrast, the agreement between D IRECT and the counterfactual prompts is meager (<0.2).
These results reinforce our ﬁnding that the model may behave similarly regardless of the actual type/value of the symbols.
Analysis of employing in-distribution symbols in thoughts.
We also delve into the details of generated answers for GSM-8 K using p and C symb_ood p
p q .
As Table 2 delineates, C symb_ood p
p q prompts for GSM-8 K contain questions/thoughts with simple decimals.
We investigate whether such prompts help to improve the solve rate for questions with decimals preferentially.
Surprisingly, we observe that such prompts did not help with generating correct answers for questions with decimals (details in Appendix G).
3.1. A TTENTION A NALYSIS
While counterfactual prompting naturally treats the target model as a black box, we use attention as a proxy indicator to understand the inference mechanism of LLM for such prompts (details of calculation in Appendix E).
Figure 1 (additional results Figure 12) illustrates the average attention per token for randomly sampled questions across the studied datasets.
The near identical attention patterns between these variants of prompting indicate that the model presumably employs a similar inference mechanism in both cases.
4. R OLE OF P ATTERNS
We now use counterfactual prompts that modify patterns to study their role.
[Table]
No patterns v C pat_none p p q w .
We next gauge the sensitivity of the model performance to the existence of patterns.
For GSM-8 K , the distinction between text and patterns is clear—the equations in thoughts represent patterns, and everything else serves as text.
Therefore, we can construct C pat_none p p q for GSM-8 K by removing everything except equations.
For S PORTS dataset, the patterns are an implicit way of structuring the thought in the following form: “ person is a sport 1 player.
activity is part of sport 2 ”.
The answer is yes, if and only if sport 1 and sport 2 are the same.
In such cases, merely partially removing patterns is not sufﬁcient.
For example, just using “ person is a sport 1 player.
as a thought is not equivalent to a counterfactual example with no patterns.
This example resembles an experiment with a reduced pattern.
To circumvent this, we simulate C pat_none p p q for S PORTS by crafting a prompt in which several variations of thoughts are mixed.
The key insight is that if several different patterns are included in a single prompt, the induced “noise” from different examples creates a virtually equivalent variant of C pat_none p p q setup.
That is, we imitate C pat_none p p q by creating a hodgepodge of thought variants (See Table 63) without explicitly submitting to a particular pattern.
Identically, we conform to this terminology for D ATE and S ORTING .
The results in Table 3 reveal that C pat_none p
p q consistently underperforms C O T, relatively yielding similar performance as D IRECT .
This indicates that the existence of patterns in thoughts is crucial to the success of C O T. Note that in all datasets, C pat_none p
p q still outperforms D IRECT , hinting at the relevance of blending semantically correct statements in thoughts to improved model performance, which we study in Section 5.
Table 21 reveals that while the generated thoughts are structurally correct, the model can not establish an explicit connection between PERSON and SPORT , hence, attaining rather spurious outcomes.
This analysis underscores the importance of blending explicit patterns in thoughts, which corroborates with the least-tomost prompting (Zhou et al., 2022).
Pattern-only prompts v C pat_only p p q w .
Finally, we investigate counterfactual prompts in which we exclusively use patterns while wiping out the rest of the thoughts.
The results in Table 3 reveal that pattern-only prompts are futile and annul the gains of C O T.
Wrong pattern v C pat_wrong p p q w .
Examples of such wrong patterns are 5 + 3 = 7 for the GSM-8 K , and 2 days after 2/2/2022 is 2/6/2022 in D ATE .
In tasks like GSM-8 K (24.39 % ), D ATE (44.84 % ), and S ORTING (64.80 % ), the task solve rate is robust to speciﬁc mistakes.
For these tasks, the model performance
[Table]
with counterfactual prompts is on par with the vanilla C O T.
On the contrary, in the S PORTS dataset with C pat_wrong p p q , the model simply fails to form a connection between the ﬁrst and second clauses, leading to a substantially lower task solve rate (93.67 % ÞÑ 46.02 % ).
Appendix-Table 18 shows additional results for using wrong patterns in thoughts.
Qualitative Analysis of the Role of Patterns Counterfactual prompts with wrong patterns strike an interesting perspective in S PORTS and S ORTING .
In the S PORTS dataset, replacing the sport activity with a factually wrong one (e.g., “basketball” ÞÑ “soccer”) presumably disorients the model about the target task.
Therefore, the model can not elicit factual connections between the player and activity, hence, it fails to attain a correct answer.
Surprisingly, the accuracy of the model is slightly below random (50 % ), hinting at the possibility that the model “ actively ” generates incorrect answers.
We present additional analysis in Appendix H.
5. R OLE OF T EXT
Following the same methodology, we employ various tailored counterfactual prompts to retain the entire symbols and patterns, while altering the text or grammar.
We present a subset of results in Table 4, and defer additional analysis and results to Appendix I.
Text with discrepant entities v C text_diff_entities
p
p q w .
To investigate the role of entities, we modify the entities in the thoughts to be discrepant with the ones in the questions.
C text_diff_entities
p p q is exclusively relevant for the GSM-8 K and S PORTS datasets for which concrete entities exist: objects and people for GSM-8 K and players and activities for S PORTS .
For S PORTS , changing entities leads to model to disregard the prompt and achieve a similar performance as D IRECT (sample outputs in Table 23).
The task solve rate for GSM-8 K with C text_diff_entities p p q is noticeably better than D IRECT , while still failing to match the performance of C O T.
Analyzing the generated outputs for GSM-8 K (Table 24) reveals that since the substituted entities are semantically similar (e.g., toys ” cookies), the crafted prompts are still meaningful.
Interestingly, the model often generates an analogical thought (e.g., using “ sweaters ” in the thought when the question mentioned apples “ apples ”).
Establishing such connections is more challenging for S PORTS , as replacing the entities demolishes the factual correctness of the thoughts and possibly confuses the model.
Text with altered grammatical style v C text_style p p q w .
We experiment with Yodish (Kaminski, 2011; Walkden, 2012; Espindola, 2016; Pullum, 2005), a stylistic variation of English (IMDB, 1980; Wookieepedia, 2022) (less frequent on the web) 4 .
Yodish forms syntactically valid but non-colloquial English sen- tences (Honeycutt, 2019).
The sentences in Yodish typically use a “ XSV ” construct where “X” is a phrase that complements the verb “V” and “S” is a subject 5 .
In addition, the “XSV” style drifts the object (“X”) apart from the verb (“V”), which leads to a more challenging sentence structure.
We experiment with three variants of prompting by reconstructing (a) C text_yodathoughts p p q : thoughts, (b) C text_yodaquestions p p q : questions, and (c) C text_yoda
p p q : both questions and thoughts.
We ﬁnd that the C text_yodathoughts p p q (remaining results Appendix I) has moderate (GSM-8 K ) to drastically negative repercussion (S PORTS and D ATE ) on task solve rate.
We attribute this variation in the model performance to the relation between text and patterns.
In some tasks, texts and patterns are intertwined (S PORTS and D ATE ), and the answer is derived from the text, whereas in other tasks (GSM-8 K ), patterns are more explicit.
In the S PORTS dataset, the “XSV” Yodish style entails the model to place the sport activity ﬁrst.
For instance, the model is compelled to generate “ A baseball player Bryce Harper is ” instead of generating “ Bryce Harper is a baseball player ”.
We provide samples of generated thoughts in Tables 28 and 29.
We also experiment with thoughts of varying degrees of randomness, and ﬁnd that the amount of randomness directly affects the model performance (more random text is worse, detailed results in Appendix I).
6. S YMBIOSIS BETWEEN T EXT AND P ATTERNS
The analysis in the preceding sections suggests that patterns 6 and text form a symbiotic and harmonious relationship.
Text without patterns is insufﬁcient to instruct the model to the correct answer (Sectionsand 4).
Conversely, patterns without text cannot successfully recoup the model performance (Section 5).
This section attempts to deliver a qualitative analysis and tangible examples to elucidate this association.
We analyze samples in which C O T(p) yields the correct answer, whereas both C pat_none p p q and C pat_only p p q are wrong, and summarize the main ﬁndings below.
C O T is more effective in solving questions with more patterns.
Questions with more patterns require more intermediate steps to arrive at correct answers.
Thus, C O T is expected to help more in such cases.
For GSM-8 K , for example, we ﬁnd that the average number of entities in questions solved exclusively by
C O T(p) is 3.98 compared to the overall average of 3.62, a statistically signiﬁcant difference (difference of means t-test p = 0.04)— (cid:229) Q 1 - (cid:8) T v C O T w in Appendix-Table 30 shows an example.
Similarly, the presence of patterns not only leads to meaningful thoughts, but also assists the model in not neglecting the symbols.
Without patterns, the model has intrinsic tendency to prematurely arrive at a conclusion ( (cid:229) Q 2 , (cid:229) Q 6 - (cid:8) T v C pat_none p p q w in Appendix-Table 30), which is more probable to be inaccurate.
For D ATE , we observe several scenarios in which the C pat_only p p q prevents the model from generating requisite information (e.g., “ 10 days ago ” (cid:229) Q 4 in Appendix-Table 30) and complex patterns (e.g., “ day before yesterday ” (cid:229) Q 5 in Appendix-Table 30) out of questions.
Text helps in eliciting semantic commonsense knowledge.
For example, we observe that in (cid:229) Q 3 of
Appendix-Table 30 the model with C O T infuses commonsense knowledge about animals (e.g., “ 5 dogs have 4 legs each . . . 2 cats have 4 legs each . . .
10 birds have 2 legs each ”).
For D ATE , the model articulates the exact date for “ Christmas Eve ” in the generated thought ( (cid:229) Q 6 - (cid:8) T v C O T w in Appendix-Table 30), helping the ﬁnal answer.
7. CC O T: C ONCISE C HAIN O F T HOUGHT
Based on our initial ﬁndings, this section explores the idea of engineering effective prompts tailored to remove ineffectual tokens.
The beneﬁt of such tailored design is multi-fold: (1) “reducing noise”: reducing
[Table]
noise from prompts could potentially lessen the confusion for the model and lead to a higher task solve rate and (2) “potential energy savings”: the reduced number of tokens in the prompts instruct the model to generate less number of tokens per question.
Less number of generated tokens directly translates to reduced runtime and carbon footprint per inference (Strubell et al., 2019).
This is especially crucial because of the signiﬁcant compute cost of large language models.
Our proposed prompts, dubbed as CC O T, distill the essence of our key ﬁndings by supplying minimal requisite information to drive the model to factual conclusions.
Table 5 compares C O T and CC O T across different tasks and four variants of LLMs.
In all tasks, except GSM-8 K , we use identical examples in the prompt and only rephrased the thoughts to use less number of tokens.
For GSM-8 K , we could not ﬁnd a systematic mechanism to shorten the thoughts.
Instead, we randomly harvest questions from the training set whose thoughts are shorter than C O T. Overall, CC O T outperforms C O T while employing prompts with fewer tokens.
The task solve rate of CC O T remains relatively high as we scale the model to PaLM-540 B , highlighting the efﬁciency of CC O T. AppendixTable 36 compares the average number of input/output tokens between C O T and
CC O T.
On average, CC O T (our approach) reduces the number of input (1.39 ˆ ) and output tokens (1.58 ˆ ).
8. R ELATED W ORK
Appendix B provides a detailed discussion of the related work. This paper intersects with a growing body of work on (1) large language models (Brown et al., 2020; Chowdhery et al., 2022; Chen et al., 2021a), (2) few-shot prompting (Zhou et al., 2022; Kojima et al., 2022; Ling et al., 2017; Sun et al., 2019; Rajani et al., 2019; Shwartz et al., 2020; Madaan et al., 2021; Nye et al., 2021;
Gu et al., 2022; Wei et al., 2022; Wang et al., 2022b; Min et al., 2022; Kim et al., 2022; Reynolds & McDonell, 2021; Liu et al., 2021a; Poesia et al., 2021; Le Scao & Rush, 2021), and (3) machine learning explainability via counterfactual examples (Jacovi & Goldberg, 2020; Ribeiro et al., 2016; Lundberg & Lee, 2017; Liu et al., 2021b; Jain & Wallace, 2019;
Pruthi et al., 2020; Wiegreffe & Pinter, 2019; Mothilal et al., 2020; Stepin et al., 2021; Verma et al., 2020;
Poyiadzi et al., 2020; Goyal et al., 2019; Laugel et al., 2019; Slack et al., 2021).
This work differs from prior work, as we study the success/failures of LLM through counterfactual prompting and propose systematic methods to better understand the underpinnings of these methods.
9. C ONCLUSIONS
This work evaluates the capacity of C O T to elevate complex reasoning in three state-of-the-arts LLMs,
PaLM, GPT-3, and C ODEX .
We systematically assembled a series of controlled counterfactual experiments.
Our results show the initial inklings of connection between text, patterns, and reasoning in LLMs.
Our study indicates that the symbiosis of text and patterns bears more weight in the chain of thought reasoning process.
In addition, we assert that text is a channel to extract semantic patterns, unlocking the ability of these models to mold correct answers.
R EPRODUCIBILITY S TATEMENT
We take the following steps to enable the reproducibility of our work.
Controlling for randomness due to the order of examples.
We run each experiment with multiple random seeds to control for randomness due to the order of examples in the prompt.
We report the average and standard deviation of the results across all the random seeds.
Additionally, we conduct statistical signiﬁcance tests (McNemar’s test (McNemar, 1947)) to compare the results across different prompts.
Finally, we evaluate the agreement in output generated by different models using Cohen’s kappa ( κ ) metric.
Reproducing results.
Anonymized source code is available at https://anonymous.4open.science/r/
CoTTwoToTango-3106/ .
In addition, we have provided scripts for one-click reproduction of the results for the publicly available models in the paper.
Experiments with publicly available models.
We experiment with three different language models: PaLM,
GPT-3 ( text-davinci-002 ), and C ODEX ( code-davinci-002 ).
PaLM is not publicly available as of submission time, but the provided source code is compatible with OpenAI API v0.23.0, and can work with any OpenAI models.
Finally, C ODEX is free to use as of submission time that further helps with the reproducibility of the results.
Prompts and outputs.
All the prompts are included in the prompts/ directory.
In the anonymous repository, we have included the outputs from GPT-3 and C ODEX in the zip ﬁle outputs.zip .
Each output ﬁle follows a standard naming convention: task _ name _ model _ name _ sseed.jsonl .
Each line of the output ﬁle is a json with three ﬁelds: 1.
prompt + test question ( “question” ), 2.
generated answer ( “generated_answer” ), 3.
true answer ( “answer” ) .
A shortened output example is as follows: 1 { "question": "Q: Is the following sentence plausible? ’ Jonas
Valanciunas beat the buzzer. ’\nA: Jonas Valanciunas is a basketball player.
Beating the buzzer is part of basketball.
The answer is yes...
Q: Is the following sentence plausible? ’
Malcolm Brogdon banked the shot in.
’\nA:
Malcolm Brogdon is a basketball player.
Banking the shot in is part of basketball.
The answer is yes.\n\n\nQ: Is the following sentence plausible? ’
Sam Darnold passed the puck.
’\nA:
Sam Darnold is an American football player.
Passing the puck is part of hockey , not American football.
The answer is no.\n\n\nQ: Yes or no: Is the following sentence plausible? \"Javi
Martinez launched the desperation heave .\"\nA
:", "generated_answer": " Javi Martinez is a soccer player.
Launching a desperation heave is part of basketball , not soccer.
The answer is no.", "answer": "no", "is_correct":
E THICS S TATEMENT
Disseminating reasoning into machines has numerous beneﬁts and applications, from algorithmic reasoning (Li et al., 2022) to code generation (Chen et al., 2021b; Poesia et al., 2021) and formal veriﬁcation (Wu et al., 2022).
While this research does not directly enhance the reasoning capabilities of large language models, it identiﬁes several systematic behavioral patterns in the functioning of few-shot models.
Similar to any technological advances, this work has risks of detrimental societal impact.
However, anticipating potential future downsides of such methods is challenging.
More than ever, the research community’s utmost responsibility is to acknowledge these risks candidly and reﬂect on practices and strategies to prevent potential harm.
Environmental impact.
Training large language models devour a nontrivial amount of compute resources, a limiting factor for frequent training.
Few-shot prompting is an appealing solution for mitigating the unfavorable environmental impact of large language models by evading additional iterations of training and dataset collection.
Ours and similar studies may lead to more effective prompting techniques and bring technological innovation to the architecture of large language models, especially regarding their reasoning capabilities.
Therefore, we hope that the signiﬁcant compute used in this work can help promote positive environmental outcomes.
Finally, aligned with the credible concerns of the research community, we recognize the longer-term risk of Excellence in artiﬁcial intelligence, primarily when it boils down to human reasoning.
While distilling comparable human reasoning to machines offers many beneﬁts, undisciplined and uncontrolled progress in this area could be alarming, especially in the presence of bad actors.
Effective reasoning in machines as a result of our study, even though not directly, can lead to algorithmic advances that may facilitate bad actors in developing malicious software and systems with human-level capabilities.
We also want to acknowledge a large body of researchers that has greeted innovations in large language models and steady scaling of models with skepticism, questioning the connection between human reasoning and large language models (Han et al., 2022; Binz & Schulz, 2022).
Additionally, recent efforts have started exploring the relationship between the structure of contemporary language models and the human mind (Schrimpf et al., 2021; Tang & Ha, 2021; Whittington et al., 2021).
Despite their progress, the rationale behind C O T’s mimicking human reasoning, and any potential connection with linguistics is so far an uncharted territory.
Appendix Table of Contents A Extended Background
A.1 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
B Extended Related Work and Discussion
C Details on Studied Reasoning Tasks
D Computational Resources and Models
E Attention Analysis
E.1 Per-layer Attention Analysis . . .
E.2 Specialized Attention Heads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
F Results on 540 B and
F.1 Results on C ODEX , GPT-3, PaLM-540 B . . . . . . . . . . . . . . . . . . . . . . . . . .
F.2 Signiﬁcance tests for PaLM-62 B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
G Additional Experiments on Role of Symbols
G.1 Counterfactual Prompts for Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . .
G.2 Role of symbols: Output Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
G.3 Attention Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
G.4 Role of Dataset for Effective Chain of Thought . . . . . . . . . . . . . . . . . . . . . . .
G.5 Additional Experiments for S PORTS . . . . . . . . . . . . . . . . . . . . . . . . . . . .
[Table]
[Table] [Table]
I Additional Experiments on Role of Text
[Table] [Table] Text and Patterns: Additional Examples and Qualitative
[Table]
[Table] [Table]
[Table]
[Table]
[Table]
A. E XTENDED B ACKGROUND
Background on chain of thought .
This work broadly investigates the premise of in-context few-shot prompting in large language models (LLM).
In these methods, the input to the model is a prompt p consisting of k in-context examples in the form of x input ÞÑ x i , output ÞÑ
y i
y tuples 7 .
Each x x i , y i y alludes to the target task.
For example, in math solving problems Cobbe et al.
(2021), an input is math question ( If three apples were added to a basket which had two apples, how many apples are in the basket now? ), and the output supplies the answer ( 5 ).
Wei et al.
(2022) additionally supplement in-context few-shot prompting with Chain Of Thought (C O T) method, improving the performance of LLM in solving several reasoning tasks.
In particular, C O T additionally preﬁxes each output with a thought , creating triplets x x i , t i , y i y .
The “chain of thought” t i describes the intermediate steps and/or results required to derive the output y i from x i .
Therefore, the prompt is assembled in the form of p ” x x 1 ¨ t 1 ¨ y 1 y } x x 2 ¨ t 2 ¨ y 2 y } . . . x x k ¨ t k ¨ y k y , where “ ¨ ” and “ } ” are indicator symbols.
The role of ¨ is to separate elements of an example, whereas } indicates the boundary of an example.
The intuition behind chain of thought prompting is that catering the outputs/answers with intermediate steps/results present additional in-context information to the model (Ling et al., 2017;
Amini et al., 2019; Chen et al., 2021b; Cobbe et al., 2021; Nye et al., 2021).
This additional in-context information presumably improves accuracy in solving various reasoning tasks.
At inference time, C O T appends an unseen question ˆ x to the prompt p and supplies the extended prompt to a LLM.
The model completes the prompt to generate a relevant thought ˆ t and an answer ˆ y .
To assess the performance of LLM, C O T only compares the post-processed generated answer with the ground truth.
Gauging the correctness of the generated thought ˆ t is not straightforward because ground truth thoughts are unavailable.
Nonetheless, the generated thought can be further analyzed to infer the possible mechanisms, allowing an analogy with the human thought process, with which the model attains the answer.
Background on counterfactual explanation.
Counterfactual explanations seek to explain the behavior of a model by conducting “ what if ” analysis on examples for which the expected outputs of the model is known (Mothilal et al., 2020; Stepin et al., 2021; Verma et al., 2020; Poyiadzi et al., 2020; Goyal et al., 2019; Feder et al., 2021).
Speciﬁcally, let p x, y q be a tuple where x is the input to a model M that estimates an output distribution p p¨ | x q , and y „ p p¨ | x q .
Counterfactual explanations utilize variants C f p x, b, a q of the inputs that differ from the original input x in all except one feature f .
Here, b and a denote the before and after values of the feature f in x .
For instance, consider an image x of a camel with a brown background labeled correctly by a classiﬁer.
A counterfactual C bg p x, brown, green q example is an identical image with only a different background color, green, in this example.
By virtue of comparing p p¨ | x q with p p¨ | C bg p x, brown, green qq for a sufﬁciently large sample of images, one may infer certain facts about the classiﬁer, for example its reliance on the background color.
A.1. L IMITATIONS
While the counterfactual approach has its advantages, its assumptions induce limitations summarized below.
There is potential for uncharted and bafﬂing artifacts that the model could be exploiting, leading to potentially misleading observations.
For instance, symbols, patterns, text and the outcome may be spuriously correlated.
A classic example of such spurious correlation is confounding: the phenomenon of an observed correlation between X and Y induced by their common cause Z. Despite taking methodical and scientiﬁc measures, to the best of our knowledge, to ground our hypothesis in in-depth empirical analysis, the discrete and multiplicative nature of language understanding tasks implies that no such study can be completely thorough.
In addition, this work is limited to a subset of common tasks and datasets, including math (Cobbe et al., 2021), commonsense reasoning (BIG-bench Collaboration, 2022), and symbolic reasoning.
Our conclusions in this work may not apply to other reasoning and question-answering tasks, or even to other datasets from the same task category.
We experiment with large language models that are either not publicly available (e.g., PaLM) or are either available using API calls.
Existing documentation (Brown et al., 2020; Chen et al., 2021a) does not fully describe the details of these models, such as the pretraining corpus, model size, and model biases.
B. E XTENDED R ELATED W ORK AND D ISCUSSION
Broadly, this paper intersects with a growing body of work on prompting and large language model reasoning (Brown et al., 2020; Chowdhery et al., 2022).
Below, we review the most relevant work in these directions.
Exploring the role of examples in few-shot setup With the growing interest in few-shot prompting, several works have explored the role that in-context examples play in the success of few-shot prompting.
Notably, Min et al.
ﬁnd that label correctness is not crucial for the success of the models, and even random labels might lead to competitive performance.
Building on this work, Kim et al.
ﬁnd that the role of the correctness of the labels might be task-dependent.
Our ﬁndings concur with these methods on label correctness—for GSM-8 K , label correctness is not material, whereas it plays a larger role for S PORTS .
Surprisingly, we ﬁnd cases where wrong examples can improve performance by being better indicators of the end task.
Our work goes beyond label correctness explored by these methods and teases apart the role of placeholder symbols, patterns, and text in the success of few-shot models.
Finally, in addition to comparing the ﬁnal results (outcome), we also focus on the mechanism (attention patterns), allowing us to reveal instances where model reasoning is identical.
Our results also resonate with the work of Reynolds & McDonell (2021), who found that one of the key roles played by the prompt is to remind the model of the underlying task.
Razeghi et al.
ﬁnd that pre-training term frequencies can somewhat explain the success of few-shot methods.
In line with their work, our experiments on S PORTS also show that D IRECT prompting method is most suited for easy questions (involving personalities and activities found on the web).
Finally, Xie et al.
(2021) show that in-context learning enables a large model to infer a shared concept between the examples, possibly leading to better task understanding.
Our studies on the role of prompt, especially examples where wrong examples lead to better output (e.g., for S ORTING ), add more empirical evidence to this ﬁnding.
Further, we show that a symbiotic relationship between text and patterns allows a more efﬁcient inference of task instruction.
Least to most prompting.
Zhou et al.
(2022) help the model generate a chain of thought by ﬁrst asking the model to generate the sub-questions for the given problem.
Next, the model is asked to answer the subquestions, and ﬁnally, the sub-questions, along with sub-answers, are combined to generate the ﬁnal result.
This work is closely related to Kojima et al.
(2022), the latter distinguished by generating the rationale from a large language model directly.
We posit that Zhou et al.
(2022) derives its key strengths from its ability to generate useful sub-steps.
This resonates with our ﬁnding that the key contribution of CoT is the extraction of meaningful sub-steps.
Rationale generation as an intermediate step The idea of generating rationales as an intermediate output for reasoning and structured generation tasks has shown promising results for ﬁne-tuned models (Ling et al., 2017;
Sun et al., 2019; Rajani et al., 2019; Shwartz et al., 2020;
Madaan et al., 2021; Nye et al., 2021; Gu et al., 2022).
Recently, Wei et al.
(2022) proposed chain-of-thought prompting, which shows that few-shot setups can also be improved by making the model ﬁrst generate an understanding of the output.
As a natural extension to C O T, Wang et al.
(2022b) seek to improve C O T using over-generation using selfconsistency .
They sample multiple outputs, and take a plurality vote (i.e. most frequently generated answer) to arrive at the ﬁnal answer.
This general idea of enforcing the model outputs to be consistent has also been explored for symbolic-commonsense reasoning (Kassner et al., 2021).
The efﬁcacy of this approach is Q1.
What if we replace all the symbols in the prompt with abstract placeholders, can the required task still be discerned?
Q2.
What if the examples in the prompt were incorrect, will it affect the correctness of the outputs?
Q3.
What if we remove all patterns from the input, will C O T continue to be effective?
Q4.
What if the linguistic style of the prompt was different than that of the questions, will it hamper the performance?
corroborated by Wang et al.
(2022a), who report that taking multiple samples helps a model become robust to settings in a few-shot setup.
Our work looks at understanding the efﬁcacy of C O T in the standard setup of generating a single output per input.
Prompt selection.
Several works have recently explored the design of the prompt—a process often called “prompt engineering” (Le Scao & Rush, 2021; Liu et al., 2021c).
The methods include dynamically creating prompts based on the question (Liu et al., 2021a;
Rubin et al., 2021; Poesia et al., 2021), formatting the prompt as a list or questions (Mishra et al., 2021;
Rubin et al., 2021), improving order of examples in the prompt (Lu et al., 2022), and providing instructions in the task (Ouyang et al., 2022).
Unlike these techniques, C O T is relatively robust to minor changes in the prompt design.
Thus, the ﬁndings of our work might be more generally applicable.
Explaining model behavior using counterfactual prompts and attention.
As noted by Jacovi & Goldberg (2020), an explanation of a deep learning system typically serves two different purposes: i) plausibility, which aims to provide an interpretation of system outputs that is convincing for humans, and ii) faithfulness, which aims to capture the actual reasoning process of a model.
Our study requires both and uses different means to achieve them.
We utilize counterfactual prompts to interpret the system outputs to aid human understanding.
This is similar to using posthoc analysis tools (Ribeiro et al., 2016; Lundberg & Lee, 2017; Liu et al., 2021b), which also focus on analyzing outputs without concern for the details of the model.
To get a glimpse of the model’s inner workings, we leverage attention (Vaswani et al., 2017), a ubiquitous mechanism in NLP.
While the broader question on the utility of attention for posthoc analysis is still open (Jain & Wallace, 2019; Pruthi et al., 2020), there is some evidence to show that attention can act as an explanation (Wiegreffe & Pinter, 2019).
Finally, the utility of any explanation mechanism is closely tied to the users and application domain (Kaur et al., 2020; Burkart & Huber, 2021).
As our analysis shows, attention adds intuition and insights to the empirical ﬁndings.
Counterfactual explanations seek to explain the behavior of a model by performing a what if analysis on examples (Mothilal et al., 2020; Stepin et al., 2021; Verma et al., 2020; Poyiadzi et al., 2020; Goyal et al., 2019).
While counterfactuals can be misleading due to artifacts (e.g., see (Laugel et al., 2019; Slack et al., 2021)), they offer a tractable solution for probing large models like PaLM and GPT-3.
Notably, unlike ﬁnetuned methods, the most important examples for generating the model output are readily available.
Thus, counterfactual inputs that show a consistent and systematic change in the model performance are more likely to reﬂect the model’s behavior.
C. D ETAILS ON S TUDIED R EASONING T ASKS
In this work, we evaluate counterfactual prompting on the following reasoning tasks:
1. (cid:16) M ATHEMATICAL (cid:17) We experiment with GSM-8 K (Cobbe et al., 2021) (1319 samples).
The dataset contains math word problems geared toward an average middle-school curriculum.
2. (cid:16) C OMMONSENSE (cid:17)
We use date understanding (D ATE , 349 samples) and sports understanding (S PORTS , 980 samples) as representative tasks for commonsense reasoning, both derived from BIGbench Collaboration (2022).
3. (cid:16) S YMBOLIC (cid:17)
We experiment with sorting (S ORTING , 500 samples) a list of single-digit integers.
We do not associate explicit instruction (e.g., sort these numbers ) with the questions.
Instead, we frame the questions as a challenging setup in which the model should ﬁgure out the task and the requisite information to solve it.
D. C OMPUTATIONAL R ESOURCES AND M ODELS
In this work, we neither train any of the PaLM models, nor performs ﬁnetuning.
We solely perform inference on PaLM variants using TPU v4 (Jouppi et al., 2021; 2017).
For PaLM-62 B , we use 4 ˆ 4 ˆ 4 TPU v conﬁguration, whereas, for PaLM-540 B we use 4 ˆ 4 ˆ 16 mesh conﬁguration.
To account for the variation in results caused by the order of examples in the prompt, we conduct each experiment three times, each with different seeds, and report the average task solve rate.
Following Wei et al.
(2022), we evaluate each task using accuracy i.e. fraction of examples where the output matched the expected result.
Public large language models.
We use OpenAI API 8 to conduct experiments with GPT-3 ( text-davinci002 ) and C ODEX ( code-davinci-002 ).
[Table]
[Table] [Equation] Q’ T’
A’
E. A TTENTION A NALYSIS
Attention mechanism (Vaswani et al., 2017) is now ubiquitous in NLP.
While the broader question on the utility of attention for posthoc analysis is still open (Jain & Wallace, 2019; Pruthi et al., 2020), there is some evidence to show that attention can act as an explanation (Wiegreffe & Pinter, 2019).
Finally, the utility of any explanation mechanism is closely tied to the users and application domain (Kaur et al., 2020; Burkart & Huber, 2021), and as our analysis shows, attention provides concurring evidence that adds intuition and insights to the empirical ﬁndings of this work.
Note that while we conduct empirical experiments with PaLM, GPT-3, and C ODEX , we only conduct attention-related ablations with PaLM as the GPT-3 and C ODEX were only available to us via API.
Attention for autoregressive models.
Consider a sentence: my dog loved the toy .
Modern NLP methods divide each sentence into tokens, a decision dictated by the underlying tokenization library.
PaLM uses
SentencePiece (Kudo & Richardson, 2018) for tokenization.
For simplicity, we assume a tokenizer that divides the sentence into tokens based on the whitespace.
This yields the following list of tokens: [my, dog, loves, treats].
Let BOS be a special beginning of sequence token present in all sentences, and p θ be a language model with the parameters θ .
Decoder-only language models such as PaLM estimate the likelihood of a sequence such as my dog loved the toy using an autoregressive factorization popularly known as the chain-rule: p θ p BOS , my, dog, loves, treats q “ p θ p my | BOS q ˚ p θ p dog | BOS , my q ˚ p θ p loves | BOS , my, dog q ˚ p θ p treats | BOS , my, dog, loves q
Estimating these conditional probabilities (e.g., p θ p my | BOS q ) requires a stack of transformer layers, each containing an attention module.
Thus, this factorization also implies that tokens attend to the left (Figure 3), with a token w i at location i attending to all tokens w ă i .
Let w s be the source token (current input to the model).
The set of target tokens, or tokens that w s will attend to, thus are: w 0 , w 1 , . . . , w s ´ 1 .
PaLM-62 B has 64 layers, each containing the self-attention mechanism with 32 heads.
Focusing on a single layer and head, let a st be the attention score from w s to w t , where ř s ´t “ 0 a st “ 1 .
Analyzing the important components of a C O T prompt.
We leverage attention scores as an additional signal to help uncover the important components of a prompt.
To this end, we calculate the attention scores from the source tokens that are part of the Q’ , T’ , or A’ to the target prompt question Q i , thought T i , and answer A i (Figure 2).
Note that the same prompt is used for all the questions in the test set.
Thus across questions, the set of target tokens remains the same.
Our goal in attention analysis is to uncover important tokens and spans used by PaLM to solve a task.
Since the distribution of attention scores a s is typically long-tailed, recording the attention score between every pair of source-target tokens might lead to noise and spurious patterns (Nan et al., 2021).
To remedy this, we take inspiration from nucleus sampling (Holtzman et al., 2019) and set all values below the k th largest attention value to 0 (we use k “ 10 ).
Let Q 1 j be the j th question in the test set Q of the questions to be evaluated.
Recall that the same prompt is used for all the questions, and we calculate the attention scores from the source tokens (tokens in the inference question) to the target tokens (those in the prompt).
Let a st be the attention from token w s to w t .
We calculate the attention importance I t of a token w t in the prompt as the average max attention it has received across the set Q of inference questions. [Equation] [Figure] [Equation]
[Figure]
[Equation]
The spectrum plots show a comparison of I t for all tokens in the prompt for two different prompts: vanilla C O T prompt and C symb_abs
p p q prompt.
The pattern vs. text prompts group the target tokens by their type: the tokens that belong to a pattern vs. tokens belonging to the text.
The attention importance values are then shown.
The bos by layer plots investigate the total attention importance for the BOS token.
[Figure]
E.1. P ER - LAYER A TTENTION A NALYSIS
The main draft provides spectrum plots averaged over heads and layers.
Figure 10 shows the same question for three different datasets averaged across layers.
Figure 4, Figure 5, and Figure 6 provide the same plots, per layer.
We ﬁnd that the spectrum of I s values is identical between C O T(p) and C symb_abs p p q across layers, showing that averaging is not leading to spurious correlations.
E.2. S PECIALIZED A TTENTION H EADS
Fine-tuned models can be expected to learn attention patterns that facilitate solving a task.
Does the same hold for few-shot models?
To our knowledge, the question of attention in a few-shot setup has not been explored.
Surprisingly, we ﬁnd that the model consistently uses certain heads and layers for attending over [Figure] [Figure] certain semantic parts of the inputs.
We ﬁnd such specialized head-layer pairs manually [Figure] and plot the average I t for 100 questions for them in Figure 8.
The I t values show a clear tendency for the head to favor either past tense ( would, yesterday ) or future tense ( will ).
Analyzing a large-language model’s attention patterns in detail is an interesting future work.
Symbiosis in attention scores We have explored different semantic components of prompts, namely patterns (including symbols) and text.
A logical next question is whether patterns or text confer differential importance.
While importance can be measured via various approaches, we use attention scores as a reasonable proxy.
For GSM-8 K (where the distinction between patterns and text is clear), we calculate attention mass on patterns and text across several layers and average it over their attention heads.
Figure 10 compares these average scores, normalized between patterns and text.
Our ﬁndings show that the model pays approximately equal attention to both, indicating similar importance.
These results concur with our ﬁndings that text and patterns contribute equally to the success of C O T. [Figure] [Figure] [Table]
[Table]
[Figure] [Figure] thus, higher attention mass on bos may be interpreted as the model paying equal attention to the rest of the sequence, indicating an evolving global representation. Figure 15-a
Pattern
Text
C/A 0 0.56340131 0.43659869 1.
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
, the attention patterns closely match for vanilla and symbolic C O T, implying that few-shot models leverage patterns to a larger extent.
F. R ESULTS ON C ODEX , GPT-3, P A LM-540 B AND S TATISTICAL S IGNIFICANCE T EST F.1. R ESULTS ON C ODEX , GPT-3, P A LM-540 B
We show results from four models: C ODEX (Chen et al., 2021a), GPT-3 (Brown et al., 2020), and two variants of PaLM (Chowdhery et al., 2022) (PaLM-62 B and PaLM-540 B ).
Note that we could not get results on all variations of prompts for GPT-3 because of usage limits by OpenAI.
Such cases are indicated with a hyphen (-).
Similarly, due to the rate limitations, we experimented with two seeds for all variations on C ODEX and had to use a single seed for some variations.
The ﬁndings are shown in Table 8 (GSM-8 K ),
Table 9 (D ATE ), Table 10 (S PORTS ), and Table 11 (S ORTING ).
We ﬁnd that all the ﬁndings hold across models: correctness of patterns is immaterial, abstract and OOD symbols are still helpful, and the sensitivity to text is proportional to the degree of randomness.
Finally, CC O T matches or outperforms C O T despite being 20% shorter.
[Table]
(Table 38)
[Table]
q (Table 48)
[Table]
(Table 43)
[Table]
(Table 55)
[Table]
(Table 57)
[Table]
(Table 70)
[Table]
(Table 79)
[Table]
(Table 82)
[Table]
(Table 68)
[Table] [Table]
Table [Table] Table
[Table]
Table 42)
[Table]
Table
[Table]
Table 62)
[Table]
Table
[Table]
Table
[Table]
(Table 84)
[Table]
[Table]
[Table]
Table 50)
[Table]
(Table 45)
[Table]
[Table]
Table 63)
[Table]
[Table] Table
[Table]
Table 80)
[Table]
Table 83)
[Table]
[Table]
[Table]
Table 51)
[Table]
Table 44)
[Table]
Table 65)
[Table]
Table
[Table]
F.2. S IGNIFICANCE TESTS FOR P A LM-62 B
In this section
[Table]
we present detailed results for experiments on PaLM-62 B .
Each experiment was repeated thrice using three different values of the random seed.
We use McNemar’s test (McNemar, 1947) to calculate the statistical signiﬁcance of differences in the performance of a given Counterfactual prompt with C O T(p), and Cohen’s kappa (Cohen, 1960) to measure the degree of agreement between the outputs generated by a counterfactual prompt and C O T(p). [Table] [Table] [Table]
0.6655 C symb_ood_verbalized p p q (Table 87) 24.03 % 22.74 % 24.11 % 23.63 % 0.
[Table]
0.5193 C symb_ood_neg p p q (Table 52)
[Table] [Table] [Table] [Table]
C
[Table]
[Table]
_large p p q (Table 53)
[Table]
[Table]
p p q (Table 48)
[Table]
1935 C
[Table]
p q (Table 55)
[Table]
[Table]
[Table]
q (Table 57)
[Table]
[Table]
p p q (Table 68)
[Table] [Table] ă [Table]
C
[Table] _rand p p q
(Table 76) [Table] [Table] [Table]
ă
[Table]
C
[Table]
yoda p
p q (Table 73)
[Table]
[Table]
4056 C
[Table]
yodathoughts p p
q (Table 70) 24. [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
Table
[Table]
[Table]
[Table]
[Table]
(Table 45)
[Table]
Table
[Table]
Table
[Table]
Table
[Table]
Table 64)
[Table]
Table 63)
[Table]
[Table]
Table 71)
[Table]
Table
[Table]
Table
[Table]
(Table 80)
[Table]
Table 83)
[Table]
[Table]
[Table]
[Table]
Question: It is 4/19/1969 today.
What is the date 24 hours later in MM/DD/YYYY?
[Table]
[Table]
Thought: Today is 04/19/1969 .
24 hours later is one day after today, which would be 04/20/1969 .
[Table]
Thought: Today is DATE .
24 hours later is one day after today, which would be DATE .
[Table]
[Table]
Thought: Today is 04/30/3069 .
24 hours later is one day after today, which would be 04/31/3069 .
[Table]
Question: 7 , 8 , 4 , 1 , 2 , 9 , 3 , 6 ,
[Table]
Thought: 1 ă 2 ă 3 ă
[Table]
ă
[Table]
[Table]
Thought: ς < φ < γ < δ < ζ
[Table]
χ < (cid:15) <
π < υ
[Table]
[Table]
A DDITIONAL E XPERIMENTS G. A DDITIONAL E XPERIMENTS ON R OLE OF S YMBOLS
This section provides additional experiments and ablations omitted from the main text due to space constraints.
Table 16 shows the detailed results.
Appendix G.2 analyzes the generated outputs.
The attention plots are provided in this section at a higher resolution, and we also investigate how artifacts of datasets can affect performance.
G.1. C OUNTERFACTUAL P ROMPTS FOR S YMBOLS
Abstract symbols v C symb_abs p p q w .
We ﬁrst experiment with the role of symbols by creating C symb_abs p p q , a modiﬁed variant of prompt p in which some or all the symbols are replaced with an abstract placeholder, while preserving all other components of p .
An example of the corresponding modiﬁed thought for each task is shown in Table 16.
The results in Table 16 illustrate that the performance has little to no impact when the symbols are replaced with abstract placeholders.
Note that for the S PORTS dataset, we also experiment with changing sportsperson and sport activity, which mutates the baseline thoughts to vague and ungrammatical.
Our experiment shows that in this scenario, the task solve rates decreases to merely 52.96 % .
We summarize the outcome of additional experiments with C symb_abs p p q on the S PORTS dataset in Table 17.
The logical next question that emerges is the potential contributions of symbols, if any, and the relevance of their types to the model performance.
The results suggest that the presence of symbols and their locations in the text presumably inform the model on how , and from where to replicate them in the output.
Speciﬁcally, we attempt to answer the following research questions: “ Do symbols bear an operational utility?
Are the symbols potent in informing the model on how to perform a task?
Are either type or value of symbols pivotal to generate cogent answers?
We conduct an additional experiments with counterfactual prompting to answer these questions
Out of distribution symbols v C symb_ood p p q w .
To test the operational utility of symbols we design counterfactual prompts C symb_ood p
p q , in which the symbols are sampled from a distribution different from the distribution of symbols in question.
We experiment with out-of-distribution counterfactual prompts across the studied datasets as follows:
GSM-8 K : The symbols in the question are small integer ( ď 100), with „ 40% between 1-5 and 80% single digit integers (See Figure 13).
ÞÑ We replace numbers in the prompt with decimal numbers, large numbers, negative numbers, and fractions.
S PORTS : The symbols for each prompt represent a sportsperson and a sports activity.
ÞÑ Deﬁning out-ofdistribution prompts for this dataset is not straightforward and well-deﬁned.
To craft out-of-distribution counterfactual prompts for this dataset, we replace the sportsperson with a randomly generated name and change the activity to a non-sport one (e.g., “passing the soda”).
The intuition here is that this modiﬁcation should still be sufﬁciently instructive for the model to elicit structures from thoughts.
D ATE : The symbols generally present dates from the past, present, or near future.
ÞÑ We replace the symbols with dates from the distant future.
For example, 02/07/2022 is replaced with 02/07/3022.
The answers are accordingly adjusted.
S ORTING :
The numbers in the questions are positive single-digit integers and zero.
ÞÑ To create C symb_ood p
p q for this dataset, we simply include examples with more signiﬁcant integer numbers ( ě 10).
This setting appears to be similar to C symb_abs p p q
However,
note
that under this setting, the symbols are closer to the initial symbols in vanilla C O T.
For example, in the GSM-8 K and S ORTING datasets, the symbols used for vanilla C O T and C symb_ood
p p q both belong to the set of real numbers.
For these cases , the questions are intuitively more similar to the prompt, in contrast to C symb_abs p p q in which Greek symbols are employed in
the prompt.
The results summarized in Table 16—indicated with C symb_ood p
p q —reinforce our initial ﬁnding that the type of symbols is mostly immaterial to the model performance.
A notable exception is S PORTS , where including artiﬁcial names and activities closes the gap between D IRECT and C O T.
However, surprisingly, even with completely artiﬁcial names and activities in the S PORTS dataset, the model performance is marginally better than direct.
Another interesting exception occurs in the S ORTING dataset.
Compared to vanilla C O T, using larger integers ( ě 10) considerably improves the task solve rate (60.6 % Ñ 80.0 % ).
We postulate that in this scenario, the modiﬁed thoughts more effectively inform the model about the underlying task of sorting numbers.
These results indicate that placeholders and abstract values can do merely as well at eliciting effective thoughts.
However, the follow-up questions are whether the symbols are useful to any extent in solving a task and whether we can simply strike them out from thoughts without observing any repercussions for the model performance.
Evidently, in tasks like S PORTS and S ORTING , removing symbols (e.g., sport/activity type and person in S PORTS and numbers in S ORTING ) from thought is akin to D IRECT prompting.
That is, the remaining tokens in prompts after removing symbols are non-explanatory.
For example, removing numbers in the S ORTING dataset molds the thoughts into mere commas.
In other datasets like GSM-8 K and D ATE , deleting all the symbols (e.g., numbers and dates) nulliﬁes the gains carried over by C O T.
In this scenario, the solve rates drop to 11.97 (D IRECT = 9.6) and 29.95 (D IRECT = 31.6) in GSM-8 K and D ATE , respectively.
In summary, while the exact symbol types and values are presumably not critical to the model performance, without symbols, the model effectively neglects the semantic value of prompts.
G.2. R OLE OF SYMBOLS : O UTPUT A NALYSIS
Nature of generated answers.
We observe that the task solve rates are relatively unaffected for both C symb_abs p p q and C symb_ood p
p q .
In hindsight, it is not apparent whether systematic differences exist in the generated answers.
To quantify this, we compute the Cohen’s κ agreement score (Cohen, 1960) between the predictions generated by p and various counterfactual prompts.
Similarly, we compute this score between D IRECT and the same counterfactual prompts.
The results (Appendix F.2) show that there is a moderate (>0.4) to substantial (>0.61) agreement between C O T and symbolic counterfactual prompts across all datasets.
In contrast, the agreement between D IRECT and the counterfactual prompts is meager (<0.2) in all cases.
These results reinforce our ﬁnding that the model may behave similarly regardless of the actual type/value of the symbols.
Analysis of employing in-distribution symbols in thoughts.
We also delve into the details of generated answers for GSM-8 K using p and C symb_ood p
p q .
As Table 16 delineates, C symb_ood p
p q prompts for GSM-8 K contain questions/thoughts with simple decimals.
We investigate whether such prompts help to improve the solve rate for questions with decimals preferentially.
Surprisingly, we observe that such prompts did not revive the ability of the model to generate correct answers for questions with decimals.
Out of 1319 questions, we identify 124 questions with at least one decimal.
The performance breakdown of p vs. C symb_ood p
p q with decimals on the two class questions is as follows.
Either of the prompts solves 50 questions, out of which both prompts solved 26.
Out of 24 questions solved exclusively by either, p solved 18, whereas C symb_ood p
p q merely solved six.
Heeding the results, it is plausible to conjecture that the model was not able to leverage the presence of in-distribution symbols (decimal numbers) in the prompts to attain correct answers.
As a whole, these results reinforce our ﬁndings portrayed for the C symb_abs p
p q setup that the model performance is moderately insensitive to the exact value and type of symbols in the thoughts.
G.3.
A TTENTION A NALYSIS
The analysis heretofore empirically hints at the limited potential of the values/types of the symbols in steering the model to generate correct answers.
While counterfactual prompting naturally treats the target model as a black box, we are also keen to understand the inference mechanism of LLM for such prompts.
We use the attention mechanism as a proxy indicator to compare the inference process of such models across different inputs.
If our initial hypothesis about the limited role of symbols in generating effective patterns holds, we expect to observe similar attention patterns across different prompts.
Figure 12 illustrates the average attention per token for a randomly sampled question across the studied datasets.
The top and bottom heatmap bars for each dataset show the average attention scores for vanilla C O T and C symb_abs p p q , respectively.
The near identical attention patterns between these variants of prompting indicate that the model presumably employs a similar inference mechanism in both cases.
To glean further insights into the attention patterns, we glance at the mapped average attention scores to the model input in Figure 12.
Intriguingly, the model (a) GSM-8 K : p vs. C symb_abs p p q 0. [Figure] ATE
[Figure]
symb
[Figure]
p p
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
attends to similar tokens with relatively similar attention scores.
We expound on the attention calculation mechanism and provide additional results on attention scores in Appendix E.
No symbols ablations.
While symbols can be abstracted, and the patterns composing them are misleading, they cannot simply be deleted.
Removing all the symbols erases gains provided by C O T. We ﬁnd that the accuracy for D ATE drops to 29.94%, while for GSM-8 K , it drops to 11.98%.
G.4. R OLE OF D ATASET FOR E FFECTIVE C HAIN OF T HOUGHT
As shown in experiments of Wei et al. (2022) and Zhou et al.
(2022), the gains obtained by C O T are inconsistent and heavily depend on the dataset.
In this section, we highlight a few peculiarities of popular datasets that either allow C O T to be successful (GSM-8 K ) or ineffective ( CSQA ).
Role of small numbers for maths.
We ﬁnd that, by and large, the capabilities of language models at doing math are limited to simple operations between small numbers, as previously observed by Cobbe et al.
A simple ablation experiment reinforces this: We take a set of 25 randomly sampled questions on which C O T achieves perfect (100%) accuracy using a PaLM-62 B .
Then, we change each question to increase the magnitude of numbers in the question to large numbers like those in hundreds.
All the other factors, like text in the question, order of prompt, and the model are kept intact.
For example, a question Janet’s ducks layeggs per day.
She eats three for breakfast every morning and bakes mufﬁns for her friends every day with four.
She sells the remainder at the farmers’ market daily for $2 per fresh duck egg.
How much in dollars does she make every day at the farmers’ market? is replaced with Janet’s ducks lay 863 eggs per day.
She eats 97 for breakfast every morning and bakes mufﬁns for her friends every day with four seventy three.
She sells the remainder at the farmers’ market daily for $611 per fresh duck egg.
How much in dollars does she
[Table]
, β more computers were added .
So β * four = λ computers were added .
α + λ is π .
The answer is π .
Q : Michael had α golf balls .
On tuesday , he lost β golf balls .
On wednesday , he lost λ more .
How many golf balls did he have at the end of wednesday ?
A : Michael started with α golf balls .
After losing β on tuesday , he had α - β = π .
After losing λ more , he had π - λ = μ golf balls .
The answer is μ .
Q :
Olivia has $ α .
She bought five bagels for $ β each .
How much money does she have left ?
A : Olivia had α dollars .
5 bagels for β dollars each will be 5 x β = λ dollars .
So she has α - λ dollars left .
α - λ is π .
The answer is π .
[Figure] [Figure]
make every day at the farmers’ market?
We ﬁnd that as magnitude increases, the accuracy drops from 100% to 12%, a 10x loss in accuracy.
Even with PaLM-540 B , the largest available language model, the accuracy went down to 16%, showing that scale is not particularly helpful.
We then use a calculator to evaluate the ﬁnal outputs and ﬁnd that the accuracy still dwindles at 30%, a 3x reduction in performance.
What then explains the success of models like PaLM in solving math word problems?
We ﬁnd that part of the answer lies in the values of digits that are commonly found in these datasets, as shown in Figure 13.
[Figure]
[Table] Since language models are not really doing math [Table] do the numbers in the prompt even matter?
As results in Section 3 show: the answer is no.
Changing the distribution of numbers completely has no bearing.
G. . A DDITIONAL E XPERIMENTS FOR S PORTS
Abstracting out the hidden symbol.
Table 16 shows that replacing PERSON and ACTIVITY has negligible impact on the accuracy.
However, abstracting out the sports abstract drops the performance below D IRECT levels (Table 17).
What explains such a discrepancy?
Note that both PERSON and ACTIVITY are already present in the question [Table] and are copied in the thought.
Thus, replacing them even with abstract tokens does not harm the performance, as the extraction pattern is retained.
On the other hand, the SPORT is not present in the question
[Table]
and has to be extracted.
Consider a question: Is the following sentence plausible?
“Norman Powell committed a blocking foul.”
For abstracted sports prompts, the model generates Norman Powell is a SPORT1 player.
Committing a blocking foul is part of SPORT2.
So the answer is no.
Whereas for the abstract person or the activity prompt, the model correctly infers the pattern (i.e. copy the person or the activity, and generate the corresponding sport) and generates: Norman Powell is a basketball player.
Being a blocking foul is part of basketball.
So the answer is yes.
This example also clearly drives the notion of symbiosis: only generating the right pattern or only having the right text.
Both are required for effective C O T.
Cases where patterns help.
As empirical results show, combining patterns with language reveals their true potential.
However, our analysis uncovers interesting reasons for the superiority of patterns only, for the small number of cases.
Interestingly, we found a few cases where forcing the model to generate some text leads to errors.
Primarily, they were caused by the model trying to make faulty connections.
Detailed examples are located in Table 37.
H. A DDITIONAL E XPERIMENTS ON R OLE OF P ATTERNS IN C O T
This section provides a detailed description on identifying patterns, includes detailed results (Table 18), and presents qualitative analysis that was omitted from the main text for brevity.
Identifying patterns Intuitively, the formation and structure of patterns seem to be effective in deducing correct answers.
Patterns either composition of symbols (e.g., 20 - 12 = 8 ) or structure in prompt that reinforces task understanding (Section 2 (Table 1)).
To achieve this, we systematically construct counterfactual prompts for patterns.
However, identifying patterns is not always explicit, unlike symbols, and generally depends on the target task.
For example, in the D ATE dataset, it is not straightforward to tease apart the exact structure of patterns.
Nonetheless, for certain tasks like S PORTS , the structure of the patterns are apparent: all the examples involve inferring whether a sportsperson (e.g., Michael Jordan) and a sport activity (e.g., basketball) belong to the same sport, and further adhere to a common sentence structure.
In this particular scenario, we can isolate patterns as follows: “ person is a sport 1 player.
activity is part of sport 2 ”.
Here, sport 1 and sport 2 represent sports like basketball or baseball.
For a sentence to be plausible, sport 1 and sport 2 must reserve an identical value.
We postulate that the structured nature of the patterns enables C O T to achieve considerable gains.
To evaluate the role of patterns, we craft and study various counterfactual prompts as described below.
Wrong pattern v C pat_wrong p p q w .
The results in Section 3 show that the value and type of symbols are mostly immaterial to the model performance.
Since patterns are a superset of symbols, it is tempting to infer that replacing patterns with abstract patterns should be equally effective as abstracting symbols.
However, what if the patterns are concrete but incorrect or factually wrong?
Instances of incorrect patterns are 5 + 3 = 7 and 2 < 1 for the GSM-8 K and S ORTING datasets, respectively.
Factually wrong instances are Jamal Murray is a Baseball player in S PORTS and 2 days after 2/2/2022 is 2/6/2022 in D ATE .
Table 18 shows that in such cases, the efﬁcacy of task solve rates is contingent on the intrinsic purpose of patterns.
In tasks like GSM-8 K , D ATE , and S ORTING , the task solve rate is robust to speciﬁc mistakes.
For these tasks, the model performance with counterfactual prompts is on par with the vanilla C O T.
On the contrary, in the S PORTS dataset with C pat_wrong p p q , the model simply fails to form factual connections between the ﬁrst and second clauses, leading to substantially lower task solve rate (93.67 % ÞÑ 46.02 % ).
No patterns v C pat_none p p q w .
We next gauge the sensitivity of the model performance to the existence of patterns.
As mentioned before, the isolation of patterns in thoughts is dataset-dependent.
For GSM-8 K , the dichotomy between text and patterns is a clear cut—the equations in thoughts represent patterns, and everything else serves as text.
Therefore, experimenting with C pat_none p p q is trivial.
We can simply construct C pat_none p p q for GSM-8 K by removing everything except equations.
In the S PORTS dataset, the pattern is a speciﬁc channel in which the thought is structured.
More speciﬁcally, the patterns are the linguistic structure of the thoughts, which bear the following form: “ person is a sport 1 player.
activity is part of sport 2 ”.
The answer is yes, if and only if sport 1 and sport 2 are the same.
In such cases, merely partially removing patterns is not sufﬁcient.
For example, just using “ person is a sport 1 player.
as a thought is not equivalent to a counterfactual example with no patterns.
This example rather resembles an experiment with a reduced pattern.
To circumvent this, we simulate C pat_none p p q for S PORTS by crafting a prompt in which several variations of thoughts are intertwined.
The key insight is that if several different patterns are included in a single prompt, the induced “noise” from different examples creates a virtually equivalent variant of C pat_none p p q setup.
That is, we imitate C pat_none p p q by creating a hodgepodge of thought variants (See Table 63) without explicitly submitting to a particular pattern.
Identically, we conform to this terminology for D ATE and S ORTING .
The results in Table 18 reveal that C pat_none p
p q consistently underperforms C O T, relatively yielding similar performance as D IRECT .
This indicates that the existence of patterns in thoughts are crucial to the success
[Table]
of C O T. Note that in all datasets C pat_none p p q still outperforms D IRECT hinting at the relevance of blending semantically correct statements in thoughts to improved model performance, which we study in Section 5.
To glean additional insights about patterns [Table] we investigate the nature of results in S PORTS .
Table 21 reveals that while the generated thoughts are structurally correct, the model can not establish an explicit connection between PERSON and SPORT , hence, attaining rather spurious outcomes.
This analysis underscores the importance of blending explicit patterns in thoughts, a ﬁnding that corroborates with the least-to-most prompting (Zhou et al., 2022).
Similarly in D ATE , thoughts generally consists of two parts, an intermediate calculation followed by extracting the ﬁnal answer.
A prompt for which this consistent and explicit pattern is broken—for example by incorporating mixed examples in S PORTS (Table 63)—severely underperforms vanilla
C O T.
This analysis reinforces the importance of invigorating thoughts with explicit patterns.
Pattern-only prompts v C pat_only p p q w .
Finally, we investigate counterfactual prompts in which we exclusively use patterns while wiping out the rest of thoughts.
The results in Table 18 reveal that pattern-only prompts are futile and annul the gains of C O T.
In all datasets, the solve rate for pattern-only prompts is akin to the D IRECT performance.
Note that in S ORTING pattern-only counterfactual examples is analogous with the D IRECT method.
This is because in our deﬁnition (Table 1) both numbers and less than signs are part of pattern category.
This behavior hints at the importance of text in prompts that we explore in Section 5.
[Table]
O UTPUT A NALYSIS
Confusing model with incorrect thoughts Counterfactual prompts with wrong patterns strike an interesting perspective in S PORTS and S ORTING .
In the S PORTS dataset, replacing the sport activity with a factually wrong one (e
[Table]
g., “basketball” ÞÑ “soccer”) presumably disorients the model about the target task.
Therefore, the model can not elicit factual connections between the player and activity, hence, fails to attain a correct answer.
Surprisingly, the accuracy of the model is slightly below 50 % , compared to 72.1 % with the D IRECT method, hinting at the possibility that the model “ actively ” generates incorrect answers.
Nonetheless, the results so far indicate that C O T employs information in the prompt as a prop to guide the generation of thoughts.
Thus, as C pat_wrong p p q experiments on S PORTS show, it seems plausible to construct misleading prompts to entrap the model in generating deceptive outputs.
Table 20 presents a comparison between C O T and C pat_wrong p
p q when their generated answers are discordant.
Visually inspecting the generated thoughts, it is apparent that the model generates structurally correct thoughts—“ person is a x sports player y , activity is/not is part of x sport y ”—, though they could be factually amiss (ﬁrst three rows).
In six samples, the model fails to associate either the player or the activity with the right sport, but not both.
We also observe one sample, in which the model fails to attain proper associations for both clauses.
However, even in this case the model arrives at the correct answer because the ﬁrst and second clauses conﬂict (akin to a double negative).
Only in two samples, the model properly associates the player, sport, and activity.
Our ﬁndings, on balance, illustrate the model’s overall ineptitude for generating factual thoughts as well as its sporadic ability to arrive at the right answer.
On the contrary, in the S ORTING dataset, such a noise with C pat_wrong p p q signiﬁcantly beneﬁts the model, outperforming D IRECT method (46.0 % vs. 64.8 % ).
With vanilla C O T, the model often generates a simple sequence of integers, 1, 2, . . .
We attribute the occurrence of this behavior to the disorientation of the model in deducing the target task.
Therefore, instead of conducting a sorting operation, the model simply generates the simple sequence of integers, which is prevalent on the web.
Adding planned noise in prompts revive the ability of the model in grasping the nature of the target task (sorting).
Gaining more insights about the underlying workings of the model in these scenarios require additional in-depth analysis, which we defer to future work.
Strength in differences.
We dig into how blending a jumble of factual thought variants into prompts assists the model to fathom the underlying task.
We use the S ORTING dataset as the case study, mainly because variants of thoughts are better deﬁned.
We conceive three scenarios, thoughts with the sequence of numbers exclusively starting with either (a) “1” (Table 41), or (b) “2” (Table 66), or (c) a mixture of “1” and “2” (Table 67).
Table 19 illustrates that restricting the diversity of thoughts, as shown in scenarios (a) and (b), hinders the model’s ability to comprehend the underlying task and simply fails to generalize, leading to a meager overall task solve rate ( ď 40 % ).
On the contrary, constructing a mixture of thoughts, as shown in scenario (c), boosts the model performance (87.78 % ).
This improvement may be attributed to the more diverse and meaningful thoughts that better signal the underlying task to the model.
This concurs with the ﬁndings of Reynolds & McDonell (2021), who found that the one of the key roles played by the prompt is to remind the model of the underlying task.
In addition, these results suggest that LLMs may be merely pattern replicator, which resonates with the way our brain works Heilbron et al.
This section also contains additional examples depicting role of patterns in C O T (Table 20, Table 21).
Table 20 presents a comparison between C O T and C pat_wrong p
p q when their generated answers are discordant. Visually inspecting the generated thoughts, it is apparent that the model generates structurally correct thoughts—“ person is a x sports player y , activity is/not is part of x sport y ”—, though they could be factually amiss (ﬁrst three rows). In six samples, the model fails to associate either the player or the activity with the right sport, but not both. We also observe one sample, in which the model fails to attain proper associations for both clauses.
However, even in this case the model arrives at the correct answer because the ﬁrst and second clauses conﬂict (akin to a double negative).
Only in two samples, the model properly associates the player, sport, and activity.
Our ﬁndings, on balance, illustrate the model’s overall ineptitude for generating factual thoughts as well as its sporadic ability to arrive at the right answer. On the contrary, in the S ORTING dataset, such a noise with C pat_wrong p p q signiﬁcantly beneﬁts the model, outperforming D IRECT method (46.0 % vs. 64.8 % ). With vanilla C O T, the model often generates a simple sequence of integers, 1, 2, . . . We attribute the occurrence of this behavior to the disorientation of the model in deducing the target task.
Therefore, instead of conducting a sorting operation, the model simply generates the simple sequence of integers, which is prevalent on the web. Adding planned noise in prompts revive the ability of the model in grasping the nature of the target task (sorting). Gaining more insights about the underlying workings of the model in these scenarios require additional in-depth analysis, which we defer to future work.
[Table]
the answer is no. (cid:51) (cid:51) (cid:56)
[Table]
I. A DDITIONAL E XPERIMENTS ON R OLE OF T EXT
Table 22 presents detailed results, including ablations on various parts of the prompt.
These were omitted from the main text due to space constraints.
Following the same methodology as symbols and patterns, we employ various tailored counterfactual prompts to retain the entire symbols and patterns, while altering the text or grammar.
Note that the practicality of the counterfactual experiments for text is contingent on the nature of the target task.
In addition, because there is no text in S ORTING , we did not conduct counterfactual experiments in this dataset.
Tablesummarizes the results and shows sample thought for each counterfactual prompt.
Text with discrepant entities v C text_diff_entities
p
p q w .
We ﬁrst investigate the role of employing identical entities (e.g., names and objects) between a question and its associated thought.
To achieve this, we simply modify the entities in the thoughts to be discrepant with the ones in the questions.
C text_diff_entities
p p q is exclusively relevant for the GSM-8 K and S PORTS datasets for which concrete entities exist: objects and people for GSM-8 K and players and activities for S PORTS .
In the S PORTS dataset, we change the player name and the activity in the thought, while retaining the sport in the question.
Samples of generated outputs in Table 23 illustrate that in this scenario, the model appears to disregard the prompt, generating arbitrary thoughts and attaining a similar performance as D IRECT .
The task solve rate for GSM-8 K with C text_diff_entities p p q is noticeably better than D IRECT , while still fails to match the performance of C O
T. We delineate samples of generated outputs in Table 24 to glean more insights.
We postulate that the gains over D IRECT occur because of the following two reasons:
• Semantically meaningful thoughts.
Since the substituted entities are semantically similar (e.g., toys ” cookies), the crafted prompts are still meaningful.
Thus, the prompts are conducive to assisting the model in generating meaningful thoughts.
• The ability of the model to form abstractions
The model manages to disregard the exact value of an entity (possible using abstract embeddings) and maintain an analogically similar thought.
For example, the generated thought for Question: “ After Andrea saved some money, she then spent the rest of her money on an $11 sweater and gave her brother $ 4.
If she had $ 36 in the beginning, how much did Andrea save? ”
is Thought: “ There were 36 apples in the basket .
After saving some, she had 36 - 11 = 25.
Then she spent 25 - 4 = 21.
The answer is 21. ”.
Note that the factual mathematical reasoning and patterns in the generated thought.
In this example, the model correctly establishes connection between “ sweaters ” and “ apples ”.
This study hints at the ability of the model to form higher abstractions for entities and establish proper connections between similar entities.
Establishing such connections is more challenging for S PORTS , as replacing the entities demolishes the factual correctness of the thoughts and possibly confuses the model.
For this counterfactual experiment, we substitute each thought with a random thought, yet factual and semantically correct.
As Table 22 illustrate, employing random information in the thoughts, even though meaningful, has catastrophic outcomes in the GSM-8 K and S PORTS datasets.
For GSM-8 K , we attribute this deteriorated state to the model failing to generate meaningful thoughts, and instead arriving at irrelevant statements (Table 25).
Similarly, the S PORTS dataset with random thoughts underperforms the D IRECT method (58.01 % vs. 72.1 % ).
The output analysis (Table 26) depicts the model’s inability to generate consistent thoughts and form credible connections, which can be attributed to the lack of compatible patterns between questions and their associated thoughts.
Shufﬂed text Finally, we experiment to understand how disorienting the location of words in thoughts affects the model performance.
We create two different scenarios, when the words are shufﬂed within a thought ( C text_inter_shuf p p q ) or across thoughts ( C text_intra_shuf p p q ).
The results illustrate that the task solve rates are severely sunk in both scenarios, especially for text-dependent tasks such as S PORTS and D ATE .
Overall, C text_intra_shuf p p q outperforms C text_inter_shuf p
p q across all the tasks, especially in the GSM-8 K
[Table]
If
[Table]
Is the following sentence plausible
[Table]
perfect
[Table]
line
[Table]
Murray is
[Table]
basketball.
[Table]
is
[Table]
is a player Jamal
[Table]
basketball of
[Table]
Question: It is 4/19/1969 today.
What is the date 24 hours later in MM/DD/YYYY?
[Table]
Thought: Today is 04/19/1969.
24 hours later is one day after today, which would be 04/20/1969.
[Table]
[Table]
dataset (17.01 % vs. 10.84 % ).
We posit that this behavior in GSM-8 K is possibly attributed to the requisite of “relative ordering” between text and mathematical equations to form factually correct thoughts.
I.1. T EXT WITH A LTERED G RAMMATICAL S TYLE
We experiment with three variants of prompting by reconstructing (a) C text_yodathoughts p p q : thoughts, (b) C text_yodaquestions p p q : questions, and (c) C text_yoda
p p q : both questions and thoughts.
Table 22 illustrates an interesting divergence in the model performance across these scenarios.
The results show that changing just the questions ( C text_yodaquestions p p q ) has little to no impact on the model performance.
On the contrary, this variant even slightly boosts the model performance in D ATE .
This observation is explanatory when placed in context with the results on C text_yodathoughts p p q , which has moderate (GSM8 K ) to drastically negative repercussions (S PORTS and D ATE ) on the model performance.
We attribute this variation in the model performance to the relation between text and patterns.
That is, in some tasks, texts and patterns are intertwined (S PORTS and D ATE ), and the answer is derived from the text, whereas in other tasks (GSM-8 K ), patterns are more explicit.
We notice that for GSM-8 K (Table 27), the sentence structure is relatively simpler, enabling the model to generate Yodish sentences effectively.
For example, the model correctly generates “ 5 cookies, the pizza had ” instead of “ The pizza had 5 cookies ”.
In the S PORTS dataset, the “XSV” Yodish style entails the model to place the sport activity ﬁrst.
For instance, the model is compelled to generate “ Bryce Harper is a baseball player ” instead of generating “ A baseball player Bryce Harper is ”.
We provide samples of generated thoughts in Table 28.
For D ATE , the sentence structure of the thoughts is intrinsically complicated, because the thought requires explicitly spelling out the complex relationship between time and date.
Changing the sentence structure to “XSV”, the model further struggle to generate factual and meaningful thoughts (Table 29).
[Table]
[Table]
[Table]
The red car is 40% cheaper than the blue car.
The price of the blue car is $100.
How much do both cars cost?
[Table]
has 6 jars of sprin- kles in her pantry. Each jar of sprinkles can decorate 8 cupcakes. [Table] wants to bake enough cupcakes to use up all of her sprin- kles
If each pan holdscupcakes, how many pans worth of cupcakes should
she bake?
[Table]
sprin- kles.
[Table]
jars can decorate
[Table]
48 cupcakes.
[Table]
each
[Table]
, then 48 / [Table]  pans.
[Table]
answer
[Table]
[Table]
[Table]
Scoring in the shootout is part of hockey ( (cid:51) ). So the answer is yes. (cid:51) Yes.
[Table]
[Table]
Six of the pack- ets had eight pieces of colors each, and the other four packets had sixteen pieces of
[Table]
many colors of crayons did Beatrice buy in all?
[Table]
to a new school
[Table]
20 more friends than Lily.
If Lily made 50 friends, [Table] many friends do Lily and Amy have together?
[Table]
Question
[Table]
“Chris Paul ran out of bounds.”
Chris Paul is a basketball player.
Running out of bounds is part of basketball.
So the answer is yes.
(cid:56)
A basketball player
Chris Paul is.
Out of bounds is part of basketball running, not basketball.
So the answer is no. (cid:56) “Kyle Connor scored a freekick.”
Kyle Connor is a hockey player.
Scoring a freekick is part of soccer, not hockey.
So the answer is no.
(cid:51)
A hockey player Kyle Connor is.
A freekick is part of soccer scoring, not hockey.
So the answer is no.
(cid:51)
“Kyle Tucker launched a triple.”
Kyle Tucker is a baseball player.
Launching a triple is part of baseball.
So the answer is yes.
(cid:56)
A baseball player Kyle
Tucker is.
Part of basketball launching a triple is, not baseball.
So the answer is no.
(cid:51) “Bryce Harper hit the back shoulder fade.”
Bryce Harper is a baseball player.
Hitting the back shoulder fade is part of basketball, not baseball.
So the answer is no.
(cid:51)
A baseball player Bryce
Harper is.
Part of basketball hitting the back shoulder fade is, not baseball.
So the answer is no.
(cid:51) “Chris Godwin crossed the blue line in the Stanley Cup.”
Chris Godwin is a football player.
Crossing the blue line is part of hockey, not football.
So the answer is no.
(cid:51)
A football player Chris
Godwin is.
Part of hockey crossing the blue line is, not football.
So the answer is no.
(cid:51)
[Table]
today
[Table]
from
[Table]
was
[Table] So [Table] month
[Table]
The 29th, the last day of february is, Born on
[Table]
[Table]
[Table]
jane was.
[Table]
birthday, today is,
[Table]
ago
[Table] today was. [Table] Direct
CoT Concise CoT Abstract Symbols Pattern Only
Pattern Wrong
Text Only
Text Changed Style
Direct (a) GSM-8 K
Direct
CoT
Concise CoT
Abstract Symbols
Pattern Only
Pattern Wrong
Text Only
Text Changed Style
Direct (b) S PORTS
Direct
CoT
Concise CoT
Abstract Symbols
Pattern Only
Pattern Wrong
Text Only
Text Changed Style
Direct (c) D ATE
J. S YMBIOSIS BETWEEN T EXT AND P ATTERNS : A DDITIONAL E XAMPLES AND Q UALITATIVE A NALYSIS
In this section, we expand on additional details and qualitative analysis.
The analysis in the preceding sections suggests that patterns 9 and text form a symbiotic relationship.
This harmonious relationship enables C O T to thrive, as summarized in Figure 14.
Text without patterns is insufﬁcient to instruct the model to the right answer (Sections 3 and 4).
Conversely, patterns without text cannot successfully recoup the model performance (Section 5).
thoughts purposefully glue patterns with text, forming a symbiotic association.
Beyond the empirical results in the previous sections, this section attempts to deliver a qualitative analysis and tangible examples to elucidate this association.
J.1. C ONSTRUCTING E FFECTIVE I NTERMEDIATE T HOUGHTS
Heeding our preceding ﬁndings, this section underscores few concrete venues in which the symbiosis of patterns and text contribute to the construction of effective thoughts, consequently leading to the success of C O T. To enable a systematic analysis, we ﬁrst identify samples in which C O T(p) yields correct answer, whereas both C pat_none p p q and C pat_only p p q are wrong.
Analyzing these samples assist us in identifying probable systematic differences across these methods.
C O T is more effective in solving questions with more patterns.
In general, questions with more patterns require more intermediate steps to arrive at correct answers.
Thus, C O T is expected to help more for such cases.
We test this hypothesis by glancing into the GSM-8 K dataset.
The number of GSM-8 K questions that the model exclusively solve using C O T(p) is 140.
Note that these are the questions that neither C pat_none p
p q nor C pat_only p p q can solve.
We observe that this is roughly twice and four-times the number of questions that are exclusively solved by C pat_none p
p q (71) and C pat_only p p q (33), respectively.
Further, the average number of numerical entities in these questions is 3.98 as compared to the overall average of 3.62, a statistically signiﬁcant difference (difference of means t-test p = 0.04).
We attribute this signiﬁcant gain to the ability of C O T(p) in forming longer and more detailed intermediate steps.
These additional steps presumably steer the model to elicit meaningful structures/patterns from thoughts, arriving at a factual answer ( (cid:229) Q 1 - (cid:8) T v C O T w in Table 30).
As we mentioned, text glues patterns and language structures together.
Similarly, the presence of patterns not only leads to meaningful thoughts, but also assists the model not to neglect the symbols.
Without patterns, the model has intrinsic tendency to prematurely arrive at a conclusion ( (cid:229) Q 2 , (cid:229) Q 6 - (cid:8) T v C pat_none p p q w in Table 30), which is more probable to be inaccurate.
[Table]
The importance of intermediate steps in reviving C O T. One of the crucial intentions of thoughts is to resemble a practical intermediate solution process.
In addition, thoughts reiterate patterns and text that [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
for
[Table]
The concert was scheduled to be on 06/01/1943, but was delayed by one day to today. What is the date one year ago from today in MM/DD [Table]
One day after 06/01/1943 is 06/02/1943, so today is 06/02/ [Table] One year ago from today is 06/02/ [Table]
So
[Table]
is 06/02/
[Table]
presumably act as a beacon for the model to direct its attention (of various intensity) to particular tokens. To better understand the intricacy of repeated patterns and text in thought, we analyze the generated thoughts in the D ATE dataset.
We observe several scenarios in which the C pat_only p p q prevents the model from generating requisite information (e.g., “ 10 days ago ” (cid:229) Q 4 in Table 30) and complex patterns (e.g., “ day before yesterday ” (cid:229) Q 5 in Table 30) out of questions.
C pat_none p
p q exhibit a similar trend, without interspersed patterns in prompts, the model often arrives at a wrong outcome via semantically correct steps.
For example, (cid:229) Q 1 - (cid:8) T v C pat_none p p q w in Table 31 shows correct intermediate steps up until the very last calculation.
The last calculation— “44 times the weight of the Papillon, so 216 pounds” —however, is incorrect.
On the contrary, the model arrive at a factual and mathematically correct result when explicit patterns are employed (“ so the Mastiff weighed 44 ˆ 5 =pounds ”).
Interestingly, we observe an identical phenomenon occurs in the D ATE dataset.
Recall that for D ATE (Table 1), the patterns in the thoughts are (a) x calculation y ÞÑ a description of computation of the current state and (b) x output y ÞÑ a statement that repeats the requisite intermediate information from (a) to attain a correct answer.
Breaking such explicit pattern disorients the model from generating intermediate information, as in (cid:229) Q 5 - (cid:8) T v C pat_none p p q w in Table 31.
J.2. C OMMONSENSE E XTRACTION
This section compiles a set of ﬁndings related to commonsense extraction mechanism of C O T through analysis of the generated thoughts.
Text is a decisive component in eliciting semantic commonsense knowledge.
We conjecture that C O T unlocks a golden opportunity to bring forth supplementary commonsense knowledge from the question and generate correct intermediate steps.
Consequently, the generated commonsense knowledge assists the model to attain a factual answer.
For example, we observe that in (cid:229) Q 3 of Table 30 the model with C O T infuses commonsense knowledge about animals (e.g., “ 5 dogs have 4 legs each . . . 2 cats have 4 legs each . . .
10 birds have 2 legs each ”).
In addition, reviewing the generated thoughts in D ATE underscores the ability of the model to recast commonsense knowledge about dates into a coherent format in thoughts ( (cid:229) Q 6 - (cid:8) T v C O T w in Table 30).
In this example, the model articulates the exact date for “ Christmas Eve ” in the generated thought.
Evidently, the model conditions on “ Christmas Eve ” and “ Today is ” to form the exact date “ 12/24/1937 ”.
This is undeniably an arduous task for the D IRECT setup, as it warrants the creation of these two fragments of information in one step.
Extracting rare commonsense knowledge.
To better understand the ability of the model to extract rare commonsense knowledge, we resort to the number of Google search results, which we refer to as “Popularity Metric”, as a proxy to gauge the rarity of an entity.
Employing this metric is germane to PaLM’s training dataset, which is a web-based corpus (Chowdhery et al., 2022).
We use this metric in the S PORTS dataset because the model is required to reason about factual commonsense knowledge to arrive at the correct conclusions.
Table 32 shows a contingency table summarizing the average popularity metric of “ x Activity y ” across correct and incorrect conclusions by C O T and C pat_only p p q .
We observe that the average popularity metric of activities for which C O T exclusively yields correct answers is lower ( « 52 ˆ ) compared to the ones for which C
pat_only p p q exclusively lands correct answers.
Compared to the average popularity metric across the entire S PORTS dataset (399k), this is still « 6 ˆ lower.
We conjecture that the well-formed structure of thoughts in S PORTS —intertwined patterns and text— empower the model to extract commonsense knowledge about “ x Activity y ”, even when the activities are infrequently seen during training.
On the contrary, the lack of explanatory thoughts in C pat_only p p q (See Table 3, Table 18) disorient the model to arrive at the correct conclusion.
Therefore, in these cases, the model arrives at the correct conclusion only when the popularity of “ x Activity y ” is signiﬁcantly high (3,575 k).
[Table]
Establishing factually meaningful connections.
While generating correct clauses in thoughts is pivotal to answer a question, establishing factually meaningful connections between these clauses is the ﬁnal step to attain a correct answer.
Recall that the pattern structure for S PORTS is “ person is a sport 1 player.
activity is part of sport 2 ” (Section 4).
If sport 1 and sport 2 are concordant (discordant), the answer is yes (no).
Breaking this pattern, as in C pat_none p p q of Table 18, leads to a peculiar failure case.
While the model successfully generate correct clauses—the sport that is associated with the player and the corresponding activity—it fails to connect these clauses in a meaningful way to attain a correct yes/no answer.
For example: “ Robert Woods is a football player ( (cid:51) ).
Killing the powerplay is a part of hockey ( (cid:51) ).
So the answer is yes.
(cid:56) ”
This ﬁnding underscores another pivotal role that patterns play in prompts.
The patterns reinforce the mechanism of extracting relevant information followed by establishing factually meaningful connections between the generated clauses.
J.3. T EXT AND P ATTERNS : I T T AKES T WO TO T ANGO
Preceding sections empirically underscore the prominence of text and patterns and their intertwined pathways to generate correct answers.
We articulate our ﬁndings in the following.
Patterns are the hidden force that help generate meaningful text.
Without patterns the model is not purposefully prompted to generate intermediate meaningful text.
In addition, the presence of patterns reinforces how the model should form factual connections between different clauses in the intermediate text.
This intermediate text, which generally embody relevant information to the underlying task, drives the model to elicit knowledge and attain correct conclusions.
Text imbues patterns with knowledge and meaning.
Theoretically, patterns seem to be sufﬁcient in directing the model to solve a task.
However, patterns need explicit text to be effective in imparting the requisite information to the model about how to accomplish a task.
The role of text primarily is to elaborate the patterns and imbues them with knowledge and meaning.
The presence of text enables the model to elicit requisite information from thoughts and reach to a factual conclusion.
We empirically observe the crucial role of the text in assisting the model to solve a task, such as GSM-8 K (e.g., four-legged vs. two-legged animals), S PORTS (e.g., sports that is being talked about), and D ATE (e.g., Christmas Eve is on 24 th ). [Table] [Table] [Table]
[Table]
K. A DDITIONAL R ESULTS CC O T: C ONCISE C HAIN O F T HOUGHT K.1. CC O T P ROMPTS K.2. R EDUCTION IN L ENGTH BY CC O T
In this work [Table] we empirically show how the symbiosis between text and patterns unlocks the functional potential of C O T.
Relying on our preceding ﬁndings [Table] this section explores the idea of engineering effective prompts tailored to remove ineffectual tokens.
The beneﬁt of such tailored design is multi-fold:
Reducing noise.
Reducing noise from prompts could potentially reduce the confusion for the model and consequently lead to higher task solve rate.
Investigating the impact of noise (futile or misleading information) in prompts on the model performance is an interesting future work.
Potential energy savings.
Last but not the least, reduced number of tokens in the prompts instruct the model to generate less number of tokens per question.
Less number of generated tokens directly translate to reduced runtime and carbon footprint per inference.
This is especially crucial because of the inherently large compute cost associated with the large language models.
Our proposed prompts, dubbed as CC O T 10 , distill the essence of our hypothesis by supplying minimal requisite information to drive the model to factual conclusions.
Table 5 compares C O T and CC O T across different tasks across the four models (PaLM-62 B and PaLM-540 B ).
In all tasks, except GSM-8 K , we use identical examples in the prompt and only rephrased the thoughts to use less number of tokens.
For GSM-8 K , we could not ﬁnd a systematic mechanism to shorten the thoughts.
Instead, we randomly harvest questions from training set whose thoughts are shorter compared to C O T. Overall, CC O T outperform C O T while in general employing prompts with less number of tokens.
The task solve rate of CC O T remains relatively high as we scale the model to PaLM-540 B , highlighting the efﬁciency of CC O T.
The next logical question is to determine to what extend CC O T reduces the number of tokens.
Tablecompares the average number of input/output tokens between C O T and CC O
T.
On average, CC O T (our approach) reduces the number of input and output tokens by 1.39 ˆ and 1.58 ˆ , respectively.
This reduction in the number of tokens could potentially render as energy saving in the model inference, which we defer for future work.
To conclude, we highlight the importance of conducting theoretical and empirical studies to better understand the sensitivity of LLM performance to the number of tokens under few-shot setting.
We now present additional results and examples that were moved from the main text for brevity and clarity.
10 Concise Chain Of Thought
[Table]
Source Code 1: Python code to reproduce Sort dataset. from itertools import combinations import permutations np 4def GenerateCombinations
A
Returns subsequences of elements with length r from array A. """ return list(combinations(A, r)) 10def main
_): base_array
(10) with open ds: tsv : question answer join for
]) thought =
]) tsv_writer.writerow question, answer, thought]) I. C OMPLETE L IST OF C OUNTERFACTUAL P ROMPTS This section includes all the counterfactual prompts used in our experiments.
L. C OMPLETE L IST OF C OUNTERFACTUAL P ROMPTS
This section includes all the counterfactual prompts used in our experiments.
[Table]
were originally 9 computers
[Table]
For each of 4 days
[Table]
5 more computers were added.
So
[Table]
[Table]
computers were added
[Table]
[Table]
[Table]
There are 15 trees in the grove.
Grove workers will plant trees in the grove today.
After they are done, there will be 21 trees.
How many trees did the grove workers plant today
[Table]
There are 15 trees originally.
Then there were 21 trees after some more were planted.
So there must have been 21 -
[Table]
[Table]
[Table]
[Table]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Table]
[Table]
her sister had β .
If they ate λ , how many pieces do they have left in total?
[Table]
ÞÑ Originally, Leah had α chocolates.
Her sister had β .
So in total they had α + β
= π .
After eating λ , they had π -
[Table]
= µ
[Table]
Olivia has α .
She bought ﬁve bagels for β each.
How much money does she have left
[Table]
Olivia had
[Table]
5 bagels for β dollars each will be 5 x β = λ dollars.
So she has α - λ dollars left.
α - λ [Table] had [Table]
golf balls.
On Tuesday, he lost β golf balls.
On Wednesday, he lost λ more. How many golf balls did he have at the end of Wednesday [Table] ÞÑ Michael started with α golf balls.
After losing β on Tuesday, he had α - β = π . After losing λ more, he had π - λ = µ golf balls.
[Table] [Table]
[Equation]
Prompt
[Equation]
[Table]
[Table]
[Table]
NFC
[Table]
in is part of SPORT1. V A ÞÑ yes (cid:113)
Prompt(cid:229) Q
[Table]
Is the following sentence plausible? "
Draymond Green threw a touchdown."
(cid:8) T ÞÑ Draymond Green is an SPORT1 player.
Throwing a touchdown is part of SPORT3, not SPORT1. V A ÞÑ no
[Table]
nine
[Table]
and [Table] more computers were installed
[Table]
many computers are
[Table]
in [Table] [Table] originally
[Table]
computers
[Table]
For each of 4 days [Table] [Table] 75 more computers were added
[Table] So [Table]
[Table]
There are 15.3 trees in the grove.
Grove workers will plant trees in the grove today.
After they are done, there will be 16.5 trees.
How many trees did the grove workers plant
[Table]
were planted
[Table]
started with
[Table]
After losing 2.
[Table]
on [Table]
he had
[Table]
[Table]
he
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] removed
[Table]
So [Table] [Table] [Table]
There
[Table] [Table] in the grove. Grove workers will uproot trees in the grove today.
After they are done, there will be 10 trees [Table] many
[Table]
the grove workers uproot
[Table]
There are 15 trees originally. Then there were 10 trees after some were uprooted. So the number that is uprooted must have been 10, as 15 - 5 = [Table]
ÞÑ Michael started with 58 golf balls. After losing 57 on Tuesday, he had 58 - 57 = 1. After losing 2 more, he had 1 - 2 = -1 golf balls. So he lost 2 - 1 = 1 more balls.
[Table]
[Table] [Table] [Table] [Table]
Q ÞÑ Is the following sentence plausible?
“Draymond Green threw a touchdown.”
(cid:8) T
[Table]
is an basketball player.
Throwing a touchdown is part of football, not basketball.
[Table]
[Table]
Grove
[Table]
they
[Table]
more were
[Table]
[Table]
[Table]
[Table]
[Table] [Table]
ÞÑ
So the date 24 hours later will be 03/13/2002.
V A ÞÑ 03/13/
[Table]
[Table]
[Table]
[Table]
[Table] [Table] 1 , 2 , 4 , 6 , 7 , 8 ,(cid:113) Prompt(cid:229) Q ÞÑ 7 , 6 , 8 ,(cid:8) T ÞÑ 1 < 6 < 7 <
V A ÞÑ 1 , 6 , 7 ,
[Table]
[Table]
[Table]
Shawn
[Table]
ﬁve toys.
For Christmas, he got two toys each from his mom and dad.
How many toys does he have now?
[Table]
With 5 toys, Shawn started.
2 toys each from his mom and dad, if he got, then that is 4 more toys.
[Table]
had 20 lollipops.
He gave Denny some lollipops.
Now Jason has 12 lollipops.
How many lollipops did Jason give to Denny
[Table]
With 20 lollipops, Jason started.
12 after giving some to Denny, then he had.
So he gave Denny
[Table]
[Table]
Q ÞÑ There were nine computers in the server room.
Five more computers were installed each day, from Monday to Thursday.
How many computers are now in the server room?
[Table]
ÞÑ Originally 9 computers, there were.
For each of 4 days, added, 5 more computers were.
So 5 *= 20 computers were added.
9 + 20 is 29.
[Table]
There are 15 trees in the grove.
Grove workers will plant trees in the grove today.
After they are done, there will be 21 trees.
How many trees did the grove workers plant today?
(cid:8)
[Table]
ÞÑ 15 trees originally, there are.
21 trees after some more were planted, then there were.
Been {+
+} = 6, so there must have.
[Table]
(cid:229) Q ÞÑ Leah had 32 chocolates and her sister had 42.
If they ate 35, how many pieces do they have left in total?
(cid:8) T ÞÑ Originally,32 chocolates, Leah had.
42, her sister had.
32 + 42 = 74, so in total they had.
After eating 35,74 - 35 = 39, they had.
[Table]
(cid:229) Q ÞÑ Olivia has $23.
She bought ﬁve bagels for $3 each.
How much money does she have left?
[Table] ÞÑ 23 dollars, Olivia had.
5 x 3 = 15 dollars, 5 bagels for 3 dollars each will be.
23 - 15 dollars left, so she has.
23 - 15 is 8.
[Table]
Q [Table] Michael had 58 golf balls.
On Tuesday, he lost 23 golf balls.
On Wednesday, he lost 2 more.
How many golf balls did he have at the end of Wednesday?
[Table]
ÞÑ With 58 golf balls, Michael started.
After losing 23 on Tuesday,58 - 23 = 35, he had.
After losing 2 more,35 - 2 = 33 golf balls, he had. [Table]
[Table] A hockey [Table] Kyle Palmieri is
[Table]
Called for slashing is part of hockey being
[Table]
the following [Table] set the pick and roll
[Table]
An American football player Carson Wentz is. Part of basketball pick and roll is, not football [Table] Is the following sentence plausible?
“Malcolm Brogdon banked the shot in [Table] ÞÑ A basketball player Malcolm Brogdon is. In is part of basketball banking the shot.
[Table]
yes [Table] A ÞÑ no
[Table]
It
[Table]
today.
What is the date 24 hours later in MM/DD [Table] [Table] [Table]
1969, today is.
Later is one day after today, 24 hours, 04/20/1969, which would be.
[Table]
The concert was scheduled to be on 06/01/1943, but was delayed by one day to today.
What is the date 10 days ago in MM/DD/YYYY?
[Table]
06/02/1943, one day after 06/01/1943 is, 06/02/1943, so today is.
05/23/1943, 10 days before today is.
[Table]
The ﬁrst day of 2019 is a Tuesday, and today is the ﬁrst Monday of 2019.
What is the date today in MM/DD/YYYY?
[Table]
ÞÑ Tuesday, if the ﬁrst day of 2019 was, a Tuesday, then 01/01/2019 was.
The ﬁrst Monday, today is, six days later, would be.
01/07/2019, so today is.
[Table]
Q ÞÑ Jane was born on the last day of February in 2001.
Today is her 16-year-old birthday.
What is the date yesterday in MM/DD/YYYY?
[Table]
ÞÑ The 28th, the last day of February is, Born on 02/28/2001, so Jane was.
Her 16-year old birthday, today is,02/28/2017, so today is.
02/27/2017, so yesterday was.
[Table] [Table]
Q [Table] 2015 is coming in 36 hours.
What is the date one week from today
[Table]
ÞÑ Coming in 36 hours, if 2015 is, coming in 2 days, then it is.
12/30/2014, 2 days before 01/01/ is, 12/30/2014, so today is.
01/05/2015, so one week from today will be.
[Table]
Jane thought today is 3/11/2002, but today is in fact Mar 12, which is 1 day later. What is the date 24 hours later in MM/DD/YYYY?
[Table]
ÞÑ 03/12/2002, today is. Later will be 03/13/2002, so the date 24 hours.
[Table]
[Table]
had. Denny some lollipops, he gave. 12 lollipops, now Jason has. Jason give
[Table]
Denny, how many
[Table]
did
[Table]
With 20 lollipops, Jason started.
12 after giving some to Denny, then he had. So he gave Denny
[Table]
Nine computers in the server room, there were.
Installed each day, ﬁve more computers were, from Monday to Thursday.
Now in the server room, how many computers are?
[Table]
ÞÑ Originally 9 computers, there were. For each of 4 days, added, 5 more computers were. So 5 *= 20 computers were added. 9 + 20 is
[Table]
(cid:229)
[Table]
ÞÑ 15 trees in the grove, there are.
Grove workers will plant trees in the grove today.
Done, after they are,21 trees, there will be.
The grove workers plant today, how many trees did?
(cid:8)
[Table]
ÞÑ 15 trees originally, there are.
21 trees after some more were planted, then there were.
Been {+
+} = 6, so there must have.
V A ÞÑ(cid:113)
Prompt(cid:229) Q ÞÑ 32 chocolates and her sister had 42, Leah had.
35, if they ate, they have left in total, how many pieces do?
(cid:8) T ÞÑ Originally, 32 chocolates, Leah had.
42, her sister had.
32 + 42 = 74, so in total they had.
After eating 35,74 - 35 = 39, they had.
V A ÞÑ(cid:113) Prompt(cid:229) Q ÞÑ $23, Olivia has.
Five bagels for $3 each, she bought.
She have left, how much money does? (cid:8) T ÞÑ 23 dollars, Olivia had.
5 x 3 = 15 dollars, 5 bagels for 3 dollars each will be.
23 - 15 dollars left, so she has.
23 - 15 is 8.
V A ÞÑ(cid:113) Prompt(cid:229) Q ÞÑ 58 golf balls, Michael had.
On Tuesday,23 golf balls, he lost.
On Wednesday,2 more, he lost.
He have at the end of Wednesday, how many golf balls did? (cid:8) T ÞÑ With 58 golf balls, Michael started.
After losing 23 on Tuesday,58 - 23 = 35, he had.
After losing 2 more,35 - 2 = 33 golf balls, he had.
V A ÞÑ
[Table]
American
[Table]
is.
Part of hockey passing the puck is, not American football
[Table]
was
[Table]
A hockey player Kyle Palmieri is
[Table]
Called for slashing is part of hockey being
[Table]
Is the following sentence
[Table]
pick and roll Carson Wentz set
[Table]
ÞÑ An American football player Carson Wentz is.
Part of basketball pick and roll is, not football
[Table]
is the following sentence plausible?
In “Malcolm Brogdon banked the shot.”
[Table]
ÞÑ A basketball player Malcolm Brogdon is.
In is part of basketball banking the shot.
[Table]
yes
[Table]
ÞÑ is the following sentence plausible [Table]
A touchdown “Draymond Green threw.”
(cid:8) T ÞÑ An basketball player Draymond Green is.
A touchdown is part of football throwing, not basketball. no
[Table] [Table]
[Table] [Table]
[Table]
originally [Table]
For
[Table]
were
[Table]
were
[Table]
There are 15 trees in the grove
[Table]
Grove workers will plant trees in the grove today.
After they are done, there will be 21 trees [Table] many trees [Table]
the grove workers
[Table]
There 15 originally trees
[Table]
planted were some 21 more Then after there trees were.
must So there been
[Table]
[Table]
[Table]
[Table]
[Table]
was perfect
[Table] line [Table] ÞÑ perfect player.
basketball. a Murray part basketball is of Being Jamal is the from line [Table] Q
[Table]
Is the following sentence plausible?
“Joao Moutinho caught the screen pass in the NFC championship.” [Table] American soccer. Moutinho not soccer Joao is part player.
NFC The football, a championship is of
[Table]
the
[Table]
Beating is
[Table]
is
[Table]
basketball
[Table]
Is the following sentence
[Table]
“Sam Darnold passed the puck.”
[Table]
ÞÑ football is puck the hockey, player.
not football.
Darnold part a American of American Passing
[Table]
Is the following sentence plausible?
“Kyle Palmieri was called for slashing.”
(cid:8)
[Table]
ÞÑ hockey.
of Palmieri slashing Kyle Being a player.
called part hockey is is for
[Table]
ÞÑ Is the following sentence plausible?
“Carson Wentz set the pick and roll.”
(cid:8) [Table] ÞÑ is football not football. Carson Pick roll basketball, and part Wentz American an of player.
is
[Table]
Is the following sentence plausible? “Malcolm Brogdon banked the shot in.”
[Table]
ÞÑ player. is of a in Brogdon basketball Banking shot the basketball.
part Malcolm is
V A ÞÑ yes (cid:113) Prompt(cid:229)
Q ÞÑ Is the following sentence plausible?
“Draymond Green threw a touchdown.”
(cid:8) T ÞÑ of an a is touchdown football, Green basketball Draymond basketball. not player. is part Throwing V A ÞÑ
no
[Table] [Table] today [Table]
The
[Table]
was scheduled to be on 06/01/1943, but was delayed by one day to today.
What is the date
[Table]
days ago
[Table]
[Table]
The ﬁrst day of 2019 is a Tuesday, and today is the ﬁrst Monday of 2019.
What is the date today in MM
[Table]
ÞÑ 01/01/2019 today Monday, would Tuesday, 2019 ﬁrst is Tuesday.
day of later.
then So was 01/07/2019.
ﬁrst be days Today the a six was is If the
[Table]
Q ÞÑ Jane was born on the last day of February in 2001.
Today is her 16-year-old birthday.
What is the date yesterday in MM/DD/YYYY?
[Table]
ÞÑ last was her 16-year is 02/28/2001.
So of so 02/27/2017.
is birthday, is The February on yesterday was old today Today 02/28/2017.
so day born 28th, the Jane
[Table]
[Table]
2015 is coming in 36 hours.
What is the date one week from today in MM/DD/YYYY?
[Table]
ÞÑ in 2015 one hours, 01/01/2015 36 so it days.
today 12/30/2014, then 2 week 2 is 12/30/2014.
is be days in is 01/05/2015.
So from today coming coming before is will If
V A ÞÑ
[Table]
[Table]
Q ÞÑ Jane thought today is 3/11/2002, but today is in fact Mar 12, which is 1 day later.
What is the date 24 hours later in MM/DD/YYYY?
[Table]
ÞÑ later
So date Today 03/13/2002.
hours 24 is 03/12/2002.
the be will
[Table]
[Table]
[Table]
[Table]
twenty
[Table]
some
[Table]
were
[Table]
had thirty two chocolates and her sister had 42.
If they ate thirty ﬁve,how many pieces do they have left in total?
[Table]
T
[Table]
Originally, Leah had thirty two chocolates.
Her sister had 42.
So in total they had thirty two + forty two = 74.
After eating thirty ﬁve, they had seventy four - thirty ﬁve = 39.
[Table]
Olivia has twenty three.
She bought ﬁve bagels for three dollars each.
How much money does she have left?
[Table]
ÞÑ Olivia had twenty three dollars.
ﬁve bagels for three dollars each will be ﬁve x three = ﬁfteen dollars.
So she has twenty three - ﬁfteen dollars left.
twenty three - ﬁfteen is 8.
[Table]
Michael had ﬁfty eight golf balls.
On Tuesday, he lost twenty three golf balls.
On Wednesday, he lost two more.
How many golf balls did he have at the end of Wednesday?
[Table]
ÞÑ Michael started with ﬁfty eight golf balls.
After losing twenty three on Tuesday, he had ﬁfty eight - twenty three = 3ﬁve.
After losing two more, he had thirty ﬁve - two = thirty three golf balls.
[Table]
[Table]
SPORT1.
[Table]
PERSON
an SPORT
V A ÞÑ
no
[Table]
Being
[Table]
the
[Table]
is part of basketball.
[Table]
is a basketball player
[Table]
the following sentence
[Table]
[Table]
Joao Moutinho caught the screen pass in the NFC championship
[Table]
The NFC championship is part of American football, not soccer.
Joao Moutinho is a soccer player
[Table]
part
[Table]
basketball.
Jonas Valanciunas is a basketball player
[Table]
Sam Darnold passed the puck
[Table]
part of hockey, not American football.
Sam Darnold is a American football
[Table]
was
[Table]
for
[Table]
ÞÑ Being called for slashing is part of hockey.
Kyle Palmieri is a hockey player
[Table]
the
[Table]
and
[Table]
and roll is part of basketball, not football.
Carson Wentz is an American football player
[Table]
Is the
[Table]
[Table]
Malcolm Brogdon banked the shot in
[Table]
ÞÑ Banking the shot in is part of basketball.
Malcolm Brogdon is a basketball player.
[Table]