Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse
Authors: Ryan Liu
Jiayi Geng
Addison J. Wu
Ilia Sucholutsky
Tania Lombrozo
Thomas L. Griffiths
Venue: ICML 2025 poster (2025)
Keywords: chain of thought
psychology
cognitive science
Primary area: applications->neuroscience_cognitive_science
TL;DR: We compare human verbal thinking with model chain-of-thought.

Abstract: Chain-of-thought (CoT) prompting has become a widely used strategy for improving large language and multimodal model performance. 
However, it is still an open question under which settings CoT systematically reduces performance. In this paper, we seek to identify the characteristics of tasks where CoT reduces performance by drawing inspiration from cognitive psychology, focusing on six representative tasks from the psychological literature where deliberation hurts performance in humans. In three of these tasks, state-of-the-art models exhibit significant performance drop-offs with CoT (up to 36.3\% absolute accuracy for OpenAI o1-preview compared to GPT-4o), while in others, CoT effects are mixed, with positive, neutral, and negative changes. While models and humans do not exhibit perfectly parallel cognitive processes, considering cases where thinking has negative consequences for humans helps identify settings where it negatively impacts models. By connecting the literature on human verbal thinking and deliberation with evaluations of CoT, we offer a perspective for understanding the impact of inference-time reasoning.

Mind Your Step (by Step): on Tasks where Thinking Makes Humans Worse
Chain-of-Thought can Reduce Performance
Ryan Liu"! Jiayi Geng"! Addison J. Wu
Ilia Sucholutsky
Tania Lombrozo 3 Thomas L. Griffiths 13
Abstract
formance of models in many tasks, particularly those in- volving symbolic reasoning (Sprague et al., 2025). More generally, inference-time reasoning has become default component of the newest LLMs and LMMs such as Ope- nAl's o-series models (OpenAI, 2024a) and Claude's web interface and thinking models (Anthropic, 2025). However, there also exist cases where CoT decreases performance, but the literature has not identified any patterns as to when this happens. As inference-time reasoning becomes the new norm in deployed frontier models, is imperative under- stand and predict when CoT has negative effect on model performance.
Chain-of-thougee (CoT) prompting has become a widely used strategy for improving large language and multimodal model performance. However, it is still an open question under which settings CoT systematically reduces performance. In this pa- per, we seek to identify the characteristics of tasks where CoT reduces performance by drawing in- spiration from cognitive psychology, focusing on six representative tasks from the psychological literature where deliberation hurts performance in humans. In three of these tasks, state-of-the-ar models exhibit significant performance drop-offs with CoT (up to 36.3% absolute accuracy for Ope- nAI ol -p-reeveww compared GPT-4o), while in others, CoT effects are mixed, with positive, neu- tral, and negative changes. While models and humans do not exhibit perfectly parallel cognitive processes, considering cases where thinking has negative consequences for humans helps identify settings where negatively impacts models. By connecting the literature on human verbal think- ing and deliberation with evaluations of CoT, we offer perspective for understanding the impact of inference-ting reasoning.
A key challenge for determining the limits of CoT is the sheer variety of tasks for which LLMs and LMMs are used. While the machine learning community has dedicated great efforts towards developing a large set of benchmarks for these models (c.g., Hendrycks et al., 2021; Suzgun et al., 2023), applications of models extend beyond benchmarks to diverse contexts and variations of tasks that could all potentially affect performance. Exploring this vast space to identify settings where CoT has negative effects is a daunt- ing problem. This motivates the need to develop heuristics to help us identify risky cases that could pose challenges for inference-time reasoning.
In this paper, explore the effectiveness of heuristic formed by drawing a parallel between CoT reasoning and hu- mans engaging in verbal thought (Lombrozo, 2024). Specifi- cally, we explore whether tasks for which thinking decreases human performance are also tasks where CoT harms model performance. This heuristic is motivated by the idea that in some cases, the tasks themselves, in conjunction with shared traits between humans and models, result in verbal thinking negatively impacting performance. However, models and humans also have different capabilities and consequently different constraints for some tasks (Griffiths, 2020; Shiffrin & Mitchell, 2023; McCoy et al., 2024). For example, LLMs have context lengths that exceed human memory limitations. Thus, we do not expect our heuristic to predict model per- formance perfectly, but rather allow us quickly identify cases where CoT has significant negative impact.
1. Introduction
Chain-of-thought (Wei et al., 2022; Nye et al., 2021) is widely used technique for prompting large language and multimodal models (L.MMs and LMMs), instructing models to "think step-by-step" or providing other structure that should be incorporated into the response. Large meta- studies have shown that this technique improves the per-
'qqual contribution 'Department of Computer Science, Prince- ton University, Princeton, NJ. USA 'NYU Center for Data Sci- ence, New York University, New York, NY, USA 'Department of Psychology, Princeton University, Princeton, NJ, USA. Corre- spondence to: Ryan Liu <jiayig @ princeton.edu>.
edu>, Jiayi Geng
To explore our approach, we first draw on the psychology literature for a collection of tasks where verbal thinking hurts human performance (Schooler & Engstler-Schooler,
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Mind Your Step (by Step)
ISL
Does following string also follow the same set of rules?
NLI
If you press triguer, then always the case that bullet fired it is not the case that bullet is fired
TUMLL, "FDOMOFC YPOMOUC
DZZCUUCC X
PZABQC
statements be true same time?
FR
SI
X
WM
CDE
Secarndil kurrig
Encpptinn
B
B
Coke Veow
Gommanky-
Lighing
Lightng
Spors-
Iucateb lanting
Spice
Neggloohood
Kiche-
Trangertiooo
Pirce
Edccution
Cardes+
Figure 1. Tasks evaluated for reductions in performance from COT. Left: Implicit Statistical Learning (ISL): Classifying if strings were generated by an artificial grammar. Face Recognition (FR): Recognition of a face from a set that shares similar descriptions. Classifying Data with Exceptions (CDE): Learning labels in the presence of exceptions. Right: Natural Language Inference (NLJ): Recognizing a logical inconsistency. Spatial intuitions (SI): Tilting water glasses. Working Memory (WM): Aggregating features for decision. Humans show reductions in performance when engaging in verbal thinking in all tasks, and LLMs and LMMs show similar effects in the first three.
1990; Dijksterhuis, 2004; Van den Bos & Poletick, 2008, inter alia). We then organized these into six prominent task archetypes, and selected the most representative study from each (see Figure 1). We adapted these tasks to properly evaluate LLMs and LMMs and measured performance with and without COT. In our experiments, we found large perfor- mance decreases from CoT in three archetypes: tasks that involve implicit statistical learning, tasks where language is ill-suited to represent stimuli, and tasks that involve learning labels that contain exceptions to generalizable rules. For the three other archetypes, we did not see consistent decreases in performance from COT. For these, we suggest explana- tions for why CoT does not decrease performance based on meaningful differences between humans and models.
stead mixed collection including positive and neutral in- stances. For these tasks, we suggest plausible explanations for why models do not perform worse while humans do. When L.MMs had access to relevant priors that humans lacked, such as in the logical inconsistency task where partic- ipants were not adept in formal logic, performance generally improved using CoT as LLMs could tap into existing logical frameworks while reasoning whereas human participants could not. Conversely, when models lacked access to rele- vant priors, such as in a task where motor simulation was responsible for improved performance (relative to verbal thinking) in humans, performance was roughly equal be- tween conditions. Finally, having access to long context windows compared to human working memory synergized with CoT to improve model performance in preference aggregation task involving myriad of features. These cases highlight the importance of understanding differences be- tween humans and models when translating psychological results to predictions about model performance.
In representative tasks for each of the first three archetypes, CoT drastically decreased performance across models. First, in implicit statistical learning, we observed drop of 36.3% in the absolute performance of ol-preveew compared to directly prompting GPT-4o, along with consistent reduc- tions in accuracy from CoT across eight other state-of- the-art LLMs. Next, for a task involving visual stimuli ill-represented by language, we found that CoT reduced performance in all six vision-language models tested. And third, when learning labels that contain exceptions to gen- eralizable rules, CoT increased the number of iterations it took all models to learn the correct labels, by up to 331% % more.
To evaluate whether to use CoT on unseen tasks, one could search for stimulus patterns (such as reliance on another modality) that are based in psychological findings, and pre- dict that e.g., due to verbal overshadowing, performance using CoT would also be poor. While our work does not cre- ate a definitive classifier for when CoT should not be used, our results provide valuable scientific observations support- ing parallel between human deliberation and model CoT failures. These failure cases we derive are uniquely infor- mative for studying the limits of CoT because of existing
In contrast, for the latter three archetypes, we did not ob- serve consistently negative effects caused by CoT, but in-
2

---

Mind Your Step (by Step)
psychology literature that explains why these failures hap- pen (see Section 3). Such parallels also suggest that these failures we observe are not arbitrary, but rather reflect deeper patterns in reasoning.
availability of these datasets, and the fact that they often consist mainly of text and/or images, have led to these tasks from the psychology literature to quickly become popular methods for evaluating and understanding LLMs and LMMs (e.g., Binz & Schulz, 2023; Coda-Forno et al., 2024). Re- cent studies that leverage insights or datasets from psychol- ogy have evaluated the representational capacity of LLMs (Frank, 2023), explored how RLHF and CoT lead to dif- ferent outcomes when trying to make models helpful and honest (Liu et al., 2024), and compared human and machine representations via similarity judgments (Peterson et al., 2018; Marjich et al., 2023a;b; 2024a). Studies have also found that LLMs over-estimate human rationality (Liu et al., 2025), identified incoherence LLM probability judgments (Zhu & Griffiths, 2024), identified susceptibility to linguis- tic illusions in LLMs (Marjich et al., 2024b), and uncovered LLMs' underlying social biases (Bai et al., 2025). Other works have used storytelling to understand episodic memory in LLMs (Cornell et al., 2023), constructed prompts using theories of metaphor (Prystawski et al., 2023), discovered cross- cosss-linguistic variability in LLM representations (Nieder- mann et al., 2023), and probed the roles of language and vision for in-context learning in VLMs (Chen et al.. 2024). Many of these studies start with phenomenon in human cognition and explore whether there is an analog to it in Al models. Our work follows this approach by associating the well-studied impact of deliberation on human performance with the effects of CoT on model performance.
The remainder of the paper is structured as follows: We cover related work surrounding CoT and intersections be- tween AI models and psychology in Section 2. We ground our approach in the psychology literature and identify six archetypes of tasks for which thinking reduces human per- formance in Section 3. In Section 4, we describe the six representative tasks corresponding to each archetype, how we implemented these tasks as model evaluations, and sum- marize our findings. We then contextualize our findings and discuss limitations in our work in Section 5.
2. Related work
2.1. Inference-time reasoning
Chain-of-thought prompting aims to improve the perfor- mance of language-based models by encouraging them to generate an intervening string of tokens that increases the probability of producing the correct answer (Wei et al., 2022: Nye et al., 2021). This approach can result in significant performance improvements in language (Zhang et al., 2023) and vision (Zhang et al., 2024) tasks, hypothesized to be consequence of exploiting local structure in language (Prys- tawski et al., 2024). However, a recent metastudy suggests that gains from using CoT are primarily in mathematical and symbolic reasoning tasks, and other areas such as text classi- fication can instead see decreases in performance (Sprague et al., 2025). Yet, even the reasoning capabilities on sym- bolic tasks are also fragile (Mirzadeh et al., 2025). In the literature, there are no fine-grained patterns that explain under which cases CoT performs poorly. In settings such as planning, studies have shown little benefit from using CoT (Kambhampati et al., 2024). Furthermore, CoT can also increase the frequency of harmful outputs (Shaikh et al., 2023). Despite these results, the default expectation is that CoT improves performance. For example, a recent update to popular benchmark cited the fact that CoT results in improvement on the new benchmark but decreased perfor- mance on the original as an indicator that the new bench- mark is better (Wang et al., 2024). This expectation has likely driven the tendency towards default use of CoT and inference time reasoning in the latest models.
3. Approach: When thinking impairs humans
A large body of psychological research has investigated effects of verbal thinking (often explicit "deliberation" "" on memory, learning, judgment, and decision-making. Very often these effects are positive. For example, people who spend more time deliberating are more likely to respond correctly on questions that initially trigger an intuitive but incorrect response (Travers et al., 2016). However, there are also cases in which verbal thinking can impair performance, often involving a mismatch between the representations or types of processing that best support the task and those induced by verbal thinking (Schooler, 2002).
A canonical setting for such effects implicit statistical learning. Examples include artificial grammar learning stud- ies, where participants are presented with sequences of let- ters or phonemes that conform to some structure (c.g., fi- nite state grammar) and are asked to recognize werm-formed sequences. Studies found that participants can differentiate well well-formed sequences from those that are not, but cannot verbalize the basis for their judgments (Aslin & Newport, 2012; Romberg Saffran, 2010). Some (but not all) studies further find that receiving explicit instructions to identify rules in verbal form impairs performance (Reber, 1976).
2.2. Psychological methods as a tool for studying models
Since the introduction of LLMs, there has been growing interest in understanding the connections between models and human minds (Hardy et al., 2023). Human cognition is often studied using well-controlled tasks involving carefully curated datasets designed to test specific hypotheses. The
3

---

Mind Your Step (by Step)
In this task archetype, the implicit statistical structure of learned patterns are often better recognized without verbal descriptions. We use the original artificial grammar learning task shared by Reber & Lewis (1977), Whittlesea & Dorken (1993), and Van den Bos & Poletiek (2008) to test models.
participants less likely to recognize a logical inconsistency (Khemlani & Johnson-Laird, 2012). However, these partici- pants are typically untrained in formal logic, whereas LLMs are familiar with logical manipulation via their vast training corpuses. Prior work has also found that verbal thinking can be less accurate than visual or motor simulation (Schwartz & Black 1999; see also Aronowitz & Lombrozo 2020; Lom- brozo 2019). but this is consequence of information en- coded in visual and motor representations that are likely not available to models. Finally, humans sometimes make poor choices when they deliberate over multi-dimensiona aggregation problems with numerous features (Dijksterhuis, 2004)- - pllausibly a consequence of memory limitations that are not faced by LLMs. We anticipate that for tasks like these, CoT is less likely to reduce performance. We adapt one task from each of these archetypes (explanations, motor simulation, memory) and also apply them to test LLMs.
Another class of cases concerns a phenomenon termed ver- bal overshadowing. In a classic demonstration, instruc- tions verbalize a face led impaired facial recogni- tion compared to a condition in which participants did not (Schooler& & Engstler-Schooler, 1990). Such effects have been found for other perceptual stimuli (Fiore & Schooler, 2002; Melcher & Schooler, 1996), but do not extend to stimuli that are easy to verbalize (such as a spoken state- ment) (Schooler& & Engstler-Schooler 1990) or to logical problem solving (Schooler et al., 1993). This task archetype features stimuli that are better processed in another modal- ity, and consequently humans perform worse when they are forced to solve these tasks verbally. We adapt the seminal facial recognition demonstration from Schooler & Engstler- Schooler (1990) to test multimodal models.
4. Experiments
For each of the six representative studies on human verbal thinking described in Section 3, we test the performance of collection of LLMs or LMMs with and without CoT. For each task, we massively scale up the study stimuli and adapt the task to appropriately test these models. We release the six scaled-up task datasets as a human overthinking bench- mark that ML practitioners can use, available at https: / giihhbb. com/ wawpaayiieenn/ooooeeeeeeee
As third category, studies find that asking people to gen- erate verbal explanations for their observations supports the discovery of broad and simple patterss (Edwards al., 2019; Walker et al., 2017; Williams& & Lombrozo, 2010; 2013). But when the stimuli are designed such that these broad and simple patterns contain exceptions, participants who were prompted to explain learned more slowly and made more errors (Williams et al., 2013). These effects are thought to arise from a mismatch between the representa- tions processes induced by form of thinking (in this case, explaining) and those which best support task performance (Lombrozo, 2016). To test models, we adapt experiments from Williams & Lombrozo (2013)-1 the landmark study that demonstrated these failure effects.
4.1. Implicit statistical learning
Task. The first class of tasks we examine are those involv- ing implicit statistical learning. As described in Section 3, psychology studies found that data that contain statistical patterns can sometimes be better generalized by humans when those patterns not linguistically described. We explore whether this also holds for LLMs by replicating the task of artificial grammar learning (Reber & Lewis, 1977; Whittlesea & Dorken, 1993; Van den Bos & Poletick, 2008). In this task, artificial "words" are constructed using finite- state grammars (FSGss and participants are tasked with identifying which words belong to the same pattern (are generated by the same FSG).
The effects reviewed so far plausibly concern impairments that arise from the representational limitations of language and the generalization of patterns found in language: lan- guage is not well-suited to encoding fine-grained perceptual discriminations (c.g., in face recognition), and language readily encodes some kinds of relationships (such as de- ductive entailment, or simple and broad patterns) but is less well-suited or employed for others (such as complex grammars, or patterns with arbitrary exceptions). Given that LLMs are likely to share limitations that arise from lan- guage and generalization, we might expect L.MMs to exhibit patterns of impairment that mirror those found for humans on these tasks. We test these predictions using the selected study paradigms in Section 4.
To test this, we constructed 4400 classification problems corresponding to 100 randomly sampled unique FSGs that were structurally similar to those in Fallshore & Schooler (1993). Each classification problem consisted of 15 training examples generated from the grammar, and the model was given a new example and asked classify it. For each FSG, models were asked to classify 44 words, where 22 words belonged to the FSG and 22 did not. Words not belonging to the grammar were generated by replacing one letter of an ex- isting word in the grammar. Details on problem generation are provided in Appendix A.
Prior work has documented additional impairments in hu- mans from verbal thinking, but for some it is less clear if they should generalize to LLMs. For example, explaining how inconsistent statements could be true makes human
4

---

Mind Your Step (by Step)
To test whether our results generalize across the broader task archetype, we also examined the effects of CoT iteratively simpler versions of the problem; we reduced the complexity of our 6-node FSG to {5, 4, 3) nodes via edge contraction (see Appendix A.5), creating a series of distinct implicit statistical learning tasks of varying difficulty.
Table I. For artificial grammar learning, CoT greatly reduces per-
formance in majority of models. Random performance is 50%.
Zero-shot
CoT
decrease (absolute)
g-value
GPT-4o (subset)
94.00/%
36.307%
0,0001
ol-preview (subset)
57.70%
GPT-40
87.50%
64.40%
23.10%
0.0001
Claude 3 Opus
70.70%
62.70%
8.00%
0,0001
Claude 3,5 Sonner
65.90%
67.70%
-1.80%
0.980
Human failure. In the artificial grammar learning task, humans prompted verbalize performed more poorly than those who were not told to do so (Fallshore & Schooler, 1993). We investigate whether CoT will similarly reduce LLM performance.
Gemini 1.5 Pro
68.00%
61.95%
6.05%
< 0.0001
Llama 8B Instruct
59.70%
57.90%
1.80%
0.05
Llama 70B Instruct
60.50%
58.30%
2.20%
Q.05
Llana 3.1 8B Instruct
53.52%
51.54%
1.98%
0.0001
Llama 3.1 70B Instract
65.90%
57.10%
8.80%
< 0.0001
Models and prompts. We test several open- and closed- source models: OpenAl ol -p-peeiew, GPT-40, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, Llama 3.1 70B & 8B Instruct, and Llama 70B & 8B Instruct. Due to cost limitations, we test ol -previiew on a subset. We could not re- move ol's 's inference-time reasoning, and thus used GPT-4o as its zero-shot comparison. We mainly evaluated zero-shot and CoT prompts (Appendix A.2), but also experimented with tree-of- tree-of-thoughh thought (Yao et al., 2023, Appendix A.4), ro- bustness to sampling temperature (Appendix A.6), and in- context steering towards reasoning methods more suited for the statistical structure of the task (Appendix A.7).
same person from set of candidates (Schooler & Engstler- Schooler, 1990). While psychological studies often include a distractor task between the initial face and the candidates to increase the difficulty, we did not use these for LMMs due to their weak performance. We scale this task from one recognition problem to a novel synthetic dataset of 500 problems across 2500 unique faces. For each problem, all faces were given the same described attributes for seven features: race, gender, age group, eye color, hair length, hair color, and hair type. We then generated a pair of images of the same person and four images of other people match- ing this description using stable-image-ulte (StabilityAL. 2024), which we selected based on generation quality. We adjusted the generation process to ensure that the pair clearly consisted of the same person, while the others clearly did not (see Appendix B. for further details). One of the pair was selected to be the initial stimulus, while the other was shuffled with the four images to create the set of candidate answers, Models were asked to identify which candidate matched the person from the initial stimulus.
Results. We find large reductions in performance when using CoT compared to zero-shot prompting, as shown in Table 1. When on randomly selected subset of 440 problems, ol ol-preview, with inference-time reasoning built into its responses, has 36.3% absolute accuracy decrease compared to GPT-4o zero-shot on the same subset. Simi- larly, while there is limited performance change between conditions for Claude 3.5 Sonnet, both conditions perform worse than Claude 3 Opus zero-shot. Across other mod- els, we find consistent decreases in performance with CoT: 23.1% % in GPT-4o, 8.00% in Claude 3 Opus, 6.05% in Gem- ini, and 8. 8.80% in Llama 3.1 70B Instruct. Weaker models such as Llama 3.1 8B Instruct have smaller drops in absolute performance, but these are still substantial when contextual- ized against random chance (50%). As such, the differences caused by CoT remain statistically significant.
To test whether our results generalize across the broader archetype, we also tested a less difficult case where the images' descriptions within the answer options were not required to be the same (see Appendix B.4).
Human failure. Participants prompted to verbally de- scribe the faces performed worse than those who were not (Schooler & Ingstler-Schoolle 1990). We investigate if CoT similarly reduces performance on the task in LMMs.
Additionally, CoT reductions in performance extend across task complexity (Appendix A.5), and while both tree-of- thought and in-context reasoning steering helps perfor- mance, neither method meaningfully bridges the gap to- wards zero-shot performance (Appendices A.4, A.7).
Models and prompts. We evaluated this task on several open- and closed-source LMMs: GPT-4o, Claude 3.5 Son- net, Claude Opus, Gemini 1.5 Pro, InternVL2 26B, and Intern Llama3 76B. Other multimodal models were not considered they did not support multiple image input. We mainly evaluated zero-shot and CoT prompts (avail- able in Appendix B.2), but also experimented with a binary classification prompt (i.e., "is this the same person?") to rulc-out explanations of CoT disrupting image ordering (Ap- pendix B.5).
4.2. Facial recognition
Task. The second archetype of tasks we identify from the literature involves verbal overshadowing. We study this case using& a classic face recognition task, where participants are first shown a face and then asked to select an image of the
5

---

Mind Your Step (by Step)
Table CoT reduces performance for facial recognition across all six LMMs. Random performance is 20%.
Table 3. Average number of rounds for models to learn labels greatly increases when using COT. GPT-40 with CoT often never completely learned labels (15 rounds).
decrease (absolute)
decrease (relative)
Zero-shot
CoT
p-value
Rounds increase (absolute)
Rounds increuse (relative)
Direct
p p-value
GPT-4o
64,00%
51.20%
12.80%
20.00%
0.01
Claude 3 Opus
44.00%
29.60%
14.40%
32.73%
0.0001
GPT-40
2.9
9.6
331%
0.0001
Claude 3.5 Sonnet
97.80%:
94.80%
3.00%
3.07%
0.05
Clande 3.5 Sonnet
2.3
6.4
4.1
178%
0.0000
Gemini 1,5 Pro
66.00%
54.60%
11.40%
17.27%
0.15
Clande Opus
2.4
5.5
3.1
129%
0..5
Intern VL2 268
9.20%
6.00%
3,20%
34.78%
0.0G
Intern V1.2 3amma 76B
15.77%
13.77%
2.00%
12.68%
12-
Results, We find that all six LMMs tested show a drop in performance when performing CoT (see Table 2). Weaker models often answered that "all images are of the same per- son", resulting in accuracies below the random chance rate of 20%. However, even in these cases we observe decreases in performance due to CoT. We find similar reductions in the reduced difficulty and binary classification settings, which I) demonstrates generalizability across the task archetype; and 2) suggests that CoT affects the reasoning process, rather than causing surface-level ordering failures.
Aruney in of Comer Gerppin
Biiver Proeppting Cra Thoug
12
Banation
Figure 2. Number of correct objects classified per round for GPT- 4o. CoT stagnates at 80%, the proportion which are not exceptions.
4.3. Classifying data with rules that contain exceptions
Task. A third class of tasks where CoT may harm per- formance is learning to classify exemplars when there are exceptions to generalizable rules. As mentioned in Section 3, when humans try to explain the category membership of emplars, they tend to hypothesize simple classification rules, which can lead to inefficient learning when data contain arbitrary exceptions to these rules.
ten vehicles each - and measured LLMs' abilities to learn the labels of each list across up to 15 passes (see Appendix C.1 for details). Memory was implemented by including previous problems, guesses, and feedback in context.
To test the robustness of our results, we also tested an easier where the feature without exceptions (license plate) is replaced with binary oracle feature (see Appendix C.5).
To study this phenomenon extends to CoT, we replicate multi-tum vehicle classification task from Williams al. (2013), in which participants try correctly assign binary labels to a list of vehicles. Participants are given feedback after each prediction, and conduct multiple passes over the list until they label all vehicles correctly in a single pass or exceed the maximum number of tries. Vehicles in the task contained one feature that was almost fully correlated (80%) with the classification label, three features with no relation to the label, and one feature that individually identified the vehicle. Thus, participants could either try to learn generalizable rule from the highly correlated feature but fail due to the exceptions, or they could learn the individual mappings from the identifying feature to the corresponding label. Human participants who were prompted to explain the classification of exemplars performed worse because they tended attempt the former strategy.
Human failure. When learning, people tended to reason about generalizable rules when conducting explanations, increasing the time needed to learn labels for the entire list (Williams et al., 2013).
Models and prompts. We evaluated this task on GPT- 4o, Claude 3.5 Sonnet, and Claude 3 Opus. Other models such as Llama 3.1 70B Instruct were insufficient at multi- turn long context conversation, which made outputs unus- able. We mainly used direct and CoT prompts (see Ap- pendix C.2), and also investigated the effects of a series of steering prompts with varied strength (Appendix A.7).
Results. We find that CoT drastically increases the num- ber of passes for the model to learn all labels correctly, Averaged across the 240 lists, GPT-4o with CoT needed more than four times the number of passes to learn the la- bels compared to direct prompting, while Claude 3.5 Sonnet and 3 Opus both needed more than double (see Table 3).
Participants in the treatment condition of the original study were prompted to deliberate after receiving feedback. To more explicitly include inference- titeeeenee-eeeeeeee reasoning, we modify the point at which deliberation is prompted, instead asking the LLM to perform CoT before making each prediction. In total, we constructed 2400 vehicles - split into 240 lists of
We also investigated the per-round accuracy of GPT-4o and found that direct prompting resulted in the model attaining
6

---

Mind Your Step (by Step)
combination of three datasets: Stanford Natural Language Inference (SNLI), Multi-Genre Natural Language Inference (MNLD, and a synthetic LLM-generated dataset of 100 entailment pairs. We filtered these datasets for pairs that were labeled "entailment" (i.e., A entails B). In addition,
Table Comparing zero-shot and CoT the logical inconsisteney
task using stimuli from 3 datasets. Random performance is 50%.
MNLI
SNLI
Synthttc
Zero-shot
Col
Zero-shot
CoT
Zoro-shot
Col
al ol-preview (subset)
86.5%
GPT-4a
53.2%
93.9%
51.4%
94.3%
51.0%
74.0%
we limit the format of A and B such that the template forms coherent sentences. In total, we evaluate on 1608
Claude 3.5 Sonnet
65.2%
67.5%
67.4%
69.8%
56.7%
57.8%
Claude Opus
62.7%
58.8%
66.2%
58.7%
54.5%
51.8%
Gemini 1.5 Pro
73.2%
68.2%
8%%
63.99%
601.5%
61.8%
(A, B) pairs: 675 from SNLI, 833 from MNLI, and 100 synthetic. Each pair was used to construct two classification problems, one consistent and one inconsistent, for total of 3216 problems in our dataset. For more details on problem generation see Appendix D.I.
Llama 3.1 70B Instract
55.6%
81.6%
50.456
82.3%
50.0%
65.8%
perfect classification the second or third iteration, while with CoT the model was only able to correctly classify 8/10 objects on average after 15 iterations (see Figure 2). The model was unable to surpass this degree of accuracy, likely due to COT biasing the model to rely on the seemingly gen- eralizable rules from the exemplars, while nown-weighing the usefulness of contextual tokens that explicitly contained all of the correct answers.
We evaluated a suite of state-of oftteoofthee-eee art LLMs on this task: ol -r-prevvew (on a subset of 30 synthetic questions due to cost constraints), GPT-4o, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 70B Instruct. We used one zero-shot prompt and two conditions of CoT: one where the model is simply asked to reason before answer- ing, and another that follows the original experiment by asking the model to explain the inconsistency directly (see Appendix D.2 for details). Results were very similar across these two conditions, and we report an average over both.
We found similar reductions of performance from CoT even in the case with a binary oracle feature (Appendix C.5), but CoT performance reached zero-shot levels with strong steering prompts that explicitly told the model to match license plates (Appendix C.6). This demonstrates that CoT reasoning is indeed able to match zero-shot performance, but its default does not match the optimal reasoning space.
An important difference between models and humans in this setting is priors over logic: participants in Khemlani & Johnson-Laird (2012) were recruited intentionally without logical expertise, but LLMSs have access to numerous logical puzzles and knowledge in their training corpuses, creating a synergistic effect with the extra tokens CoT provides. As such, CoT often improved performance (see Table 4). This was especially pronounced in GPT-40,, where CoT improved performance by over 40% on pairs from MNLI and SNLI. However, in the models that performed best with zero-shot prompting, Gemini 1.5 Pro and Claude 3 Opus, we did see decreases in performance with CoT, potentially mirroring some of the effects in what is seen with humans.
4.4. Tasks with a mismatch between humans and models
We also found three tasks for which humans do worse when performing verbal thinking, but where this effect does not cleanly translate to models with COT. One unifying expla- nation for these effects is humans and models possessing different limitations for task-relevant abilities, such as ac- cess to different kinds of information or memory resources.
Explaining a logical inconsistency. When human partici- pants are shown a pair of logically inconsistent statements and asked to explain their coexistence, they become worse at judging whether the statements are indeed logically in- consistent (Khemlani & Johnson-Laird 2012). In the task, participants are provided with two sentences following the template: "If A then it is always the case that B", and either "A, but it is not the case that B" or "It is not the case that B" The former introduces a logical inconsistency, while the latter does not. In one condition humans were first asked to verbally explain why an inconsistent pair could coexist before providing the inconsistency judgement, while in the other they conducted the explanation after the judgement. Performance was significantly worse in the former.
Spatial intuitions. Psychologists have documented cases involving spatial reasoning in which humans generate more accurate responses after visual or motor simulation com- pared to verbal thinking. To investigate whether this applies to models, we replicate cup-tilting task from Schwartz & Black (1999). In the task, participants are shown an image of two rectangles with varying dimensions, representing two cups - one empty and one that contains water. Partici- pants are asked to estimate the height of water that should be added to the empty cup so that when tilting both cups, water will reach both rims at the same angle (see Figure 1, SI). While the original task had participants draw water level on the empty cup, LMMs were unable to do this, so we turned the task into a multiple choice question by adding markings A D to the side of the empty cup and asking the model to select one. Incorrect options were generated by adding Gaussian noise to the correct answer while satisfying
The original human experiment contained 12 unique (A, B} pairs. To scale this task for LLMs, we leverage existing entailment pairs in natural language inference tasks, which we use as A and B to fill in the sentence templates. We used
7

---

Mind Your Step (by Step)
Table Comparing zero-shot and CoT on the spatial intuition task.
Table 6. Results for apartment selection task across four models and three ranges of A.
Performance (absolute)
Performance (relative)
Zem-shot
CoT
p-value
10.1. 0.3]
[0.3, 0.5)
[0.5, I]
GPT-40
38%
40%
+2%
+5.00%
Zero-shot
CoT
Zero-shot
CoT
Zero-shot
Col
Claude 3.5 Sonnet
42%
38%
.4%
-10.53%
0.28
GPT-40
47%
45%
57%.
56%
80%
87%
Clande 3 Opus
42%
38%
-4%
-10.53%
0.28
Claude 3.5 Sonnet
50%
62%
62%
72%
81%
95%
Gemini 1.5
35%
36%
+1%
+2.78%
Claude Opus
35%
50%
57%
58%
72%
84%
VL.2 Llama3 76B
39%
31%
-8%
-25.81%
0.67
lama 3.1 70B Instrnet
42%
6%
44%
5%
43%
20%
the constraint that options must be a certain distance apart. We scaled up this task by varying the dimensions of cup sizes and water height, creating a total of 100 problems, each with a code-drawn image containing the cups and mul- tiple choice answers (see Appendix E.1). We evaluated with zero-shot and CoT prompts on several open- and closed- source LMMs: GPT-4o, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and InternVL2 Llama3 76B.
the CoT condition, reducing performance.
In this setting, there were meaningful differences in work- ing memory between humans and models. Humans were forced to rely on their aggregate impressions of each apart- ment due to the large amount of information. However, even after increasing the number of contextually relevant statements over six-fold, models were able to access all feature statements in-context. Thus, we observed mostly positive effects from CoT (see Table 6). When observing reasoning traces, we found that the availability of context turns the problem into summing up the importances of the features. More capable models were able to leverage CoT to get these approximately correct, despite known issues in long-contex retrieval (Liu al., 2023), greatly surpass- ing human memory. This highlights the need to consider fundamental differences in capabilities between models and humans when searching for parallel effects between them.
In this setting, it is unlikely that large multimodal mod- els would share the same motor simulation capabilities as humans due to lack of representations built from motor experience. The improved performance in the non-verbal thinking condition for humans required spatial motor in- tuition, and thus for models we did not observe significant differences between zero-shot and CoT prompts (see Ta- ble 5). Generally, we expect this to extend to other tasks for which models lack telk-relevant priors that humans possess.
Aggregating features for a decision. The final category of tasks we consider are tasks that overwhelm human work- ing memory. Dijksterhuis (2004) found that humans made poor choices when deliberating over apartment options if provided with a large amount of information about various apartment features in short amount of time. In the study, participants were shown 48 statements for four seconds each, where each statement described a positive, negative, or neu- tral aspect of one of four apartments. They were then asked to select the best apartment after either deliberating or com- pleting a distractor task. Authors found that the distractor task actually improved performance over deliberating.
4.5. Effectiveness of our heuristic
Our heuristic based on human overthinking identified three tasks with significant ToT-induedd performance drops, but mixed or negligible effects were observed in the other three tasks. It is thus reasonable to ask whether selecting tasks based on human behavior reliably isolates tasks where CoT reduces performance, or merely samples randomly from all tasks. To evaluate this, we conducted a bootstrap test using dataset of 378 comparisons between zero-shot and CoT across different models and tasks from Sprague et al. (2025). From our results, we selected all 50 numerical dif- ferences across the six types of tasks across all models'. Resampling 100,000 times with a sample size of 50, none of the 100,000 bootstrapped samples had a greater average de- crease in performance than results, strongly supporting the hypothesis that our heuristic is more efficient than past endeavors at finding failures in CoT based on effect size.
To scale this task up for LLMs,, we generated 80 unique apartment features with four statements per feature: one positive, one negative, and two neutral. We then asked GPT- 4o to rate the impact each statement would have on the im- pression of an average tenant in [-5, 5]. We randomly sam- pled apartments by choosing one statement per feature and constructed sets of four where the best apartment had a per- feature average score A {[0.1, 0.3), |0.3, 0.5), [0.5, }]} higher than the next-best option. We sampled 300 such sets (100 per A range) to form choice tasks (see Appendix F.I). We tested several open- and closed-source LLMS with zero- shot and CoT prompts: GPT-4o, Claude 3.5 Sonnet, Claude 3 Opus, and Llama 3.1 70B Instruct. Llama 3.1 70B Instruct was often unable to return an answer after deliberating in
In addition to effect size, we also test if our heuristic finds a higher occurence of performance decreases (irrespective of magnitude). Using the same bootstrapping procedure and data, we find that only 11 of 100,000 resampled cases yielded equal or greater reductions in performance than sample (yielding an estimated p < 0.00011). These results
For the classifying data with exceptions task, we use differ- ences in accuracy reported from Figure 2 to match other inputs.
8

---

Mind Your Step (by Step)
confirm that our heuristic also disproportionately identifies CoT-iiduced failures irrespective of effect size. For more detailed explanations of both analyses, see Appendix G.
in models but not humans. Thus, we envision our contribu- tion to be complementary existing evaluation methods in natural language processing.
As we've seen across our six tasks, knowledge of what drives a decrease in performance in humans can be lever- aged to generate predictions about the effects of CoT, but this remains an inferential step that requires careful reason- ing and an understanding of model capabilities. Despite these limitations, our method can be used to identify large and consequential failures of COT, as documented in our three failure cases. It also offers valuable cross-domain insight that can help build intuitions and contribute to our overall understanding of inference-ting reasoning. On the flipside, the existence of capable L.MM/LMM systems also allows us to better understand why human performance can be degraded by deliberation. By considering when CoT's effects mirror humans and when they do not, we can distin- guish when the task or mechanisms shared by humans and models are responsible for failures, versus when the issues arise from uniquely human strategies or limitations.
5. Discussion
Chain-of-though prompting is an effective way to expand the capacities of large language and multimodal models. However, knowing that CoT significantly decreases perfor- mance in specific settings is important for considering when it should be deployed, and especially whether it should be deployed by default. By using cases where verbal thinking decreases human performance as a heuristic, we successfully identify three settings where CoT results large decreases in model performance, leading to implications for choosing when CoT should be deployed.
While we draw connection between human cognition and LMMs we do not claim that these systems operate in the same way or that models should be anthropomorphized. Rather, we see this connection as a tool for identifying set- tings where the structure of the task or shared limitations result negative effects of verbal thinking. Our exploration is guided by considering not only whether verbal thinking reduces human performance, but also whether there are differences between humans and models that must be con- sidered. Our results provide evidence that CoT can result in large decreases in performance when human verbal thinking leads to similar failures, illustrating that we can use the psy- chology literature to find cases that are informative about the performance of COT. We now turn to limitations and future directions.
Alternative explanation for mismatch between CoT and humans. Another explanation for why we do not see con- sistent drops in performance in the latter three tasks is that how we implemented the tasks for LLMs removed the fail- ure effect. It's possible that with other implementations (e.g., using memory-based LLM agent for the apartments task) we might in fact see decreased performance mirror- ing humans. While we explored prompt variations for each task, these were not exhaustive due to the endless space of changes to prompts. In other words, because the tasks were inevitably changed scale up the evaluation and match more realistic applications of models, it's also possible that these changes are what explain the human-model mismatch.
Types of inference-time reasoning. Since the invention of CoT, researchers have developed new prompting strate- gies suited for application domains, as well as more elabo- rate cmneral-purpose prompts such as tree-of-thought (ToT; Yao et al., 2023) and self-consistency ( (Wang et al., 2023). We tested the effectivvnees of ToT on GPT-4o for the im- plicit statistical learning task (see Appendix A.4). While ToT improved accuracy (64.55% vs. 62.52%), this was still far from GPT-4o's zero-shot performance of 94.00%, suggesting that our findings extend across inference-time reasoming techniques. However, future work is required to determine whether this generalizes to other task domains and methods of eliciting verbal thinking in models.
Future directions. Overall, we envision studying how to evaluate and improve models as a collaborative effort be- tween machine learning methods, psychological insights, and a burgeoning literature connecting and comparing hu- mans and models. By sharing knowledge and building strong collaborations between these disciplines, we can utilize rich insights from decades of studying humans to advance the domain's 's intuitions about models and analyze an even broader array of tasks and applications for Al.
Scope of application. While our psychology- heuris- tic offers strategy for identifying failure cases of CoT, it is unlikely to cover all cases where CoT decreases perfor- mance. Existing psychological research has been guided by a variety of theoretical and practical considerations, but does not offer an exhaustive or representative sample of all tasks, and will miss cases that are uniquely interesting to study

---

Mind Your Step (by Step)
Acknowledgments
Dijksterhuis, A. Think different: the merits of unconscious thought in preference development and decision making. Journal of Personality and Social Psychology, 87(5))586, 2004.
Experiments were supported by Azure credits from a Mi- crosoft AFMR grant. This work and related results were made possible with the support of the NOMIS Founda- tion. Special thanks to Howard Chen, Jian-Qiao Zhu, and Mengzhou Xia for providing invaluable feedback on this project.
Edwards, B. J., Williams, J. J., Gentner, D., and Lombrozo, T. Explanation recruits comparison in a category task. Cognition, 185:21-388 2019.
Fallshore, M. and Schooler, J. W. Post-encoding verbal- ization impairs transfer on artificial grammar tasks. In Proceedings of the Fifteenth Annual Conference of the Cognitive Science Society Erlbaum: Hillsdale, NJ, pp. 412-416 416, 1993.
Impact Statement
This paper presents work whose goal is to advance the un- derstanding of inference-time reasoning between humans and large language and multimodal models. While there are many potential societal consequences of our work, we are explicit about the takeaways we intend for our paper (see Section 5) and thus we don't forsee that any additional details need to be highlighted here.
Fiore, S. M. and Schooler, J. W. How did you get here from there? Verbal overshadowing of spatial mental models. Applied Cognitive Psychology, 16(8):897-910, 2002.
Frank, M. C. Baby steps in evaluating the capacities of
References
large language models. Nature Reviews Psychology, 2(8): 45 451-452, 2023.
Anthropic. Claude's extended thinking, 2025.
URL https: /wwp: https://www.atttrop.... anthropicc wwc.ec...o//nii cow/news/ witblle-extenddd-thhnkkiii 2025-06-01.
Griffiths, T. L. Understanding human intelligence through human limitations. Trends in Cognitive Sciences, 24 I): 873-883, 2020.
Accessed:
Aronowitz, S. and Lombrozo, T. Learning through simula- tion. Philosophers' Imprint, 20, 2020.
Hardy, M., Sucholutsky, I., and Thompson, B. Large lan- guage models meet cognitive science: LLMs as tools, models, and participants. In Proceedings of the Annual Meeting of the Cognitive Science Society, 45 (45), 2023.
Aslin, R. N. and Newport, E. L. Statistical learning: From acquiring specific items to forming general rules. Cur- rent Directions in Psychological Science, 2103):77-1176, 2012.
Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song. D., and Steinhardt, Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021.
Bai, X., Wang. A., Sucholutsky, I., and Griffiths, T. L. Ex- plicitly unbiased large language models still form biased associations. Proceediiggs of the National Academy of Sciences, 122(8):e441622812) 2025.
Kambhampati. S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L., P., and Murthy, A. B. Position: LLMs can't plan, but can help planning in LLM-modulo frameworks. In Forty-first International Conference on Machine Learning, 2024.
Binz, M. and Schulz, E. Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences, 12066):22185221200 2023.
Chen, A., Sucholutsky. I., Russakovsky, O., and Griffiths, T. L. Analyzing the roles of language and vision in leam- ing from limited data. Proceedings of the Annual Meeting of the Cognitive Science Society, volume 46, 2024.
Khemlani, S. S. and Johnson-Laird, P. N. Hidden conflicts:
Explanations make inconsistencies harder to detect. Acta Psychologica, 139(3):486-491, 2012.
Kirchner, W. K. Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55(4):352, 1958.
Coda-Forno, J., Binz, M., Wang, J. X., and Schulz, E. Cog- bench: large language model walks into psychology lab. In Forty-first International Conference on Machine Learning, 2024.
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F,, and Liang, P. Lost in the middle: How language models use long contexts, 2023, URL https:
Corell, C., Jin, S., and Zhang, Q. The role of episodic memory in storytelling: Comparing large language mod- els with humans. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 46, 2023.
/ //arxiv. org/ abs 2200 031770017..
Liu, R., Sumers, T., Dasgupta, I., and Griffiths. T. How do large language models navigate conflicts between honesty
10

---

Mind Your Step (by Step)
and helpfulness? In Forty-first International Conference on Machine Learning, 2024.
Niedermann, J. P., Sucholutsky, I., Marjieh, R., Celen, E., Griffiths, T. L.,, Jacoby, N., and van Rijn, P. Studying the effect of globalization on color perception using mul- tilingual online recruitment and large language models. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 46, 2023.
Liu, R., Geng, J., Peterson, J., Sucholutsky, I., and Griffiths, T. L. Large language models assume people are more ra- tional than we really are. In The Thirleenth International Conference on Learning Representations, 2025.
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:21 2021.
Lombrozo. T. Explanatory preferences shape learning and inference. Trends in Cognitive Sciences, 20(10):44--759 2016.
Lombrozo, T. 'Learning by thinking' in science and in everyday life. The Scientific Imagination, pp. 230-249, 2019.
OpenAL. Learning to reason with LLMs, 2024a. URL
https:/ /opp/ai//pp/...////// comp moom/inne index/ seriinng-t--rasoo-viii
Lombrozo, T. Learning by thinking in natural and artificial minds. Trends in Cognitive Sciences, 2024.
OpenAL. DALL-E 2, 2024b. URL https:://peeaat
com http//nnxx///// (afdex/daal-- dall-a-llllle- e-- Accessed: 2024-10-01.
Marjich, R., Rijn, P. V., Sucholutsky. I., Sumers, T., Lee, H., Griffiths, T. L., and Jacoby, N. Words are all you need? Language as an approximation for human similar- ity judgments. In The Eleventh International Conference on Learning Representations, 2023a.
Peterson, J. C., Abbott, J. T.,, and Griffiths, T. L. Evaluating (and improving) the correspondence between deep neural networks and human representations. Cognitive Science, 42(8):2648-2666 2018
Prystawski, B., Thibodeau, P., Potts, C., and Goodman, N. haahooooiccally-nfomone chain-of-thought prompts for metaphor understanding large language models. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 45, 2023.
Marjich, R., Sucholutsky, I., van Rijn, P., Jacoby, N., and Griffiths, T. L. What language reveals about perception: Distilling psychophysical knowledge from large language models. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 45, 2023b.
Prystawski, B., Li. M., and Goodman, N. Why think step by step? Reasoning emerges from the locality of experience. Advances in Neural Information Processing Systems, 36, 2024.
Marjieh, R., Sucholutsky, I., van Rijn, P., Jacoby, N., and Griffiths, T. I.. Large language models predict human sen- sory judgments across six modalities. Scientific Reports, 14(1):21445, 2024a.
Reber, A. S. Implicit learning of synthetic languages: The role of instructional set. Journal of Experimental Psy- chology: Human Learning and Memory, 2(1):88, 1976.
Marjieh, R., van Rijn, P., Sucholutsky, I., Lee, H., Griffiths, T. L., and Jacoby, N. A rational analysis of the speech-to- song illusion. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 46, 2024b.
Reber, A. S. and Lewis, S. Implicit learning: An analysis of the form and structure of body of tacit knowledge. Cognition, 5(4):333-361, 1977.
McCoy, R. T., Yao, S., Friedman, D., Hardy, M. D., and Griffiths, T. L.. Embers of autoregression show how large language models are shaped by the problem they are trained to solve. Proceedings of the National Academy of Sciences, 121(41):e2332242012 2024.
Romberg, A. R. and Saffran, J. R. Statistical learning and language acquisition. Wiley Interdisciplinary Reviews: Cognitive Science, 1(6)9906-914, 2010.
Schooler, J. W. Re-representing consciousness: Dissocia- tions between experience and meta-consciousness. Trends in Cognitive Sciences, 6(8):339-344, 2002.
Melcher, J. M. and Schooler, J. W. The misremembrance of wines past: Verbal and perceptual expertise differentially mediate verbal overshadowing of taste memory. Journal of Memory and Language. 35(2):231-255 1996.
Schooler, J. W. and Engstler-Schoolee T. Y. Verbal over- shadowing of visual memories: Some things are better left unsaid. Cognitive Psychology, 22(1):33-77) 1990.
Mirzadeh, S. I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., and Farajtabar, M. GSM-Symbolic: Under- standing the limitations of mathematical reasoning in large language models. In The Thirteenth International Conference on Learning Representations, 2025.
Schooler, J. W., Ohlsson, S., and Brooks, K. Thoughts be- yond words: When language overshadows insight. Jour- nal of Experimental Psychology: General, 122(2):166, 1993.

---

Mind Your Step (by Step)
Schwartz, D. L. and Black, T. Inferences through imagined actions: Knowing by simulated doing. Journal of Experi- mental Psychology: Learning, Memory, and Cognition, 25(1):116, 1999.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F, Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35: 24824-24837, 2022.
Shaikh, O., Zhang, H., Held, W., Bernstein, M., and Yang, D. On second thought, let's not think step by step! Bias and toxicity in zero-shot reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pp. 4454-4470, 2023.
Whittlesea, B. W. and Dorken, M. D. Incidentally, things in general are particularly determined: An episodic- processing account of implicit learning. Journal of Ex-
perimental Psychology: General, 122(2):227, 1993.
Shiffrin, R. and Mitchell, M. Probing the psychology of ai models. Proceedings of the National Academy of Sci- ences, 120(10):e2300663 20, 2023.
Williams, J. and Lombrozo, T. The role of explanation in discovery and generalization: Evidence from category learning. Cognitive Science, 34(5):776-806 2010.
Sprague, Z. R., Yin, F., Rodriguez, J. D., Jiang, D., Wadhwa, M., Singhal, P., Zhao, X., Ye, X., Mahowald, K., and Durrett, G. To CoT or not to CoT? thoughtf helps mainly on math and symbolic reasoning. In The Thirteenth International Conference on Learning Repre- sentations, 2025.
Williams, J. J. and Lombrozo, T. Explanation and prior knowledge interact guide learning. Cognitive Psychol- ogy, 66 66(1):55-2) 84, 2013.
Williams, J. Lombrozo, T., and Rehder, B. The hazards of explanation: Overgeneralization in the face of exceptions. Journal of Experimental Psychology: General, 142(4);
Atability/L. Stability AI developer plaffooon. API reference, 2024. http:tttt//ppttttttotooo stabiltabiiee webbiiiiyyyy. aii docs pocc/api-rree htt//aaprrreeeeeeeeeeeee Accessed: 2024-09-28.
1006, 2013.
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K. R. Tree of thoughts: Deliberate problem solving with large language models. In Thirty- seventh Conference Neural Information Processing Systems, 2023.
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung. H. W., Chowdhery, A., Le, Q.. Chi, E., Zhou, D., and Wei, J. Challenging BIG-bench tasks and whether chain-of-thi of-thought can solve them. In Findings of the Association for Compulational Linguistics: ACL 2023, Toronto, Canada, 2023, Association for Computational Linguistics.
Zhang, Z., Zhang. A., Li, M., and Smola, A. Automatic chain of thought prompting large language models. In The Eleventh International Conference on Learning Representations, 2023.
Travers, E., Rolison, J. J., and Feeney, A. The time course of conflict on the cognitive reflection test. Cognition, 150: 109-118, 118, 2016.
Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., and Smola, A. Multimodal chain-of-thought reasoning in language models. Transactions Machine Learning Research, 2024.
Van den Bos, E. and Poletick, F. H. Intentional artificial grammar learning: When does it work? European Jour- nal of Cognitive Psychology, 20(4):793-806, 2008.
Zhu, J.-Q. and Griffiths, T. L.. Incoherent probability judg- ments in large language models. In Proceedings of the Annual Meeting of the Cognitive Science Society, vol- ume 46, 2024.
Walker, C. M., Lombrozo, T., Williams, J., Rafferty, A. N., and Gopnik, A. Explaining constrains causal learning in childhood. Child development, 88(1) (8(1):2299244) 2262 2017.
Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Chi, E. H., Narang, S., Chowdhery, A., and Zhou, D. Self- consistency improves chain of thought reasoning in lan- guage models. The Eleventh International Conference on Learning Representations, 2023.
Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., Li, T., Ku, M., Wang, K., Zhuang, A., Fan, R., Yue, X., and Chen, W. MMLU-pro: A more robust and challenging multi-task language understanding benchmark. In The eiiht Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
12

---

Mind Your Step (by Step)
A. Implicit Statistical Learning Task
To study cases involving implicit statistical learning, we consider an artificial grammar learning task. In the task, LLMs are provided with letter string train examples that belong to the same category, and are tasked to classify whether new string belongs the same category.
A.1. Generation of Artificial Grammar Learning Dataset
In the original psychology experiments (Fallshore & Schooler, 1993; Reber & Lewis, 1977), participants performed the classification task on strings generated by fixed finite state grammar (FSG) constructed by the researchers (see Figure 3), A string is generated by the FSG if it corresponds valid path along the directed edges from the source node the sink node where the letters on the path are appended together.
In our experiments, we expand the experiments massively to 100 randomly sampled FSGs that follow the same rough structure of those used in the experiment. To scale up the dataset, we construct and sample from all possible FSGs that obey following rules. For visual representation please see Figure
6 nodes total, including source 8, sink f, and four nodes ,1:...,,4.
Edges (s, (1); (s, .r3), (x2, t) and (#4, always present.
Edge (21, x22 is always present to avoid isomorphisms and the null case where no paths exist from s
The remaining middle edges ((=1,=0)) (#1,74)) (12, x1), (x2, (12, (13,:1), (23, #2), (13) (14, (2), (za, 233} can either exist or not, for total of 2 combinations.
Each zy can have self-loops, c.g., (I1, 21), for total of 2 combinations.
Letters on each edge are randomly selected from the capital alphabet, for a total of 26 combinations.
Each FSG should be able generate least 37 unique strings with length < 8.
The construction of the FSG unique with respect to the three graphical isomorphisms that each FSG satisfying the rules could have.
For each FSG, we sampled paths of up to length 8 and used them as stimuli for the experiment. Following Fallshore & Schooler (1993), we sampled 37 to use in the experiment, assigning to be training examples and 22 to positive test examples. We also constructed negative test examples by sampling a random string from the FSG, perturbing one letter in randomly selected position to another letter that exists some edge of the FSG. We ensured that the negative examples did not belong to the FSG.
In total, this yielded 4400 individual questions asked to the large language models. Each question was asked individually after the 15 training examples. See the next section for the specific prompts.
X
X
S
X
P
P
X
X
T
Figure 3. The FSG used in Fallshore & Schooler (1993) and Reber Lewis (1977), two classic studies artificial grammar learning.
This FSG was used the generate strings for all participants in both studies. We form our dataset using FSGs that follow similar structure.
13

---

Mind Your Step (by Step)
Z
x
R
X
Figure 4. The potential FSGs used in our dataset. Directed edges that always exist are in black, while the others that could exist are dashed and gray. Bi-directional arrows denote two potential directed arrows. The letters on each edge represent a random sample.
A.2. Prompts
For our experiments, we prompted the models using one zero-shot prompt and one CoT prompt. The zero shot prompt shown in Table For Claude, GPT, and Gemini models, we use temperature 0.0. For ol, the beta version limited its usage to temperature 1.0. For open-source models, we use temperature = 0.0. Max tokens was set to 10 for zero-shot and 1000 for COT. The remaining hyperparameters were set at their default values: top-p, top_k, seed, min etc.
Table 7. Example prompt for artificial grammar learning task, zero shot.
Prompt:
These strings generated according to certain set of rules. Does the following string also follow the same set of rules?
[test example]
Please ONLY answer "Yes" or "No".
The CoT prompt uses one of the most common prompting methods for chain-of-thought, replacing the last line with, 'Please reason about your answer before answering "Yes" or "No".
When conducting pilot experiments, we also tried version of the prompts where we asked models "memorize the following letter strings" in the first line of the prompt as this was in-line with the original human experiment. We found that results were extremely similar to the more general version shown above, and thus discarded this more specialized case.
A.3. CoT Failure Example
An example CoT prompt and output where GPT-4o fails for the artificial grammar learning task is in Table 8.
A.4. Tree-of-thought Experiments
To analyze whether our hypotheses about model chain-of-thought extend other types of inference-time reasoning, we evaluated the performance of GPT-4o with tree-of-thought (ToT) (Yao et al., 2023) on subset of 10 artificial grammars, totaling 440 examples. Given the input prompt, we asked the LLM to generate five different thoughts explain whether the string also followed the same set of rules as the in-context examples. Then, five votes were conducted to select the best thought, which we parsed and compared to the ground truth label. We found that ToT resulted in small improvement over the task (64.55% vs. 62.52% accuracy, see Table 9), but this performance was still much worse compared to GPT-4o zero-shot accuracy (9..00%) This suggests that the reduction performance is not only associated with CoT, but also other types of inference-time reasoning.
A.5. FSG Complexity Reduction Experiments
To show generalizability across variations of each task, we conduct additional experiments varying the problems in each of the failure cases. For artificial grammar learning, we varied the complexity of underlying FSG. Specifically, we reduced
14

---

Mind Your Step (by Step)
nodes
nodes
4 nodes
nodes
Figure 5. Different complexities in the underlying finite state grammar for implicit statistical learning.
the nodes in the finite state automata that generated the artificial grammars. While the original had 6 nodes, we iteratively reduced to 5, 4, and nodes using edge contraction (see Figure 5). Across all valid FSGs with no unused nodes, we observed following accuracies:
nodes, zero-shot 0.886, CoT 0.766
4 nodes, zero-shot 0.837, CoT 0.665
3 nodes, zero-shot 1.000, CoT = 1.000
Excluding the reasonably trivial case three nodes, where we could not detect a difference between the zero-shot and CoT conditions, our results are consistent across variations in difficulty. This provides further confidence in the generalizability of the findings, that CoT hurts performance not in just one-off task, but generalizable class of tasks in implicit statistical learning.
A.6. Robustness Sampling Temperature
To further the robustness of our results, we conducted ablatons on different temperatures (t 0.5, 0.7) across the full 4400 problems for the artificial grammar learning task GPT-4o. The resulting accuracies were follows:
t= 0, zero-shot 87.5, CoT = 64.4 (original results for reference)
t 0.5, zero-shot = 88.3, CoT 63.6
t= 0.7, zero-shot = 87.8, CoT 63.6
Thus, our results seem be robust to variations in temperature sampling.
15

---

Mind Your Step (by Step)
A.7. In-context Steering to Improve CoT for Artificial Grammar Learning
To study whether explicitly prompting the model to focus on an optimal reasoning method improves performance, we conducted additional experiments using the same subset of problems evaluated with the OpenAl ol, assessing performance on GPT-4o. Specifically, the prompts were designed to explicitly instruct the model to identify and utilize bigram patterns when classifying words as belonging to or not belonging to FSGs, which is advantageous because of the nature in which accepting strings are generated. Under these conditions, we obtained the accuracy of 68.43%, compared to 87.50% in the zero-shot condition and 64.40% in the CoT condition. Consistent with our expectations, this result indicates that while explicit bigram instruction did improve CoT performance, it still did not reach that of zero-shot prompting.
B. Facial Recognition Task
To study tasks where language impairs the recognition of visual stimuli, we focus on facial recognition task, where VLMs are asked to select one of five candidate images that matches the face of a provided image. The original experiment in Schooler & Fngstler-Schooler (1990) had participants view a 30-second video of an individual robbing a bank and then perform a 20-minute distractor task, before either writing down descriptions of the robber's face or doing a distractor task for 5 minutes. Participants were then provided with verbally similar faces to choose from, and those who performed the written description performed much worse (38% vs. 64% accuracy) identifying the robber.
B.1. Generation of Facial Recognition Dataset
To adapt this task testing models, we made a few design decisions. First, we chose to replace the initial video stimuli with an image of the person's face allow for the testing of vision language models. Next, we chose to remove the distractor tasks. This decision was based on pilot results indicating that common psychology distractor tasks such the n-back task (Kirchner, 1958) resulted in large amounts of noise in model outputs, while other distractors were of limited effect on the model due to it being able to retrieve the earlier stimuli in-context. Furthermore, even without the distractor, models already showed a large difference in performance across zero-shot and CoT conditions. Thus, our task was simplified to a facial matching task, where a model was given a human face as input and responded with the index of the matching face image as its output.
To generate the faces for the facial recognition dataset, we use table-imagee-ulta (StabilityAI, 2024). We experimented with other models such as DALL-E 2 (OpenAI, 2024b) and DALL-E 3, but found generation capabilities were significantly less realistic than stable-image-ulraa This difference was especially pronounced in generating realistic facial images of people in racial minorities.
To cover a diverse set of human faces, we prompt models to generate faces with features age [young, middle-aged, old), race/ethnicity (asian, black, hispanic, White], gender ( (man, woman), eye color (brown, blue, green), hair color (brown, black, blonde, red, gr hair length {long, short), and hair type {curly, wavy, straight]. We removed some low-probability combinations such red hair with asian ethnicity due to poorer quality of image generation. Then, we randomly sampled combinations of features to form descriptor set.
One issue with stable-image-ulres is that when asked naively to generate an image of the same person as another image, it would alter some details such as ear shape, nose shape, or other facial ratios that would make it impossible to be the exact same person. We addressed this issue by prompting the cubbl-image-ulteee image generation model to
"Generate two realistic images of the same person, one on the left and one on the right. The person should have the following description: [description]".
After doing so, we were able to manually check and verify that the faces shown in the two images is clearly the same the naked eye. One of these images assigned to be the initial stimuli shown, while the other would be shuffled into the list of answers.
We also ensured that the other remaining images were 1) clearly not of the same person as the image, and 2) the pose of the person, which was often similar between the pair of generated images, was also replicated in the other fake answers. This was achieved using the following prompt with the edit structure task in the set of image control API calls from StabilityAl:
"Generate an image of unique person with the same pose and style as the image provided. The person should
16

---

Mind Your Step (by Step)
Figure 6. An example of the six images generated for problem. The first contains of the pair of generated images. The first image in the second row contains the other image in the pair, and the remaining four images are incorrect answers generated from this image.
have the following description: description".
image imput: [correct answer image]
Once all the answers were generated, we manually verified the quality of generated images, and ensured that each of 1) and 2) were satisfied. An example of the images generated for problem are shown in Figure 6.
B.2. Prompts
To evaluate models on the facial recognition task, we used one zero-shot prompt and one CoT prompt. The zero shot prompt shown in Table 10. For all models, we use temperature = 0.0. Max tokens was set to 10 for zero-shot and 1000 for CoT. The remaining hyperparameters were set at their default values: top p. top k, seed, min tokens, etc.
The CoT prompt uses the most original chain-of of--nnoo--uuhhh prompting method by appending "Let's think step by step" the end of the zero-shot prompt, with no other changes.
B.3. CoT Failure Example
example CoT prompt and output where GPT-4o fails for the facial recognition task Table 11.
B.4. Facial Recognition Difficulty Reduction Experiment
To show generalizability across variations each task, we conduct additional experiments varying the problems in each of the failure cases. For the facial recognition task, we relax the constraint that the faces need have the same verbal description, and instead we sample 5 faces with different descriptions form each recognition problem. For a visual representation, Figure 7.
Across 100 randomly selected sets within this easier paradigm, we find that CoT continues to drastically reduce model performance. GPT-4o has direct prompting accuracy of 0.61, but CoT accuracy is only 0.32, corroborating our findings that CoT reduces performance across facial recognition tasks.
B.5. Does CoT distort surface-level prerequisites instead of reasoning?
One possible explanation for CoT reducing LMM performance is that verbal processing distorts the ordering of the image options. order to determine whether CoT prompting introduces confusion related to image ordering, we conducted a simpler version the facial recognition task with just binary classification. We evaluate GPT-4o on 100 facial recognition problems, each consisting of only two images, and prompt the model judge whether they represented the same person.
17

---

Mind Your Step (by Step)
Figure Reduced difficulty in the facial recognition task; images are from original corpus but do not match the same description.
The dataset we test on consists of exactly half positive ("yes") and half negative ("no") answers.
In this simplified setting, GPT-40 achieved an accuracy of 72% under zero-shot prompting. However, accuracy decreased to 62% when using CoT prompting. These results align with our hypothesis that CoT can impair performance, even in simplified binary classification version of the task, suggesting that CoT disrupts the reasoning process rather than surface-level problem attributes in facial recognition problems.
C. Data with exceptions task
In this task, we analyze the effect that CoT prompting has on the ability of LLMs to learn a classification of objects that appear to follow pattern, but with exceptions. In these types of settings, Williams et al. (2013) reveal that when humans are given opportunities deliberate after receiving feedback, they learn more slowly and make more errors compared those who do not deliberate. The active form of thought mentally ingrains incorrect patterns that shift when exposed to successive unexpected answers, altogether leading to the creation of many deceptively incoherent lines of reasoning throughout the learning process that hinder the ability to directly keep track of the correct labels even after multiple passes.
C.1. Vehicle dataset generation
We build off of the experimental set-up in Williams et al. (2013) where in each trial, we first create a list of objects (vehicles) that are either warm- or cold-climate, which is the label which we want models to learn. Based on this label, we generate one feature that correlates with this target label completely (see Column 2 of Table 12), and flip this 20% of the time to create exceptions in the data.
In addition this discriminating feature, following Williams et (2013), we also include 1) one unique feature which different for each object and 2) three additional features whose values randomized and have no connection with the object class. The unique feature in the original experiment was vehicle color, which we replaced with the license plate for realism. An example setup is depicted in Table 12.
We sampled 240 sets of 10 vehicles each and prompt the model to learn the labels of the vehicles in multi-turn setting, which we detail below.
C.2. Prompts
Models are provided with text descriptions of vehicle's features one vehicle at time, iterating through the full set of ten vehicles repeatedly up to 15 times. Each time the model given a set of features, predicts the corresponding label and subsequently receives feedback for its answer. In contrast to previous experiments, the problems, the model's previous guesses, and the feedback given the model are all stored in-context and provided the model in its next prediction.
18

---

Mind Your Step (by Step)
In each iteration, the vehicles' order shown to the participant is shuffled. Prompting stopped when the model correctly classified all of the vehicles in one iteration, reached 15 iterations without performing this successfully. We used one zero-shot prompt and one CoT prompt. The zero-shot prompt was as follows in Table 13.
In the CoT condition, instead of replicating the human study and asking the model to deliberate after each piece of feedback, we modify the prompt asking the model to make a prediction. Specifically, we replace "Only answer with 'A' or "B'." with "Let's think step by step and answer with either 'A' or 'B', If you are unsure, feel free to guess and explain your reasoning".
We append the last sentence because we observed that sometimes the model would refuse to answer based on lack of information. While we could have also implemented deliberations after each feedback stay more faithful to the human experiment, our ultimate goal is to inform thainh,f- and CoT is most often applied during the process of asking questions to the model rather than having it reflect by itself. Furthermore, we believe that these settings are approximately cquivalent: Deliberation in human experiments would focus on explaining the feedback provided, but this is also the case in this paradigm because the model would perform reasoning on the previous feedback provided when performing CoT during the prediction of the next label.
For all models, we use temperature 0.0. Max tokens was set to 10 for zero-shot and 1000 for CoT. The remaining hyperparameters were set at their default values: top.p. top.k k, seed, min tokens, etc.
C.3. Per-round accuracy analysis
Figure depicts the aggregate accuracy (correctly predicted examples out of 10) of GPT-4o with direct and CoT prompts over 15 iterations through the list. Although CoT performs better than direct on the first iteration of the list, direct prompting quickly surpasses the performance of CoT by attaining perfect classification ability on the third iteration. Chain-of-thougg prompting stagnates in performance at an accuracy level equivalent to the percentage of exemplars whose class designation adheres to the corresponding first-glance generalizable rule (80%). This suggests that the verbal thinking of CoT biases the model towards predicting via generalizable rules, even when there are more useful features that map exactly to correct answers in context.
It is worth noting that CoT's tendency towards generalizable rules is often very helpful in other settings. For example, CoT does benefit from this tendency in the predictions of the first pass when all stimuli previously unseen. This is in line with our conclusion that different strategies for prompting should be chosen based on the task, and neither is always better than the other.
C.4. CoT Failure Example
An example CoT prompt and output where GPT-4o fails for the classifying data with exceptions task is in Table 14.
C.5. Robustness Analysis: Reducing Difficulty via Binary Oracle
We analyze how we can steer GPT-4o to attain consistently perfect list classification in fewer rounds with chain-of-thouge but at the same time, steering it to focus less on the distracting factor, namely, the pattern-related features as depicted in Table 12.
Initially, we performed an environmental intervention where we injected an oracle feature for each vehicle that directly mapped to the vehicle type (Class A or Class B) by altering the license plate feature to be 'license plate type', a binary feature (either '0' or I') instead of the original randomized six-character-long alphanumeric string. While an inconsistency between the pattern-related feature (hot climate or cold climate vehicle) and the vehicle type was still present as formed initially, Class A vehicles always had license plate type of 'O', and Class B vehicles always had license plate type of 'I' Direct prompting was able to consistently achieve perfect list classification in an average of 1.84 iterations - in several cases, direct prompting was able to achieve perfect classification on the first iteration due to an initial correct guess by chance that likely allowed the LLM to internalize the direct connection between license plate type and vehicle class. However, CoT prompting still performed considerably worse, failing to achieve perfect classification in the maximum number of iterations alloted (7) in 29 out of 30 trials. Notably, CoT prompting only allowed the LLM to pick up on the relationship between license plate type and vehicle class in only one trial, allowing the LLM achieve perfect list classification on the second iteration.
19

---

Mind Your Step (by Step)
C.6. Steering Analysis: Focusing the Feature without Exceptions
Next, we analyzed the use of prompt steering explicitly nudge GPT-4o not consider the distracting pattern-related feature in its CoT reasoning chains. For this series of experiments, used the original features shown in Table 12. For the first attempt, the end of both Direct and CoT prompts shown in Tables 13 and 14, append the following prompt:
Focus on the license plate of the vehicle and not the other features.
We run 30 independent trials each with maximum of 15 list iterations alloted. Direct prompting eventually achieved perfect classification in every trial in an average of 4.17 trials. CoT prompting still performed worse, achieving perfect classification in only 11 trials, although with a slightly yet statistically significant fewer 10.47 average trials (8.19, 12.74) compared to the original average 12.5 trials needed (p < 0.05).
To make the steering even more explicit, we also try appending the following prompt:
When trying to predict. first carefully check if any previously seen vehicles match the same license plate. Do your best to keep track along the way.
We run 15 independent trials each with a maximum of 15 list iterations alloted. Direct prompting achieved perfect classification every single trial in either iterations, with an average of 2.2. We also observe noticeably improved performance with CoT prompting, attaining perfect classification in 14 of the 15 trials, with an average of 3.6 trials needed.
In attempt to achieve equally high performance with direct and CoT prompting, we append the following promopt which is a combination of the above two:
Focus on the license plate of the vehicle and not the other features. When trying to predict, first carefully check if any previously seen vehicles match the same license plate. Do your best to keep track along the way.
We run 15 independent trials each with a maximum of 15 list iterations alloted. Both direct and CoT prompting manage to achieve perfect classification on the second iteration in each trial.
These findings suggest that, with appropriately designed steering prompts, chain-of-thousee reasoning can overcome spurious feature distractions and match the efficiency of direct prompting. In particular, explicitly instructing the model to track and prioritize license plate information effectively eliminates reliance on irrelevant pattern-related features.
D. Logical inconsistency task
Here, participants were tasked to evaluate whether a set of two statements were logically inconsistent. Statement pairs followed two forms: The first statement was always of the form A + B, where - denotes implication, and the second statement was either of the form B or -B, where A denotes the boolean AND operation, and denotes boolean negation. If the second statement was of the form -B, the pair inconsistent, whereas the second statement was of the form -B, the pair consistent. Khemlani Johnson-Laird (2012) found that if you ask humans to deliberate specifically as why A B plausible, they would subsequently be less accurate identifying logical inconsistencies between the statements.
D.1. Logic dataset generation
To construct the dataset for the task, we first assigned claims A and B, and then filled in the template to construct the actual statements. To do the first part, we took statements where A + B made logical sense following Khemlani & Johnson- (2012). While the original authors simply hand-constructed 12 pairs of claims, we use a combination of natural language inference (NLJ) datasets where pairs of statements are filtered to be of the "entailment" condition: MNLI, SNLI, and a synthetic datset generated by prompting GPT-4o using the prompt:
Generate a list of 100 true statements of the format "if A then B". For each statement generate the result in JSON format with separate fields for index, and B.
20

---

Mind Your Step (by Step)
To construct the actual statements, fit A and B into the templates in Table 15.
In addition, to avoid having entailment pairs where statements are more than one sentence long contain multiple clauses, we limited the maximum amount of words per claim (A or B) to seven. This allowed the sentences in the problem to flow smoothly, while still maintaining a large population of entailment pairs. In total, we conducted experiments on 675 pairs from SNLI, 833 pairs from MNLI, and 100 pairs of claims that were synthetically generated, for a final sum of 1608 pairs of {A, BJ. This corresponded to 3216 questions asked per model, over which we calculated model accuracy.
D.2. Prompts
We prompted models using one zero-shot prompt and two CoT prompts. The prompt in the zero-shot condition was as follows:
The two offthhutttthh prompts altered the last line in the prompt to the following two sentences, respectively:
Can both of these statements, as explicitly stated, be true at the same time? Please reason about your answer and then answer "Yes" or "No""
Can both of these statements, as explicitly stated, be true at the same time? Please first explain why statement 2 could be true and then answer "Yes" or "No".
Here, the first prompt follows the standard "reason about your answer before answering" CoT request, whereas the latter is more specific request aimed at more closely replicating the human study.
For all models, we use temperature = 0.0. Max tokens was set to 10 for zero-shot and 1000 for COT. The remaining hyperparameters were set at their default values: top-p, top_k, seed, min_tokens, etc.
E. Spatial intuition task
In this task, participants were given drawings of two drinking glasses, one filled with water and one empty. They were asked to estimate the level of water that the second glass would need to be filled to such that the two glasses, when tilted to certain degree, would have the water they contain reach the rim of the glass at the same angle (Schwartz & Black, 1999).
To simplify the task for the model, we changed the task from drawing a line (image manipulation) to multiple choice (text output) by marking four separate heights on the side of the empty glass, labeling them A through D, and asking the model to select letter.
F. Motor simulation task dataset generation
To scale up our dataset, instead of fixing the dimensions of the glass that contains water, we varied the width and height in (2, 3. 4] and (4, 5. 6) respectively (units are per 100 pixels). Then, following Schwartz & Black (1999), we created scenarios where the width and height of the empty cup was (wider, less wide, same width) and (taller, less tall, same height). We also varied the amount of water that was in the original glass between (1) 1 3 of its total height, Altogether, this resulted in 243 unique combinations of problems compared to the original 9.
2 4)
For each problem, we computed the exact height h that the empty cup would need to be filled with water to in order to get the water to the rim at the desired angle. Then, we sampled from Gaussian noise
24 ~ N(
in order to generate the other answer choices {a, = h + i E (1,2,3)}, where o² is half the distance from the correct answer to the maximum height of the glass. Furthermore, we ensured that none of the answer choices provided were above the maximum height of the cup, below zero, or within distance e of each other. was an empirically determined parameter that controlled the difficulty of the problem, while also having a lower bound due to a limit for how closely the multiple choice letter options could be to each other on the graphical representation of the empty glass. A visual representation of the final problem setup in Figure 8.
21

---

Mind Your Step (by Step)
<
Figure An example the water problem presented to large multimodal models. The glass on the left filled with water, and the task to determine which letter choice the empty glass should be filled to such that when the two glasses tilt to the same angle, water reaches each their rims at the same time.
E.2. Prompts
We use one zero-shot prompt and one CoT prompt. The zero-shot prompt is shown in Table 17. For the CoT prompt, replaced "Do not include anything else" with "Let's think step by step".
For all models, we use temperature = 0.0. Max tokens was set to 10 for zero-shot and 1000 for COT. The remaining hyperparameters were set at their default values: top-p, top.k, seed, min_tokens, etc.
F. Working memory preference task
In this task, participants were shown individual statements about one of four apartments frequently in succession. Each statement describes different aspect of one apartment, and participants' tasks were to determine which apartment was overall most favorable. However, due to limits human working memory, their performance in identifying the most beneficial apartment decreased when they tried to reason about the features of each apartment.
F.1 Apartment dataset generation
To extend this task to LLMs, we scaled up the number of stimuli to hopefully induce an increased pressure on the long-context capabilities of the model. Towards this effort, we first tested the limit of the amount different features an apartment could have with the help of GPT-4o. We found that the model started repeating aspects of apartments after around 80 unique features. Then, we asked the model to generate positive, negative, and more neutral versions of statements regarding these features:
"Generate 80 positive statements about different aspects of an apartment. None of the statements should be about the same aspect."
"The following are 80 positive statements about aspects of an apartment. For each, generate a corresponding negative statement that is the exact opposite. Make sure that all of the negative statements can coexist with positive statements that are not its direct correspondent. [positive statements]"
"The following 80 positive [...] For each, generate a corresponding neutral statement that is about the same aspect, is worse than the positive version, but is not negative. Make sure that all of the neutral statements can coexist with positive statements that are not its direct correspondent. [positive statements]"
"The following 80 negative [...] For each, generate corresponding neutral statement that is about the same aspect, is better than the negative version, but is not positive. Make sure that all of the neutral statements can coexist with negative statements that are not its direct correspondent. [negative statements]"
We then manually considered conflicts between pairs of statements that were not of the same feature, and manually replaced
22

---

Mind Your Step (by Step)
the only feature statement that had a conflict with another feature. Thus, cohesive descriptions of apartments could be sampled by randomly selecting one of the four statements for each of the 80 features.
Next, we asked GPT-4o to rate the importance of each statement based on how much the "statement affects the desirability of the apartment for the average tenant, from -5 to 5, with 5 being most desirable". Based on this, we could estimate the ground truth quality of each apartment by making the assumption that the features' utilities sum up linearly. We then randomly sampled apartments with one statement per feature, and computed the score of an apartment as the mean of the feature scores. We then constructed sets of four apartments where the best apartment had at least an average score A {[0.1, 0.33, [0.3, 0.5), [0.5, 1]] higher than the next-best option. This was to ensure that there is a clear best apartment for the average tenant while not making the task too simple, which were also requirements in the original human study (Dijksterhuiss 2004). Intuitively, A can be considered as a difficulty level, where apartments are closer in rating for lower A problems and are thus harder to get correct.
Sampling randomly, this led to a total of three datasets corresponding to three ranges of A, each containing 100 sets of four apartments.
Separately, we note that our implementation of this task favors models over humans due to humans being unable to reference the statements after viewing them for the initial 1 second. We recognize that there are other implementations of this task that would be similarly less favorable to models, including simulating partial forgetting by masking some of the sentences. However, since there are no guarantees that performing something like this would be functionally equivalent to how humans process the provided statements, we opted for what we believe is closest to how present models would solve this task in practice.
F.2. Prompts
For this task, we used one zero-shot and one CoT prompt in our evaluations. The zero-shot prompt is shown in Table 18, The CoT prompt replaces "Respond with only the number of the apartment, do not include anything else." with "Let's think step by step".
In our pilot experiments, we also tried variety of prompts such as replicating the distractor task using a verbal n-back task, setting time limit for the model (i.e., "you have three minutes to think about the problem") or using phrases such "very carefully think" that were present in the original experiment, but the first resulted in too much noise whereas the latter two did not change the results.
For all models, we use temperature = 0.0. Max tokens was set to 10 for zero-shot and 8000 for CoT because reasoning chains did not finish in 1000. The remaining hyperparameters were set at their default values: top-p, top_k, seed, min_to,ens, etc.
G. Details on Bootstrapping Analyses for the Effectiveness of our Heuristic
We conduct two analyses to examine the effectiveness of our psychology- inspired heuristic for finding CoT failures: magnitude-based performance decreases and frequency-based decreases irrespective of magnitude. In both bootstrapping analyses, we use the following protocol:
For the larger population, we take all evaluations that compare zero-shot and CoT in a recent metastudy, Sprague et al. (2025), for total of 378. Models evaluated include Llama 2 7b, Mistral 7b, Ilama 3.1 8b, Llama 3.1 70b, Gemma 2 9b, Phi-3 Small 8k, Qwen 2 7b, Qwen 2 72b, GPT-4o Mini, Gpt-4o, Claude-3 Haiku, Claude-3.5 Sonnet, Gemini 1.5 Flash, and Gemini 1.5 Pro. Tasks evaluated span various domains such as mathematical reasoning (e.g., GSM8k-Hard), commonsense reasoning (e.g., CommonsenseeAA, soft reasoning (c.g., AGIEval LSAT AR Soft Reasoning), and various commonly used benchmarks (e.g., MMLU-Pro, Big-Bench Hard).
For our experiments, we take all comparisons between zero-shot and COT in our 6 tasks, for a total of 50. These are exactly all of the comparisons that we list in tables 1-6 in the main paper. For task 3, our main metrie was number of rounds and not accuracy, so we replaced this with the difference in classification accuracy (e.g., y-axis of Figure 2).
For each comparison, we take the percentage accuracy decrease (consistent with Sprague et al. (2025)) and use this as the value of the datapoint. We then bootstrap 100,000 samples of size 50 from the population and compute the mean percentage
Note that this is sometimes untrue; e.g, close proximity to grocery store is much more meaningful when an apartment has a kitchen.
23

---

Mind Your Step (by Step)
accuracy decrease. None of these 100,000 means lower than the average percentage mean that we obtained in experiments.
For each comparison, we labeled accuracy decreases from CoT compared to zero-shot. Separate to the previous analysis, we bootstrapped 100,000 samples of size 50 and counted the number of accuracy decreases. Only 11 of the 100,000 samples had more instances of performance decreases than the 50 datapoints in our experiments.
Both results show that our heuristic is highly statistically significant at finding CoT failure cases.
24

---

Mind Your Step (by Step)
Table 8. Example CoT prompt and GPT-4o output for artificial grammar learning task.
Prompt:
Consider the following letter strings:
FDMQUQC FDQMUMR DZGBDMR DGUMQQR DGUMUMR FDXDC FDMUXDC DGUMQR FDQXDC DGBDC FDMQQR DGQQR DZZGR DGQQUC FDMUMR
These strings were generated according to certain set of rules, Does the following string also follow the same set rules?
DGBDQC
Please reason about your answer before answering "Yes" or "No".
Model output:
determine whether the string "DGBDQC" follows the same set of rules as the given strings, we need to analyze the patterns and rules that might be present in the provided strings.
Let's break down the given strings identify any patterns:
**Lnegth Consistency All strings have a length of characters.
2. **Character Positions**;
The first character can be either 'F' or 'D'.
The second character can be 'D', 'G', 'Z'.
The third character can be 'M', 'Q", 'G', 'X', or 'Z'.
The fourth character can be 'Q', , 'U', 'X', 'B', 'D'.
The fifth character can be 'Q', 'U', 'X', 'B' D
The sixth character can be 'C', 'R', or 'D".
The seventh character can be 'C', 'R', DD'
3. **Reeetition and Pairing**;
Some strings have repeated characters, such as "FDMQUQC" and "DGUMQQR".
Some strings have specific pairs of characters that appear together, such as "UM" in "DGUMUMR" and "FDMUMR".
4. **Character Sets**:
The characters used are from the set D, M.
Now, let's check the string "DGBDQC":
1. *#nggg***: The string "DGBDQC" has 6 characters, which is different from the 7-character length of the given strings.
2. **Character Positions** **;
The first character 'D' is valid.
The second character 'G' is valid.
The third character 'B' is valid.
The fourth character 'D' is valid.
The fifth character 'Q' valid.
The sixth character 'C' is valid.
However, the string is missing seventh character.
Given that the string "DGBDQC" does not match the 7-character length rule observed in all the provided strings, it does not follow the same set of rules.
Therefore, the answer is *****
Ground truth: Yes
25

---

Mind Your Step (by Step)
Table 9. Results comparing zero-shot, CoT, and ToT on a subset the artificial grammar learning task.
Performance Performance decrease (ToT)
p-value (CoT)
p-value
Zero-shot
CoT
ToT
decrease (CoT)
(ToT)
GPT-4o
94.00%
62.52%
64.55%
31.48%
29.45%
0.0001
0.0001
Table 10. Example prompt for facial recognition task, zero shot.
Prompt:
Here is image of a person.
jimage of initial person
Select the image that contains the same person the person in the first image.
five images of possible matching l'aces]
Table IL. Example CoT prompt and GPT-4o output for facial recognition task.
Prompt:
Here is an image of person.
Let's think step by step before selecting the image that contains the same person the person in the first image.
Image
Image 3
Image
Image
Model output:
To identify the person from the first image. closely compare distinct features such as hairstyle and the visible part of clothing. The first and the third image show the same ear shape and hairstyle. Hence, same person appears the third image.
Ground truth:
Image (2nd row, leftmost)
Unique features
Pattern-related features
Irrelevant features
License Plate
Colld' (Class War'mrm'' (Class B) climate
Transmission
Seat covers
Doors
A23BCD
Drives on glaciers
Manual
Cloth
Two
B34ERG
Made in Norway
Automatic
Vinyl
Two
C4SHIJ
Used in mountain climbing
Automatic
Vinyl
Four
D56KLMM
Drives jungles
Manual
Vinyl
Four
E67NOP
Has treads
Manual
Cloth
Two
F78QRS
Heavily insulated
Manual
Vinyl
Four
G89TUV
Made in Africa
Manual
Cloth
Four
H90WXY
Has wheels
Automatic
Cloth
Two
J12ZAB
Lightly insulated
Manual
Vinyl
Two
K23CDE
Used safaris
Automatic
Vinyl
Two
Table Sample vehicle classification list. Boldened features indicate flipped labels that break the initial classification pattern.
26

---

Mind Your Step (by Step)
Table Example prompt for vehicle classification task, zero shot. [Chat history including previous prompts, model predictions, and feedback]
Prompt:
The vehicle description follows:
License plate: [license plate]
Descriptor: [descriptor]
Transmission: [transmission]
Seat Cover: [seat cover]
Doors: [doors]
Is this vehicle likely he Class A Class B vehicle? Only answer with 'A' 'B'.
27

---

Mind Your Step (by Step)
Table 14. Example CoT prompt and GPT-4o output for vehicle classification task.
Iteration 0:
[Previous conversation, including questions, outputs, and feedback]
Question
The vehicle description is follows:
License plate: G5IXVQ
Descriptor: Drives jungles
Transmission: Automatic
Seat Cover: Cloth
Doors: Two
Is this vehicle more likely to be Class A or Class B vehicle? Let's think step by step and answer with 'A' or 'B', If you are not sure, feel free to guess and explain your reasoning.
Model output 2:
Given the new vehicle description:
The vehicle drives in jungles, which suggests may be specialized vehicle with off-road capabilities. It has an aulomatic transmission, which is common in both commercial and personal vehicles.
The seat cover is cloth, which is more common in personal vehicles but not exclusive to them.
has two doors, which could indicate smaller vehicle, but this is not definitive. Based this description alone, there is clear indication that the vehicle is designed for heavy commercial use, which would typically be associated with Class A vehieles.
The descriptor of driving in jungles implies a need for maneuverability and possibly a smaller size, which might more characteristic of Class B vehicle.
However, since the previous vehicle with similar non-commercial descriptor was stated to be Class A vehicle, despite the expectations, it's possible that the classification being used here does not follow the typical commercial vehicle classification system there may be additional context that not provided.
Given the lack of information and the previous correction stating that the glacier-driving vehicle is Class A vehicle, it's difficult to make an accurate guess.
However, if follow the same logic as before, we might assume that this jungle-driving vehicle could also be classified as Class A vehicle, despite the description not aligning with typical Class A characteristies. Therefore, based on the previous correction, the guess would be:
But please note that this is guess and the actual classification could vary based on the specific criteria used for this classification system.
Feedback 2:
Your answer is incorrect! The vehicle is in class B.
Question
Iteration
Table 15. Sentence template for logical inconsistency task. Statement 1: If [A]. then it always the case that [
Statement 2 (conflict): [A], but it is not the case that [B].
Statement (no conffict): It not the case that [B].
28

---

Mind Your Step (by Step)
Table 16. Example prompt for logical inconsistency task, zero shot.
Prompt:
The following are statements:
1. Statement 1]
2. Statementt 2]
Can both of these statements, as explicitly stated, true at the same time? Please ONLY answer with "Yes" or "No".
Table 17. Example prompt for spatial intuition task, zero shot.
Prompt:
On the left of the figure provided, there is rectangular glass with its water level indicated with the blue line. the right, there a target glass. The target glass has four height markings (A, B, C, D), each indicating different water level. Which marking should the target glass be filled to that both glasses pour at the same angle?
Provide your answer using the letter that matches the correct height marking. Do not include anything else.
Prompp image)
Table 18. Example prompt for working memory apartments task, zero shot.
Prompt:
You are Al assistant designed to evaluate the desirability of four upartments for a potential tenant. You will be given list of statements about the apartment candidates and how much the tenant likes or dislikes an apartment with the quality described by the statement. Your task is to determine which apartment is the most desirable based on the given critcria. The statements follows: [statements]
Which apartment is most desirable to the tenant? Respond with only the number of the apartment, do not include anything elsc.
29


Reviews:
[{"review_id": "7jgsL2Ffub", "reviewer": "Reviewer_HPne", "summary": "As Chain-of-thought (CoT) prompting becomes a widely used practice, this paper aims to answer the limitations of the approach. Authors propose a \"heuristic\" for determining limitations of CoT by drawing a comparison between CoT prompting and humans engaging in verbal thought. Inspired by psychological literature, six tasks where it's claimed that verbal thought perils performance were chosen and adapted them to evaluate LLMs and LMMs. \n\nExperiments show that three of these tasks see a drastic decrease in performance caused by adding CoT to zero-shot prompting, and the effect is more pronounced in stronger models. In other tasks, however, performance was either improved or not affected by CoT. Some bootstrapping results were provided to prove the effectiveness of the heuristic of task choosing.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Methods And Evaluation Criteria 2:**\n\n“I'd like to see more overall evaluations on whether the given heuristic is providing trustworthy predictions of CoT impairment.”\n\nWe agree with the reviewer that more depth is better. We add **4 additional experiments**:\n1. Artificial grammar learning, varying complexity of underlying FSG\n2. Facial recognition, varying level of similarity between faces\n3. Classifying data with exceptions, binary oracle feature\n4. Ablation on temperature sampling\n\nAcross all these experiments, we find large decreases in CoT performance compared to zero-shot, demonstrating the within-category robustness of our findings. Please see our response to reviewer 3 (fD9f) for details on each additional experiment. \n\n**Claims And Evidence 1, Methods And Evaluation Criteria 1 & 3**\n\n“How the 6 tasks were chosen based on the heuristic [...] \"cannot provide verbal basis\"”\n\n“How should the heuristic be applied to a broader task pool”\n\n“practicability and informativeness”\n\n**Choosing tasks based on the heuristic:**\n\nIn Section 3, we provide an overview of the verbal thinking literature, including that artificial grammar experiments found that humans “cannot provide verbal basis” for their judgments. However, in section 4.1, paragraph “Human failure”, we specify that “In the artificial grammar learning task, humans prompted to verbalize performed more poorly than those who were not so prompted (Fallshore & Schooler, 1993)”. For each of the other tasks, we also justify the choice in section 4. \n\nWe recognize that this could be better structured for readers, and we have redrafted section 3 for the camera ready to provide better clarity on why each task was chosen. \n\n**How the heuristic can be applied to a broader task pool:**\n\nWe chose the tasks based on the six most well-studied categories of human verbal thinking failures. However, each category does not only consist of one such task, but instead a broad range (e.g., verbal overshadowing does not only apply to facial recognition, but also phenomena like wine-tasting! (Melcher and Schooler, 1996)) Thus, for other tasks, we could search for stimulus patterns (such as reliance on another modality) that are based in psychological findings, and predict that e.g., due to verbal overshadowing, performance of that task with CoT would also be poor. \n\nWe do acknowledge that this approach is limited in that it only covers patterns or categories that are studied in the psychology literature. At the same time, it nicely complements existing ML approaches (e.g., Sprague et. al., 2024). We highlight this limitation in our discussion (“Scope of application”, page 8).\n\n**Claims And Evidence 2:**\n\n“Brevity of the bootstrapping section. There's no explanation of the setting and illustration of complete results to make this section convincing enough”\n\nWe agree that this could be more detailed. We have updated the writing to include the following points (we omit the full subsection due to space constraints). \n\nFor the larger population, we take all evaluations that compare zero-shot and CoT in a recent metastudy, Sprague et al., (2024), for a total of 378. \nModels evaluated include Llama 2 7b, Mistral 7b, Llama 3.1 8b, Llama 3.1 70b, Gemma 2 9b, Phi-3 Small 8k, Qwen 2 7b, Qwen 2 72b, GPT-4o Mini, Gpt-4o, Claude-3 Haiku, Claude-3.5 Sonnet, Gemini 1.5 Flash, and Gemini 1.5 Pro. Tasks evaluated span various domains such as mathematical reasoning (e.g., GSM8k-Hard), commonsense reasoning (e.g., CommonsenseQA), soft reasoning (e.g., AGIEval LSAT AR Soft Reasoning), and various commonly used benchmarks (e.g., MMLU-Pro, Big-Bench Hard). \n\nFor our experiments, we take all comparisons between zero-shot and CoT in our 6 tasks, for a total of 50. These are exactly all of the comparisons that we list in tables 1–6 in the main paper. For task 3, our main metric was number of rounds and not accuracy, so we replaced this with the difference in classification accuracy (e.g., y-axis of Figure 5). \n\nFor each comparison, we take the percentage accuracy decrease (consistent with the Sprague et. al., paper) and use this as the value of the datapoint. We then bootstrap 100,000 samples of size 50 from the population and compute the mean percentage accuracy decrease. None of these 100,000 means were lower than the average percentage mean that we obtained in our experiments. \n\nFor each comparison, we labeled accuracy decreases from CoT compared to zero-shot. Separate to the previous analysis, we bootstrapped 100,000 samples of size 50 and counted the number of accuracy decreases. Only 11 of the 100,000 samples had more instances of performance decreases than the 50 datapoints in our experiments.\n\n**Other Responses**\n\nDue to rebuttal length limitations, we were not able to provide all our answers in the box provided. For answers to the remaining questions, we provide the following anonymous link: https://cryptpad.fr/doc/#/2/doc/view/YYLHK9InmafRO6yRL0legsg2tB4IUv6KY-3Whm+ddhw/", "claims_and_evidence": "The major claim of the paper is the effectiveness of the proposed heuristic: LLMs/LMMs perform worse with CoT in tasks where verbal thinking perils humans. Authors claim that, although not perfect due to intrinsic differences between humans and models, the heuristic allows for a more efficient identification of tasks unfit for CoT. The claim was supported by choosing 6 tasks guided by the heuristic, evaluating and analyzing effects of CoT, and conducting bootstrapping experiments to prove that the effectiveness of the heuristic is beyond random selection.\n\nThe claim is fairly intuitive and, given the significant experiment and bootstrapping results, largely convincing. The experiments involved a wide variety of models and produced results with high confidence in some scenarios; there are also experiments corresponding to scenarios where authors predicted the heuristic would be less convincing; bootstrapping against random selection is considered too.\n\nHowever, I find it hard to be convinced by a qualitative heuristic. It's not explicitly stated, not to mention theorized, how the 6 tasks were chosen based on the heuristic, and nor is how should the heuristic be applied to a broader task pool. Therefore I don't see how the experiment results conclude the effectiveness of the heuristic. And this is not obvious: for some of the tasks, psychological literature only claim that participants \"cannot provide verbal basis\" rather than \"perform worse when providing verbal basis\"; why are such tasks chosen based on the heuristic?\n\nAnother thing is the brevity of the bootstrapping section. I imagine ablation studies in proving this claim is quite important, but there's no explanation of the setting and illustration of complete results to make this section convincing enough.", "methods_and_evaluation_criteria": "The method proposed is to detect unfit-for-CoT tasks based on whether verbal contemplation perils human performance in these tasks. Evaluation criteria is a subset of intuitive reasoning tasks in psychology literature, on which experiments were performed to study the effect of CoT. \n\nAs mentioned before, the \"heuristic\" proposed is a little confusing as a method: it lacks in practicability (i.e. how to determine extents to which tasks are appropriate for CoT in an arbitrary setting) and informativeness (i.e. what aspects and what extents of human under-performance under verbal deliberation signals a unfit-for-CoT task).\n\nEvaluation was more focused on performance change of models with respect to CoT addition in separate tasks. Indeed promising results that align with authors' prediction were provided, but I'd like to see more overall evaluations on whether the given heuristic is providing trustworthy predictions of CoT impairment. Otherwise, this paper would seem more like a case study of discrete tasks that's a little less powerful in supporting the method.\n\nI'm aware of the bootstrapping study, but the setting and results are still a little hazy. What does the 378 comparisons consist of? Are the 6 tasks newly selected to the same as before? If so, why sample multiple times again, as it is already done in the main experiments?", "theoretical_claims": "I didn't see explicit theoretical claims and/or proofs, except for the comparison between human underperforming with verbal contemplation and LLM/LMM underperforming with CoT. This claim is comprehensive in that multiple defects of the comparison were discussed, and that it's based on solid psychological evidence, but I don't think it's a provable theoretical claim, at least it's not framed to be one.", "experimental_designs_or_analyses": "The main experiments were conducted with 6 tasks based on the psychological heuristic on models with ranging capabilities. Results for each model in zero-shot prompting and CoT prompting were provided, as well as p-values. Detailed analyses were present in both experiments where the hypothesis hold true and false. \n\nHowever, some details of the results were not in line. In some experiments, o1-preview was used as the CoT version of GPT-4o, while other experiments directly used GPT-4o with CoT; in some tasks, some models were evaluated on subsets of the problems while others were evaluated on the entire problem set; p-values were provided in some studies but not others. Reasons for these disparities were not explicitly stated (or maybe I missed them?)\n\nThe analyses were a little lacking to me as well. For example, it's claimed that in a certain task “CoT often improved performance, attributable to both the low base performance and the logical reasoning component\", but can these be solved as well by few-shot prompting? Does CoT improve performance in few-shot settings as well? Moreover, sound explanations for situations where performance drop caused by CoT is less pronounced (even negligible) for weaker models were lacking.", "relation_to_broader_scientific_literature": "Previous studies on limitations of CoT have focused on the computational expenses, problems caused by its sensitivity to prompting and problems in training (i.e. over-fitting, difficulties in evaluation). This paper provides a new light on the intrinsic incompatibility of some tasks, encouraging carving a clear boundary for applications of CoT.  Should the authors solidify the method (i.e. the proposed \"heuristic\"), I believe this would be a novel line of work to build upon.", "essential_references_not_discussed": "I'm not aware of any.", "other_strengths_and_weaknesses": "The introduction of psychological studies and the parallel between human performance and predictions with LLM is brilliant; authors also removed previous assumptions that CoT's perils arise mainly from sensitivity to prompts, and focused on tasks themselves.\n\nClarity of writing needs improving, especially in that figures showing experiment results and settings are lacking. The lack of theoretical framework also seems like a problem.", "other_comments_or_suggestions": "Please refer to the previous sections.", "questions_for_authors": "I've stated some of mine confusions in the 'Claims', 'Method' and 'Experiments' section; It'd be great if these confusions can be answered. One of the more important questions is clarification on the setting and results of your bootstrapping experiment.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DMWe7VceDY", "reviewer": "Reviewer_28yY", "summary": "This work delves (no, not written by an LLM) into the conditions under which CoT works. Often in ML research, only positive results are presented, and the many things that don't work never see the light of day.  The contribution of this work is firstly to make explicit that CoT doesn't always works, and more importantly to test a hypothesis of why/when it doesn't work. As a good starting point, the authors look at the behavioral psychology literature, and shortlist 6 tasks for which human performance drops with (more) thinking. Proxy tasks are then performed by a number of SOTA LLMs/LMMs to investigate if CoT also causes performance drops in these models.\n\n## Update after rebuttal:\n\nIn review of the additional experiments and the reasonable responses, I upgrade my review from 2 to 3. However, overall while I like the approach and sympathize with the conclusions, I think the piece of work could still be improved significantly. **If there were a 2.5 rating option, I would select that.**\n\nRegardless of acceptance/rejection, I think the work may not be entirely convicing to a more traditional ML/CS readership. I urge the authors to consider how their case can be made more convincing, and whether perhaps a different venue (if rejected) or additional venue for follow-on work (if accepted) with a different readership/slant may be better.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful review. We are also grateful that the reviewer appreciates our approach of providing our full results rather than cherry-picking cases.\n\n**Claims and Evidence 1:**\n\n“While the paper purports to \"understand and predict\" when CoT has a negative effect, the paper doesn't really predict in the rigorous or quantitative sense.”\n\n**Response:**\n\nThe sense in which we are able to predict cases where CoT has negative effects is that we are able to identify risky tasks based on the psychological literature, for which we do find large negative effects from CoT in model performance. \n\nWe do agree that this is a looser interpretation of “predict” — however, the failure cases that we do find are within structured categories that share psychological explanations, thus allowing us to rigorously predict reductions in performance for altered variations of these tasks.\n\nTo illustrate this, we provide results from **3 additional experiments**: \n1. Artificial grammar learning, varying complexity of underlying FSG\n2. Facial recognition, varying level of similarity between faces\n3. Classifying data with exceptions, binary oracle feature\n\nThese additional experiments each have changes that significantly differentiate them from their original counterparts by varying difficulty/complexity. Across all these experiments, we find large decreases in CoT performance compared to zero-shot, demonstrating the within-category robustness of our findings. Please see our response to reviewer 3 (fD9f) for details on each additional experiment. \n\n**Claims and Evidence 1, pt 2:**\n\n“It would have been much stronger if ultimately a large variety of tasks were performed by the LLMs/LMMs with and without CoT, and some sort of predictive computational model trained. In other words, what we really want (but the authors didn't do) is that given a new/unseen task, predict accurately whether CoT should be used or not.”\n\n**Response:**\n\nWe agree that this a valuable goal for future research. However, \nit’s also worth mentioning that this is incredibly hard: In decades of overthinking research in psychology, there is still no progress towards a generalizable “overthinking” classifier for humans. Thus, it seems reasonable that building something similar for LLMs would also be very difficult. \n\nThe current state of literature on CoT largely focuses on tasks developed in the NLP literature such as MMLU. Our paper builds towards the goal the reviewer suggests by exploring a novel set of tasks inspired by the psychology literature, which we find result in a number of large negative effects of CoT. Such cases are uniquely informative for understanding the limits of CoT, and relevant to developing better predictive models. \n\nAt the same time, we certainly do not claim to have found the predictive computational model that the reviewer proposes. We highlight this in our discussion (“Scope of application”, page 8):\n\n“While our psychology-based heuristic offers a strategy for identifying failure cases of CoT, it is unlikely to cover all cases where CoT decreases performance. Existing psychological research has been guided by a variety of theoretical and practical considerations, but does not offer an exhaustive or representative sample of all tasks, and will miss cases that are uniquely interesting to study in models but not humans. Thus, we envision our contribution to be complementary to existing evaluation methods in natural language processing.”\n\n\n**Claims and Evidence 2:**\n\n“If only 3 of the 6 tasks had clear negative effects in models (i.e. like humans), then there are very mixed results, and it's not clear whether the work can make any strong claims, other than just reporting these results.”\n\n**Response:**\n\nIn previous studies of CoT, negative impacts are much rarer than those seen in the six cases we considered. To support the claim that our tasks resulted in a higher failure rate, we conducted a bootstrapping significance test that found that our method of searching for CoT failures is more effective than previous attempts. This includes quantifying by both failure magnitude (estimated p < 0.000001) and failures irrespective of magnitude (estimated p < 0.00011). See Section 4.5 in the paper for details. Thus, we believe that we can make the claim that our method for exploring CoT failure tasks is more efficient than (and also complementary to) previous endeavors.\n\n**Claims and Evidence 3:**\n\n\"strongly supporting the hypothesis that our heuristic is better than random at finding failure in CoT\"... this seems to be a very weak and uninteresting claim.\n\n**Response:**\n\nWe agree this claim was not particularly strong in its original statement. We have tried to clarify our claim by replacing this with “our heuristic is much more efficient than past endeavors at finding failures in CoT”, which is also more precise. We welcome further suggestions from the reviewer for how to phrase the takeaways of this section more clearly.", "claims_and_evidence": "1) I like the goal of this work, but ultimately it's an observational / analytical report. While the paper purports to \"understand and predict\" (Introduction, pg 1 right column top para) when CoT has a negative effect, the paper doesn't really predict in the rigorous or quantitative sense. It would have been much stronger if ultimately a large variety of tasks were performed by the LLMs/LMMs with and without CoT, and some sort of predictive computational model trained. In other words, what we really want (but the authors didn't do) is that given a new/unseen task, predict accurately whether CoT should be used or not.\n\n2) If only 3 of the 6 tasks had clear negative effects in models (i.e. like humans), then there are very mixed results, and it's not clear whether the work can make any strong claims, other than just reporting these results.\n\n3) On page 8, first para, the authors write: \"strongly supporting the hypothesis that our heuristic is better than random at finding failure in CoT\"... this seems to be a very weak and uninteresting claim.", "methods_and_evaluation_criteria": "Methods and evaluation criteria were generally sound.", "theoretical_claims": "Not applicable.", "experimental_designs_or_analyses": "Experimental designs / analyses were generally sound.", "relation_to_broader_scientific_literature": "Interesting contribution to investigate deeper the phenomenon of CoT, where many papers and model use it, but never discuss or show the setting / expts in which CoT doesn't help or makes things worse.\n\nThis is a negative (or unclear) result in the sense that of the 6 tasks tested, only 3 fulfilled the (implicit) expectation or hypothesis that CoT would produce performance reductions, as they did with humans. I mean this as a neutral comment (negative results can be valuable).", "essential_references_not_discussed": "Nil.", "other_strengths_and_weaknesses": "Nil.", "other_comments_or_suggestions": "Nil.", "questions_for_authors": "Nil.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kgQ18OMoqs", "reviewer": "Reviewer_fD9f", "summary": "This paper aims to explore the settings where CoT reduces performance from the perspective of cognitive psychology. It focuses on six tasks where extensive thinking affects human performance. For three of these tasks, the authors find that current models experience a performance drop when allowed more reasoning. In the other three tasks, models exhibit both positive and negative effects. This work demonstrates how insights from psychology literature can inspire the evaluation of LLMs and inference-time reasoning.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their encouraging review! \n\n**Weakness 1:**\n\n“It is difficult to determine the generalizability of the findings, especially since each task is specifically designed for its respective category.”\n\n**Response:**\n\n**A. Generalizability within categories**\n\nThe studies we choose are the most classic or well known studies in the psychology literature with respect to its category. In our experimental design, we strived to create the most representative task with minimal edits to the original study. In addition, we did not cherry-pick results — all the results represent the first iteration of testing said category. \n\n**B. Generaliability with respect to variations**\n\nTo show generalizability across variations of the task, we conduct **3 additional experiments** varying the problems in each of the failure cases. We find that our results are consistent across variations in difficulty, which provides further confidence in the generalizability of the findings:\n\n1. Artificial grammar learning, varying complexity of underlying FSG:\n\nWe conduct an ablation reducing the nodes in the finite state automata that generate the artificial grammars. While the original had 6 nodes, we iteratively reduce to 5, 4, and 3 nodes by merging nodes (see https://imgur.com/a/VCBQYEB). Across all valid FSGs with no unused nodes, we observe the following accuracies:\n\n5 nodes, zero-shot = 0.886, CoT = 0.766\n\n4 nodes, zero-shot = 0.837, CoT = 0.665\n\n3 nodes, zero-shot = 1.000, CoT = 1.000\n\nWe see across varying complexity that CoT consistently hurts performance.\n\n2. Facial recognition, varying level of similarity:\n\nWe conduct an ablation reducing the difficulty of the task. Instead of similar faces, we sample 5 faces with different descriptions. For a visual representation, see https://imgur.com/a/1PsqIzd. \n\nAcross 100 randomly selected sets, we find that CoT continues to drastically reduce model performance. GPT-4o has a direct prompting accuracy of 0.61, but CoT accuracy is only 0.32, corroborating our findings that CoT reduces performance. \n\n3. Classifying data with exceptions, binary oracle feature:\n\nIn this task, the oracle feature was license plates, which mapped to the correct category without exceptions. However, LLMs may find it difficult to build a map from 6 character license plates to a binary class. \n\nWe conduct an ablation where we change the oracle feature to a binary feature. We replaced “license plate” with “license plate type”, a feature with labels {0, 1}. Other features remained the same.\n\nWe evaluated GPT-4o with 25 trials for up to 7 passes over the list. Direct prompting took an average of 1.84 passes to get all labels correct, while only 1 / 25 CoT trials achieved perfect classification within 7 passes, so CoT took >250% more passes to learn all labels. This mirrors our findings that CoT hurts performance in this type of task. \n\n**C. Generalizability across categories**\n\nOur generative process for the categories of human failure tasks was as follows:\n\n1. Two senior cognitive scientists generated all cases they could come up with in which explicit or verbal thinking impairs human performance on some task. Our list branches cognitive psychology (e.g., task 1), perception (task 2), educational psychology (e.g., task 3), and spatial cognition (task 5). \n\n2. We categorized these under themes and chose tasks that were most representative of each category based on the literature. \n\n3. We then adapted these tasks to an LLM, ensuring that our dataset matches ML standards of scale and LLM / LMM applications, yielding our final 6. \n\nThus, we have reasonable belief that our list of categories is generalizable across the psychology literature. At the same time, we acknowledge that we are limited by the coverage of psychological literature, and thus generalizability across types of tasks is restricted (see Discussion, “Scope of application”). \n\n\n\n\n**Weakness 2:**\n\n“Most models are evaluated with a temperature of 0 during inference. Since the authors also mention advanced prompting methods such as self-consistency, it would be interesting to see whether the findings still hold when sampling multiple times using temperature sampling.”\n\n**Response:**\n\nThis is a great suggestion. We conducted some ablatons on different temperatures (t = 0.5, 0.7) across the full 4400 problems for the artificial grammar learning task on GPT-4o. Accuracies were as follows:\n\nt = 0, zero-shot = 87.5, CoT = 64.4 (original results for reference)\n\nt = 0.5, zero-shot = 88.3, CoT = 63.6\n\nt = 0.7, zero-shot = 87.8, CoT = 63.6\n\nThus, our results seem to be robust to variations in temperature sampling.", "claims_and_evidence": "The claims made in the paper are well supported by both psychological analysis and evaluation results.", "methods_and_evaluation_criteria": "The evaluation is the main contribution of this work and is well-performed. The paper also provides a detailed description of the evaluation data curation.", "theoretical_claims": "No theoretical claims or proofs were discussed in this work.", "experimental_designs_or_analyses": "The experimental design heuristics are derived from psychology literature, making them a reasonable choice for studying LLM behavior. The experiments are conducted on a variety of models but lack a consistent model size scaling.", "relation_to_broader_scientific_literature": "This work provides a fresh perspective on cases where CoT may fail to improve performance. It represents a novel contribution to the community and could inspire further exploration of the underlying reasons and test-time inference to mitigate such issues.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "**Strengths:**\n1. The paper is well-written and easy to follow, even for readers without a background in psychology.\n2. It draws inspiration from psychological findings and offers novel insights. The studied topic is of interest to the community.\n3. The overall evaluation is comprehensive and sound.\n\n**Weaknesses:**\n1. It is difficult to determine the generalizability of the findings, especially since each task is specifically designed for its respective category.\n2. Most models are evaluated with a temperature of 0 during inference. Since the authors also mention advanced prompting methods such as self-consistency, it would be interesting to see whether the findings still hold when sampling multiple times using temperature sampling.", "other_comments_or_suggestions": "NA", "questions_for_authors": "1. Is there a correlation between performance drop and model size? The current selection of models appears somewhat arbitrary, making it difficult to observe a clear scaling relationship. Have you conducted any analysis to examine this trend?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wCy5duH7Fp", "reviewer": "Reviewer_c7Md", "summary": "The paper investigates the conditions under which CoT prompting, a widely used technique to improve the performance of LLMs/LMMs, can actually reduce model performance. \n\nThe authors draw inspiration from cognitive psychology, focusing on six tasks where verbal thinking (deliberation) has been shown to impair human performance. They adapt these tasks to evaluate the impact of CoT on state-of-the-art models, finding that CoT significantly reduces performance in three of the six tasks (implicit statistical learning, face recognition, and classification of data with exceptions), while the effects are mixed or negligible in the remaining three (logical inconsistency, spatial intuitions, and working memory). \n\nThe paper suggests that tasks where verbal thinking harms human performance may also be problematic for models using CoT.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful review. \n\n**Essential References:**\n\n“Include a more thorough discussion of prior work on the limitations of CoT prompting (e.g., Kambhampati et al., 2024).”\n\n**Response:**\n\nThis paper is already cited in our related work (L124-126): “In related settings such as planning, there is little benefit from CoT prompting (Kambhampati et al., 2024)”. Feel free to suggest more references if others are missing.\n\n**Weakness 1 & 4:**\n\n“The proposed heuristic is reasonable, but lacks rigorous testing or validation.”\n\n“Could the authors provide more evidence or experiments to validate this heuristic?”\n\n**Response:**\n\nWe agree with the reviewer that more depth is better. We add **4 additional experiments**. We vary the difficulty of tasks 1–3, and conduct an ablation on temperature.\n\n1. Artificial grammar learning, varying complexity of FSG:\n\nWe conduct an ablation reducing nodes in the finite state automata that generate the artificial grammars. While the original had 6 nodes, we iteratively reduce to 5, 4, and 3 nodes by merging nodes (see https://imgur.com/a/VCBQYEB). Across all valid FSGs with no unused nodes, we observe the following accuracies:\n\n5 nodes, zero-shot = 0.886, CoT = 0.766\n\n4 nodes, zero-shot = 0.837, CoT = 0.665\n\n3 nodes, zero-shot = 1.000, CoT = 1.000\n\nWe see across varying complexity that CoT consistently hurts performance.\n\n2. Facial recognition, varying level of similarity:\n\nWe conduct an ablation reducing task difficulty: Instead of similar faces, we sample 5 faces with different descriptions. For a visual example, see https://imgur.com/a/1PsqIzd.\n\nAcross 100 randomly chosen sets, CoT continues to drastically reduce performance. GPT-4o has a zero-shot accuracy of 0.61, but CoT accuracy is only 0.32, corroborating our findings that CoT reduces performance.\n\n3. Classifying data with exceptions, binary oracle feature:\n\nIn this task, the oracle feature was license plates, which mapped to the correct category without exceptions. However, LLMs may find it difficult to build a map from 6 character license plates to a binary class.\n\nWe conduct an ablation where we change the oracle feature to a binary feature. We replaced “license plate” with “license plate type”, a feature with labels {0, 1}. Other features remained the same.\n\nWe evaluated GPT-4o with 25 trials for up to 7 passes over the list. Direct prompting took an average of 1.84 passes to get all labels correct, while only 1 / 25 CoT trials achieved perfect classification within 7 passes, so CoT took >250% more passes to learn all labels.\n\n4. For our temperature ablation, please see our response to reviewer 3 (fD9f).\n\n**Weakness 2:**\n\n“The tasks where CoT does not harm performance are not analyzed as thoroughly as the tasks where it does.”\n\n**Response:**\n\nWe focused on the negative cases because the vast majority of the literature so far has focused on the positive impacts of CoT. Understanding the negative cases can help us identify the settings where CoT is likely to fail in the future.\n\n**Weakness 3:**\n\nThe paper does not discuss alternative prompting strategies like Tree-of-Thought or Self-Consistency, which have been shown to improve reasoning in some cases.”\n\n**Response:**\n\nWe discuss both Tree-of-Thought (ToT) and self-consistency in the paper, and we also conducted an ablation on ToT for artificial grammar learning. See Discussion, L427-438, “Types of inference-time reasoning”.\n\nFor artificial grammar learning, ToT improved accuracy on GPT-4o (64.55% vs. 62.52%), but was still far from zero-shot performance (94.00%), providing support that our findings extend across these techniques.\n\nWe are happy to conduct more analyses before the camera ready if the reviewer believes it is imperative.\n\n**Other Comments:**\n\n“it is better to provide a more detailed discussion of the implications of the findings for the design of LLMs and LMMs and their reasoning prompting.”\n\n**Response:**\n\nGreat idea! We add the following to our discussion. Please feel free to suggest changes.\n\nImplications for the design of LLMs, LMMs, and prompts\nIn our experiments, we observe that CoT can also perform worse at certain types of tasks. This suggests that models should be flexible in choosing when to use CoT. Towards this, one promising direction is rational metareasoning — when people are faced with a task, they often make trade-offs between increased costs of reasoning and marginal benefits they would attain. We could prompt LLMs to do the same before solving a task. In this direction, De Sabbata et. al., (2024), trained LLMs to use intermediate reasoning steps only when necessary by incorporating a term into the reward function that penalizes unnecessary reasoning. Future works may further incorporate types of reasoning failures such as the ones we study into these training objectives.\n\nDe Sabbata, C. Nicolò, Theodore R. Sumers, and Thomas L. Griffiths. \"Rational metareasoning for large language models.\" arXiv preprint (2024)", "claims_and_evidence": "The claims made in the paper are generally supported by clear and convincing evidence. The authors conduct extensive experiments across multiple state-of-the-art closed-sourced models (e.g., GPT-4, Claude, Gemini) and provide detailed results showing significant performance drops when CoT is applied to tasks like implicit statistical learning and face recognition. \n\nIt is better to provide a deeper exploration of why CoT fails in these specific tasks, as the current analysis is somewhat surface-level.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are appropriate for the problem at hand. The authors adapt well-known psychological tasks to evaluate LLMs/LMMs, scaling them up to modern use cases. The evaluation metrics (accuracy, number of passes, etc.) are standard and well-suited to the tasks.", "theoretical_claims": "The paper does not make strong theoretical claims. The authors primarily focus on empirical results and the heuristic that connects human cognitive limitations to model performance under CoT.", "experimental_designs_or_analyses": "The experimental design is sound. The authors adopt tasks from psychological studies to fit the capabilities of LLMs/LMMs. \n\n- The paper does not explore the impact of different CoT prompting strategies in depth, which could provide more insights into why CoT fails in certain tasks.\n\n- The tasks where CoT does not harm performance (e.g., logical inconsistency) are not analyzed as thoroughly as the tasks where it does.", "relation_to_broader_scientific_literature": "The paper connects well with the broader literature on CoT prompting and cognitive psychology. It builds on prior work showing that CoT can improve performance on certain tasks (e.g., symbolic reasoning) but also acknowledges cases where CoT can be detrimental.", "essential_references_not_discussed": "It is better to include a more thorough discussion of prior work on the limitations of CoT prompting. \n\nFor example, recent studies have shown that CoT can increase harmful outputs or fail in tasks requiring planning (e.g., Kambhampati et al., 2024). Additionally, the paper does not discuss alternative prompting strategies like Tree-of-Thought or Self-Consistency, which have been shown to improve reasoning in some cases. Including these references would provide a more comprehensive view of the current state of CoT research.", "other_strengths_and_weaknesses": "Strengths:\n\n- The paper tackles the significant and underexplored issue of when CoT might harm model performance.\n\n- The experiments are well-designed and provide clear evidence of CoT's negative impact in certain tasks.\n\n\nWeaknesses:\n\n- The paper lacks a deeper theoretical explanation for why CoT fails in certain tasks. Although the proposed heuristic is reasonable, it lacks rigorous testing or validation.\n\n- The analysis of tasks where CoT does not harm performance is less thorough. It raises some questions about the generalizability of the findings.\n\n- It is better to include a broader discussion of alternative prompting strategies and their potential to mitigate the issues identified.\n\n- The heuristic connecting human cognitive limitations to model performance is interesting but not rigorously tested. Could the authors provide more evidence or experiments to validate this heuristic?", "other_comments_or_suggestions": "- The paper is well-written and easy to follow, but it is better to provide a more detailed discussion of the implications of the findings for the design of LLMs and LMMs and their reasoning prompting.\n\n- The authors should consider exploring the impact of different CoT prompting strategies (e.g., Tree-of-Thought).", "questions_for_authors": "Please kindly see the weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper investigates an interesting question: when CoT might harm the performance of LLMs. The authors draw inspiration from cognitive psychology and identity tasks where humans tend to perform worse when they verbalize their thinking process. They adapt six such tasks to test whether models also show performance drops with CoT in these situations. CoT achieves significant worse results on three tasks but mixed results on the others. The psychology-inspired approach is novel and the work studies an important but often overlooked aspect of CoT. This new perspective is appreciated by the reviewers. Concerns about generalization beyond these specific tasks and need for more analysis also exist, but this work presents a step in an interesting and under explored direction.
Decision: Accept (poster)