Title: Does Thinking More Always Help? Mirage of Test-Time Scaling in Reasoning Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Reasoning Models

Test-time Scaling
Abstract: Recent trends in test-time scaling for reasoning models (e.g., OpenAI o1, DeepSeek R1) have led to a popular belief that extending thinking traces using prompts like “Wait” or “Let me rethink” can improve performance.  This raises a natural question: Does thinking more at test-time truly lead to better reasoning? To answer this question, we perform a detailed empirical study across models and benchmarks, which reveals a consistent pattern of initial performance improvements from additional thinking followed by a decline, due to "overthinking". To understand this non-monotonic trend, we consider a simple probabilistic model, which reveals that additional thinking increases output variance—creating an illusion of improved reasoning while ultimately undermining precision. Thus, observed gains from "more thinking" are not true indicators of improved reasoning, but artifacts stemming from the connection between model uncertainty and evaluation metric. This suggests that test-time scaling through extended thinking is not an effective way to utilize the inference thinking budget. Recognizing these limitations, we introduce an alternative test-time scaling approach, parallel thinking, inspired by Best-of-N sampling. Our method generates multiple independent reasoning paths within the same inference budget and selects the most consistent response via majority vote, achieving up to 20% higher accuracy compared to extended thinking. This provides a simple yet effective mechanism for test-time scaling of reasoning models.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates whether increasing the amount of reasoning tokens (i.e., length of the model's chain-of-thought generation) at test time improves performance in large language models. The authors empirically analyze the effects of extended thinking on model accuracy and output entropy across three reasoning benchmarks: GSM-8K, MATH-500, and AIME 2024.\n\nThe study finds that while moderate increases in chain length initially improve accuracy, excessively long reasoning traces can lead to higher output entropy and degraded performance. The authors attribute this to growing variance in the model’s output distribution.\n\nTo address this, the paper proposes a \"parallel thinking\" strategy, which samples multiple reasoning paths and aggregates answers through majority voting. This approach is shown to be more robust to overthinking and maintains or improves accuracy as the thinking budget increases.\n\n**Key Contributions:**\n- Empirical evidence that longer chain-of-thoughts can hurt model accuracy due to increased output entropy.\n- Comparative analysis of sequential vs. parallel test-time reasoning strategies.\n- Demonstration that majority voting over multiple reasoning paths can mitigate overthinking effects.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- **Clear empirical findings.** The paper demonstrates the effects of overthinking with well-designed experiments and clear manipulations of reasoning length.\n- **Intermediate explanation.** The authors use a probabilistic framework and probe the entropy of the final answer to explain why longer chains of thought may degrade performance—namely, due to increased entropy.\n- **Practical solution.** The authors propose a simple yet effective method to mitigate the issue: splitting a long reasoning trace into multiple, independently sampled chains of thought and using majority voting to aggregate the results. This preserves accuracy while keeping each reasoning path within a manageable length.\n\n**Weaknesses:**\n\n- **Ambiguity in the notion of “thinking more.”** The title *“Does thinking more always help?”* is potentially misleading. In practice, thinking more could mean reasoning with greater diversity, exploring deeper cognitive paths, or acquiring more relevant information—not just generating longer token sequences. The paper focuses solely on the “resource” aspect (token length) of reasoning, which narrows the scope significantly and may overclaim its implications.\n- **Superficial explanation of degradation.** While the authors provide an entropy-based explanation for why longer reasoning hurts performance, the analysis may not go deep enough. The phenomenon of performance drop with longer input could reflect a broader issue in LLMs—such as difficulty managing long contexts—rather than being unique to reasoning. Prior work shows that LLMs tend to become more uncertain as context length increases. It would be more convincing to disentangle whether the degradation is due to reasoning difficulty or general limitations in long-context processing. A potential experiment could be to append repeated short chains to reach a longer sequence length. If entropy still rises and performance drops despite no new information, it would suggest that the issue lies in context accumulation, not reasoning per se.\n- **Lack of difficulty analysis.** The paper does not investigate whether the optimal reasoning length varies across different problem difficulties. It would strengthen the findings to show, for instance, that more complex problems require longer reasoning traces, while simple problems suffer from overthinking. This would make the core effect more interpretable.\n- **Limited originality in proposed solution.** The parallel sampling and majority voting method is not novel. Similar ideas have been proposed in earlier work, most notably in Wang et al. (2022), who introduced the *self-consistency* method. Their approach samples multiple reasoning paths and aggregates answers via majority voting. The current paper does not cite or acknowledge this, which may undermine the originality of its solution.\n\n**References:**\n\n- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2023). *Lost in the middle: How language models use long contexts.* arXiv preprint arXiv:2307.03172.\n- Braverman, M., Chen, X., Kakade, S., Narasimhan, K., Zhang, C., & Zhang, Y. (2020). *Calibration, entropy rates, and memory in language models.* In *International Conference on Machine Learning* (pp. 1089–1099). PMLR.\n- Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., ... & Zhou, D. (2022). *Self-consistency improves chain of thought reasoning in language models.* arXiv preprint arXiv:2203.11171.\n\nQUESTIONS:\n- How many parallel chains of thought are generated? Is it a fixed or flexible number (based on the total budget)?\n- If it is flexible, how do the authors decide each CoT's token length? Is it the measured optimal length? If so, how to ensure the setting generalizes in a novel, even without a ground truth scenario?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submission analyzes the efficacy of test-time scaling for reasoning in large language models (LLMs) by extending their \"thinking\" duration. The authors conduct a systematic empirical study and find that performance does not improve monotonically. Initially, it ascends with an increase in cognitive tokens but subsequently declines, a phenomenon known as \"overthinking.\" The submission utilizes a probabilistic model indicating that prolonged contemplation amplifies the variance of the model's output distribution, which may be advantageous at first but ultimately diminishes accuracy.  Informed by this insight, the authors advocate for \"parallel thinking\": the creation of multiple independent reasoning pathways and the application of a majority vote.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. The paper talks about a real-world problem with using reasoning models.  Many tests on three different datasets and three different open-source models with different parameter scales support the main idea of \"overthinking.\"  Figs. 2 and 3 clearly and consistently show the non-monotonic performance trend.\n2. The reason for overthinking based on variance makes sense and works.   The \"parallel thinking\" method that is suggested is a simple but helpful way for practitioners to get clear advice on how to best use their inference budget.\n3. The paper's writing is mostly clear and easy to follow.   The authors make it clear what their hypothesis, methods, and results are.   The numbers, in particular, do a good job of showing the main results.\n\n**Weaknesses:**\n\n1. The main weakness is that the proposed \"parallel thinking\" method is functionally similar to well-established techniques like self-consistency sampling. The submission frames it as an effective way to allocate a budget compared to sequential thinking, but it could be more clearly placed in the context of previous work to better show what it adds.\n2. The tests are done on small models with up to 8 billion parameters. While the results are consistent within this scope, it remains an open question whether the \"overthinking\" phenomenon and the relative benefits of parallel thinking generalize to much larger state-of-the-art models (e.g., QwQ-32B, DeepSeekR1, or OpenAI o3). The authors acknowledge this limitation in Section 6.\n3. The probabilistic model used to explain the phenomenon is a helpful illustration, but does not constitute a formal theoretical proof. It simplifies both the model's output and the reward structure. This is noted as a limitation by the authors.\n\nQUESTIONS:\n1. The proposed \"parallel thinking\" coupled with majority voting is fundamentally synonymous with self-consistency.  Could you clarify the distinction?  Does the primary contribution lie in the direct, cost-effective comparison with sequential reasoning, thereby establishing it as a superior budget allocation strategy? A more explicit discussion in the paper could strengthen the positioning of this contribution.\n2. Your study provides convincing evidence on models up to 8B parameters. Do you have a hypothesis on how the \"overthinking\" phenomenon might manifest in much larger models (e.g., QwQ-32B or DeepSeek R1)? It would be helpful to hear about any early experiments or ideas you have on this.\n3. The study uses cues like \"Wait\" to make people think longer.  Have you thought about how the type of this prompt might affect the result?  For example, would a more structured prompt, like \"Let's go over the steps again,\" have a different effect on the output variance and the overthinking curve than a general one?\n4. The example probabilistic model assumes that the reward function is smooth and has only one peak, but the actual reward in these reasoning tasks is a sparse, binary signal.  Could you talk about how this simplification might make it harder to apply the model's intuition to a real-world situation?  Is the idea behind this that the model's preference landscape (the chance it gives to answers) is smoother than the external reward function?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper questions whether the performance gains from test-time scaling continue to improve indefinitely as more “thinking” (i.e., sequential computation) is applied. To this end, the paper uses some tricks to force the model to think more (e.g., \"Wait\"), and observes that the model's performance degrades at some point. Thus the author propose joint usage of parallel thinking is effective.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength**\n\n(1) Well-motivated and great topic that should be explored\n\n(2) Overall analysis is well conducted \n\n(3) The writing is clear\n\n**Weakness**\n\n(1) I have a concern as it is an out-of-distribution scenario. While I do agree that S1 [1] has shown \"Wait & Think more\" is effective for test-time scaling, we are not sure that this is an optimal way for scaling. Specifically, the R1 model family is not trained to scale based on that condition; rather, it is just an interesting finding (e.g., we don't know whether this works for OpenAI o-series model or other models as well). \n\n(2) I think it would be great if the authors could consider more model families than R1, e.g., QwQ-32B. Since this paper is an analysis paper, it should consider a broad range of model families. I think it is worth reporting even if the trends are inconsistent from R1.\n\n(3) I think we still need test-time-scaling that is non-parallel if the answer is hard to aggregate. For example, coding [2,3] or summarizing problems makes it hard to use parallel test-time-scaling. Therefore, I think this may limit the scope of the paper. \n\n(4) Extending the question from (3), I think it is worth investigating the test-time-scaling trend in coding [2,3] datasets where parallel thinking is hard to apply.\n\n(5) Considering more math benchmarks are required. This paper is mainly about analysis. However, the paper has only considered three datasets. Considering some benchmarks like AIME 2025, AMC2023, and GPQA Diamond is encouraged.\n\n------------------\n\nOverall, I think the paper has strengths to be accepted. If the concerns are well-addressed, I am willing to increase the score.\n\n[1] s1: Simple test-time scaling\\\n[2] Program Synthesis with Large Language Models\\\n[3] Evaluating Large Language Models Trained on Code\n\nQUESTIONS:\nI think it is quite interesting that test-time scaling works for somewhat relatively easy datasets (e.g., GSM-8K, MATH). Do the authors have any explanation why test-time scaling does not work for hard datasets such as AIME?\n\nLIMITATIONS:\nSee the weakness above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors revisit test-time “think more” prompting and discover a non-monotonic accuracy curve: after a few added “Wait/Think more” tokens, accuracy peaks and then falls—an overthinking regime that shows up on GSM-8K, MATH-500 and AIME 2024 for three open-source models (Fig. 2): DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, and DeepSeek-R1-Distill-Llama-8B model.\n\nA simple 1-D Gaussian argument plus measured entropy curves explain this mirage effect  in test-time scaling.\n\nMotivated by this, the paper proposes parallel thinking: split the same token budget into several short, independent reasoning paths and majority-vote. Under a fixed 16 K-token budget this outperforms sequential “Wait & Think more” by ~22 pp and never drops below the peak.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1) **Clear empirical observation:** Figure 2 consistently shows the overthinking dip across three math benchmarks and three model sizes, correcting prior “more is always better” claims.\n\n2) **Insightful explanation**: The Gaussian overlap toy model and entropy analysis tie the phenomenon to variance growth.\n\n3) **Simple, actionable fix**: Parallel thinking is simple yet yields large gains and avoids tuning a stopping point.\n\n\n**Weaknesses:**\n\n1) **Model scale:** Results stop at 8 B params; a single run on a 30-ish B model (e.g., Qwen-30B) would help confirm that overthinking—and the parallel fix—persist at scale.\n\n2) **Predictive power missing:** The variance story/ schematic discussed in Sec. 3 and Fig.4 is qualitative ; it doesn’t give a formula to anticipate the overthinking peak across tasks or model scales. Without a way to estimate the tipping point in advance, practitioners still have to run experiments to find it.\n\n3) **Decoding details:** Entropy trends depend on sampling temperature/top-p; explicitly reporting those settings, or adding a robustness table, would strengthen Section 2.\n\n4) **Latency & throughput:** Because parallel branches can be batched, it is likely faster than an ultra-long single trace, yet the paper only plots accuracy vs. tokens. A small wall-clock latency plot would make the practical win clear.\n\n5) **Domain range:** The study is math-only; benchmarks on commonsense or coding tasks would show generality.\n\n\nQuestion for my own understanding: Does the non-monotonic “overthinking” curve and parallel-thinking advantage still hold when you add a verifier?\n\nQUESTIONS:\nPlease see strengths and weaknesses section.\n\nLIMITATIONS:\nYes, addressed in Sec. 6.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-06"}