Do NOT Think That Much for 2+3=? On the Overthinking of Long Reasoning Models
Authors: Xingyu Chen
Jiahao Xu
Tian Liang
Zhiwei He
Jianhui Pang
Dian Yu
Linfeng Song
Qiuzhi Liu
Mengfei Zhou
Zhuosheng Zhang
Rui Wang
Zhaopeng Tu
Haitao Mi
Dong Yu
Venue: ICML 2025 poster (2025)
Keywords: Large Language Models
OpenAI-o1
QwQ
R1
Reasoning
Math
Overthinking
Efficiency
Primary area: deep_learning->large_language_models
TL;DR: The article addresses overthinking in long reasoning models and propose optimizations to enhance efficiency without compromising performance.

Abstract: The remarkable performance of long reasoning models can be attributed to their ability to emulate human-like long-time thinking during inference. These models employ extended chain-of-thought (CoT) processes, exploring multiple strategies to enhance problem-solving capabilities. However, a critical question remains: How to intelligently and efficiently scale computational resources during testing. This paper presents the first comprehensive study on the prevalent issue of overthinking in these models, where long reasoning models generate redundant solutions that contribute minimally to accuracy and diversity, thereby wasting computational resources on simple problems with minimal benefit. We introduce novel efficiency metrics from both outcome and process perspectives to evaluate the rational use of computational resources by long reasoning models. Using a self-training paradigm, we propose strategies to mitigate overthinking, simplifying reasoning processes without compromising accuracy.  Experimental results show that our approach successfully reduces computational overhead while preserving model performance across a range of testsets with varying difficulty levels, such as GSM8K, MATH500, GPQA, and AIME. Our code is open-source and available at https://github.com/galaxyChen/overthinking.

Do NOT Think That Much for 2+3=? On the Overthinking of Long Reasoning Models
Xingyu Chen 12 Jiahao Xu 2 Tian Liang"
Zhiwei He 12
Jianhui Pang? Dian Yu Linfeng Song?
Qiuzhi Liu
Mengfei Zhou?
Zhuosheng Zhang²
Rui Wang'
Zhaopeng Tu Haïtao Mi
Dong Yu*
Abstract
1100
ol-like models
987
The remarkable performance of long reasoning models can be attributed to their ability to emulate human-like long- time thinking during inference. These models employ extended chain-of of-thought (CoTT) processes, exploring multiple strategies to enhance problem-sol capabilities, However, a critical question remains: How intelligently and efficiently scale compulational resources during testing. This paper presents the first comprehen- sive study on the prevalent issue of overthinking in these models, where long reasoning models generate redundant solutions that contribute min- imally to accuracy and diversity, thereby wast- ing computational resources on simple problems with minimal benefit. We introduce novel effi- ciency metrics from both outcome and process perspectives to evaluate the rational use of com- putational resources by long reasoning models. Using a self-training paradigm, we propose strate- gies to mitigate overthinking, simplifying reason- ing processes without compromising accuracy. Experimental results show that our approach suc- cessfully reduces computational overhead while preserving model performance across range of testsets with varying difficulty levels, such as GSMSK, MATH500, GPQA, and AIME. Our code is available at galaxychees/ovetth.kkki.
conventional models
901
Generated Tokens
440
220
218
113
74
13
Qwen2.5 2oen2.5572BB 5-72B
L. L.LMMa333 3-8
DeepSeek Maak--Mhh--- 7B
Qwen2.5 5-5225555. 5-Math-71
Qwen2 Qwen2.5-Mathh S-Mtth--Mtt---- --BB
72B
GPT-4o
GPT-4o-mini Gemini Pro
Claude-3.5 3.
0l Dl-Preview
1-minn
DeepScok-RI QwQ-32B-Preevee
Figure Illustration overthinking issue: long reasoning mod- els (in red color) spend much more tokens than conventional LLMs (in blue color) on simple question "What the answer plus 3?"
soning. Their success largely attributed to mimicking human-like long-term thinking before responding to ques- tion. Specifically, long reasoning models cultivate a long chain of thoughts (CoT), explore multiple strategies, break down complex steps, and perform double-checking, which ultimately enhance their ability to tackle intricate reasoning tasks. This approach, known as "scaling rest-time compute", involves allocating computational resources during the model's inference phase to generally yield more accurate responses.
com/
1. Introduction
While effective, a critical yet underexplored question re- mains: Are we scaling test-time compute efficiently and intelligently? This study provides an initial exploration of this problem. We first observe that long reasoning models exhibit significant overthinking issues. Specifically, they tend to expend excessive compute (in terms of tokens thinking rounds) questions that are exceptionally simple or for which the answer is already evident. For example, Figure I compares the token usage of long reasoning mod- els with conventional models when answering the question, "what the answer of plus 3?" On average, long reasoning
The OpenAl ol model (OpenAI, 2024) and its repli- (Qwen, 2024; Guo al., 2025; Kimi et al., 2025; DeepSeek, 2025) exemplify the tha in Al rea-
Equal contribution Shhnnhaai Jiao Tong Univer- Zhaopeng
sity, China "Tencent. Correspondence <zptu@tencent.com: Rui Wang <wangruil2 120 @esjuu.du..c....
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
Lam (7 tokens)
SWW0-22B-Previee (901 tokens)
The 2
2 plus 3.,, that's pretty striightffowand urithmetic problem. I think I cun handle this, So, let's 2 plus means adding numbers fogether. I know that when you add two and three, you get
Solution1 39 tokems
Qwen2.5-72B (7 tokens)
Hut mayhe should think shout stop by step make sure understand the process. First, what is addition! Addition is mathematical operation where you combine two or total sum. this case, I'm dealing with two numbers: and So, starting with 2, ifT add 3 to il, I'm essemially moving dince staps forward on the number line. Let visualize that. If start 2 and move step the right. reach 3. Another step
The answer to 2 plus 5.
Solution2 109 tokens
more cumbers
Qwen2.5-Math-722 (45 tokens)
to 4, one more step takes to 5, 2 plus equals
1. Start with the number 2.
find the answer of plus we can follow these steps:
Solution3 39 tokens
Aermatively, I can think of in temss of objects, Suppose I have two apples and then I get three more apples. many apples do equals live apples.
total? Well, apples plus three apples
2. Add the number to the number 2.
we get: { = So, the answer (5)))
When we add
Solution4
Another way to look at it using fingers. I can hold up two fingers on one hand and three the other, then
34 tokens
fingers together, which gives five fingers.
Solution5 42
I can also break down using smaller numbers. For example, plus 2 equals 4, and them makes 5. 2 is 4, plus 5, which the 2 plus
adding
GPT-4o tokens)
Solution6 26 tokens
Moreover, if consider the number properties, and are both positive integers, and their integer, which is 5.
should also
The unswer
Gemini Pro (5 tokens)
Solution8 29
Purthermore, algebmm if have an equation like x then solving for would give which sligns with equils
equals
Solution10 tokens
Albernatively, using Roman sumerals, 2 is ll and 3 III Adding them together gives which
Claude-3.5 (7 takens)
The answer 243 is
In concllssion, the
i.
Figure 2. An example of overthinking issue for QwQ-32B-Preview model's output response that consists of 13 solutions. Each colored box contains solution with the same explicit answer 5. The model repeatedly produces solutions with identical answers, demonstrating redundancy and a lack of diversity. We also list the outputs of other conventional LLMs for reference.
models consumed about 20x more tokens than conventional models to reach the same answer. Figure 2 illustrates concrete example where excessive long thinking results in generating 13 solutions for this trivially simple question. Across extensive analyses of mathematical benchmarks, we found these overthinking patterns: (I) contribute minimally to improving accuracy, (2) lack diversity in reasoning strate- gies, and (3) occur more frequently with simple problems.
100
Decpseek-R R1
95
Ours
Accuracy (%)
90
QwQ-32B-Previce
85
Qwen2.5-Math-72B
80
The overthinking observed in long reasoning models reveals inefficiency in inference and highlights fundamental limi- tations in their reasoning and decision-making processes. We assert that reasoning involves not only accuracy but also the application of the appropriate level of complexity based on the problem's requirements. This insight motivates our exploration of studying and mitigating overthinking. To address this, we propose two metrics from both outcome and process perspectives to evaluate long reasoning models' efficiency. These metrics help provide a comprehensive assessment of the efficiency of long reasoning models, aug- menting the commonly-used effectiveness metries.
75
Llama3.3-708
Llamma3 1-405B
70
0
750
1500
2250
3000
# Tokens
Figure 3. Token-accuracy plot on MATH500. Our method reduces the overthinking issue when applicd Mw-Q-32B-Previcet
reflexivity. Experimental results across testsets of varying difficulty levels (e.g., ASDIV, GSM8K, MATH500,, GPQA, and AIME) demonstrate our approach's 's effectiveness and robustness in mitigating overthinking issues. For instance, as shown in Figure 3, our approach can reduce token output by 44.5% while maintaining accuracy on the widely-used MATH500 testset as applied QwQ-32B-Prent Previee.
To mitigate overthinking without introducing external in- formation, we adopt self-training paradigm that teaches the model to produce more concise responses using its own generated outputs. We simplify the generated responses by removing redundant solutions while maintaining basic
2

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
50%
QwQ: ASDIV
QuQ: MATH500
40%
RI: ASDIV
Distribution Ratio
RI: MATH500
30%
20%
10%
24
0%
2
5
27
Number of Solutions
Figure 4. Distribution of solution counts in generated responses for different test sets and models (QwQ-32B 1---32--rreevew ("QwQ") and
DeepSeek-RI ("RI")).
In summary, our contributions are three-fold:
Solution Density Tokens(K)
Solution Density Tokens(K)
1. We present the first study offering both definitive expla- nation and comprehensive analysis of the overthinking issue, showing that long reasoning models often expend unnecessary computational resources on redundant solu- tions that contribute minimally to final outcomes.
3.8
Solution Density
3.9
3.6
3
3.2
4
#Tokens (K)
2.0
2.8
2
2.1
2. We introduce metrics considering both outcome and pro- cess aspects to assess the efficiency of long reasoning models.
1.4
1 2
4 5
4 Level (Difficultyt) (b) DeepSeck-RI
1
3
Level (Difficultyt) (a) QwQ-32B-Preview Preview
3. We explore several strategies to mitigate overthinking, significantly reducing generated tokens while maintaining model performance across testsets of varying difficulty.
Figure 5. Average solution density (the number of solutions per 1000 tokens) and the number of tokens in generated responses across different difficulty levels the MATH500 test set.
2. Observing Overthinking Issues
In this section, we present comprehensive analysis of out- puts generated by long reasoning models. First, provide basic illustration of the solution distribution in responses from these models ($ 2)... We then identify two inefficien- cies in long reasoning responses: their limited contribution to accuracy ($ 2.2) and diversity ($ 2.3). To evaluate these inefficiencies empirically, we propose two efficiency metrics based on our observations. Finally, we present empirical results in 2.4 and conclude that long reasoning models often overthink, particularly with easier math problems.
Solution Distribution In this paper, we define solution as part of the full model generation that contains an answer explicitly. For example, in Figure 2, each solution in the QwQ-32B 1-32--reeieww generation contains the answer 5. We use the Llama-3.3-70B model to separate solutions from generated responses. Figure 4 shows the distribution of so- lutions in generated responses. Generally, long reasoning models produce 2 to 4 solution rounds for most instances, covering 56% to 65% of cases for Maw---3--rreiieee across the test sets and 57% to 62% for DeepSeek-RR Regarding different test sets, long reasoning models tend to generate more solutions on average for easier test sets. Since these models naturally produce
2.1. Solution Distribution of long reasoning Models
Experimental Setup We use two test sets, including AS- DIV (Miao et al., 2020) and MATH500 (Hendrycks al., 2021). ASDIV is an elementary-level math test set, while MATH500 is a high school-level math test set. We mainly investigate two widely recognized long reasoning mod- els featuring a visible thinking process: Qwen-QwQ-2 Q-Q-32B- Preview (Qwen, 2024) and DeepSeek-RI (DeepSeek, 2025).
tokens for harder problems, we define Solution Density as the number of solutions per 1, 1,000 tokens to compare the extent of overthinking across different difficulty levels. The solution density of QwQ 32B-Previi Peevrrevew is 5.5 on ASDIV and 2.7 on the most difficult MATH500 test set. Similarly, DeepSeek-R2 achieves solu-
3

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
extended CoT might not significantly enhance accuracy. For instance, the average length of the first round of solutions for QwQ-32B-Preview on the ASDIV test set is 287 tokens, constituting only 38.7% of the entire response. These re- sults suggest that later solutions marginally contribute to improvements accuracy.
100%
QwQ: ASDIV
First Correctrcess Distri.
QwQ: MATH500
80%
RI: ASDIV MATH500
60%
40%
20%
Outcome Efficiency Metric Based on the above observa- tion, we propose an outcome efficiency metric to empirically evaluate how effectively later solutions contribute to accu- racy improvements. The outcome efficiency metric, denoted Eo. is defined by the following formula:
16
0%
1
2
24
Index of Solution
Figure 6. Distribution of occurrences for the first correct answer. Later solutions marginally contribute improvements in accuracy
T TT.
Σ Eo N
(1)
T
tion densities of 4.2 and 2.4, respectively. As shown, both MM0--2B--reeveee and DeepSeck-Ri -R1 exhibit more severe overthinking in simpler problems.
where N is the number of instances in a given test set. of denotes whether the evaluated model can produce a correct answer in the response:
To further empirically validate this finding, we conducted an analysis across various difficulty levels in the MATH500 test set, as illustrated in Figure 5. Both PwQ-32B--reviees and
f1, 1, if at least one solution in response is correct 0,
0, otherwise
R1 generate more tokens for difficult problems, and their solution density decreases as question difficulty increases. 1-Q-32B-Previeee achieves a solution density
T. is the total number of tokens produced for the i-th in- stance, and T. denotes the efficient tokens that contribute to reaching the correct answer:
For example, of 3.8 for the simplest (level I) questions, indicating that, on average, every 1,000 tokens contain more than three solutions, whereas the solution density for level problems drops to 1.7. Although RI performs slightly better, still produces multiple solutions for simple questions. These results support our claim that long-reasoning models tend to overthink simpler problems.
T. #tokens to first arrive at correct answer, Ti
= 1
0 = 0
Intuitively, if a model correctly answers at an early stage, the tokens generated thereafter do not contribute to improving accuracy and are considered inefficient. Consider Figure 2 as an example: The first solution correctly addresses the problem with f 39. Consequently, ξo 901 which can be considered extremely inefficient.
2.2. Efficiency on Accuracy Improvements
39
4.3%,
Intuition In the example in Figure 2, we observe that the initial round of solutions already yields the correct answer. Subsequent solutions, which account for the majority of generated tokens, do not enhance accuracy. Based on this observation, we empirically investigate whether later so- lutions contribute accuracy improvements. Specifically, for all cases where long reasoning models produce the cor- calculate the distribution
2.3. Efficiency on Diverse Thinking
Some researchers might argue that while solv- ing an easy math problem may appear straightforward, ap- proaching it from different perspectives derstanding and build flexibility in mathematical thinking, which is also valuable. Consider the example output of QwQ QwQ-32B-B (-Q-22--Preeiii Preview in Figure 2: Solution states the basic fact that plus equals 5; Solution 2 breaks the addition into smaller steps; Solution 3 uses a counting objects analogy. These three solutions provide different reasoning strategies. However, Solution 4 repeats Solution 3, and Solution 5 repeats Solution 2 using similar reasoning strategies. In this section, we empirically examine the diversity among solutions within a response.
Intuition
deepen un-
rect answer in the response, of occurrences for the first correct answer, termed the "first correctness distribution". If more correct answers appear in carlier solutions, then the subsequent solutions contribute minimally accuracy improvement, indicating reduced efficiency.
Observation Figure 6 illustrates the first correctness dis- tribution across the test sets and models. In more than 80% of cases, the initial round of solutions produces the correct answer. Notably, the first round generally comprises less than 60% of the total tokens generated, suggesting that the
Observation
To empirically evaluate whether later solu- tions provide new reasoning strategies, we introduce the
4

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
100%
DeepScek-RI
QWQ: ASDIV QwQ: MATH500
80%
RI: ASDIV MATI1500
73.2%
9%
Distincteess
60%
70% 0%
65. 6%%
Efficiency
40%
60%
56 16%
20%
51.%%
50.3%
52.0%
52,8%
50%
0%
1
2
Process Efficiency
Index Solution
40%
Outcome Efficiency
1
2
3
Figure 7. The ratio of diverse solutions at each solution index. Later solutions tend to repeat earlier ones.
MATH500 Level (Difficulyy))
Figure 8. Efficiency results across different levels of MATH500.
"distinctness ratio" as the measure for the ratio of distinct solutions for each data index. Suppose the response to the i-th problem from a given test set can be divided into M, solutions. We denote the set of solutions as R {s 8 ...., s Let S {s.'..... 8% ...., } be the set of m-th solutions in the responses of all instances in the test subset. The distinctness ratio is defined as:
Process Efficiency Metric Based on the above observa- tion, propose a process efficiency metric to empirically evaluate the contribution of later solutions to solution diver- sity. The process efficiency metric is calculated as:
N Er
D. T.
(2)
N
E TE
Dism
where D, represents the number of efficient tokens that con- tribute to the solutions' diversity. Here, we intentionally exclude the factor o; to concentrate on diversity, indepen- dent of correctness. Let T.m denote the number of tokens in solution s!" We define:
K
where
1,
if (($)) & (0(s))...,0ff
Tk 10, (0, 0, otherwise
M
D =
T TT
In this context, ((8)") is the reasoning strategy of m. In- tuitively, Th² = if the m-th solution introduces a new reasoning strategy, and Dis" is the average number of new reasoming strategies solution position m. We use GPT-4o to cluster the solutions for each instance into groups via prompt like (Ye et al., 2024).
m==1
Intuitively, the tokens in distinct solution are regarded as process efficient tokens, and &P is the average ratio of tokens associated with diverse reasoning strategies. In the example shown in Figure 2, the 13 solutions are categorized into 7 distinct reasoning strategies. Consequently, tokens in Solutions 1, 2, 3, 7, 8, 9, and 12 are efficient, resulting in E (39+1094 39 29 +19+-9) 35.8%.
Figure 7 displays the distinctness ratio for each solution index. Intuitively, the ratio for Solution#1 is always 100%, as it has no preceding solutions, thus 7 = 1 for all instances. Generally, the ratio decreases with higher indices, indicating that later solutions often repeat earlier ones. For example, the average distinetness ratio for Solution# across test sets decreases by 7.5% compared to Solution#3. The ratio of Solution#2 significantly decreases compared to Solution#11. By reviewing outputs, we find that Solution#2 often double- checks answers from Solution#1 using the same reasoning strategy. Subsequently, Solution#3 attempts to solve the problem using new reasoning strategy.
901
2.4. Empirical Efficiency Results
Table I presents the results model efficiency. For com- parison, we include two representative conventional LLMs: Llama-3.3-70B -Inttrctt and Qwen2.5- Mathh-22- These conventional L.LMs produce only a single solution, meaning that 4. # 1. Therefore, in these cases, the outcome efficiency metric Eo 1/2 Σ dg equals ac- curacy, and the process efficiency metric & = 1.0. In comparison, long reasoning models generate significantly longer responses, which are less efficient in improving ac- curacy and solution diversity. We refer to the inefficient use of generated tokens as the "overthinking issue". The
IIf a response does not contain the m-th solution (i.e. M; M))m), that response is excluded from the set, hence K does not necessar- ily equal the number of test set instances N. Refer to Appendix A.1 for clustering prompt and results.
5

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
Table Model efficiency results of strong LLMss.
Response
Efficiency
Models
Accuracy
#Solution
#Token
Outcome
Process
ASDIV
-lama-3.3-70B--ml. Instruct
95.6
1.0
166.4
95.6%
100.0%
0wen2.5-Marth72B--nnttte
96.3
1.0
213.0
96.3%
100.0%
96.5
3.5
713.7
53.7%
69.0%
Qwen/QwQ-33
Q-33BB
97.1
3,4
1171.1
57.6%
69.1%
QwennQQwen32255BBAA22
97.2
3.8
1399.2
56.3%
67.9%
DeepSeck/R1
97.1
3.3
840.0
57.5%
70.2%
GSM8K
Mlma--33-77B--mm. Instruct
92.6
1.0
220.3
92.6%
100.0%
Qwen2.5-Mait Instrrct
95.8
1.0
317.4
95.8%
100.0%
Qwen/QwQ-332B-Pree Preview
94.7
2.9
756.1
11.8%
75.2%
Owen/QwQ-32B
96.1
4.2
1835.8
53.3%
61.8%
QwennQQeen-2235B-A2222
95.7
5.3
2419.4
47.9%
58.6%
DeepSeek/R1
96.4
2.9
1053.1
67.2%
73.2%
MATH500
Llama-3.3- 3.0---.8-7---trruue
75.4
1.0
553.4
75.4%
100.0%
Dwen2.5-Math-7221
86.8
1.0
593.1
86.8%
100.0%
Wwen/OQwQ-32BB-Previiee
91.2
4.3
2398.5
51.4%
70.3%
Qwen/QwQ-32B
96.2
6.2
4365.4
44.4%
61.1%
Qwenn/wwen-2235BAAD0
92.6
6.2
5810.2
43.5%
60.9%
DeepSeek/R1
95.4
4.2
2696.4
53.0%
70.7%
experimental results demonstrate that while long reasoning models have the capacity to generate multiple solutions, their efficiency is hindered by the overthinking issue.
Table 2. Statistics on different types of generated responses based the training data. "Greedy" denotes responses generated via greedy search; "Shortest" and "Longest" indicate the shortest and longest responses among 10 samples.nspecctielly.
Figure 8 presents the detailed efficiency results of DeepSeek- RI across various difficulty levels of the MATH500 test set. Notably, DeepSeek-RI performs poorly on the simplest Level problems, achieving about 50% outcome efficiency. These findings underscore that the overthinking issues faced by long reasoning models are pronounced with simpler math problems.
Efficiency
Response
#Solution
#Token
Outcome
Process
Greedy Shortest Longest
3.1
1434.8
55.6%
72.6%
2.5
T031.
69.8%
80.3%
4.1
2258.7
46.0%
66.4%
3. Mitigating Overthinking Issues
In this section, we explore strategies to improve the effi- ciency of long reasoning models. Our approach involves three key steps: (1) generating long reasoning traces on mathematical dataset, (2) constructing shorter reasoning traces by pruming unnecessary solutions from the long traces, and (3) applying preference optimization to train the model on (short, long) reasoning pairs. Following prior work, we use the PRM12K dataset (Lightman et al., 2024) as our training set. Our experiments are conducted on the QwQ Pre-Pre model, leveraging self- self-training strategy (Ze- likman et al., 2022; Ho et al., 2023) in which the model generates its own training data.
3.1. Generating Long Reasoning Traces
We begin by generating long reasoning traces for the PRM12K dataset through sampling. Using QwQ-32B- Preview model with a temperature of 1.0, we produce 10 reasoning traces per question and retain only those yielding correct answers. Table 2 presents the statistics of different types of generated responses. Our analysis of these sampled responses reveals that the shortest response performs better in terms of both outcome and process efficiency, using fewer rounds and tokens. These findings support our initiative to enhance model efficiency through simplifying responses.
6

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
3.3. Length Preference Optimization
Table Statistics different types of positive examples. "#S"
denotes the number of solutions.
For each question, construct a paired training dataset where:
Efficiency
Positive Example
#S
#Token
Outcome
Process
The positive example is the short reasoning trace
Shortest Response
2.5
1051.3
69.8%
80.3%
FCS
1.1
681.0
99.5%
99.1%
The negative example is the longest response from the sampled outputs
FCS + Ref.
1.9
878.7
78.4%
82.4%
GDS
1.6
856.8
86.8%
94.2%
We then apply preference optimization to these pairs, train- ing the model to favor concise, efficient reasoning paths. Our experiments focus on the following approaches:
3.2. Simplifying Responses to Enhance Efficiency
Building on the observations in Section 2, where earlier solutions demonstrate greater efficiency, we propose three strategies for simplifying response generation by construct- ing concise reasoning traces:
Direct Preference Optimization (DPO; Rafailov et al. 2024): This method trains a model directly on human- preferred responses increase the likelihood of preferred responses over unpreferred ones.
Reasoning Preference Optimization (RPO; Pang et al. 2024; Liu et al. 2024): This approach modifies the DPO loss by adding negative oog-likelhhoo term the pre- ferred response. RPO enhances DPO training stability by preventing decreased probability of selected responses.
First-Correct Solutions (FCS): We retain the portion of the response from the beginning until the first occurrence of a correct answer.
FCS+Reflection: Since most responses yield the correct answer in the first solution (see Figure 6), retaining only the First-Correct Solutions might cause long-reasoning models to revert to conventional LLM behavior. To miti- gate this, we extend the approach by including the second correct solution, thereby preserving the model's 's reflective capability while maintaining efficiency.
Simple Preference Optimization (SimPO; Meng et al. 2024): This method addresses the discrepancy between the reward function and the generation metric during inference found in other preference optimization methods. SimPO incorporates techniques like adaptive margin and length regularization into DPO training.
Greedily Diverse Solutions (GDS): Figure 7 shows that the distinctiveness of the second solution diminishes significantly, as it often double-checks the first solu- tion using the same reasoning strategy. Consequently, FCS+Reflection may compromise process efficiency. To address this, introduce simple heuristic that greed- ily selects solutions with new reasoning strategies. This approach would include additional solutions when the second solution introduces novel reasoning, thereby en- hancing diversity.
We also explore Supervised Fine-Tuning (SFT; Wei et al. 2022a) the positive examples, which enables the model to directly learn how to generate shorter responses.
4. Experiments
4.1. Datasets
We use the following test sets for our experiments:
ASDIV (Miao et al., 2020): an English math word prob- lem corpus with 2,305 instances, typically found in ele- mentary schools.
We also use the Shortest Response among the sampling results as the baseline of simplifying responses.
GSM8K (Cobbe et al., 2021): a dataset of high-quality, linguistically diverse grade school math word problems created by human problem writers. The test set includes 1,319 problems, with solutions often involving sequence of elementary calculations using basic arithmetic.
Table 3 presents the statistics for these simplified responses. Notably, all simplified responses enhance efficiency com- pared to the shortest response. "FCS" is the most efficient, both in terms of outcome and process, using the fewest num- ber of solution rounds and tokens. "FCS+Reflection" incor- porates reflection, requiring approximately one additional solution round, which reduces both outcome and process efficiencies. "Greedily Diverse Solutions" serves as com- promise, balancing the number of solutions and tokens, and achieving moderate to high efficiency.
MATH500 (Hendrycks et al., 2021): a challenging dataset consisting of problems from high school math competitions across seven subjects (e.g.. Prealgebra, Al- gebra, Number Theory) and difficulty levels based on AoPS (ranging from 1 to 5).
7

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
Table 4. Experimental results of the proposed efficiency enhancing methods.
Response
Efficiency
Methods
Accuracy
#Solution
#Token
Outcome
Process
ASDIV
AMQ-32B-Previee
96.5
3.5
713.7
53.7%
69.0%
#SimPOrcS-Reelettinn
96.6
1.9
381.5
82.5%
87.9%
GSM8K
BwQ-32B-Preview +SimPO
94.7
2.9
756.1
11.8%
75.2%
95.9
1.8
416.5
86.0%
91.0%
MATH500
Qw QwQ-32B- Q-32B-Previeew
91.2
4.3
2398.5
51.4%
70.3%
SFT Shortest Response
92.6
4.4
2359.0
59.7%
72.8%
+ DPO Shortesi Response
93.2
3.4
1928.8
64.3%
77.8%
+ +RPO Shortest Response
90.2
3.5
2015.2
64.7%
76.6%
SSimPOshortes Shortest Response
SSmPOOs
91.0
3.5
1871.5
64.7%
78.1%
+SimPOp First Solution
90.4
1.3
1015.6
85.5%
96.3%
Reffeettooooooition (Ours)
91.4
2.4
1330.3
79.1%
88.9%
+SimPOGreedily Diverse Solations
91.2
1.7
1285.8
80.1%
90.2%
GPQA
Swven.5-MMtth722--mme PW@Q-32BBPreviiew PSimmOOpc+Retteetie
46.5
1.0
811.7
46.5%
100%
59.1
6.0
32226.4
38.6%
66.4%
59.6
3.0
2084.5
57.9%
80.2%
AIME24
Swen2.5-Math-. Ins-trrt
23.3
1.0
1204.5
23.3%
100.0%
QwQ-32B Preview
46.7
3.7
9480.6
37.5%
79.5%
+SimPOOpcsReffettio
43.3
2.0
5154.4
39.3%
91.6%
The overall difficulty levels of the test sets are ASDIV GSM8K MATH500.
<
Performance of Length Preference Optimization Meth- ods We first evaluate all methods described in Section 3.3 under the Shortest Response setting. While supervised fine- tuning (SFT) yields modest reductions in the number of solu- tions and tokens compared to the vanilla QwQ-32B B-e--iew model, preference optimization methods demonstrate signif- icantly greater effectiveness. Among these, SimPO achieves the best performance, reducing generated tokens by 22% on MATH500 while maintaining comparable solution qual- ity. Based these results, we select SimPO our default method for subsequent experiments.
We also validate method using more challenging test sets, specifically GPQA and AIME:
GPQA (Rein et al., 2023): graduate-level dataset sisting of multiple-choice questions in subdomains of physics, chemistry, and biology. For our experiment, we select the highest quality subset, known as GPQA Dia- mond (composed of 198 questions).
AIME24 (MAA Committees): dataset from the Ameri- can Invitational Mathematics Examination, which tests mathematical problem solving across multiple areas (c.g. algebra, counting, geometry, number theory, and proba- bility).
Performance of Response Simplification Methods As anticipated, the First-Correction Solutions (FCS) strategy achieves the greatest length reduction. However, this ap- proach degrades performance on the challenging MATH500 test set, likely due its need for more extensive reflec- tion. The FCS+Reflection variant addresses this limitation, outperforming standard FCS through an additional reflec- tion round. While the Greedily Diverse Solutions strat- egy maintains a balance between performance and token efficiency, it underperforms FCS+Reflection supporting our hypothesis that the difficult MATH500 problems ben-
4.2. Experimental Results
Table 4 presents the main results of the proposed methods. We first conduct a detailed comparison on MATH500 to identify the most effective approach, and then validate the proposed method on additional benchmarks.
8

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
efit from the deep inference capabilities of long reasoning models. We consequently establish FCS+Reflection our default response simplification method. The performance of the FCS+Reflection method across different test sets demonstrates its effectiveness. Accuracy improves slightly on ASDIV, GSM8K, and MATH500, with nearly half the tokens and solutions compared to the MwW-32B-Previees baseline. Additionally, FCS+Reflection enhances efficiency, as reflected in the efficiency metrics.
et al., 2024). Advanced models combine these patterns into extensive chains-of-thought, improving accuracy with more reasoning time (OpenAI, 2024).
5.2. Efficient Thinking
Scaling the search space and scaling human- thinking in- volves two distinct aspects of efficcincyy efficient search and efficient thinking. However, few works specifically focus on efficient thinking in LLMs. Kimi et al. (2025) leveraged the long to short strategy to compress generation context. Zhao et al. (2024) encourages the model to terminate rea- soning by saying "I don't 't know" when the problem is hard to solve. Han al. (2024) introduces token- Miken-budget-wwree reasoning, where the model is prompted with a specified token budget to guide its reasoning process. There are also several contributions (Damani et al., 2024; Wang et al., 2024; Xu et al., 2024) made to predict the distribution of the computation budget and allocate the computation power based on the prompt's 's difficulty. Another line of work em- phasizes the early stopping strategy to save computation budget while reasoning (Manvi et al., 2024; Li et al., 2024). Moreover, multi-agent framework utilizes large LLMs for difficult tasks while small LLMs for simple tasks (Kirchner et al., 2024; Damani et al., 2024)
Results on Challenging Test Sets Our approach signifi- cantly improves efficiency on simpler benchmarks (ASDIV and GSM8K) while maintaining comparable performance. However, a critical concern remains: could this length pref- erence optimization method compromise the models' capac- ity for complex, long-term reasoning? To directly address this, conduct evaluations on challenging datasets requir- ing extended reasoning chains (GPQA and AIME24). The results demonstrate that our approach preserves the mod- els' complex reasoning capabilities while reducing token usage- performance on GPQA and AIME24 only drops slightly with improved efficiency. This confirms that our method achieves robust generalizability without sacrificing the fundamental strengths of long reasoning models.
5. Related Work
In summary, all the aforementioned works consider conven- tional models rather than long reasoning models with longer chains-of-thought, In contrast, our work first identifies the overthinking problem in long reasoning model. Addition- ally, instead of limiting the reasoning space or leaving the token budget to be specified by the user, we aim to train the model to learn how to think efficiently.
5.1. Scaling Test-Time Compute
Enhancing model performance on complex tasks can be achieved by scaling test-time compute, which involves:
Expanding Search Space LLMs have strong reasoning abilities, but their auto-regressive decoding often misses optimal solutions. Self-consistency generates multiple responses and use majority voting to select the best an- swer (Wang et al., 2023b). Other approaches include best- of-n decoding, minimum Bayes risk decoding (Lightman et al., 2024; Li et al., 2023; Khanov et al., 2024; Heineman et al., 2024; Wu al., 2024), and structured search methods such as Thee-hf-Thought, Carlo Tree Search (Yao et al., 2024; Besta et al., 2024; Luo et al., 2024; Tian et al., 2024; Wan et al., 2024).
6. Conclusion
This study identifies a key challenge in long reasoning mod- els - efficient and intelligent scaling of test-time com- putational resources. We have presented a comprehensive analysis of the overthinking issue in long reasoning models. By highlighting the overthinking phenomenon and propos- ing efficiency metrics, we enhance our understanding of resource utilization in ol-like models. Our self-training based approach effectively mitigates overthinking, reducing unnecessary computation while maintaining performance across reasoning benchmarks of varying difficulty levels.
Traph-of-Thoughhe
and Monte
Human-Like Thinking Patterns LLMs often use natu- ral language reasoning. Techniques like chain-of-thought encourage step-by-step reasoning instead of direct an- swers (Wei et al., 2022b; Kojima et al., 2022). This has been expanded with methods like debating, self-correction self- critique, and plan-and-solve (Liang et al., 2024; Du et al., 2024; Xiong ct al., 2023; Kumar et al., 2024; Kamoi et al., 2024; Ke et al., 2023; Lin et al., 2024; Yu et al., 2024; Wang et al., 2023a). Recent studies also explore latent space rea- soning to mimic human cognition (Hao et al., 2024; Goyal
This work not only boosts efficiency in long-reasoning mod- els, but also lays critical groundwork for future research aimed at optimal allocation of computational resources in Al reasoning tasks, which is crucial for the development of sustainable and cost-effective Al systems. Future directions include exploring adaptive compute strategies that dynami- cally adjust to problem complexity and refining efficiency metries for broader model generalization.
9

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
Impact Statement
Heineman, D., Dou, Y., and Xu, W. Improving minimum bayes risk decoding with pultpp In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 2--222255225 22545, 2024.
The paper reveals the overthinking issue in long reasoning models. Our findings call for the community to research on models' efficient thinking capabilities, which could signifi- cantly influence future developments in this field. We see no harmful impacts of this work.
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E.,, Song, D., and Steinhardt, J. Mcasuring mathematical problem solving with the MATH dataset. In NeurlPS, 2021.
References
Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Pod- stawski, M., Gianinazzi, L., Gajda, J.. Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial In- telligence, volume 38, pp. 17682-17690, 2024.
Ho, N., Schmid, L., and Yun, S.-Y. Large language models are reasoning teachers. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 14852-14882, 2023.
Kamoi, R., Zhang, Y., Zhang, N., Han, J., and Zhang, R.
When can llms actually correct their own mistakes? critical survey of self-correctinn of llms. Transactions of the Association for Computational Linguistics, 12:1417- 1440, 2024.
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. arXiv:2110. 14168, 2021.
Ke. P., Wen, B., Feng, Z., Liu, X., Lei. X., Cheng. J., Wang, S., Zeng, A., Dong, Y., Wang, H., et al. Critiquellm: Seal- ing Ilm-as-critic for effective and explainable evaluation of large language model generation, corr, abs/2311, 187702. detection for generative large language models. In Pro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 9004-9017, 2023.
Damani, M., Shenfeld, I., Peng. A., Bobu, A., and An- dreas, J. Learning how hard to think: Input-adaptive allocation of Im computation, 2024. URL https: //arxiv.org// /tt.s///21..00000
DeepSeck. Deepseck-rl: Incentivizing reasoning ca- pability in Iims via reinforcement learning. 2025.
Khanov, M., Burapacheep. J., and Li, Y. Args: Alignment soward-guided search. In The Twelfth International Conference on Learning Representations, 2024.
URL attps://apil semanticscholar org/ CorpusID:275789955
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning, 2024.
Kimi, T., Du. A., Gao, B., Xing, B., Jiang, C., Chen, C.. Li, C., Xiao, C., Du, C., Liao, C., al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501. 200.25111.2000 /2599, 2025.
Goyal, S., Ji, Z., Rawat, A. S., Menon, A. K., Kumnr, S., and Nagarajan, V. Think before you speak: Train- ing language models with pause tokens. In The Twelfth International Conference on Learning Representations, 2024. URL id=ph04cRkPdC.
Kirchner, H., Chen, Y., Edwards, H., Leike, J., McAleese, N., and Burda, Y. Prover- verifier games improve legibility of Im outputs, 2024. URL https: /ttps:///xxvvvvvee abs 22472.1.....2 13692.
http///pppeeeeeeeeeeeeeee/eee/ee www.reewwwwwwweeeeeeeeeeeeeeeeee net/ forum?
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. Large language models are zero-shot reasoners. Ad- vances in neural information processing systems, 35: 22199-22213, 2022.
Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-rl: In- centivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J. D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., et al. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409. 12917. 2024.
Han, T., Fang, C., Zhao, S., Ma, S., Chen, Z., and Wang, Z. Tokke--uuget-awarre Ilm reasoning. arXiv preprint arXiv:2412.185477 2024.
Hao, S., Sukhbaatar, S., Su. Di, Hu, Z., Weston, J., and Tian, Y. Training large language models to reason in a continuous latent space, 2024. URL https:// abs/////....22 2412.00 06769.
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
a arxiv v. orgo abs /tt///2/
10

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
Li, Y., Lin, Z., Zhang, S., Fu, Q. Chen, B., Lou, J.-G., and Chen, W. Making language models better reasoners with step-aware verifier. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 6lst Anmual Meet- ing of the Association for Computational Linguistics ( (Vol- une 1: Long Papers), pp. 5315-5333, Toronto, Canada, July 2023. Association for Computational Linguistics. URL https: //aclanthology..o...o wtg.:y.rrggggaa acl wwweeel-oonn..... 291.
mid-generation, 2024. URL BRRthtss://erxiv.om Abs/2410.0272
MAA Committees. Aime problems and solutions. wotp://emmmoiii.i..ooooo......
index.ppp). Bttp:/AAAMMPPPooobeeee sott
Meng. Y., Xia, M., and Chen, D. Simpo: Simple preference optimization with reference-free reward. In Advances in Neural Information Processing Systems (NeurIPS), 2024,
doi: 10.18665w11/00233cllooogg
Li, Y., Yuan, P., Feng, S., Pan, B., Wang, X., Sun, B., Wang, H., and Li, K. Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning. In The Twelfth International Conference on Learning Representations, 2024. URL https::/ http:::///ppeeeeviiiie net/ forum? id=ndR8Ytrzhh.
Miao, S.-Y., Liang, C.-C., and Su, K.-Y. A diverse corpus for evaluating and developing english math word problem solvers. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, 2020.
OpenAL llms.
Learning to reason with https: /ttp:///pppa...// com/ index/ learning to rewson with 1mm, 2024.
Liang, T., He, Z., Jiao, W. Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., and Tu, Z. Encouraging diver- gent thinking in large language models through multi- agent debate. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 17889-17904, Miami, Florida, USA, November 2024. Association for Computational Linguisties. doi: URL https:// ac aclanthology org wwwp//g/////....//// emn 1p main. 992.
Pang. R. Y., Yuan, W., He, H., Cho, K., Sukhhaatar, S., and Weston, J. E. Iterative reasoning preference optimiza- tion. In The Thirty-eighth, Annual Conference on Neural Information Processing Systems, 2024. URL https: // //pprrevveew... net/ www.mmmdddeiiiee......
Qwen. Qwq: Reflect deeply on the boundaries of the unknown, November 2024. URL pestps://qwenn http://wwa--2--peeeeeee
10.
Lightman, H., Kosaraju, V., Burda, Y.. Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URL https: httpa:///ppeeevveeeeee net/ forum? 1d=v8L0pN6E01.
Rafailov, R., Sharma, A., Mitchell, E., Manning, D., Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Ad- vances in Neural Information Processing Systems, 36. 2024.
Rein, D., Hou, B. L., Stickland, A. C., Petty, J., Pang, R. Y., Dirani, J., Michael, J., and Bowman, S. R. Gpqa: A graduate-level google-proot q&a benchmark. arXiv preprint arXiv:23 120022, 2023.
Lin, Z., Gou, Z., Liang, T., Luo, R., Liu, H., and Yang, Y. CriticBench: Benchmarking LLMs for critique-corres reasoning. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 1552- 1587, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18553/v1/024. findings-acl.91. URL https:/ accanthooogccyy... org/202 2024 4. findings-acl. 91.
Tian, Y., Peng, B., Song, L., Jin, L., Yu, D., Mi, H., and Yu, D. Toward selfoimmprvemmnt of 1lm via imagination, searching, and criticizing. arXiv preprint arXiv:2444. 12253, 2024.
Wan, Z., Feng, X., Wen, M., McAleer, S. M., Wen, Y., Zhang, W., and Wang, J. Alphazero-like tree-search can guide large language model decoding and training. In Forty-first International Conference on Machine Learn- ing. 2024.
Liu, Z., Lu, M., Zhang, S., Liu, B., Guo, H., Yang, Y., Blanchet, J., and Wang, Z. Provably mitigating overopti- mization in rlhf: Your sft loss is implicitly an adversarial regularizer. arXiv preprint arXiv:2405. 16436, 2024.
Wang. L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.- W., and Lim, E.-P. Plan-and-solve prompting: Improv- ing zero-shot thain-of-thought reasoning by large lan- guage models. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Vol- Toronto, Canada,
Luo, I.,, Liu, Y. Liu, R., Phatale, S., Lara, H., Li, Y., Shu, L., Zhu, Y., Meng, L., Sun, J., et al. Improve mathemati- cal reasoning in language models by automated process supervision. arXiv preprint arXiv:2406. 06000,06662226. 2024.
Manvi, R., Singh, A., and Ermon, S. Adaptive inference- time compute: Llms can predict if they can do better, even
ume I: Long Papers), pp.

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
July 2023a. Association for Computational Linguistics. doi: 10.18655//1/2023aac-looggg 147. URL https: //aclanthology http://ooy..rg//....o.o. acl long.147.
Ye, Gu, J., Zhao, X., Yin, W., and Wang, G. Assessing the creativity of lims in proposing novel solutions math- ematical problems. arXiv preprint arXiv:2410.18336, 2024.
Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Chi, E. H., Narang, S., Chowdhery, A., and Zhou, D. Self- consistency improves chain of thought reasoning in lan- guage models. In The Eleventh International Confer- ence on Learning Representations, 2023b. URL https: h/t////oppeeee /oppereeveex
Yu, Y., Chen, Z., Zhang, A., Tan, L.,, Zhu, C., Pang, R. Y., Qian, Y., Wang, X., Gururangan, S., Zhang, C.,, et al. Self- generated critiques boost reward modeling for language models. arXiv preprint mrXiv:2411.16644 2024.
(eaerrummi=11PPINNN 1d=1PLINIMM....
Zelikman, E., Wu, Y., Mu, and Goodman, N. Star: Boot-
strapping reasoning with reasoning. Advances Neural Information Processing Systems, 35:15476-15488 2022.
Wang, X., Feng, S., Li, Y., Yuan, P., Zhang, Y., Pan, B., Wang, H., Hu, Y., and Li, K. Make every penny count: aulfiuult-adaapive self-consistency for cost-efficient rea- soning, 2024. URL https://arxiv.co org/ abs/ 2408.
Zhao, Z., Dong, H., Saha, A., Xiong, C., and Sahoo, Automatic curriculum expert iteration for reliable I1m reasoning, 2024. URL https: / /arxiv.. oorggii.orr..... /abs/////// 2410. 07627.
Wei, J., Bosma, M., Zhao, V., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., and Le, Q. V. Finetuned lan- guage models are zero-shot learners. In International Conference Learning Representations, 2022a.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of- thoutt prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022b.
Wu, I. Fernandes, P., Bertsch, A., Kim, S., Pakazad, S., and Neubig, G. Better instruction-folloiin through minimum bayes risk. arXiv preprint ar.0v22410.02902, 2024.
Xiong, K., Ding, X., Cao, Y., Liu, T., and Qin, B. Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. In Findings of the Asso- ciation for Computational Linguistics: EMNLP 2023, pp. 7572-7590, 2023.
Xu, M. Li, Y., Sun, K., and Qian, T. Adaption-of- thought: Learning question difficulty improves large language models for reasoning. In Al-Onaizan, Y., Bansal, M., and Chen, Y.NN. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Nat- ural Language Processing, pp. 5468-5495, Miami, Florida, USA. November 2024. Association for Compu- tational Linguistics. doi: 10.186533w1122244emmpp..mm 313. URL https: htt////aannoooyy.oo.aaoooca emnlp-main.31.
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024.
12

---

Do NOT Think That Much for 2+3=? On the Overthinking of long reasoning LLMs
A. Appendix
clusterl [Solution 1, Solution Solution 11] stating or affirming the hasic arithmetic fact that 2 plus 3 equals 5. cluster2 [Solution 2, Solution5] breaking down the addi- tion into smaller, simpler steps reach the result. cluster3 [Solution 3, Solution 4] using practical analogy of counting objects to explain the addition.
A.1. Prompts for Clustering Solutions
Inspired by (Ye et al., 2024), we leverage GP to cluster the solutions for each instance into groups with the follow- ing prompt:
cluster4 [Solution 7] using subtraction as a reverse check to verify the addition result.
Criteria for clustering the mathematical solutions: If the solutions used arrive the solutions are fundamentally different from each other, such as algebraic manipulation versus geometric reasoning, they be considered novel;
cluster5 [Solution 8] using algebraic manipulation and solving simple equations to confirm the result.
cluster6 [Solution 9, Solution 10] converting numbers into different systems (binary and Roman numerals) to verify the result.
2. Even if the results are the same. if the intermediate steps or processes involved in reaching those solutions vary significantly, the solutions can be considered different;
cluster7 [Solution 12, Solution 13] considering specific contexts or frameworks like modular arithmetic pro- gramming which could change traditional addition results.
3. If the solutions relies on different assumptions or conditions, they should be considered different from each other;
4. A solution might generalize to broader class of problems, while another solution might be specific to certain conditions. In such cases, they are considered distinct;
If one solution is significantly simpler or more complex than the others, can be regarded essentially novel, if they lead to the same result.
Given the following mathematical problem: **obbememm**** ***
Solutions: Solution ...
Solution 2: ...
Please output the clusters strictly following the following format, each row containing a cluster. names, and reasons.
Do not include any additional text or explanations outside of this format:
clusterl [solution names] reason for cluster cluster2 [solution names] reason for cluster
cluster3 [solution names] reason for cluster
For example:
clusterl [Solution 1, Solution 3, Solution 5] similar alge- braic approach using the volume formula and canceling terms or directly solving for the height.
cluster2 [Solution 2, Solution 4] verifying the correctness and consistency of the formula and solution and consider- ing unit checks or logical reasoning on how volume relates to base area and height.
The clustering results for the BwQQ-3BB-Previee response in Figure 2 are:
13


Reviews:
[{"review_id": "6CZA5dKtqQ", "reviewer": "Reviewer_KedN", "summary": "This paper investigates the phenomenon of \"overthinking\" in recent \"o1-like\" large language models (LLMs), where models expend excessive computational resources (generating many tokens and solution steps) on simple problems with minimal benefit to accuracy or reasoning diversity. The authors provide the first comprehensive analysis of this issue, introducing novel outcome and process efficiency metrics to quantify the inefficiency.\n\n# update after rebuttal\n\nSince most of my concerns remain and significant revisions are still needed, I strongly suggest another round of revision. Thanks.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your insightful questions and suggestions.\n\nQ1: Provide the code for reproduction.\n\nA1: We will make our code publicly available soon.\n\n---\nQ2: The term \"o1-like models\" isn't precisely defined.\n\nA2: Thank you for pointing this out. We acknowledge that our original description of “o1-like” models lacks precision. A clearer term would be “Long Reasoning Models”, which refers to models that generate detailed CoT reasoning by iteratively producing intermediate reasoning steps and sequentially refining solutions until reaching a final conclusion.\n\n---\nQ3&Q4:More details about the temperatures.\n\nA3&A4: We set the temperature to 1.0 for generating training data to promote diversity. For benchmark evaluations, we used a temperature of 0 to ensure reproducibility. \n\nWe followed your suggestion and conducted a more detailed analysis regarding the impact of temperature settings. The table below summarizes the results of QwQ-32B-Preview evaluated on the Math Level 1 and 5. Clearly, the temperature hyperparameter has only a marginal effect on the number of generated solutions, further confirming our assertion that \"overthinking\" is a fundamental behavior inherent to these models.\n\n|   Level |   temperature |   solution_num |\n|--------:|--------------:|---------------:|\n|       1 |           0   |            3.7 |\n|       1 |           0.3 |            3.9 |\n|       1 |           0.5 |            3.6 |\n|       1 |           0.7 |            3.8 |\n|       1 |           1   |            3.6 |\n|||\n|       5 |           0   |            4   |\n|       5 |           0.3 |            3.9 |\n|       5 |           0.5 |            3.9 |\n|       5 |           0.7 |            4   |\n|       5 |           1   |            3.9 |\n\n---\nQ5: Table 4 is hard to interpret. It is not sure whether the proposed method has a big advantage.\n\nA5: Both approaches demonstrate advantages in simplifying long CoT processes and mitigating overthinking. Nonetheless, considering benchmark performances and their suitability for challenging problems, we prefer \"SimPO+FCS+Reflection\" as our final recommended approach.\n\n---\nQ6:The insight is limited and the overall observations are superficial.\n\nA6: Our results indicate that \"overthinking\" mainly happens in Long Reasoning Models, which learn advanced reasoning methods (e.g., reflection and backtracking) through reinforcement learning. When they fail to stop reasoning at the right time, they start overthinking.\n\nWe confirmed our findings by evaluating two additional models, DeepSeek-R1 and QwQ-32B, which were released after our original submission, further proving that our results are robust. One important observation was that the formal (full-release) versions of the models provided higher accuracy than their preview versions, but exhibited much stronger overthinking behaviors. These results further demonstrate that the overthinking problem is a fundamental behavior inherent to these models.\n\n|Model|Acc|#Solutions|Length|\n|---|---|---|---|\n|R1-Preview|93.4|2.8|2168.6|\n|R1|96.4|4.3|2704.3|\n|QwQ-Preview|93.0|3.2|2407.9|\n|QwQ|96.4|7.1|4799.4|\n\nHowever, more in-depth research is still needed to fully understand the root causes of overthinking. While our current method effectively reduces this phenomenon, it doesn't completely solve the underlying problem. We plan to investigate more fundamental solutions in future work.\n\nFinally, it is worth noting that our primary contribution is presenting the first comprehensive study  that clearly explains the overthinking issue. We show that Long Reasoning Models frequently waste computational effort by generating unnecessary or repeated reasoning steps that add very little benefit. This notable contribution has been recognized positively by the other reviewers.\n\n---\nQ7: There is no formal definition of the \"Overthinking\" issue.\n\nA7: Formally speaking, we define \"overthinking\" as a phenomenon where a long reasoning model repeatedly evaluates and revisits solutions within the CoT reasoning process, rather than converging promptly to a final answer. We will incorporate this formal definition explicitly in the Introduction section of our forthcoming manuscript revision.\n\n---\nQ8: The proposed solution seems incremental. \n\nA8: We acknowledge that our proposed method does not completely resolve the fundamental cause of the \"overthinking\" phenomenon. Our primary contribution instead lies in providing the community’s first systematic analysis and explicit characterization of the overthinking problem, introducing quantitative metrics to assess efficiency specifically in Long Reasoning Models, and proposing viable mitigation strategies. We aim for our findings and benchmarks to provide a valuable framework to enhance the community’s overall understanding of these models' behavior and guide future research toward addressing overthinking more comprehensively.", "claims_and_evidence": "Fine", "methods_and_evaluation_criteria": "Fine", "theoretical_claims": "Fine", "experimental_designs_or_analyses": "Fine", "relation_to_broader_scientific_literature": "Fine", "essential_references_not_discussed": "Fine", "other_strengths_and_weaknesses": "Hope the feedback is valuable for the authors and helps improve the quality in the revision or camera ready. I am happy to update my score after rebuttal if necessary. Thanks!\n\nPros:\n1. Addresses a pertinent and relatively unexplored inefficiency (\"overthinking\") emerging in the latest generation of powerful reasoning LLMs.\n2. Effectively demonstrates and quantifies the overthinking issue through clear examples (Fig 1, Fig 2) and quantitative analysis (solution distributions, token usage vs. difficulty).\n3. Proposes novel \"Outcome Efficiency\" (ξo) and \"Process Efficiency\" (ξp) metrics that provide a more nuanced evaluation of LLM reasoning beyond simple accuracy, considering resource usage and solution diversity.\n4. Develops and evaluates concrete self-training based methods (preference optimization + response simplification) that demonstrably improve efficiency without significant accuracy loss.\n\nCons:\n1. It is suggested that the authors provide the code for reproduction.\n2. The central concept relies on \"o1-like\" models, but this term isn't precisely defined beyond citing specific models (QwQ, DeepSeek-R1, Kimi). The paper could benefit from explicitly stating the defining characteristics of this model class (e.g., explicit multi-solution generation, specific internal reflection mechanisms cited by the model providers) that lead to the observed overthinking, rather than just relying on the \"o1\" name recognition.\n3. It is not very clear how the authors set the hyperparameters such as temperatures. It suggested that the authors provide more details about all the experiment setting. Although temperature is briefly mentioned in \"We generated 10 samples for each instance in the training dataset with a temperature of 1.0\" (line 312), more explanations about the selection of the temperature and the temperature setting for the other parts of the study are desired.\n4. It is not very clear about the impact of hyperparameters such as temperatures on the \"Overthinking\" issues. It is suggested that the authors conduct a more detailed analysis on the sensitivity of the hyperparameters.\n5. Table 4 is hard to interpret. \"+SimPOFCS+Reflection (Ours) 92.8 1.9 1330.7 80.0% 89.5%\" does not have much difference compared to \"+SimPOFirst-Correct Solution 91.0 1.4 1016.0 88.7% 98.1%\" and \"+SimPOGreedily Diverse Solutions 91.8 1.7 1286.1 84.3% 93.6%\". It is not sure whether the proposed method has a big advantage.\n6. The insight is limited and the overall observations are superficial. Why Some LLMs have the \"Overthinking\" issues? Do all reasoning LLMs have the \"Overthinking\" issues? What are the fundamental cause of the \"Overthinking\" issues? Does the proposed method solve the fundamental cause of the \"Overthinking\" issues?\n7. There is no formal definition of the \"Overthinking\" issue. It is suggested the authors provide a formal definition of the \"Overthinking\" issue. Otherwise, it is not very clear.\n8. The proposed solution seems incremental. It is mostly based on SimPO. It seems that the proposed method does not solve the fundamental cause of the \"Overthinking\" issues.", "other_comments_or_suggestions": "Typo:\n1. Figure 1 is referred to as \"Figure 1(a)\" in the text (line 041), but the figure itself doesn't have an \"(a)\" label.", "questions_for_authors": "See Weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nQzylaZKmI", "reviewer": "Reviewer_xCey", "summary": "This paper studies the over-thinking problem of o1-like models. The overthinking refers to the scenario where unnecessary compute resource is used for simple question. \nIt first compares the average tokens needed by traditional LLM and the o1-like reasoning models. It finds that o1-like models spend much more tokens on simple problems with minimal benefits and lack diversity in reasoning strategies. Based on this observation, it proposes several metrics to evaluate the efficiency of LLM reasoning models\nIt then adopts a self-training paradigm to mitigate this issue by removing redundant solutions. Results on GSM8K, MATH500, GPQA, and AIME demonstrate effectiveness and robustness", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your questions. Below we address each point mentioned in your review.\n\nSince most of your questions pertain to Section 3, we provide additional clarifications on our methods as follows:\n\n1. Our proposed methods focus primarily on constructing suitable training data. Specifically, Section 3.2 describes how we generate paired training data for preference optimization algorithms.\n\n2. Section 3.1 describes the preference optimization algorithms employed in our experiments. As you pointed out, these algorithms were not originally designed for length optimization. Nevertheless, we explore their potential effectiveness in addressing the length optimization task.\n\n3. We evaluated the performance of various preference optimization algorithms across different training datasets. Ultimately, we chose SimPO trained with the FCS+Reflection dataset as our final method, marked as \"ours\" in Table 4.\n\n4. The total number of training examples is approximately 11,000. Both SFT and preference optimization methods were trained using a learning rate of 1e-7, with a cosine learning rate scheduler, and a batch size of 16. We trained the SFT models for four epochs and the preference optimization models for one epoch. We will include these details in the appendix of the next version.\n\n5. For simplicity and clarity, we first compared multiple training algorithms using the \"Shortest Response\" dataset, finding SimPO performed the best. Subsequently, we tested SimPO on a broader range of training datasets and selected \"FCS+Reflection\" as our best performing dataset configuration. The complete experimental results will be provided in the appendix in our revised manuscript.\n\nThank you again for your insightful questions and valuable suggestions. We will improve the clarity and presentation of Section 3 accordingly.", "claims_and_evidence": "This paper claims that o1-like reasoning models have the overthinking issue. It investigates this issue based on the proposed Outcome Efficiency Metric, distinctness ratio, and Process Efficiency Metric. Their claims are supported by the comparisons between traditional LLM and o1-models, which shows that their longer responses are less efficient in improving accuracy and solution diversity.", "methods_and_evaluation_criteria": "To investigate the overthinking, it proposes new evaluation metrics Outcome Efficiency Metric, distinctness ratio, and Process Efficiency Metric to evaluate the efficiency and diversity of the solution. One concern for the diversity is that it is independent of the correctness. High diversity but an incorrect final response may not make too much sense.\n\nTo mitigate overthinking, it finetunes the model towards short and correct solutions. It also finetunes the model on simplified responses.", "theoretical_claims": "There is no theoretical claim in the paper.", "experimental_designs_or_analyses": "The experiments in Section 3 lack the necessary details\n\t• What are the sizes of different kinds of training examples in Table 2 and Table 3?\n\t• What are the training details (such as hyperparameters) for models in Section 3?\n\nThe descriptions in Section 3.1 and 3.2 are confusing. Which part is the method proposed by this paper? \n\t• Section 3.1 lists several self-training methods such as SFT and DPO, but these methods are not particularly designed for Length Preference Optimization. What does Length Preference Optimization refer to here? Is there a specific dataset for length optimization?\n\t• The purpose of Streamlining Responses in Section 3.2 is unclear. Are they used during the inference time to select the best response? Or are they the strategies for constructing self-training datasets?\n\nWhy experimental settings in different datasets are different? For example, on MATH500, there are 7 settings, while for other datasets, there is only one. Similarly, Qwen2.5-Math-72B-Instruct is one of the baselines for GPQA and AIME24, but their performance on ASDIV, GSM8k, MATH500 is not mentioned. \n\nWhat does \"ours\" mean in SimPO FCS+Reflection (Ours) ? It looks like this one is the final method proposed by this paper and the other settings are just baseline or ablation studies. If that is the case, this should be explicitly mentioned.", "relation_to_broader_scientific_literature": "The overthinking issue is interesting and important for analyzing the performance of current o1-like models. It provides insights on how many of the tokens in the extremely long chain of thought contribute to the final accuracy.", "essential_references_not_discussed": "Necessary references have been discussed in their related work", "other_strengths_and_weaknesses": "Strength: The detailed and comprehensive investigations on the overthinking scenario of o1-like models provide more insight into the performance of o1-like models.\n\nWeakness: The methods and experiments to mitigate the overthinking issue are not convincing.", "other_comments_or_suggestions": "Section 3 should be reorganized to make it clearer. Necessary experimental details should be added for reproduction.", "questions_for_authors": "In the Outcome Efficiency Metric and Process Efficiency Metric, why does the first consider correctness and the second one independent of correctness?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rlQwc1N7FA", "reviewer": "Reviewer_JarS", "summary": "This paper addresses the \"overthinking\" issue observed in o1-like reasoning models, which expend excessive computational resources during inference, especially for simple problems. The authors first analyze this phenomenon by prompting LLMs with trivial questions (e.g., \"What is the answer of 2 plus 3?\"), observing significant redundancy in generated solutions. They identify that models often produce numerous redundant or minimally diverse solutions, which do not substantially contribute to improving accuracy. To quantify this, the authors propose novel efficiency metrics from both outcome (accuracy contribution) and process (solution diversity) perspectives. Leveraging these metrics, the authors introduce a self-distillation approach that samples diverse solutions via temperature sampling and recombines selected efficient solutions to streamline inference. Experiments across benchmarks (e.g., GSM8K, MATH500, GPQA, AIME) demonstrate the proposed approach significantly reduces token usage while maintaining accuracy, highlighting its effectiveness in enhancing CoT efficiency.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful questions. We address each question raised in your review individually below.\n\nQ (Weakness): While the metrics and simplification methods are effective, the generalizability to domains beyond mathematical reasoning tasks remains unclear.\n\nA: This is a valuable point. In this paper, we primarily focus on the mathematical reasoning domain because verifying the correctness of answers to mathematical questions is straightforward and objective. Nevertheless, we have noticed similar \"overthinking\" phenomena in general reasoning tasks, such as repeatedly asking the same questions or recurring assumptions. A deeper and more comprehensive analysis of such phenomena in more general tasks is warranted, which we plan to explore in future work.\n\n---\nQ: Could the authors clarify how they categorize this type of reflective or \"aha moment\" behavior in their framework?\n\nA: The \"aha moment\" described in R1-Zero refers to a training-time behavior, in which a model autonomously acquires reasoning strategies via reinforcement learning. In contrast, our framework specifically targets inference-time behaviors in models executing long reasoning processes. In our work, a \"solution\" explicitly denotes portions of the model-generated outputs containing a final answer. Therefore, the concept of the \"aha moment\" aligns more closely with reflective behaviors (e.g., reflection, self-verification, or backtracking) that bridge two solutions. When the model continues reasoning beyond obtaining an answer within a solution — whether through reflection, self-verification, or backtracking — we characterize this behavior as \"overthinking\".\n\n---\nQ: Should a solution that emerges from a reflective or backtracking process be retained during the self-distillation process, or would it be considered redundant and thus removed?\n\nA: Whether a reflective or backtracking-derived solution should be retained during self-distillation depends on the specific training data construction policy described in Section 3.2. For example, our \"First Correct Solution\" (FCS) policy retains only the first solution generated, whereas the \"FCS+Reflection\" policy retains one additional reflective solution alongside the initial FCS solution.", "claims_and_evidence": "The claims presented in the paper are generally supported by clear and convincing evidence. Specifically, the paper makes three primary claims: \n(1) o1-like large language models exhibit a significant \"overthinking\" issue, especially on simple problems.\n(2) novel efficiency metrics (outcome and process metrics) effectively quantify this inefficiency.\n(3) self-distillation strategies leveraging these metrics substantially mitigate overthinking without compromising performance.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria effectively address the stated problem. The authors introduce intuitive outcome and process efficiency metrics, clearly capturing the extent and nature of the \"overthinking\" phenomenon in o1-like models. Their chosen benchmark datasets (ASDIV, GSM8K, MATH500, GPQA, AIME) are appropriate and well-aligned with the goal of assessing reasoning efficiency across a range of difficulty levels. Additionally, the self-training approaches (SimPO combined with streamlined solutions like FCS+Reflection) logically target inefficiencies identified by the metrics, making them well-suited for the study's objectives. Overall, both the methods and evaluation criteria are well-motivated and sensible for the problem at hand.", "theoretical_claims": "This is not a theory paper. It conducts emperical study.", "experimental_designs_or_analyses": "The experimental designs and analyses presented in the paper are generally sound and valid. The authors provide thorough and rigorous experiments across multiple well-established benchmark datasets of varying difficulty (ASDIV, GSM8K, MATH500, GPQA, and AIME), clearly demonstrating the effectiveness of their proposed methods in mitigating overthinking.", "relation_to_broader_scientific_literature": "The paper clearly situates its contributions within the broader literature. While prior research has extensively studied methods like self-consistency, Tree-of-Thought, and early-stopping to optimize test-time computation and improve model reasoning, this paper uniquely identifies the \"overthinking\" issue specific to o1-like LLMs. The authors bridge a relevant gap by proposing targeted efficiency metrics and self-training strategies (SimPO, FCS+Reflection) that explicitly address inefficiencies in extended chains-of-thought.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Strengths:\n\n1.The paper addresses a novel and practically significant problem—the \"overthinking\" inefficiency in o1-like LLMs—providing valuable insights into the intelligent allocation of inference-time resources.\n\n2.The proposed metrics (outcome and process efficiency) are original, clearly defined, and intuitively appealing.\n\n3.Experiments are comprehensive, covering multiple datasets and difficulty levels, enhancing confidence in the robustness of the findings.\n\n\nWeaknesses:\n\n1.While the metrics and simplification methods are effective, the generalizability to domains beyond mathematical reasoning tasks remains unclear.", "other_comments_or_suggestions": "None.", "questions_for_authors": "The paper does not discuss the \"self-reflection\" or \"aha moment\" behavior recently appeared in \"Guo, Daya, et al. \"Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.\" (2025).\", which describes LLMs revising earlier reasoning after reflecting or backtracking.\n\nQuestion 1:\nCould the authors clarify how they categorize this type of reflective or \"aha moment\" behavior in their framework? \n\nQuestion2:\nShould a solution that emerges from a reflective or backtracking process be retained during the self-distillation process, or would it be considered redundant and thus removed?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IgF8dOZe78", "reviewer": "Reviewer_yXZs", "summary": "The authors observe that these models often allocate excessive computational resources to simple problems, leading to inefficiencies. To address this, they introduce novel efficiency metrics from both outcome and process perspectives and propose self-training strategies to streamline reasoning without compromising accuracy. Experimental results demonstrate that their approach reduces computational overhead while preserving model performance across various test sets.\n\n## update after rebuttal\n\nI will keep my positive rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your questions! We will answer the questions mentioned in your review.\n\nQ: Do you have the results for the proposed efficiency enhancing methods with more models?\n\nA: Yes, we have conducted experiments on the R1-Distilled-Qwen-32B model, and our efficiency-enhancing methods remain effective.\n\n| Model                 | Dataset | Acc  | #Solutions | Length | Outcome Efficiency | Process Efficiency |\n| --------------------- | ------- | ---- | ---------- | ------ | ------------------ | ------------------ |\n| R1-Distilled-Qwen-32B | ASDIV   | 96.2 | 1.6        | 308.6  | 84.4               | 95.0               |\n| +SimPO+FCS+Reflection | ASDIV   | 96.4 | 1.2        | 209.1  | 87.0               | 97.4               |\n|                       |         |      |            |        |                    |                    |\n| R1-Distilled-Qwen-32B | GSM8k   | 94.1 | 1.3        | 328.3  | 80.9               | 96.4               |\n| +SimPO+FCS+Reflection | GSM8k   | 93.3 | 1.1        | 254.4  | 82.5               | 98.1               |\n|                       |         |      |            |        |                    |                    |\n| R1-Distilled-Qwen-32B | MATH500 | 90.2 | 4.7        | 3210.0 | 54.2               | 71.0               |\n| +SimPO+FCS+Reflection | MATH500 | 91.4 | 3.4        | 2735.2 | 63.9               | 77.2               |\n\n\n---\n---\nQ: Have you checked the evaluation on other benchmarks beyond mathematical/scientific reasoning?\n\nA: We have not performed evaluations on benchmarks beyond mathematical and scientific reasoning. There are two primary reasons behind this choice: (1) Our paper specifically focuses on mathematical and scientific reasoning tasks, as they represent cutting-edge domains that strongly reflect aspects of human intelligence. (2) Benchmarks in general domains often lack clearly defined and verifiable answers, which makes objective evaluation challenging.\n\nInvestigating the overthinking phenomenon in more general tasks by leveraging an LLM-as-a-judge evaluation approach is an interesting direction that we intend to explore in future work.", "claims_and_evidence": "The primary claim is that o1-like LLMs exhibit overthinking, especially on simple problems, resulting in unnecessary computational resource usage. The authors support this by analyzing model responses to simple arithmetic questions, where o1-like models consumed significantly more tokens than conventional models to reach the same answer. They also introduce efficiency metrics to quantify this overthinking and demonstrate that their self-training strategies can mitigate it without sacrificing accuracy.", "methods_and_evaluation_criteria": "The authors employ a self-training paradigm to reduce overthinking, streamlining reasoning processes without compromising accuracy. They evaluate their approach using novel efficiency metrics from both outcome and process perspectives, assessing the rational use of computational resources by o1-like models. Experimental results show that their approach successfully reduces computational overhead while preserving model performance across a range of test sets with varying difficulty levels.", "theoretical_claims": "The paper does not focus on theoretical claims but rather on empirical observations and practical solutions to the overthinking problem in o1-like LLMs.", "experimental_designs_or_analyses": "Yes. The experimental design involves evaluating o1-like models on various mathematical benchmarks to assess overthinking patterns. The authors introduce efficiency metrics to quantify overthinking and apply self-training strategies to mitigate it. They demonstrate that their approach reduces computational overhead while preserving model performance across different test sets. In general, the experiments are solid and convincing.", "relation_to_broader_scientific_literature": "This work contributes to the broader scientific literature by addressing the inefficiencies in o1-like LLMs due to overthinking. By introducing efficiency metrics and self-training strategies, the authors provide insights into optimizing computational resource allocation during inference, aligning with ongoing research on improving the efficiency and effectiveness of LLMs.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "A potential weakness is the generalization of the proposed phenomenon and algorithm. More models and general evaluations can be helpful.", "other_comments_or_suggestions": "NA", "questions_for_authors": "Do you have the results for the proposed efficiency enhancing methods with more models?\n\nHave you checked the evaluation on other benchmarks beyond mathematical/scientific reasoning?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper tackles the overthinking problem in long reasoning models or o1 like reasoning models. It was found that the models generate unnecessarily long and repetitive reasoning chains for simple problems. The authors measures this issue and propose self-training methods to reduce the redundant computations without hurting accuracy. The work focuses on a practical problem and the experiment results are significant. But some reviewers felt that the analysis lacks depth and the evaluation and the evaluation is only limited to math tasks. There is definitely room for improvement but the paper could be interesting to some ICML audiences.
Decision: Accept (poster)