Title: Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Reasoning

Large Language Models

Efficiency
Abstract: While recent success of large reasoning models (LRMs) significantly advanced LLMs' reasoning capability by optimizing the final answer accuracy using reinforcement learning, they may also drastically increase the output length due to *overthinking*—characterized by unnecessarily complex reasoning paths that waste computation and potentially degrade the performance.
We hypothesize that such inefficiencies stem from LRMs' limited capability to dynamically select the proper modular reasoning strategies, termed *thinking patterns* at the right position. To investigate this hypothesis, we propose a dynamic optimization framework that segments model-generated reasoning paths into distinct thinking patterns, systematically identifying and promoting beneficial patterns that improve the answer while removing detrimental ones. Empirical analysis confirms that our optimized thinking paths yield more concise yet sufficiently informative trajectories, enhancing reasoning efficiency by reducing attention FLOPs by up to 47% while maintaining accuracy for originally correct responses. Moreover, a non-trivial portion of originally incorrect responses are transformed into correct ones, achieving a 15.6% accuracy improvement with reduced length.
Motivated by the improvement brought by the optimized thinking paths, we apply a preference optimization technique supported by a pairwise dataset contrasting suboptimal and optimal reasoning paths. Experimental evaluations across multiple mathematical reasoning benchmarks reveal that our method notably reduces computational overhead while simultaneously improving reasoning accuracy, achieving up to a 12% accuracy improvement and reducing token usage from approximately 5,000 to 3,000 tokens.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the \"overthinking\" issue in Large Reasoning Models (LRMs), where they produce overly long and complex reasoning, harming efficiency. The authors propose a Dynamic Thinking pattern Optimization (DTO) framework. This framework aims to segment reasoning paths into \"thinking patterns,\" then identify and promote useful ones while removing those that are detrimental. The DTO process involves finding good stopping points for reasoning using Monte Carlo estimation and then pruning unhelpful segments with an auxiliary Large Language Model (LLM). The authors further refine LRMs by using preference optimization, creating a dataset of DTO-improved trajectories versus unoptimized ones to guide the model. Experiments across mathematical reasoning tasks show their method can reduce computational load (like FLOPs and token count) while keeping or even boosting accuracy. For instance, it reduced attention FLOPs by up to 47% while maintaining accuracy for initially correct responses, and improved accuracy by 15.6% for some initially incorrect ones. With preference optimization, they report up to a 12% accuracy gain and a token reduction from about 5,000 to 3,000.\n\nSTRENGTHS AND WEAKNESSES:\nI think the paper is reasonably spund. The approach of optimizing thinking dynamics in LRMs by segmenting reasoning into \"thinking patterns\" and then refining them is quite interesting. The empirical results, showing reductions in computational overhead like FLOPs and token usage while often maintaining or even improving accuracy, appear promising. I particularly noted the ability of the DTO framework to sometimes correct initially flawed reasoning paths, which is a valuable outcome. The reported efficiency gains, such as reducing attention FLOPs by up to 47% and token usage from around 5,000 to 3,000 while also achieving accuracy improvements, are significant.\n\nHowever, I have a few points:\n\n1.  It seems the DTO framework has two main stages: identifying an optimal termination point using Monte Carlo estimation, and then pruning redundant segments with an auxiliary LLM. I'm wondering if there was an ablation study performed to understand the individual contribution of each of these stages. It would be insightful to know which component contributes more to the overall improvement, or if both are equally critical.\n\n2.  The maximum response length was set to 8192 tokens for the experiments. I'm a bit concerned whether many of the baseline models might not have fully generated their responses within this limit, potentially truncating their reasoning, leading to incorrect answer (which is a super critical problem). This could affect the comparison, especially if the DTO-optimized models inherently produce shorter responses.\n\n3.  Based on my experience, smaller models like 1.5B or even 7B parameters can sometimes (even often) get stuck in loops or fail to finalize their answers. I think a simple but potentially strong baseline could involve forcing the model to generate an answer once it reaches a certain token limit, perhaps using a similar \"finalization\" mechanism as described in the paper. This might offer a more direct comparison for the conciseness and termination aspects of DTO. \n\n4. While I understand compute limitations might restrict extensive experiments on much larger models, I do have some slight reservations about the generalizability of these findings to significantly larger models without further evidence (considering small models' may have limited \"self-correction\" ability).\n\nQUESTIONS:\nSee weakness\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a response optimization method for LRMs (large reasoning models). Specifically, it first uses Monte Carlo sampling to truncate responses and then employs an auxiliary LLM to remove redundant segments, aiming to reduce length without compromising accuracy. Preference optimization based on the optimized responses achieves competitive efficiency.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The idea of optimizing the individual contributions of reasoning segments is insightful, and the authors make a clear effort to achieve this goal through their proposed method.\n2. The experimental results are competitive and the statistical analysis is well-executed and easy to appreciate.\n3. The paper is content-rich, and the method section is clearly described.\n\n**Weaknesses**\n\n1. The proposed method is less efficient in collecting training samples compared to several baselines like FCS.\n2. The experiments are limited to two 1.5B models, making it unclear whether the proposed method remains effective on larger LRMs.\n\nQUESTIONS:\n1. I'm confused about how DTO is applied to incorrect responses, as discussed in Sections 3.3 and Section 4.1.\n    1. In Section 3.3, DTO relies on the gold answer in three ways: to truncate the given response, to get the shortest correct completion, and to remove redundant segments. Then the optimized response should achieve 100% accuracy if a correct trajectory exists in the sampling space. Therefore, Lines 192-196 are *not interesting*, as this is an unfair comparison with the initially yielded responses.\n    2. In Section 4.1 Lines 216-217, it is not guarenteed that applying DTO to a randomly sampled incorrect response will lead to an optimized response. How do you handle this?\n2. How do you segment a response y into a sequence of distinct thinking patterns? How many segments does a response have in average? And calculating all of these, how many completions need to be sampled per response to get an optimized response using DTO?\n3. FCS employs QwQ-32B. Can your proposed method show efficiency in larger LRMs, at least on 7B models?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce Dynamic Thinking Pattern Optimization (DTO), a mechanism designed to solve the 'overthinking' of large reasoning models (LRMs). DTO first segments a generated reasoning trace into modular thinking patterns and locates the earliest point where the model can produce the correct answer, yielding a truncated path. Then, DTO elicits a minimal answer token string, which produces an optimized trajectory by removing inefficient patterns. These optimized reasoning traces achieve reasoning efficiency by reducing attention FLOPs by up to 47%. The authors also provide preference optimization with DTO, which also enhances the accuracy performance.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1) While current studies have focused on making longer reasoning traces, the authors' motivation to shorten and optimize the length of reasoning is fascinating. It is also intriguing to formulate this as a constrained optimization problem.\n\n2) By delivering simultaneous gains in both efficiency and accuracy, DTO outperforms baseline methods, except for #Tokens of Instruct ver. DTO's ability to generate minimal yet sufficient reasoning paths not only lowers inference cost but also supplies high-quality supervision for further model improvement.\n\n3) I can see that DTO is not tied to arithmetic reasoning alone, but can compact and preserve the correctness of chains-of-thought in diverse subject areas from Table 4. Although the authors mentioned mathematical-focused validation as one of their limitations, I am pretty confident that DTO would reduce the reasoning token lengths or attention FLOPs in more varied benchmarks.\n\n\n## Weaknesses\n\n1) DTO's pruning stage, after determining the 'finalize' thinking pattern, requires an auxiliary LLM, which is different from the original LRM. The auxiliary LLM judges and removes redundant thinking patterns. The authors use a 70B model for an auxiliary model, whereas the original model's size is 1.5B. I wonder whether it is fair to use the auxiliary model of such a large size, when the baselines seem to only use 1.5B, which is the same size as the original LRM. However, I do not see any justification for using the large auxiliary model for DTO.\n\n2) DTO splits the reasoning trace with words such as 'Wait' or 'Alternatively'. DTO cannot split if such words are not contained in the reasoning trace. I presume the paper needs a more sophisticated method to segment a reasoning trajectory in a sequence of thinking patterns. If the current method is sufficient, please explain the reason.\n\n3) Tables 1--4 contain Instruct ver. as a baseline, which is an instruction-tuned version of LRMs. I noticed the accuracy of this baseline is low, but it has the lowest #Tokens. For instance, if we examine the #Tokens of Instruct ver. for GSM8K in Table 2, it shows approximately 3 times fewer tokens than DTO, even with on-par accuracy scores. I wonder if a simple in-context learning technique, such as providing few-shot demonstrations or Chain-of-Thought (CoT), would be more powerful than DTO in both accuracy and #Tokens. The authors should also analyze the result of #Tokens of Instruct ver. compared to DTO.\n\nQUESTIONS:\n1) Can DTO handle queries where no golden label is available? I wonder whether DTO is applicable in such settings.\n\n2) Can we use the same LRM as an auxiliary LLM? For instance, in this paper, can we use DeepSeek-R1-Distill-Qwen-1.5B or DeepScaleR-1.5B-Preview? If so, why didn't the authors conduct the experiments? If not, it would be helpful to explain the reason.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper present a novel method to improve quality of the reasoning in a dynamic manner. In particular, after model generating examples in the RL procedure, it refine these examples by cutting in the middle to only preserve the first-half part that are believed to be sufficient to derive the final answer. Additionally, it also presents another method to remove unnecessary reasoning steps in the middle, by testing if the correct answer can be obtained by removing the middle steps. Experimental evaluation shows that the proposed method effectively reducing the reasoning length while improving the reasoning accuracy as well.

STRENGTHS AND WEAKNESSES:
The idea of refining the reasoning patterns seems reasonable to me, but the method to achieve this goal is not quite convincing. The model eventually learns from generated reasoning patterns and as a result improving the patterns is useful. In terms of the method proposed, I think the method of removing in-the-middle reasoning steps is dangerous to me.

QUESTIONS:
Removing in-the-middle reasoning steps:

Consider this reasoning pattern: A->B->C->D->Result. The D may contain sufficient information to obtain Results (many LLMs explicitly or implicitly generate results in D before finally conclude the results). But the fact is the B is necessary to obtain C and then D. Removing B is not OK and can ruin the model. So I think the method proposed in the paper is faulty.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03