ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning
Authors: Bairu Hou
Yang Zhang
Jiabao Ji
Yujian Liu
Kaizhi Qian
Jacob Andreas
Shiyu Chang
Venue: Accepted by TMLR ()

Abstract: We present ThinkPrune, a simple yet effective method for pruning the thinking length for long-thinking LLMs, which has been found to often produce inefficient and redundant thinking processes. Existing preliminary explorations of reducing thinking length primarily focus on forcing the thinking process to early exit, rather than adapting the LLM to optimize and consolidate the thinking process, and therefore the length-performance tradeoff observed so far is sub-optimal. To fill this gap, ThinkPrune offers a simple solution that continuously trains the long-thinking LLMs via reinforcement learning (RL) with an added token limit, beyond which any unfinished thoughts and answers will be discarded, resulting in a zero reward. To further preserve model performance, we introduce an iterative length pruning approach, where multiple rounds of RL are conducted, each with an increasingly more stringent token limit. We observed that ThinkPrune results in a remarkable performance-length tradeoff on the AIME24 dataset, the reasoning length of DeepSeek-R1-Distill-Qwen-1.5B can be reduced by half with only 2% drop in performance. We also observed that after pruning, the LLMs can bypass unnecessary steps while keeping the core reasoning process complete.

Published in Transactions on Machine Learning Research (Dec/2025)
ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning
Bairu Hou' Jacob Andreas
Yang Zhang² Shiyu Chang!
Jiabao Ji'
Yujian Liu'
Kaizhi Qian²
UC Santa Barbara
MIT-IBM Watson Al Lab
MIT CSAIL
Reviewed on OpenReview:
Abstract
We present TRINKPRUNE, simple yet effective method for pruning the thinking length for long-thinking LLMs, which have been found to often produce inefficient and redundant thinking processes. Existing preliminary explorations of reducing thinking length primarily focus on forcing the thinking process to early exit, rather than adapting the LLM to optimize and consolidate the thinking process, and therefore the length-per pertt--pefoommccc tradeoff observed so far is sub-optimal. To fill this gap, THInKPRuNE offers simple solution that continuously trains the tingthhikking LLMs via reinforcement learning (RL) with an added token limit, beyond which any unfinished thoughts and answers will be discarded, resulting in zero reward. To further preserve model performance, we introduce an iterative length pruning ap- proach, where multiple rounds of RL are conducted, each with an increasingly more stringent token limit. We observed that THINKPRUNE results in a remarkable performance-lenge tradeoff on the AIME24 dataset, the reasoning length of can be reduced by half with only 2% drop in performance. We also obscrved that after pruning, the LLMs can bypass unnecessary sleps while keeping the core reasoning process complete. Code is available at https://github www/UCSt CLA//UUSSEN-LPPMMAE
ThinkPrune,
1 Introduction
Recent advances in large language models (LLMs) have demonstrated the effectiveness of in inference-iime scaling through reinforcement learning [DeepSeek-A]] 2025 OpenAl 2024 Liu & Zhang 2025 Zeng al., 2025a), where LLMs learn to produce long and sophisticated reasoning behaviors such self-refcction and verification, significantly increasing their performance on wide range of benchmarks. However, one key challenge of inference-time scaling is the significant number of tokens produced at inference time, leading to high computational and memory overhead. For example, on the MATH500 (Lightman et al. 2023) benchmark, the DeepSeek- BenSeekkRR-Distiee Ri-Distil1-Qwen 1.5 5B model generates solutions with more than 15,000 tokens on average, while many of the questions could have been solved with fewer than 1,000 tokens by regular I.L.s. This highlights the issuc of over-thinking, where many reasoning steps might be redundant or incfficient (umaa
et al. 2025 Chen et al. 2024 Sui et al. 2025).
There are now some preliminary explorations of limiting the generation length via budget-foreing Fuu et al, 2024 Mucnnighoff et al. 2025), where, when reaching a given token limit, the thinking process is forced to early exit, e.g., by appending an end-of-thinking token and producing an answer right away. Figure aa) (left) shows an example output of S1 (Muennighoff et al. 2025), a budget-forcing method, under a low budget (2000 thinking tokens), where the thinking process spent 453 tokens just understand the problem, and therefore could not complete its second round of thought within the budget. Accordingly, non-trivial performance drop would occur as the budget gets tighter, as shown in Figure [b) (blue lines).

---

Published in Transactions on Machine Learning Research (Dec/202))
Suzanne walks four miles every third day. What fewest number of miles she walk in Fobruary?
pween1.SBddistill RLL AIME ACc v.s. thinking Length
Budget- faaccing
ThinkPrune
Probiem onderstendieg: 451
Problase understandleeg: 1é6 Aodous
Fins, February can have 28 29 days... Wail... let me doubie-check. Hutl before that, let me ro-read the problern...
u. let me think step by siop...ssumm it's nour-kap your with 28 days.
Problem-sshinng anragt 2: tokens
Problen-sohhing atterpe 337 tokons
she walks every third dey... then 7, 7, 22, 25, 28.
February has days through 28... So starting from day 1, adding time: 1,4 7... Therefore miles
Wait, that's days in 28 days. Wait.. Let me think again... Therefore, the lowest number of milles... ... is 40 miles.
x
deepscaler Length
Uoce Maae
Seff-rortffc enlonn 99/ nokour
Sett-veritoatoon: 324 nakens
But wait, ler me verify... Wait, but hold on, ler me think... Therefore, the minimal is 40.
me doublle-hhcc that... wait.... Altematively...
Problen-wohingg aneret 2: 204 nskens
Protlen-oofring atteggr 2: set tolmms
Wait...cur can we have lower number? So. day 3. 6, 9, Wais, but that's coniticting
But wait. there staris on day 2... But le me choek...
she walk fewer miles? she
my earlier thought.
Wais, now I'm confused. So, let's clunty, it's every third day--think>> Final Amswer: (40)
Hence, the answer is 36. <ithink>
ki2 1204
2228
2091
200
(a) Example Reasoning Chains
(b) Accuracy vs. Think Length.
Figure 1: Comparison between budget forcing and TuNKPRRUNE (a) Example Reasoning Chains: Under a 2000-token thinking budget, applying budget-forcing on the original model uses up all token budgets before identifying the mistake, leading to a wrong answer. In contrast, the model trained with THINKPRUNE solves the problem more efficiently, using fewer tokens and giving the correct answer. (b) Performance of the original LLM and the model after THINK training under different thinking token budgets.
However, this is apparently sub-optimal. When the budget is low, much more sensible solution is to remove the redundant thinking or unimportant steps (such as the problem paraphrasing), rather than maintaining the inefficient thinking and getting it killed. As a result, the performance drop as thinking length shrinks may have been significantly overestimated. So far, there have not been explorations that seek to fine-tune the model to adapt to smaller thinking budgets, and thus, we still do not have a good estimate of how much redundancy there is in the long thinking process that can be removed. Specifically, the following research questions remain unanswered:
Can we finc-tune an LLM with long CoT to prune its thinking length, while minimizing the performance drop?
What would be the length-performance tradeoff when the long CoT is pruned?
What happens to the reasoning chain when it gets pruned? What steps or words are most likely to be pruned?
In this paper, we seek to fill this gap. We propose THINKPRUNE, simple yet effective for LLMs with long CoT, which enforces maximum generation length during RL training. Specifically, given an LLM with long CoT, we perform contimous RL following the same scheme of DeepSeek-R1, except that we impose a strict token limit during training (e.g., 4,000 tokens for both reasoning and answer tokens). Any tokens beyond this limit are discarded before reward computation. This means that even if the model generates a correct answer beyond the allowed length, still receives a reward of because the answer is clipped and cannot be extracted. The task performance can be further preserved with an itcrative pruning strategy, where multiple rounds of RL are conducted with increasingly more stringent length limits.
strategy
Extensive evaluation demonstrates strong trade-off between generation length and performance for THINKPRUNE. For example, THINKPRUNE reduces the average generation length from 10,355 to 3,574 tokens for the ReepSeea-RRI-Disiil--eeeer
model, while also improving average accuracy across four math benchmarks. Although there is a slight performance drop of 2% for the 1.SSSaaleA1155-PPerrre and QwQ-32B models, our method achieves a comparable reduction generation length from 5,914 3,370 tokens, and from 8,763 to 4494, respectively. Figure [b) (yellow lines) also shows the improved accuracy- length tradeoff. Further analysis shows that THINKPRUNE helps LLMs avoid unnecessary
2

---

Published in Transactions on Machine Learning Research (Dec/202)
reasoning steps while maintaining focus solving the question, shown in Figure [a) (right), where problem understanding drops to 166 tokens. These findings provide valuable insights into improving the inference-tine reasoning efficiency for long-COT LLMs.
2
Related Work
2.1
Reinforcement Learning for LLM Reasoning
Reinforcement learning (RI.) has shown strong potential in improving the reasoning abilities of L.MSs across various domains, such as math (DeepSeek-A] 2025) and coding (OpenAl 2025 Liu & Zhang 2025), and complicated browser surfing OpenAl (2024). The resulting long- COT LLMs such OpenAI-o3 (OpenAl
2024) and DeepSeek-R1 (DeepSeek-A) Al, 2025 significantly outperform short- LLMS and demonstrate that reinforcement learning with verifiable reward (RLVR) can encourage LLMs to develop deep thinking behaviors, such as broad exploration and feasibility checks (Gandhi et al.) 2025), without relying on complex reasoning data generation methods like Monte-Carlo Tree Search (Zelikman et al. 2024 Hosseini et al.) 2024). However, these behaviors often lead much longer reasoning tracos, sometimes several times longer than
those produced by short COT LLMs (Sui al. 2025 Chen al., 2025): creating an "overthinking" issue that largely increases inference costs (Kumar el al. 2025). Several recent. works have shown that this extended
reasoning often includes redundant unnecessary verification and reflection, even simple problems (Wang ct al. 2025 Ji et al. 2025). Our work follows the standard RL training pipcline without changing the reward function and shows that it is possible to retain strong reasoning performance while significantly reducing overthinking.
2.2 Efficient Long Chain-of-Thoughh LLM
Several works have aimed to improve token efficiency in long chain-of-thought (CoT) LLMs. For example, KimiTeam al. (2025), Chen et al. (2024), and (Shen et al. (2025) reduce the length of reasoning by adding
length penalty during RL training. Other works, such as Hao et al. (2024) and Geiping al. 2025 represent reasoning as an optimization over latent vectors instead of text tokens, which helps shorten the reasoning process. While these methods are effective when training long CoT LLMs from short COT ones, there have been few works on reducing the reasoning length for trained LLMs. The most relevant works to ours are test-time methods that shorten reasoning with carly exit strategics (Muennighoff al. 2025 Fu al. 2024 Zeng et al., 2025b). These methods add stopping tokens or limit the maximum number of generated tokens during testing. However, they often lead to significant performance drops, especially when early stopping occurs too in the reasoning process. In contrast, our work presents simple and effective method with further RL training to reduce reasoning length without sacrificing performance.
3 Method
3.1
Overview
Denote (-, as an LLM capable of performing long COT. Given a query q, a sample answer, along with a long reasoning chain, is sampled from the LLM's output distribution, i.e., Y Ma, (q). Our goal is fine-tune the LLM, such that its output length is reduced while the overall performance is desirably maintained.
THINK THINKPRUNE is a simple yet effective RL strategy to achieve the length reduction goal. In what follows, we will first introduce the RL framework of THINKPRUNE, and then introduce an iterative length reduction strategy to better maintain the performance.
3.2 Reinforcement Learning with Length Clipping
THINkPRUNE adopts similar RL scheme to DeepSeck-R1 (DeepSeck-Al 2025) while reducing the generation length. Specifically, we adopt the group relative policy optimization (GRPO) algorithm Shao et. al. (024
3

---

Published in Transactions on Machine Learning Research (Dec/2025)
The reward function is almost the same the DeepSeek-RI framework, except that length clipping is added. Formally, denote I, as the length limit. The reward function is defined as follows:
R(Y, qi L) =
{ 0
1
if an answer can be extracted from clip(Y, L) and is correct,
(1)
otherwisc;
where cYip(Y; L) represents clipping the output Y to length L. In other words, the only difference from the DeepSeek-R1 reward is that the model-generated output is clipped to L before the reward is evaluated. In this way, output with length above L would not be able to produce valid answer since it is cut off, thus receiving a zero reward. This clipping operation effectively encourages the model to produce answers below the length limit. Such reward design is very simple, so it does not involve any hyperparameter tuning or reward engincering, and can ensure training stability. Also, since it involves minimal tweaks to DeepSeok-RR -1'sss training strategy, the proposed solution helps to maintain the long reasoning capabilities inherent in the base model.
During training, we also append simple system prompt into each training example to explicitly tell the model the length limit, such as The oulpul of the assislant should be within L lokens, to explicitly tell the model the length limit. The full system prompt for each model is shown in Appendix A.1
3.3 Iterative Length Pruning Strategy
The succoss of the proposed algorithm relics on the choice of the length limit, L- set too stringently compared the original output length of the base model, then the task performance can be seriously compromised.
Drawing inspiration from the iterative pruming strategy for reducing model parameters (Han et al. 2015 we propose an iterative length pruning scheme. Denote L as the target length constraint, we introduce a length schedule, L > L2 > ... > L' At each iteration t, we reduce the length constraint to Lt, and further fine-tune the model 0:-1 from last iteration to Or, using the RL procedure described in Section 3.2 Such an iterative pruning strategy ensures that the LLM learns to recover the performance by gradually compacting its output reasoning chain.
A critical design choice of the iterative length pruning is the stopping criterion for each iteration of the RL training. In our implementation, we utilize AIME22 and ATMF23 as the dataset validation set to choose the best checkpoint for the next RL iteration. To better balance model performance and generation length reduction, we allow up to a relative 10% drop in pass@1 accuracy on AIME-22 and AIME-23 compared to the original model. Among the checkpoints that meet this criterion, we select. the one with the shortest average output length as the starting chockpoint.
4 Experiment
In this section, we conduct empirical evaluations to assess the effectiveness of our proposed method. We first present the experiment setup in Section 4.1 Then, we present the experiment results in Section 4.2
4.1 Experiment Setup
Backbone models. Representative open-soureed long reasoning LLMs include DoepSeck-R1 (DecpSock-A) AI, 2025) and QwQ (Qwen'Team) 2025), along with their distilled variants. In our experiments, we use three models from these families: Distill- Qweill-ween-.... 1.5B, DeepScaleR- 1etppScalee--.B-Peeeeeee Previer and QwQ-32B.
We group these models into two categorics based on the extent of their training: unsaturated and saturated models have been widely used, these models are directly
models. Specifically, while fine-tuned on the output of DeepSeek-BR -1 in a supervised fine-tuning manner instead of RL, which could limits their full potential. Previous work, such as Lso et al. (2025). has shown that further training these models with RL can improve their performance, especially on math benchmarks. Based on this, we treat Distill-Qwen-1..5 as unsaturated model and 1eepScaleeR-11B--PPeeee as its saturated version, since
4

---

Published in Transactions on Machine Learning Research (Dec/202)
it is further trained with RL. Similarly, QwQ-32B is also trained with RL and is considered saturated model. This selection of models, covering both unsaturated and saturated types, allows us to more thoroughly evaluate the effects of thinking length pruning.
Training datasets. Previous work (Ye et al. 2025) has shown that even small but high-quality training dataset can improve the LLM performance via. RL. Therefore, we utilize a small mumber of data for model training by only using the historical AIME and AMC math questions (AMC) 2025). We use the preprocessed data from Prime (Cui et al., 2025) and take the AIME-AMC subset for training. In total, the training dataset. consists of 2470 distinct training examples.
Comparisons. We compare with the original backbone models without pruming. Additionally, we include the budget-forcing method (Muennighoff et al., 2025) for length reduction, which enforces a maximum number of thinking tokens by appending the end-of-thinking token delimiter (detailed implementation is described in Appendix A.1 For our method, we report the following two variants:
One-shot length pruning: We set the maximum length to 4,000/3,,00/2,000 tokens respectively and directly perform RL to roduce the generation length of the LLM.
Iterative length pruning: Starting from a higher generation length budget, we perform multi-round RL training and gradually decrease the maximum longth after each itcration.
Implementation details. We use the Verl Sheng ct al. 2024) RL framework for nigh-performanc training.
All models are trained with a batch size of 128. The number of rollouts for each question is set to 16 following prior works (Zeng et al. 2025a Liu & Zhang 2025). We use the GROP algorithm (Shao et al. 2024). For both one-shot and iterative pruning, we employ the same checkpoint selection strategy as mentioned in Section 3.3 where the checkpoint with the shortest average output length while maintaining a relative 10% accuracy drop is selected.
Evaluation configurations. We follow prior works to include the following evaluation datasets: MATH- 500 (Lightman et al., 2003)), AIME24 (AMC) 2025). AMC23 (AMC, 2025). and OlympiadBench (He et al.) 2024]. We use the versions of these dataset hosted in the Qwen2.5-Mate GitHub repository for case of reference. Following DeepSeek RI (DeepSeek-A] 2025). we set the maximum generation length (including both the thinking tokens and answer tokens) to 32,768 tokens for all the models, far above the maximum token length during our training. For each testing question, we sample N outputs with a temperature of 0.6 and a top-p value of 0.95, and we report the average accuracy of these N outputs. The number of samplings varies depending on the model size and dataset size. Specifically, for the two 1.5B models, we use N = 64 for AIME24 and AMC23 and N 16 for MATH500 and OlympiadBench. This also refers to the evaluation pareeeparameters of DeepSeek RI, where the number of sampled responses are adjusted between and 64 depending on the test set size to balance the variance and evaluation cost. For QwQ-32B, we sample N 16 responses for each question, given its large size and high computational cost.
Since the accuracy evaluation requires complex evaluation of mathematical expressions, we adopt the math evaluator from Qwen-2.5-ma math (DeepSeek-A) Al 2025), which provides robust answer extraction and advanced expression comparison.
4.2 Main Experiment: Length Pruning
One-shot length pruning enables significant thinking length reduction. We first evaluate the effectiveness of one-shot pruning in Table (top 3 rows in each section), where the LLMs directly trained with length limit 2k/3k/4k. We highlight the following obscrvations. First, the simple strategy can effectively reduce the generation length with moderate negative effect on the performance. For the unsaturated model, Basteee--R--issti----eee---
5B, the length reduction rate can be up to 50% with one-shot length pruning. At the meantime, the average performance is well-maintained and even slightly improved. For saturated models like DeepScaleR-1. 1.B-P. and QvQ-32B, we observe a 40-50% reduction in token usage, with moderate performance degradation. This highlights the promising efficiency gains through length pruning,
5

---

Published in Transactions on Machine Learning Research (Dec/2025,
Table Performance visualization of TuNKKPRUN. The accuracy measured by sampling multiple responses from the LLMs and taking the average reduce variance.
Accuracy
Generation Length
MATII
Olympiad Bench
MATII 500
Olympiad Bench
AIME
AMC
Avg.
AIME
AMC
Avg.
DeepSeek Dist--istllllli Distilll-uee--11111
1..5
Original Model
82.9
29.4
70.3
44.7
56.8
5560
15484
10030
11526
10355
THINKPRUNED
83.8
29.0
73.6
46.5
58.2
2709
8301
4388
5529
5232
SHINKKRRUN--K
83.7
27.8
71.8
44.9
57.1
2557
7968
4096
5140
4940
82.9
27.0
72.2
45.6
56.9
2356
7574
3755
4913
4650
3k
83.9
26.9
71.4
46.0
57.1
2209
6389
3422
4229
4062
THINKPRUNE- + 3k
83.2
27.1
73.2
1938
5631
3039
3687
3574
DeepScaleR-1
5B- -B--rrviee
Original Model THINKPRUNE-4E
88.5
40.3
81.2
52.7
65.7
3084
9463
5202
5907
5914
87.1
37.2
80.2
51.4
64.0
2212
6366
3516
4055
4037
THINKPRUNE-BE
86.5
34.3
78.8
1991
5809
3122
3583
3626
THINKPRUNE22K
86.1
33.3
77.7
49.7
61.7
1880
5528
2961
3348
3429
3k
87.1
38.4
79.9
51.6
64.2
2086
5869
3278
3731
3741
THINKPRUNE-HO 3k
36.5
79.4
5301
3334
3370
QuQ-32B
Original Model
95.1
78.8
97.5
71.1
85.6
4289
13822
7442
9497
8763
THINK PRUNE-4K
94.0
76.3
95.8
68.2
83.5
2552
8787
4173
5687
5300
THINK PRNNE-K
75.0
95.8
2341
8308
5413
5001
PRHNNKPRUNB-2K THINKPRUNE-BE 3k
93.5
73.3
95.5
68.6
82.7
2133
8232
3770
5160
4824
94.0
73.8
96.1
67.6
82.9
2308
8176
3959
5301
4936
THINKPRUNE-BE 3k 2k
93.8
72.5
95.9
67.3
82.4
2162
7631
3441
4742
4494
especially when models are initially over-generating. Second, we observe a consistent tradeoff between length and performance for all models under one-shot pruning. As we lower the token limit from 4k to 2k, the number of tokens goes down and the accuracy drops slightly. This suggests that cutting more reasoning tokens aggressively may also limit the reasoning capabilitics of the LLMs.
Another interesting phenomenon is that though the model is trained with explicit length limit, it often goes beyond that limit at test time. Particularly, observe that although trained with explicit length limit, the model can still generate long responses when the problem becomes more difficult such as on the AIME dataset. This shows that our length pruning does not hurt the model's deep thinking behavior, and it maintains the ability to perform complex reasoning for difficult questions.
Iterative length pruning benefits performance preservation and length reduction. The second question we aim to explore is whether iterative length pruming can better main the original LLM performance with a docent thinking length pruning compared to onc-shot pruning. As shown in Table ] (last two rows in each section), we start with LLMs trained using a 4k length limit and then ileratively apply RL with reduced length limits from 3k 2k. Our findings are follows. First, for the two 1.5B models, iterative pruming leads to better performance with either shorter or similar response lengths compared to one-shot pruning outperforms the
under the same final length limit. For example, on DeepScaleR, one-shot 2k model by 1.5% in accuracy while using 59 fewer tokens average. Similarly, pruning from 4k to 3k leads to better results than directly pruning to 3k. Second, for the QxQ-32B model, performance drops slightly after length pruning, with an average decrease of 2.7%. Unlike the 1.5B models, iterative pruning does not help recover the lost performance. We believe this because struggles to maintain performance when the generation token budget is tight. We will discuss this in more detail the next paragraph.
Performance-length trade-off along the pruning process. We observe that the experiment results in Table indicates a positive correlation between the reasoning length and model performance on different. benchmarks. To better understand the trade-off between reasoning length and model performance, we visualize how these two change during the length pruning training process the ATMF24 dataset (see

---

Published in Transactions on Machine Learning Research (Dec/202)
DeepscaleR-1.58
QwQ-32B
77.5
Avg. Performance
Avg. Performance
75.0
38
72.5
70.0
36
67.5
65.0
CaleR RLAK CaleR RL3K
62.5
QwQ-32B RL4K
QwQ-328
32
Original Model
60.0
Original Model
6000
7000
8000
9000
6000
8000
10000
12000
14000
Avg. Output Length
Avg. Output Length
Figure 2: Performancc-length trade-off along the length pruning training process evaluated the AIME24 dataset. The generation length drops quickly in the early stages of training with minor performance drop.
Figure 2 During training, we evaluate the model every 20 steps and record both the performance and the average output length. We then identify the most effective checkpoints (the ones at the frontier of performance-length trade-off), defined as the checkpoints that achieve the best performance among all the checkpoints whose average output length is shorter than themselves. This gives us a clear picture the best performance the model can reach at each length range, helping us visualize the efficiency boundary of the pruning process.
We highlight the following observations: First, mild length pruning leads to strong efficiency gains with minimal performance drop. When reasoning length reduction is moderate, there exists favorable tradc-off: significant token reductions can be achieved with only minor drops in performance. For example, we can reduce the average generation length of QwQ-32B from 14K to 8K while maintaining its performance close to the original model. Second, for the QuQ-32B model, we observe a distinct critical threshold (marked by the vertical dotted line in Figure 2 Beyond this threshold, further reducing the reasoning length causes sharp and significant drop in performance. We also find enforcing 4k length limit. leads to even worse results than length limit on QwQ-32B. This drop partially explains why iterative pruming performs worse on QwQ-32B than smaller models like 1.5B. While optimizing hyperparameters and the training data for QwQ-32B may lead to better performance, we leave the exploration to future work due to the heavy computation cost.
Inference-time trade-off with budget- forcing. We further study how different long-
Table 2: Performance comparison between our method and the length-based rewards.
CoT LLMs perform when applying the budget- forcing method (Muennigholf et al. 2025) De- tailed budget-forcing prompt can be found in
Accuracy
Length
Kixn 1.5
TRINK
KIM--1.5
THINKPRUNE
Appendix A.I Figure 3 shows the trade-off between performance and output length on the Math-500 dataset (left) and the AIME-24 dataset (right), comparing models before and after applying THINKPRUNE with budget-forcing. significantly improves performance under limited thinking token budget. For example, THINKPRUNE- 4k-33k+2k consistently outperforms the original model when using the same number of thinking tokens. On Math-500, it reaches similar accuracy while using only about 50% of the original thinking tokens for Qwen1. 50-Distii---1111 This suggests that THINkPRuNE helps the model think more efficiently and make better use of limited token budget. Sccond, THINKPRUNt reduces more thinking tokens on casier Math-500 problems than on AIME-24 24. This suggests that redundancy in reasoning related to problem difficulty and that length pruning can adaptively remove unnecessary thinking for questions at different difficulties.
MATH550
83.0
83.2
2108
1938
AIME
27.1
27.1
6617
AMC
72.5
73.2
3519
3039
OlympiadBench
44.3
46.2
4354
3687
We highlight two key observations: First, THINKPRUNE
7

---

Published in Transactions on Machine Learning Research (Dec/2025)
Di MATH500
Qwen1.5B-Distil--R AIME
25
75
20
Accuracy
70-
15
Original model
Original model
PrintPruned- Thnncrunndo33324
ThinkPruned-o 3k
55
ThinkPruno4->
1000
1500
2000
3000
1000
2000
3000
4000
7000
Average thinking tokens
Average thinking tokens
DeepScaleR MATH500
DeepScaleR AIME
Original model
ThinkPruned==3k
30
TrinkPunnd-=33-222
Accuracy (80
(9)
25
75
20
70
Original model ThinkPrune4k
ThinkPrune4->> ThinkPrune4
10
500
750
1500
1750
2000
2250
2500
1000
2000
4000
5000
6000
Average thinking tokens
Average thinking tokers
Figure 3: Inference-tine thinking length vs. performance trade-off for different long-Col CoT LLMs with
THINKPRUNE after applying budget-forcing.
Comparison with length-based rewards. We also compare our method with existing approaches that incorporate length- length-based rewards to reduce reasoning length. Specifically, we adopt the length reward introducod Kimi-1.5 (KimíTeam et al., 2025),, which adds a length-based penalty to overlong responses. We compare this method with our iteralive pruning method, THINKPRUNB-I-- 3k- 2k. The backbone model is PhepSeek-R1-Distill--e 1... The data mixture and hyper-parametters used for the baseline training the same our method. The experiment results are shown in Table 2 Our method achieves both higher accuracy shorter output length compared to the Kimi-1.5-style length-based reward baseline. This demonstrates that our approach is not only simpler but also more effective. Additionally, we perform comparisons with more baselines on more backbone models in Appendix A.3 which consistently demonstrate that our method provides better length-accuracy trade-off compare to baseline methods.
Cross-dataset evaluation. Additionally, to further verify the generalization ability of out method beyond math reasoning tasks, we evaluate our method on GPQA, MML.U, and MML.-Pro,, which cover broader domains such as science and general knowledge. The backbone model is -en-Sock-R--DDssill---eerrr trained only on math question. The results are shown in Table
We highlight the following key observations. First, THINK domains when only trained exclusively on math reasoning data. On all benchmarks, there is only minimal performance change (within 1155% compared to the original models. Second, the reasoning length is significantly reduced across all benchmarks. For example, THINKPRUNE-4-- 3k- 2k shortens the reasoning chain by 55-65% (c.g., from 10,355 to 3,574 tokens on Math, and from 10,072 to 4,639 on GPQA), while maintaining nearly identical accuracy. These results further confirm the generalization ability of our method, which reduces redundancy while maintains the performance even unseen domains.
naturally generalize to non-math
8

---

Published in Transactions on Machine Learning Research (Dec/202)
Table The accuracy and generation length THINKPRUNE across different domains.
Accuracy MMLU
Generation Length
Math
GPQA
MMLU-Pro
Math
GPQA
MMLU
MMLU-P P-ro
Original Model
56.8
32.7
49.1
39.4
10355
10072
2816
5661
PRUNE-KK
58.2
34.5
48.8
39.9
5232
6465
1685
3515
THINK PRIEE- -K- 3k
57.1
35.6
48.8
39.4
4062
5349
1395
2982
THINK P PRUNE-AK- 3k 2k
57.4
31.2
48.3
39.7
3574
4639
1243
2687
23.9%
13.3%
6.3%
7.4%
6.9%
0%.9%
15.3%
10.1%
4%
22.2%
Understanding the Prablem
Computing Simplifying Expressions
Exploring Alternative Approaches
Reformulating the Problem Applying Khown Theorems/Properties
Substituting Known Values Or Results
Reassess and Verify Previous Reasoning
Breaking Down into SubProblems
Finalize and Present the Answer
Figure 5: Reasoning behavior change before (left) and after (right) length pruning.
4.3 Reasoning Behavior Analaysis
In this section, we study the reasoning behavior of long- LLMs after length pruning. We focus on answering the following two questions: How does the reasoning behavior of LLMs change, and what gets removed most after pruning? 2. How does the pruning affect the readability of model-generated reasoning?
Reasoning-related keyword frequency change. Figure g illustrates the frequency of specific reasoning-related keywords per re- sponse in the MATH500 dataset using the model, both before and after applying our pruning method.
Before Pruring
Count 1K Tokens
DoosSeetRR--Dsstil--eeee
1
Specifically, we count the number of occurrences of each keywords within the responsos of LLMs and then normalize the count by the number of tokens in the responses, which gives us the frequency of each word in 1000 tokens.
0.5
Walt
But walk
Held on
Attematively
Let me confimm
Double check
Remember
me compute
Therefore
The figure is divided into three sections. The left section contains phrases that signal hesitations or self-corrections, which we find undergo significant drop in frequency after pruning. This indicates that the model would hesitate less. The middle section contains phrases that signal self-verification which undergo slight drop in frequency after pruning. This indicates that the model would sometimes skip the self-verification steps, which may sacrifice a little bit of accuracy, to save the token length. Finally, the right section contains phrases that signal core computation and reasoning, which have slight increase in frequency despite the overall reduction length. This indicates that the model would maintain the reasoning process, because that the core process that contributes to the final solution.
Figure 4: Reasoning- rellatd related keyword frequency change before and after length pruning.

---

Published in Transactions on Machine Learning Research (Dec/2025)
Reasoning behavior change. We further analyze the change in reasoning behaviors that the model utilizes to solve a problem. Specifically, we first prompt GPT--4 to summarize 9 frequent problem- solving phases, such as 'understanding the problem' and 'applying known theorems// properties (complete list. A Appendix A.2). Then; for each model-generated solution, we prompt GPT-4o split the output into chunks and label each chunk with one of these phases. We measure the number of reasoning steps in each phase by counting segments separated by double newlines (*\n \n"). Figure 5 shows the distribution of these reasoning steps before and after pruning. As can be observed, the model would spend less time on relatively redundant steps, such as 'finalize and present the answer', 'reassess solution', and "explore alternative approaches'. Mcanwhile, we observe an increase in the percentage of core problern-solving steps, including 'applying known theorems/properties and 'computing simplifying expressions' These observations further confirm that the model would focus more on the core problem-solving steps and save on peripheral steps.
Table 4: Reasoning trace perplexity on different modols.
PPL
Avg. Ace
DeepSeek-RAe 5B--..i--
Original Model THINKPRUNE 1.90
1.91
82.9
83.2
DeepScaleR- 1Bw.eeR--..-------------- Prevrvv
Original Model THINKPRUNE
1.95
88.5
2.02
86.9
QwQ-32B
Original Model THINKPRUNE
2.37
95.1
2.24
93.8
Readability evaluation. One common concern in RLVR is that the heavy RL training may reduce the readability of the model's reasoning, resulting in mixed language or non-readable reasoning trace as shown in R1-Zero model (DeepSeek-Al 2025). To examine whether THINKPRUNE would also introduce similar issues, we measure the perplexity of generated reasoning traces on the Math-500 dataset for original model and THINKPRUNE-4K- 3k-> 2k using Qwen2.5-Math-
As shown in Table 4 our pruming method does not significantly affect reasoning roadability the perplexity remains nearly identical to the original model. Figure K in Appendix A.4 shows an example reasoning trace from the Distill-R1-1.5B LLM before and after pruning. As shown in the example, the original LLM repeatedly checks its previous reasoning multiple times, lcading to heavy, redundant reasoning steps. On the contrary, the reasoning trace remains fully readable and focused on solving the problem after pruning, with more efficient problem-solving steps and only one self-verification step.
5 Conclusion
In this paper, we proposed THINKPrUNE reduce the reasoning length of long CoT LLMs. THINKPRUNE introduces length constraint during RL training, which discards unfinished thoughts and answers when sampling responses. To maintain model performance, we apply iterative pruning strategy that gradually tightens the length limit over multiple rounds training. Experiments show that THINK reduces the reasoning length and achieves strong performance-lenglh trade-off. Further analysis shows that THINK PRNNKRRUNK effectively removes redundant steps while preserving key reasoning processes.
6
Acknowledgement
The work of Bairu Hou, Jiabao Ji, Yujian Liu, and Shiyu Chang was partially supported by National Science Foundation (NSF) Grant ITS-2338252, NSF Grant ITS-2207052, NSF Grant ITS-2302730, and the Open Philanthropy Research Award. Jacob Andreas was supported by NSF grants 11S-2212310 and 11S-2331117. This work additionally supported by the MIT-IBB -BM Watson AI Lab.
References
AMC. American invitational mathematics examination. http::///arrrppooooaaaooo.aa//ia hhtp/AAnerica__Ivviaaaa www.heattiss.Eaaaaae...e.aaaaaaa Exxaminaaiiin..... 2025.
AMC. American mathematics competitions (ame). http://tt/////aro//ttttttttttota 2025.
10

---

Published in Transactions on Machine Learning Research (Dec/202)
Daman Arora and Andrea Zanette. Training language models to reason efficiently. arXiv preprint or (xXx:2502.04403 2025.
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wangxiang Che. Towards reasoning era: A survey of long chain-of- thought for reasoning large language models. aiv preprint or N5N::5503.09500 2025.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like Ilms, arXiv preprinl ar 2xXic:2412.21166
2024.
Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang Hc, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502. 01456, 2025.
DeepSeek-AL. Deepseek-r1: Incentivizing reasoning capability in 1lms via reinforcement learning. arxiv preprint Xrv;v:
2025.
Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Aurick Qiao, and Hao Zhang. Efficiently serving Ilm reasoning programs with certaindex. arXiv preprint arxi:: 20992, 2024.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self- ialf-inproving reasoners, or, four habits of highly effective stars. ar preprint arXiv:2503.01307, 2025.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noachh, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Fric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 07 2024. URL https:://onodd.orggg//ooooo org///ccord///22060022222
Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. Xix preprint orXiv: 2502.05177 2025.
Song Han, Jeff Pool, John Tran, and William J. Dally. Learning both weights and connections for efficient. neural networks, 2015.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston and Yuandong Tian. Training large language models to reason in continuous latent space. arxiv preprint arXiv: 2412.06769, 2024.
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: A challenging benchmark for promoting agi with Alympiad-level bilingual multimodal scientific problems. Annual Meeting of the Association for Computational Linguistics, 2024. doi: 10.48550/ /ar// 2402...4000.14000 14008.
Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. V-star: Training verifiers for self- self-taught reasoners. arXiv preprint ar 2r 2024.
Ke Ji, Jiahao Xu, Tian Liang, Qiuzhi Liu, Zhiwei He, Xingyu Chen, Xiaoyuan Liu, Zhijie Wang, Junying Chen, Benyou Wang, et al. The first few tokens are all you need: An efficient and effective unsupervised prefix fine-tuning method for reasoning models. arXiv preprint arXiv:2503.0288 2025.
KimiTTem, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llss. ar preprini arXiv:2501. 12599, 2025.
Abhinav Kumar, Jacchul Roh, Ali Nasch, Marzena Karpinska, Mohit Iyyer, Amir Houmansadr, and Eugene Bagdasarian. Overthink: Slowdown attacks on reasoning 1ms. Xiv e-prints, pp. arXiv 22002, 2025.
11

---

Published in Transactions on Machine Learning Research (Dec/202))
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Nla Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint ar arXhv:2305.20050 2023.
Jiawei Liu and Lingming Zhang. Code-rl: Reproducing r1 for code with reliable rewards. 2025.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, Raluca Ada Popa, and lon Stoica. Deepscaler: Surpassing ol-preview with 1.5b model by scaling rl. http:://ppeett-raddooovooooooooo
2025. Notion Blog.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.1930) 2025.
OpenAL. Openai ol system card. arXiv preprint arXiv: 2412. 2024.
OpenAL. Introducing operator, 2024. /it.p//p//oooonn..c//////////aer Accossed: 2025-03-23.
OpenAL. Competitive programming with large reasoning models. arXiv preprint arXiv: 3502,06807, 2025.
QwenTeem. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlnn
holblog//./oo//////o haabbb////ooog/gg---// (atpllogg//q-300000 qwq-32b/
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv: 2402. 03300, 2024.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of- thought into continuous space via self-distillation. ar preprint arXiv: 3502.2107, 2025.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: ficxible and efficient rlhf framework. arXiv preprint arXiv: 2405.19256, 2024.
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503. 16419, 2025.
Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuoshong Zhang, et al. Thoughts are all over the place: On the underthinking of ol-like 1lss. arXiv preprint arXiv:2501.1858 2025.
Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv: 2502.03387, 2025.
Eric Zelikman, Georges Raif Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah Goodman. Quiet-star: Language models can teach themselves to think before speaking. In First Conference on Language Modeling, 2024.
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. Xivi preprint arXiv: 1589,, 2025a.
Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Yunhua Zhou, and Xipeng Qiu. Revisiting the test-time scaling of ol-like models: Do they truly possess test-time scaling capabilities? aixiv preprint or Xin: 12215, 2025b.
12

---

Published in Transactions on Machine Learning Research (Dec/2025)
A Appendix
A.1
Implementation Details
System prompt used for training. The system prompt for DeepSeek-RL-Distill-. -wee-l-.--ee---..B and Deep-
ScaleR during training is shown in Table 5 align with the original RL training of DeepSeek-RL. For QwQ-32B, we use much similar prompt, "you are helpful assistant. Your output should be within (N} tokens."
Table 5: Template for DeepSeek-RI-Distill limit for training (e.g., 2000 and 4000).
and DeepScaleR. (N} will be replaced with the length
A conversation between User and Assistant. The user asks question, and the Assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with the answer. The reasoning process and answer are enclosed within <think> </think> and <answer>> </answer> tags, respectively, i.c., <think> reasoning process here </think> <answer>> answer here </answer>. The output of the assistant should be within {N} tokens.
Implementation of budget forcing. We follow the official implementation of budget forcing in S1 Muen- nigholl el al. (2025) and made minor changes. Since the original implementation coupling with the in-harness-eval framework (Gao et al. 2024). we revise the code to remove such dependency. Also, stop the thinking process of the LLM, we append
Answer***(n (n)""" to the generation of the LLM instead of "< jim start| >answer (annsserrnFinaa (nFinal Answer:" used by the S1 model. This is because we empirically find the Dispseee-RR--issiii------s 1...-... model typically summarize its final answer starting with "** #**linal Answer:**nniy Finally, we sample multiple responses to reduce the variance in model performance instead of using greedy decoding as the original implementation.
A.2 Analyze the Reasoning Behavior Change
We use GPT-4o to analyze the reasoning behavior of LLMss by segmenting their long-form solutions into high- level problem-solving phases. The prompt used for this task is shown in Figure 7 Since the model-generated solutions are very long, we only require GPT-4o to output of the first step and last step in each chunk represent that chunk. Since the model-generated solutions are often very long, we ask GPT-4o return only the first and last reasoning steps of each phase, which serve as markers to define the boundarics of each chunk.
To align these chunks with the original model output, we use string matching to locate the start and end positions of cach chunk in the raw text. Within each matched chunk, we estimate the number of reasoning steps by counting the number of "In \nn delimiters. This gives us a step-level breakdown of how much reasoning is devoted to each phase.
A.3 Additional Comparison with Length-based Rewards
In this section, we present additional comparisons with length-based rewards. Besides length reward introduced in Kimi-1.5 (KimiTeam et al., 2025), we also include EPFICIeNTREASON (Arora & Zanette 2025) for comparison, which reduce the length via a group-relative length penalty with tunable coefficient to control the strength of the penalty. We further exlended our evaluation to larger backbone model, ReepSeek-R1-Distilll Dia--iiitiil-Qwe--TBB to provide more comprehensive evaluation. The training recipe and dataset for the 7B model are exactly the same as ReepSeckkRR--Distiill--aa.
The experiment results are in Table We group models with similar accuracy to better demonstrate the accuracy-length tradeoff. Please note that for the section, we evaluate the model accuracy with Math-verify which achieves more robust
13

---

Published in Transactions on Machine Learning Research (Dec/2025)
Table 6: Performance comparison between TuNKPRRUN and length- based rewards.
Accuracy
Generation Length
MATH
Olympiad Bench
MATH
Olympiad Bench
AIME
AMC
Avg.
AIME
AMC
Avg.
500
500
DeepSeek Backk-R1-- -Dsttil Mhtatill-uue--- 1..m-11.58 5B
Original Model
85.1
28.1
73.2
52.1
59.6
5542
16553
9507
11535
10784
APFICIENTREASON a 0.1
84.7
32.1
72.7
52.4
60.5
3178
11135
5651
7719
6920
THINKPRUNE-4K
83.8
29.0
73.6
46.5
58.2
2709
8301
4388
5529
5232
Kimi-1.5
83.0
27.1
72.5
44.3
56.7
2108
6617
3519
4354
4150
EPFICIENTREASON 0.1
82.9
27.8
70.5
50.3
57.9
2281
9525
4218
5689
5428
THINKPRUNE 3k- 2k
83.2
27.1
73.2
46.2
57.4
1938
5631
3039
3087
3574
DeepSeek-RR -Distill Beasttll--wee---- Qwen-7B
Original Model
93.6
54.9
89.6
67.8
76.5
4008
13222
6623
8871
8181
Kimi-1.5
92.8
52.9
88.9
65.8
75.1
2637
9633
1204
6127
5650
EPFICIENTREASON a 0.1
93.1
52.3
88.4
66.4
75.1
2839
10660
4706
6890
6274
EPFICIENTREASON 0.4
93.1
54.1
89.3
66.8
75.8
2558
10106
4260
6482
5852
THINKPRUNE-41
52.9
67.0
2929
9824
4504
6229
5872
THINKPRUNE-HH 3k- 2k
93.3
31.7
89.1
66.1
75.1
2464
8008
3734
5123
4832
A regular hexagon be divided into six equilateral triangles.
If the perimeter of the triangles inches, what the perimeter, inches, of the regular hexagon?
Original Model
ThinkPrune
Problem understanding: 64 tokens Okay, so have this problem here: Let me try to figure this step by step.
Problem understanding: 61 tokens Okay, so have this problem here: let try to figure this out step by step
Problem-sulring attempt-1:
Prohlem-solviing attempt-1: 230 tokons
First, let me visualize what's going
First, know a regular hexgon...
Self-seriffcation-I:: 100 tokens
Given that the perimeter....
So, it?
double-check... ...
Now a regular hexagon six sides. find
Since each side
Self-nerrificttion- 103 tokeus
Perimeter hexagon
inches.
Let me confirm with another approach
Seffveeiffcuttio: tokens
Self-veriffcntion-:: 196 tokens
Wait, another thought:
Wait, that straightforward. Let me just double- check Yep, seems correct.
Self-veriffcation--:: 112
Altematively, if think about the height the triangle
Figure 6: Example reasoning trace of Distill-R1-1.5B before and after length pruning on Math-500 question.
judgment compared to other evaluators. We did not update the numbers in the main paper to maintain consistency with the previous versions of this paper.
We highlight that our method provides better accuracy-length trade-off. Compared to both two baselines, our method achieves either higher or comparable accuracy. At the same time, the average response length of our method 30% shorter than the best baseline on RL-Distiil---en---------------- and 15% shorter on DeepSeck-R1-Diis R1--Dsttll- Dal-Distll-Qween-7E
A.4 Additional Examples
In this section, we include additional reasoning trace examples of Distill-R1-15B LLM before and after applying THINKPRUNR on Math-500 questions. As shown in Figure t and Figure the original model repeatedly checks its previous reasoning for these simple math questions, leading many redundant self- reflection steps. On the contrary, our method successfully removes these repeated steps, and helps the LLM focus the problem solving, while keeping perfect readability.
14

---

Published in Transactions on Machine Learning Research (ecc/2225)
Find all values of Sx$ that satisfy the equation Sx |lsqrt|1122x] 4S.
Original Model
ThinkPrune
Problem understanding: 98 tokens
Problem understending: 100 tokens Okay, so need to solve the equation ({ x lsqrt(11 - 2x) + Hmm, let me think about ...
equation solve: So, I'll take step time...
Alright,
Problem-solving uttemptl:: 455 tokens
Problem-sohving attempt-I: I: 418 tokens
First, let's write down the equation again
So, let's start by isolating the square term... Now, the left side x !)...
Self-verification-1 244 tokens
Wait, I have two solutions: and
Simplify the terms:
Combine like terms: Therefore, the solutions are
Self-verificution-22 417
Wait, but just make sure, let me check there are any
Self-verification:
Self-verificattonn: 3: 124 tokens
Wait, just to visualize, when x
Thus, the only solution is x 5 y. Let me verify this by plugging back into the original equation think that's only solution. the answer (( 5
Self-verification-- 4: 431 tokens
Wait, actually, at 5.5, the square root
Figure 7: Example reasoning trace of Distill-RI-1.5B before and after length pruning on a Math-500 question.
15

---

Published in Transactions on Machine Learning Research (Dec/2025)
Table Prompt for the analysis of the reasoning behavior.
### **Taak Description
Given mathematical question and its detailed solution, the task is to segment the solution into high-level problem-solving phases. The goal is to group consecutive steps into mesningful phases and output only the start and end sleps of each phase.
Note: Each reasoning step*s in the solution is separated by doubbe line break ("uu)n")**.
# ## *Reqqirements**
1. "Segment the solution into distinct problem-solving phases** based on logical progression.
2. #*ach phase should have start and an end step**.
3. **A phase can appear multiple times** in differeut parts of the solution.
4. **The order of phases is ftexible** they appear any logical sequence depending the nature of the solution.
5. **Only first and last steps of each phase should output"*, reducing redundancy.
# ## *evg Problem-Solving Phases**
Each step in the solution should belong to one of the following **ten high-level phases"*
1. "Understanding the Problem ** Identifying given data, definitions, and the goal.
2. ** Reformmlating the *roblem** Changing variables, rewriting expressions, restructuring sums.
3. Applying Known Theorems/ Saporrms/PPropeee *Properties*: Using standard formulas, identities, or mathematical principles.
4. "Breaking Down into Subproblems** Decomposing the problem into manageable components.
5. "Computnng or Simplifying Expressious** Performing algebraic manipulation numerical evaluntion.
6. SSubstttuing Known Values Results**; Using precomputed values standard mathematical constants.
7. "eeassess and Verify Local Steps**: Checking errors or incousistencies within a small part solution.
8. Reessess the Whole Solution**; **; Reviewing the entire solution for logical norrectesss and consistency.
9. "Exploring Alternative Approaches*# Considering different methods to solve the problem.
10. **Finalize and Present the Answer*s, Writing the final result and ensuring clarity.
# # **Output #ormat**
The output should consist of *'multiple phases** each represented in the following format:
Phase }: (Phase Name)
Start]:: (Text of first step in the phase)
[End]: (Text of last step in the phase]
Where:
**Phase X** represents the index of the phase (e.g., Phase 1, Phase 2, etc.). the ten high-level categories. **tart*** is the first step **nd** is the last step of the phase.
**haase Name** *
phase.
# **Exmple Output**
444
[Phase 1]: Understanding the Prohlem
[Start]: [Txx of step I]
[End': [Txx of step 3]
Phase 2):
Reformulating the Problem
Start]: Text of step 4
[Edd: [Txt of step 5)
Phase 4]: Finalize and Present the Answer
[ttrtt]: Text of step X]
Emd' [ext of step Y
16


Reviews:
[{"review_id": "VhbbCCFY3B", "reviewer": "Reviewer_aPeP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors propose ThinkPrune, a reinforcement learning (RL) framework designed to regularize the output length of long chain-of-thought (CoT) reasoning. Specifically, they introduce a strict budget constraint in the reward function, where reasoning outputs that exceed the budget receive a reward of zero. Through extensive experiments, the authors demonstrate that ThinkPrune achieves a better budget–accuracy trade-off than existing budget-forcing methods on the original model. They also present a detailed analysis of the resulting reasoning behavior.", "claims_and_evidence_yn": "Yes", "claims_explanation": "While I am leaning towards the answer 'yes' to the above questions, I have some remaining concerns. I will present my view by suggesting the strengths and weaknesses of the paper:\n\n**Strengths**\n1. The paper is clearly written and easy to follow.\n2. The approach is evaluated across three different models, supporting the generalizability and robustness of the contribution.\n3. The authors conduct thorough analyses of reasoning behavior, including keyword frequency, behavioral changes, and perplexity analysis.\n\n**Weaknesses**\n1. The main concern lies in the choice of the primary baseline. As noted by the authors, an existing RL-based length regularization method (the length-based reward in Kimi-1.5) already addresses a similar goal. Thus, the significance of ThinkPrune primarily stems from its simpler reward formulation. While Table 2 includes some comparison, it is limited to the 1.5B model. A stronger justification is needed for why the baseline comparison excludes the existing RL-based method and instead focuses on the original model, throughout the paper.\n2. In Section 4.2, the authors mention that ThinkPrune still produces long CoTs for difficult problems, but do not discuss how this affects performance. Could the reward design potentially harm performance on harder samples that require longer reasoning chains, since such cases may receive limited learning signals?\n\n3. The framing in Section 2.2 is somewhat unclear. The authors state that KimiTeam et al. (2025), Chen et al. (2024), and Shen et al. (2025) propose methods used during the RL training stage, while ThinkPrune is positioned as a method for ‘further’ RL training. It would be helpful to clarify what distinguishes ThinkPrune from these prior RL-stage methods, or why it is particularly suited for a secondary RL phase.", "audience_yn": "Yes", "audience_explanation": "Reducing the length of chain-of-thought reasoning is an important and actively studied problem. The authors present a simple yet effective method to address this issue, and I believe the findings will be of interest to at least some readers in the community.", "requested_changes": "I have listed all suggested changes in the Weakness section. Weakness 1 is critical to securing my recommendation, while others would strengthen the work in my view.", "broader_impact_concerns": "I have no concerns about the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TqK0uyTcct", "reviewer": "Reviewer_PkdD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents ThinkPrune, and RL based method to prune the CoT for LLM reasoning. Experimental results suggest that the proposed method is able to significantly reduce the thinking length of different LLMs on common math reasoning benchmarks without drastic sacrifice of the model accuracy.", "claims_and_evidence_yn": "No", "claims_explanation": "- Details of 4k->3k and 4k->3k->2k is missing. Do you use the same amount of iterations for the two experiments or same amount of iterations for each length (e.g. 1000 iterations for 4k and another 1000 for 3k (a total of 2000 iterations) vs 1000 iterations for 4/3/2k (a total of 3000 iterations)).\n- Figure 2, RL4K and RL3K definition is missing.\n- \"This suggests that redundancy in reasoning is related to problem difficulty and that length pruning can adaptively remove unnecessary thinking for questions at different difficulties.\" -> How do the authors defined unnecessary thinking?\n- Missing explanation of the performance drop at average thinking token 1000 in Figure 3 DeepScaleR AIME", "audience_yn": "Yes", "audience_explanation": "Reducing thinking length is an interesting research topic for LLM reasoning and will be interested to TMLR's audience.", "requested_changes": "- Presentation of this paper needs to be significantly improved. \n  - Please add the GRPO objective for readers.\n  - Table 1 is very crowded. To me there are three comparisons in Table 1: 1. original & one shot (row 1/2/3/4), 2. original & iterative (row 1/5/6), 3. one shot vs iterative at their corresponding lengths (3 vs 5, 4 vs 6). I would suggest color code the Table with two different colors (one shot and iterative) and color code the text. So that it is clearer for the reader which comparison you are focusing on at each paragraph.\n  - Please highlight the takeaways from the experiments more clearly. \n- It is unclear to me the goal of the paper. I think it's a cool technique to reduce the thinking lengths while maintaining the accuracy but the desired outcome of such reduced budget is not discussed by the authors. For example, it's unclear whether reducing the length provides us more budget that can correctly answer questions that we cannot previously? It's unclear whether the ability to reduce length relates to the difficulty of the problem? I think these are critical parts that needs to be shown experimentally or at least discussed in the paper.\n- Nit: p5 GROP -> GRPO", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6u10IEMtjC", "reviewer": "Reviewer_CHmk", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "ThinkPrune adds a hard length cap during RL (GRPO) and iteratively tightens it (e.g., 4k→3k→2k) so the model learns to prune redundant reasoning while preserving accuracy; shows shorter chains at similar performance on math benchmarks.\n\n**Strengths**\n\nThe paper is well written and easy to follow. The proposed idea is simple yet effective, and can be directly applied to existing reasoning models to improve efficiency.\n\n**Weakness**\n\n**[W1] Insufficient baselines**\n\nOver-thinking mitigation is already well studied, particularly in R1-distilled reasoning models. To make the contribution convincing, the paper should compare against stronger or closer baselines such as O1-Pruner [1] and efficient length-aware training methods [2]. Without these, it is difficult to disentangle improvements from the method itself versus the weakness of the selected baselines.\n\n[1] O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning, 2025\n\n[2] Training Large Language Models to Reason Efficiently, 2025\n\n**[W2] Setup concerns**\n\nTraining uses AIME/AMC subsets, while evaluation includes AIME24/AMC23, which are likely from a similar distribution. Please provide cross-dataset evaluation (e.g., train on AIME/AMC) and consider a different reasoning domain (e.g., GPQA) to confirm generalization.\n\n**[W3] Implementation details to clarify**\n\nThe paper lacks key implementation details (e.g., training steps per RL rounds)\nAre these hyper-parameters tuned separately per model (e.g., for Qwen-1.5B vs other backbones)?\nIf so, please specify the tuning ranges and selection criteria to ensure reproducibility.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Overall, the claims are well supported through a diverse set of experiments and models. However, the setup-related weakness (i.e., [W2]) requires further clarification to fully validate the conclusions.", "audience_yn": "Yes", "audience_explanation": "Potentially yes, but it is somewhat unclear. The paper needs to better justify its advantages over existing methods in order to clearly establish its contribution and relevance for the audience.", "requested_changes": "See Weakness.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)