Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL
Authors: Songjun Tu
Jiahao Lin
Qichao Zhang
Xiangyu Tian
Linjing Li
Xiangyuan Lan
Dongbin Zhao
Venue: NeurIPS 2025 poster ()
Keywords: LLM Reasoning; Adaptive Thinking; Reinforcement Learning
Primary area: applications

Abstract: Large reasoning models (LRMs) are proficient at generating explicit, step-by-step reasoning sequences before producing final answers. However, such detailed reasoning can introduce substantial computational overhead and latency, particularly for simple problems. To address this over-thinking problem, we explore how to equip LRMs with adaptive thinking capabilities—enabling them to dynamically decide whether or not to engage in explicit reasoning based on problem complexity.
Building on R1-style distilled models, we observe that inserting a simple ellipsis ("...") into the prompt can stochastically trigger either a thinking or no-thinking mode, revealing a latent controllability in the reasoning behavior. Leveraging this property, we propose AutoThink, a multi-stage reinforcement learning (RL) framework that progressively optimizes reasoning policies via stage-wise reward shaping.
AutoThink learns to invoke explicit reasoning only when necessary, while defaulting to succinct responses for simpler tasks.
Experiments on five mainstream mathematical benchmarks demonstrate that AutoThink achieves favorable accuracy–efficiency trade-offs compared to recent prompting and RL-based pruning methods. It can be seamlessly integrated into any R1-style model, including both distilled and further fine-tuned variants. Notably, AutoThink improves relative accuracy by 6.4\% while reducing token usage by 52\% on DeepSeek-R1-Distill-Qwen-1.5B, establishing a scalable and adaptive reasoning paradigm for LRMs.

Project Page: https://github.com/ScienceOne-AI/AutoThink.

Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL
Songjun Tu Jiahao Lin 1.3, Qichao Zhang 1.3, Xiangyu Tian 1.3
Linjing Lita3 Xiangyuan Lan?, Dongbin Zhao 1,2,3
Institute of Automation, Chinese Academy of Sciences "Pengcheng Laboratory Sccooo of Artificial Intelligence, University of Chinese Academy of Sciences (tusongjun20233 zhangqi chao2014 4 ac.cn
Abstract
Large reasoning models (LRMs) are proficient at generating explicit, step-by-step reasoning sequences before producing final answers. However, such detailed rea- soning can introduce substantial computational overhead and latency, particularly for simple problems. To address this overthinking problem, explore how equip LRMs with adaptive thinking capabilities, enabling them to dynamically decide whether to engage in explicit reasoning based on problem complexity. Build- ing on R1 R1-style distilled models, we observe that inserting a simple ellipsis ("...") into the prompt can stochastically trigger either a thinking no-thinking mode, revealing a latent controllability in the reasoning behavior. Leveraging this property, we propose Think, multi-stage reinforcement learning (RL) framework that progressively optimizes reasoning policies via stage-wise reward shaping. Auto- Think learns to invoke explicit reasoning only when necessary, while defaulting succinct responses for simpler tasks. Experiments on five mainstream mathematical benchmarks demonstrate that AutoThink achieves favorable accuracy-cfficiengg trade-offs compared to recent prompting and RL-based pruning methods. It can be seamlessly integrated into any RI-style model, including both distilled and further fine-tuned variants. Notably, AutoThhin Think improves relative accuracy by 6.4% while reducing token usage by 52% on scalable and adaptive reasoning paradigm for LRMs.
Baaeckk-R-iistil---eeeeee 1.5B, establishing a
Project Page: http:///tthabbbooommceeeeeee//nt
Introduction
Recently, reasoning-focused Large Language Models (LLMs), also referred to as Large Reasoning Models (LRMs) [41], have demonstrated remarkable progress solving complex reasoning tasks. Particularly, DeepSeck-RI [9] uses only outcome-baat based feedback and incentivizes explicit reasoning capabilitics through reinforcement learning (RL) with verifiable rewards. DeepSeek-RR and its distilled models typically follow the <think> and <answer> format, where the <think> process generates explicit, step-by-step reasoning sequences to support obtaining a final answer during the <answer> phase. We refer to models that follow this Chain of Thought (CoT) [37] prompting scheme as R/-syyle models. The explicit thinking process, which enables self-reflection, backtracking, and validation, is widely regarded essential for enhancing reasoning accuracy. Arising from this understanding, a popular paradigm has emerged that improves solution quality by increasing thinking token allocation during inference-time reasoning [17, 40). However, this paradigm introduces major bottleneck: excessive thinking token generation leads to high computational cost and latency, raising the overthinking phenomenon, where many reasoning steps are redundant or inefficient [29, 14].
Corresponding author t Project leader
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Manual Hybrid Reasoning
ThtoThink (Ours)
Stage
Stage
Ellipsis Prompt
Stabilize Duai Modes
Reinforce Correct Behavior
Think
For a frainiy boton:
-thiroo Wost - cftrindo on 1
stiito OK Lets <itiieao
OFF
No- Think
<thrko OK Let's c/thinio
sthinto withirto (
Random Choose
attirt
ctiinl 1/thinto
Multi Stage
<thrroo Tisttakk s/thhrroo
Stage 3
Concise Reasoning
RL
K Prune Redundant Reasoning
Long Response
thiroo (tthiio
pimpling gross
<thinlo 6666 tokens 1/thinlo
ON
Hard
-thrle -. Maybe D--. </thiroo
striitoo of (-tranko 100 takens 1/thirko cttunn 999 tolons chthinis strinto 99 tollens vrttarto
Short Response:
OFF
For Easy
thiino 66 tokens </thinlo
Adoptive Choose
Figure 1: Overview of AutoThink Compared to Prior Reasoning Paradigms.
To mitigate overthinking, recent efforts have explored hybrid reasoning and concise reasoning strategies. In the industry, Claude 3.7 Sonnet [2] introduces a controllable reasoning framework that allows the model to switch between standard and extended reasoning modes. Similarly, Qwen3 [30] proposes a thinking control scheme with a "thinking" mode (slow thinking) and "non-thinking" mode (fast thinking), and provides users with the flexibility to choose whether the model should engage in reasoning not. In the academic community, parallel research has focused on designing prompt- guided efficient reasoning [42, 24] or training pruning-based models to achieve concise reasoning [12, 6, 44]. While promising, these approaches either rely on manually predefined modes or uniformly prune reasoning steps, which may degrade performance on harder instances. A fundamental question then arises to address the overthinking issue:
Can LLMs learn adaptively determine thinking fast or slow based on given problems?
To answer this question, we propose AutoThink, a multi-stage RL framework that enables R1- style LLMs to learn adaptive reasoning behaviors. Unlike prior approaches reliant on hard-coded prompting or external control signals, AutoThink formulates reasoning as a learned dual-mode policy that determines both whether to engage the model's "thinking" process and how to generate concise reasoning. As illustrated in Figure I, AutoThink fundamentally differs from manual hybrid prompting and uniform pruning strategies by employing an ellipsis prompt and structured three-stage RL. training process that enables adaptive reasoning to emerge. In detail, an ellipsis prompt acts as a controllable entry point for optional reasoning, triggering stochastic switching between thinking and no-thinking modes in RI R1-style LLMs. Then, the proposed multi-stage RL framework shapes this behavior progressively: Stage 1 stabilizes dual-mode coexistence, Stage 2 reinforces accurate reasoning to enhance solution quality, and Stage 3 prunes redundancy via length-aware rewards. This progression enables the model to allocate reasoning effort adaptively, achieving both accuracy and efficiency. The main contributions are as follows:
We identify the ellipsis prompt, lightweight prompting scheme that activates a stochastic switch- ing behavior in RI-style LLMs between thinking and no-thinking modes.
We propose a multi-stage RL framework that trains R 1sstyle L.MMs to dynamically modulate their reasoning behaviors according to problem complexity.
Experiments on mathematical benchmarks show that AutoThink achieves accuracy-effccennc trade-offs better than existing pruning and compression methods, without sacrificing performance.
2
An Ellipsis Unlocks Random Thinking in R1-Style Models
2.1
A Surprising Effect of Minimal Prompt Modification
Recent efforts on concise reasoning aim to eliminate unnecessary thought, either via prompting that explicitly bypasses thinking [19], or RL-based training that penalizes long outputs [6]. While effective shortening responses, these methods enforce uniform brevity regardless of problem complexity. Rather than compressing by default, we pose subtler question:
Can small change, perhaps a few tokens, lead RI-style models decide whether to think?

---

Standard Prompt (Alwwys Thinking):
Accurecy under Different Prompts
Distil-RI (Before Thaining)
<chiokz
80
<thiso
Thisking Mode:
Accuracy (4)
Na-Thaerry Fromps
44.4
254
m4-10.0
No Thinklng Rahe
0.3
Summary:: First.- Finalyy
142-4-6.
46.5
finel ondwer Misseed }
10
32.
No- Thinking Prompt (Always Not Thinking):
0.1
rchilko Okay, I think heve fisiiee thicking. v/thekk
Thinking Mode:
0
"thiss Ciiay, I think have
Dishil-R1
SeepS
finighed thriking c/thinlo
Usage under Different
DishilI-R1 (After Multi- Statt Training)
Ours: Ellipsis Prompt Decf-Decidss)
Vbowed
Length
10k
10533
3180
pllp
Shandard
Thinking frompe
(o-Teeking Rats
0.8
<thinko...
7459
49925
-006
0.6
<thinoo... OK.., Woit cfthindo
Thinking Mode:
6k-
5501
-3108
0.4
Summury boowed) 3..
4k-
00 Scyeye Mold
No- Thinking
2kc
2526.
2403
0.2
<thinle.. chthaslo
00
First, final ensver bboxed(-)
Dishill-a
DeepScoleR
Difficulty Level
(a) Accuracy and token usage with standard and ellipsis prompts.
(b) No-Thinking rate by difficulty.
Figure 2: Prompting strategies shape reasoning behavior and computational cost.
To investigate this question, we explore how a minimal modification to the prompt structure can influence reasoning behaviors in RI-style models. The baseline prompt used typically includes a <think> An tag followed by fixed, detailed reasoning trace. In contrast, our modified prompt contains only single ellipsis following the baseline tag. Specifically, the final prompt we provide is: <think> \n... \n. This minimal form acts as an open-ended signal, leaving it entirely up to the model decide whether to engage in thinking. how much elaborate, and when to stop.
Surprisingly, this tiny change leads to distinct shift in behavior. Without any additional training, the model often generates closing </ </think> tag, sometimes immediately, skipping deep thinking entirely, and other times after producing a full derivation. As shown in Figure 2a, evaluation on Distill-R1-1.5B [9] and DeepScaleR [17] across five mathematical benchmarks shows that ellipsis prompting leads to a modest drop in accuracy, accompanied by substantial reduction in token usage.
Compared to the no-thinking prompt baseline [19]. which suppresses reasoning the cost of accuracy, the ellipsis prompt triggers a stochastic switch in reasoning mode and provides a more balanced trade-off by preserving reasoning when needed and reducing unnecessary computation.
2.2 Prompting Alone Does Not Enable Difficulty-Awan Thinking
The proposed ellipsis prompt seems to trigger selective reasoning: the model thinks on some inputs but not others. While this behavior appears desirable, it raises deeper question:
Does the prompt-forcing model choose to engage in deep thinking based task difficulty?
Ideally, a well-calibrated model should reason more on complex problems and skip unnecessary thinking on simpler ones. To assess this, divide MATH500 problems into 8 difficulty levels based on the average accuracy of Distill-R1 (standard prompt) over 16 rollouts, with higher accuracy indicating lower difficulty. Figure 2b (top) shows the no-thinking rate across these levels. Contrary to expectations, under the ellipsis prompt without additional training, no clear trend emerges- the flat distribution suggests that thinking unguided and unaffected by problem complexity.
decreasing no- no-thinking rate along the difficulty axis reflects a desirable reasoning pattern, which the model allocates effort based on task difficulty. However, this behavior does not emerge from prompting alone. Even with diverse prompt designs (Appendix A.2), the model failed to exhibit difficulty- aware reasoning. Yet pnompt-only control suffers from core limitation: without feedback, the model lacks a mechanism to learn when the thinking process is needed.
To address this gap, we introduce multi-stage RL framework that rewards appropriate reasoning behavior and encourages alignment between effort and difficulty. As shown in Figure 2b (bottom), the resulting distribution from our final trained model exhibits clear difficulty-awar reasoning.
3

---

3 Guiding When to Think via Multi-Stage Reinforcement Learning
We propose Auto multi-stage RL framework with three training phases that induce difficulty- aware reasoning through progressively refined reward designs. At all stages, we employ the GRPO algorithm with a token-level policy gradient loss [25, 46). The training objective is:
JARPO (0) =.) (or)
G loil
Σ ΣΣ min
(f),, -,,1,)
loi|
Here, Og denotes the i-th sampled output for a given query q: G is the number of sampled outputs per query; (0) is the token-level importance weight, defined the ratio between the new and old token probabilities; and Á., represents the estimated token-level advantage, The overall loss is normalized by the total number of tokens across all sampled trajectories. A visual overview of the reward mechanisms across the three training stages is illustrated in Figure 1. In the following subsections, we detail the reward design for each stage.
3.1 Stage 1: Preventing Mode Collapse by Batch Reward Balance
To promote efficient reasoning, higher rewards are assigned to correct answers without thinking, and stronger penalties to incorrect ones. Define think, {0, 1} indicator of whether the i-th output involves thinking, and correct; E {0, }} as an indicator of whether it yields the correct answer. Based on these variables, the naive reward assignment is:
Adj Reword vs. Think Propartion (v 0,5,
+1, if think; = 1 A correct; = 1,
0, if think; 1 A correct; O,
naive ;
Adjusted Reward
+2 if think; 0 A correct; 1,
1, if think; = correct; =
While this reward structure encourages difficulty-aware be- havior, it causes instability during early training. The model may collapse into a degenerate policy, either always think- ing always skipping, depending which yields higher expected reward in the short term. This limits exploration and hinders later optimization. To mitigate this, introduce batch-level reward balaneing:
Think Cornect
Ne- Ne-Tiikk Cerrect Ni-Tvik
Taik- Tecoract
Propartion (2)
Figure 3: Effect of r.
ad)
Let E [0, 1] denote the proportion of thinking trajectories in a training batch, and - the no-thinking proportion. A target balance ratio 7 E (0, 1) and penalty slope 2 control the strength of adjustment. For thinking and no-thinking samples, we compute soft penalty factors:
Oehink min (1, m0 (z -)) A)),
(3)
8 bothink min (1, max (0, - - A)).
Each sample i is first assigned an original reward maive E {+2, +1,0, 0, based on its thinking flag and correctness. The final adjusted reward is then:
[1 Sainn) ynaive
if think; = A correct, = 1,
(1 dahink) naavvve + Ōthink (-1);
if think; = 1 A correct, = 0, if think; 0 A correct, =
ri
(5)
(1 Snochink) ygavvve
(1 &nochink)
dnothink (-2), if think; A correct,
The adjusted reward r: introduces soft, piecewise- linear modulation over the naive reward, resembling a hinge-like transformation. Figure 3 illustrates this behavior under typical setting with
7 = 0.5 and À = 2.0. When thinking dominates (z > 7), the reward for thinking samples is softly reduced, especially for incorrect answers. Conversely, when no-thinking is overrepresented (= < 7). no-thinking rewards are suppressed. In both cases, the model is gently pushed to restore balance by favoring the less frequent behavior.

---

3.2
Stage 2: Reinforcing Reliable Behavior within Dual Modes
After establishing behavioral stability across thinking and no-thinking modes, the second stage focuses improving task performance within each mode. Specifically, the objective is to enhance reasoning quality when invoked, and to promote accurate responses in the absence of thinking.
To allow the model to refine its behavior without external constraints, we remove the batch-level balancing used in the previous stage and allow free evolution of the reasoning policy. The reward is set directly to the naive definition:
(6) In this stage, we allocate larger context budget during training, enabling longer responses when needed. Owing to the regularization established in Stage 1, the proportion of thinking in Stage remains balanced, fluctuating naturally rather than collapsing.
naive
3.3 Stage 3:
Pruning Unnecessary Reasoning Paths via Length-Aware Reward
While the relaxed setup in Stage 2 improves accuracy, it also leads to overly long responses. Building on the stability established in prior stages, we now aim to improve reasoning efficiency.
Inspired by GRPO-LEAD(SO), we introduce a length-aware reward modulation, encouraging brevity in no- thinking mode and rewarding elaboration only when warranted. Specifically, the adjusted reward in this stage is defined as:
Length-Aware Shoping 0.2.
Adjutte Roward
yuuuive e-1:) if correct; 1, correct; =
adj
+ (1 -1--5 e- By:)
where yi Li-Me is the standardized length of response i within its query group q. Here, L; denotes the response length, while By and og are the group-specific mean and stan- dard deviation of lengths, computed separately for correct and incorrect sample groups. And a and B are hyperparame- ters that control the sensitivity of the shaping term.
Corrct+ Trikk
Corract Ne-
Incernet Thine
Incornect No.
Length y)
Figure 4: Effect of a, 9 on adj
The reward decays with length for correct responses and grows for incorrect ones, encouraging concise success and thorough failure analysis, as an example illustrated in Figure 4. This final stage allows the model to adaptively regulate its reasoning depth, producing succinet responses without significantly compromising reliability.
4 Experiments
4.1
Setup
Datasets and Models We use the same training data as in DeepScaleR [17]. comprising 40K mathematically problems with varying difficulties. Following prior works [49, 33). the evaluation is conducted on five standard math benchmarks: MATH, Minerva, Olympiad, AIME24, and AMC23. We evaluate the applicability of AutoThhik on three R1 R1-style models with varying sizes and RL post-training status: DeepSeek-Re -I--ssill- ---iiiilll--en------ Qwen- 1.5B/BB (abbreviated as Distill-RI-155B/71 -rreview--..D CrepScaleR-Preview--l model obtained from Distill-RI-I. -1-5.B via context-extended RL at a training budget of up to $5,000.
and [17] (abbreviated as DeepScaleR), the state-of-the-art 1.5B reasoning
DeepScaleR-
Baselines We benchmark our approach against two classes of baselines designed to promote efficient reasoning. (1) Prompt-only baselines: we apply standard [9], no-thinking [19]. and ellipsis (ours) prompting strategies on the base models, following the description illustrated in Figure 2a. (2) RL-trained baselines: including Concise-RL RL [6], ShorterBetter [44], and ThinkPrune [12], all of which aim to shorten reasoning traces by RL, but do not explicitly account for adaptive reasoning behavior. Among these methods, only ThinkPrane provides publicly available model checkpoints; we evaluate its two representative variants, iter-2K and 4K. For Concise-Re RI. and ShorterBetter, results reported as published in their respective papers. (3) Additionally, we include set of open-source
5

---

Table 1: (Main Results) Accuracy, Token Usage, and Efficiency Comparison Across Methods.
Method
Accuracy (%)
Token Usage
[-F((%)
MATH Minerva Olympind AIME24 AMC23
MATH Minerva Olympiad AIME24 AMC23
AVG
AVG
Open-Source RI-Style Model: Truin From BibieekiR--iiiii---eeee
At1-Disiilll-0
8 or Even Larger
Open-RS3-1.5B
30.6
49.2
5578
11626
16651
11052
10497
Still-3-1.5B
849
28.4
45.0
31.0
64.6
50.8
4208
6021
9470
13399
8788
8377
FastCuRL-1L5B
3829
7U77
6571
Light-RI-DS-78
92.1
37.6
58.1
62.3
82.4
66.5
3774
5434
8362
12064
7317
7390
AReal--hoaaRRI...
93.4
37.7
62_1
65.4
85.7
68.9
4947
8290
10096
12905
8452
8933
QwQ-32B
95.1
45.3
69.0
76.7
95.5
76.3
5547
8650
9445
13970
4222
8367
Base Model:
DeepSeek-RI-DDe
Standard Prompt
83.1
26.0
43.7
27.5
62.5
48.6
5622
7688
11555
17322
10981
10633
No- Thinking Prompt
70.4
19.1
33.1
15.8
49.0
37.5
1256
628
2426
5793
2535
2528
Ellipás Prompe
78.2
21.9
38.6
25.2
57.2
44.2
4194
4336
7752.
13006
7980
7453
Concise RL
81.0
30,0
1965
6752
ShotterBetter
27.6
38.4
20,0
56.6
1147
1814
2703
1946
ThinkPrune-ite-2X
82.6
28.1
43.6
26.7
64.9
49.2
1927
2126
3683
5806
3300
3368
9.9
Phin--rune-4k
28.3
5504
8072
5040
4943
18.7
Think Itagel
79.4
21.4
40.5
27.7
$9.0
45.6
3107
3867
7212
11673
6467
64465
0.0
AutoThink- 2
27.2
31.8
8030
12117
7295
31.6
AutoThink Stage3
84.0
28.1
44.8
34.6
67.0
51.7
2195
3212
$559
9514
50.9
5108
39.6
Base Model:
DeepScaleR-
Standard Prompe
87.6
30.7
50.0
42.3
72.8
56.7
3171
4948
5967
9326
5675
5817
No- Thinking Prompt
Ellipsis Prompt
85.9
28.9
48.1
42.1
72.0
55.4
2890
4748
5416
9408
5005
5511
/
Phink-rune-ttt-2KK
38.7
55.4
1838
3254
5328
3166
3200
0.0
ThinkPrune-ai
86.5
30.6
48.5
36.5
71.8
54.8
2221
3039
4061
6624
3868
3963
0.0
AmtTThnk-- Saook----iage
82.1
27.0
33.5
50.8
2473
5372
7328
12716
$440
6666
0.0
Auto A Think-Stoge Stoge2
87.6
31.8
50.
42,9
73.9
57.3
2762
4315
5521
8567
5222
5277
7.5
Think Stage3
85.1
30.5
49.0
41.9
71.9
55.7
1897
3834
5005
9033
4696
4893
0.0
Base Model:
DeepSeek-RO
Standard Prompe
92.3
37.6
56.4
52.7
82.8
64.4
3928
5155
8815
13563
7613
7815
No: Thinking Prompt
78.2
22.1
40.2
22.7
53.7
43.4
722
486
1434
3269
1433
1496
Ellipeis Prompt
91.8
37.6
56.5
51.3
80.9
63.6
3752
4778
8643
13532
7616
7564
Concise RI.
90.3
$1.7
2041
6632
ShorterBetter
44.1
50.7
53.3
75.9
1341
3410
$288
2580
AutoThiok-Stage
89.3
31.8
53.8
52.7
78.2
61.2
1763
1717
4798
8515
4397
4274
0.0
AntoThick- Stoge-
92.2
38.5
56.2
57.1
83.7
65.5
2519
2980
S797
8676
4925
4979
7.2
AutoThink 3tage
54.8
5498
8051
RL-finetuned models based on Distill-R1-155B// as reference, including Open-RS3-1.50 [5], Still- 3-1.5B [22], FastCuRL-1.5B [28]. Light-RI-DS-7 [38], AReaL-boba-RL-7B [21]. and QwQ-32B [31]. These models are not explicitly optimized for concise reasoning and differ significantly in both training objectives and computational budgets. We report their results for contextual reference only, aiming to highlight differences in design philosophy rather than to draw direct performance comparisons.
Training and Evaluation All experiments are implemented using the verl framework [27], with most training hyperparameters retained at the default values. For all models, we set the batch size and training context length to (128, 8K) in Stage 1, (64, 16K) in Stage 2, and (64, 24K) in Stage 3. We save model checkpoints at empirically selected steps based on observed convergence throughout the procedure: 220/440/13 30 for Distill-R1-1-1 155,, 10/240/60 for DeepScaleR, and 220/450/20 for Distill-R1-7B across Stages. During evaluation, all models use a 32K context window. We sample 16 rollouts per instance with temperature 0.6 and report the average pass @ accuracy, Reward shaping hyperparameters are set to y = 0.5, À = 2.0 for Stage I, and a 3 = 0.05 for Stage 3.
4.2 Main Results
Table reports average accuracy and token usage across five mathematical benchmarks. To jointly evaluate reasoning accuracy and efficiency, we introduce the Efficiency-FI score (E-F1), defined as:
Anee
Alen Aien
E-FI =
if acc > accud and len < lenaa: else 0
Ance
where the normalized accuracy gain and token reduction are given by:
acc acCaıd
lenad len
A age acCaud
A len lengo
acCao

---

Thinking Rate Over Strae
Responde Lerglh Over Stoge
Training Accurrcy Over Stoges
Recpeeee Lenyth Over Stoges
1.0
Thikng
0.8
Tikeet Leu
3K-
Acconcy D-3
0.6
0.4
3k
0.2
Lenghb-waae
Retch Ediance
2k-
Length-turre
satth Balance
90
Strge3
Staged
Stape3
Stegeé
0.0
Stegel-Teo
StiallToo
a5o
400
(a) Effect of Stage 1 reward design on reasoning.
(b) Effect of Stage 2 in performance boosting.
Figure 5: Prompting strategies shape reasoning behavior and computational cost.
The subscripts std and no refer to the standard and no-thinking baselines. A non-zero E-FI indicates that the model improves upon the standard baseline both accuracy and token usage, capturing the extent to which pruning enhances conciseness without degrading performance.
Despite the strong performance of existing open- models, their outputs substantially longer, even reaching twice the length of ours at the same model size, suggesting that their gains stem from verbose reasoning but non-adaptive reasoning. Prompt-based baselines (no-thinking and ellipsis) reduce length at the cost of accuracy. RI--based baselines also shorten outputs, but offer limited improvements on Distill-RI and in some cases even reduce accuracy on DeepScaleR.
In contrast, AutoThink exhibits a staged progression in both accuracy and efficiency. All three stages are consistently trained with the ellipsis prompl as the base prompting strategy. Stage I primarily aims to stabilize the activation of reasoning behavior and has minimal impact on performance. Stage leads to accuracy improvements over the standard prompt across all model backbones, demonstrating length-aware aware pruning, further reducing token usage while minimizing potential performance degradation. On Distill-R1-1.51 AutoTThink-Saage achieves 51 51.7% accuracy with half the token usage of the standard prompt baseline. Remarkably, further improves performance by 0.6
effective control over when to reason. Stage 3 introduces length-
on the heavily optimized DeepScaleR, AutoTThinkSSter over the standard prompt while reducing token usage by an additional 10%. However, Stage 3 leads slight accuracy drop, likely because DeepScaleR has already undergone extensive optimization. This suggests that additional pruning may be unnecessary on fully optimized models.
4.3 Ablation Study
We conduct ablations the reward design of multi-stage RL framework Distill-R1-1.5B to assess the necessity of each stage. The performance gains achieved by Stage 2 and the pruning effect of Stage 3 are already reflected in Table 1, in terms of accuracy and token usage. Here, we focus on two key aspects: (1) the role of batch reward balance in Stage 1, and (2) whether skipping Stage and proceeding directly from Stage
Stage 3 yields comparable performance.
Batch Reward Balance Prevents Mode Collapse To assess the role of batch-level balancing in Stage examine its impact on stabilizing dual-mode reasoning behavior. Specifically, we plot the average thinking rate across training steps, as shown in Figure 5a. Under a naive reward, the model rapidly collapses into a thinking mode. Conversely, applying the length-aware reward (with a 0.05, (9 = 0) in naive reward to encourage brevity leads the model to collapse into a degenerate no-thinking mode. In contrast, the batch reward balance mechanism, by enforcing target thinking ratio via penalty slope X, helps stabilize training and supports the coexistence of thinking and no- -o-thinkiig behaviors. We observe that response length rises and then falls during training, indicating an increasing share of shorter, no- no-thinking responses. These observations imply that the model implicitly performs reasoning pruning, akin to concise reasoning.
Pruning Without Reinforcement Limits Performance We investigate the necessity of Stage 2 by applying Stage 3 directly after Stage skipping the reinforcement phase. As shown in Figure 5b, the complete training pipeline that includes Stage 2 prior to Stage 3 yields notable boost in both accuracy and response length, followed by effective pruning with minimal performance degradation. In contrast, bypassing Stage 2 results in stagnant accuracy and an eventual increase in response length after an initial decline. In contrast, skipping Stage 2 leads to stagnant accuracy and rebound in response length. With comparable response lengths, the variant achieves only 47.6% accuracy across
7

---

But
Dighill RJ 9.58
DreepSonll21150
Woit
Alternat vely
Thinking Rate (3)
Solloquize Thinking
5569
5259
Chock
50
9021
Check & Confirm
Condirm
MATH
Thonefore
Minarvo
Ronaies (lletora Trahing) (After
20
AIME24
Clympaid
210
Let
Summary Colculation
Tralingg
10
2762
6
60
100
Keyword Count 1K Tokens
Accuracy
Figure 6: Keyword usage of reasoning behaviors across thinking and no-thinking modes.
Figure 7: Accuracy vs. Thinking Rate. The numbers indicate response lengths (tokens).
five benchmarks, notably lower than the 51.7% from full training. These observations underscore the importance of Stage in establishing stable and discriminative reasoning behaviors that enable reliable pruning in the subsequent stage.
4.4 In-Depth Behavioral and Efficiency Analysis
Lexical Patterns in Two Reasoning Modes We analyze linguistic differences between thinking and no-thinking responses by quantifying the frequency of reasoning-related verbs (e.g.. "Wait", "Al- ternatively", "Check") per 1,000 tokens, capturing how explicit reasoning is manifested each mode. Following [12], we categorize these keywords into three functional groups the MATH500 bench- mark: (1) Soliloquize & Thinking, reflecting internal deliberation and self-correction, characteristic of RI-style reasoning; (2) Check & Confirm. indicating procedural verification; and (3) Summary & Calculation, marking final deduction and computational closure, As illustrated in Figure 6, Think training substantially reduces soliloquy- expressions, particularly under the no- thinking mode, indicating a decline in explicit internal deliberation. contrast, verification and computation-relate terms appear slightly more frequently in the no-thinking setting, suggesting shift toward focused conclusion and validation rather than step-by-step verbalization.
Correlation Between Task Difficulty and Reasoning Tendency We investigate the relationship between the reasoning behavior and the inherent difficulty of the tasks. As shown in Figure 7, there exists positive correlation between the thinking rate and task difficulty. To further quantify this relationship, we compute the average accuracy, thinking rate, and response length across all datasets. Here, accuracy serves proxy for dataset difficulty. The results indicate that, more challenging datasets, models tend to invoke explicit reasoning more frequently and produce longer responses. This demonstrates that stronger models do rely explicit reasoning as frequently, yet outperform weaker models, highlighting an emergent ability to reason more selectively and efficiently.
Readability and Accuracy of Dual Reasoning Modes A common concern in reinforcement fine-tuning is that reward-driven optimization may degrade the fluency or coherence of generated reasoning traces. To assess whether AutoThink introduces such effects, we follow the evaluation setup in [12] and compute the perplexity (PPL) over the <think> span traces using DistillRR--L5BB For no-thinking variants, PPL is calculated over the segment following </think>, As shown in Table 2, the think mode of AutoThink maintains PPL comparable to standard prompting, while the no-think mode achieves the lowest PPL. reflecting more concise and fluent responses. Overall, all variants remain within acceptable readability ranges. Mcanwhile, we analyze accuracy and token usage across reasoning modes. The results are also recorded in Table 2, no-thinking responses are shorter and more accurate, suggesting effective handling of simpler problems. Thinking-mode responses are longer with slightly lower accuracy, reflecting allocation to harder cases. These results indicate that AutoThink adaptively adjusts reasoning depth based on task difficulty.
Evaluating AutoThink Under Standard and No- Thinhiinng Prompts We analyze how the trained model responds to the standard and forced-no-think prompts. The forced-no-think prompt is defined as <think>\n. An... in</think> An \n, which builds upon the ellipsis prompt but enforces an immediate termination of the thinking phase. The results of Distill- 1.5--...B--- Auto- AutoThink presented in Table 3. As

---

Distill-R1-135
ThinkPrune
ActoThirk Think (Think Mode)
Auto Think Think Mode)
8.6%
11.6%
12.1%
1L2%
10.9%
12,
29,39
32.4%
4,5%
9.6%
10.4%
9.9%
17.3%
11.%%
11.3%
9.%%
Undershanding
Subshifuting Known Results
Breaking Supproblens
Reformulating the Problem
Reossess and Verify Local Shops
Reassese the Whole Solution.
Applying Khown Thesrenss/ropertics
Findliz Pressnt the
Exploring Alternative Approsches
Computing Sinpkfying Expressions
Figure 8: Distribution of Reasoning Behaviors Across Models and Reasoning Modes.
expected, the standard prompt induces longer reasoning traces and achieves higher accuracy, while the forced no-think prompt reduces token usage at the cost of slight performance degradation. These findings suggest that Auto Think has learned to internally compress its reasoning when appropriate, while retaining the ability conditionally invoke reasoning via prompting.
Table 2: PPI., Acc & Token Length.
Table 3: AutoThink Performance on Three Prompts.
Response
PPL Ace (%) Token
Prompt
MATH Minerva Olympiad AIME24 AMC23 AVG
Model: Dis-II--R--I5B
Accuracy (%)
Standard Prompt
No- To-Thinking Prompe
1.61
83.1
70.4
5622
Ellipsis
84.0
28.1
44.8
34.6
67.0
51.7
Auto Think: Think Part
2.29
56.4
5090
Standard
84.4
28.1
45.4
35.0
67.5
52.1
AmtTThink: No-Think Part
1.50
90.1
1592
Forced No-Think
83.7
27.2
44.8
32.7
65.9
50.9
Model: DeepScaleR-1.5. L.5
Token Usage
Stundard Prompt
2.19
83.1
5621
Ellipsis
2195
3212
5559
9514
5059
5108
No- Ne-Thinking Prompe
AutoThink: Think Part
1.85
2.43
70.4
63.9
5065
1256
Standard
2679
3534
5726
9862
5243
5408
No-Think
Forced So-Think
2127
2877
5143
8668
4795
4722
Reasoning Behavior Profiling To gain deeper understanding of how reasoning behaviors evolve, we annotate the generated solutions from each model with high-level problem-solving phases using GPT-4o. As illustrated in Figure 8, Distill-R1-1. 1-1.5B distributes its reasoning effort across many surface-level activities, such as "reformulating the problem" and "understanding the problem." In contrast, ThinkPrune slightly shifts focus toward finalizinaii routines, while still exhibiting dispersed reasoning patterns. Notably, AutoThink in Think Mode allocates a larger proportion of steps core reasoning phases, including "computing or simplifying expressions" and "applying known theorems," suggesting a more targeted and efficient reasoning trajectory. Meanwhile, in No-Think Mode, Think maintains strong task comprehension and delivers concise outputs, dedicating most steps to problem understanding and direct computation. These findings indicate that AutoTThink not only reduces redundant steps, but also adapts its reasoning structure based on the selected mode.
Generality Beyond Mathematical Reasoning To investigate whether AutoThink generalizes beyond mathematical reasoning, we additionally evaluate our models on three non mathematical benchmarks: (i) GPQA for scientific multi-hop reasoning, (ii) MMLU for general multi-task language understanding, and (iii) Live-Code-Bench for code generation (20250727 release).
As shown in Table 4, AutoThink retains competitive accuracy while reducing ken usage, indicating that adaptive rea- soning behaviors extend beyond math tasks. Stage 2 even surpasses the base- line in accuracy while halving response length, highlighting the transferability of our approach diverse domains.
Table Performance of AutoThink on non-math bench- marks. Each cell shows Accuracy (%) / Avg. Length.
GPQA
MMLU
Live-Code-Buun
Avg
Distill- 1.5B
3 10026
49.5/
25.2/ 13372
36.6 / 8708
SotoThink-SSagee
31.5/ 8889
47.3 1190
23.8 5653
34.3 5244
AutoThink Sunge2
37.1 / 8617
48.8/ 1743
24.2 / 9647
36.7 6669
Think-Singee
35.7 5639
48.8/ 1300
24.9/ 9054
36.5 / 5337
Additional Analysis. We further conduct additional analyses on more base models, hyperparame- ters, training cost, and case studies. Details presented in Appendix B due to space limitations.

---

5
Related Works
RL-based Post-Training for LLMs. Reinforcement fine-tuning (RFT) has been widely adopted to improve the reasoning ability of LLMs [32, 13, 9, 11, 7, 34, 35]. Recent work on RL for LLMs has focused on improving the efficiency and effectiveness of large-scale RL training. Key techniques decoupling the clipping mechanism and introducing dynamic group sampling [46], mitigating value bias over long sequences [48, 47], difficulty-aware advantage reweighting [50], model ensembling [8] and designing minimal-form credit assignment strategies for rewards [4]. In addition, RFT has been shown to explicitly promote seff-verification and self-correction behaviors [26, 18), while also wereext-eengh-exteeeee
supporting optimization of test-time compute [23]. Multi-stage, amplifies the long-chain reasoning ability of RI-style models [17, 28]. In our work, RL is applied to train RI RI-style models to adaptively control their reasoning behavior, enabling selective thinking guided by multi-stage reward shaping.
RL further
Mitigating Overthinking for LLMs. While RFT improves performance, it may induce over- thinking, causing models to generate overly verbose reasoning with limited benefit [29, 14]. [3] address overthinking in RI Rl-stlle models by using self-generated short CoT positive signals in DPO, encouraging concise reasoning. [51] mitigate overthinking by training models to terminate with "I don't know" on unsolvable problems. Recent studies have shown that inserting pseudo-thinking cues into RI RI-style prompts [19], or manually controlling reasoning based on problem difficulty [10, 39, 15], can suppress the model's thinking behavior, but resulting in reduced performance. Other studies approach the problem from different perspectives: supervised fine-tuning (SFT) with short CoT responses [45, 20), incorporate response length-aware rewards in RFT [1, 44, 12, 6, 16), or leverage smaller models guide larger ones toward faster reasoning [15, 36]. Furthermore, [43] designs an early-exit mechanism during the reasoning phase to prevent excessive thinking along an incorrect chain of thought. Inspired by these findings, we first design a minimal prompt that elicits random thinking behavior, then apply multi-stage RL guide the model to think adaptively based on task difficulty, without using external signals or teacher models.
6 Conclusion & Limitations
This work explores how RI-style LLMs can Iearn to reason adaptively. We propose AutoThink, a minimal prompting strategy paired with a multi-stage RL framework that enables task-aware thinking. Through stage-wise reward shaping, the model stabilizes reasoning patterns, reinforces effective behaviors, and prunes unnecessary steps. Experiments show that AutoThhink achieves favorable accuracy- efffcceccy trade-offs, outperforming prompting and RL baselines without compromising performance, offering scalable and controllable approach to efficient reasoning in LLMs.
While AutoThink demonstrates promising adaptive reasoning capabilities, several limitations remain:
(1) Reward Hacking:
The model may bypass the separation between thinking and answering by embedding reasoning after the </think> tag. As shown in Figure 6, reasoning-related tokens still appear in no-thinking mode, suggesting incomplete behavioral separation.
(2) Uncontrolled Reasoning Budget: AutoThink adaptively decides when to think, but cannot control overall response length. Future work could explore budget-aware CoT generation, as seen in recent systems like Qwen3 [30].
(3) Unfiltered Training Data:
We directly use the DeepScaleR dataset without filtering by task difficulty. Though simple data selection has shown utility, our focus lies in training design. Integrating curriculum-based filtering may further improve performance.
Acknowledgments and Disclosure of Funding
This work is supported by the Strategic Priority Research Program of Chinese Academy of Sciences under Grant XDA0480302, Young Scientists Fund of The State Key Laboratory Multimodal Artificial Intelligence Systems ESSP100112. National Natural Science Foundation of China 62402252, 62536003, and Guangdong High-Level Talent Programme 2024TQ08X283.
10

---

References
[1] Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning. Proceedings of the 2nd Conference on Language Modeling (COLM), 2025.
[2] Anthropic. Claude 3.7 sonnet, 2025. URL http:://wwww..aarrp.......oo
[3] Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Do not think that much for 2+ 3=? on the overthinking of ol-like llms. In Forty-second International Conference on Machine Learning (ICML), 2025.
[4] Jie Cheng, Ruixi Qiao, Lijun Li, Chao Guo, Junle Wang, Gang Xiong, Yisheng Lv, and Fei-Yur Yue Wang. Stop summation: Min-form credit assignment is all process reward model needs for reasoning. arXiv preprint 1rXiv:2504.55277 2025.
[5] Quy-Anh Dang and Chris Ngo. Reinforcement learning for reasoning in small Ilms: What works and what doesn't. arXiv preprint 2ttp:l::2553..22221. 2025.
[6] Mehdi Fatemi, Banafsheh Rafiee, Mingjie Tang, and Kartik Talamadupula. Concise reasoning via reinforcement learning. arXiv preprint arXfv:2504.05115, 2025.
[7] Yuqian Fu, Tinghong Chen, Jiajun Chai, Xihuai Wang, Songjun Tu, Guojun Yin, Wei Lin, Qichao Zhang, Yuanheng Zhu, and Dongbin Zhao. Srft: single-stage method with supervised and reinforcement fine-tuning for reasoning. arXiv preprint arXiv:2506. 19767, 2025.
Yuqian Fu, Yuanheng Zhu, Jiajun Chai, Guojun Yin, Wei Lin, Qichao Zhang, and Dongbin Zhao. Rlae: Reinforcement learning-assisted assisted ensemble for 1lms. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025.
[9] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et Deepseek-rl: Incentivizing reasoning capability in Iims via reinforcement learning. arXiv preprint arXiv:2501.119445 2025.
[10] Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware Ilm reasoning. arXiv preprint arXiv:2412.18544 2024.
[11] Jujie rics.
Skywork open reasoner se- http:///capricioou--hhhrgggga ceatiinn.itad Skywork-Oper Seaiz--OpennRReeeeer-eeer Reaonser- - Sew.aeeer--eriieeeddoeeeeeeeeeee Sccioee--.bbbeeeeeeeeeseeeeeeee3
He,
Jiacai
Liu,
al.
2025.
Notion Blog.
[12] Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long chain-of-thought of lims via reinforcement learning. arXiv preprint arXiv:2504.01226, 2025.
[13] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camey,, et al. Openai ol system card, arXiv preprint 2rXiv:2412.167200 2024.
[14] Abhinav Kumar, Jaechul Roh, Ali Naseh, Marzena Karpinska, Mohit lyyer, Amir Houmansadr, and Eugene Bagdasarian. Overthink: Slowdown attacks on reasoning llms. arXiv preprint arXiv:2502.02554, 2025.
[15] Yule Liu, Jingyi Zheng, Zhen Sun. Zifan Peng, Wenhan Dong, Zeyang Sha, Shiwen Cui, Weiqiang Wang, and Xinlei He. Thought manipulation: External thought can be efficient for large reasoning models. arXiv preprint arXiv:2504. 13626, 2025.
[16] Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length- haagthh-irmniiiingggg. fine-tuning for ol-like reasoning pruning. arXiv preprint arXiv:2501.12570, 2025.

---

[17] Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai. Jeffrey Luo, Li Erran Li, Raluca Ada Popa. and Ion Stoica. Deepscaler: Surpassing ol ol-preview with a 1.5b model by scaling rl, 2025. Notion Blog.
[18] Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, and Jia Li. S²r: Teaching Ilms self-verify and self-correct via reinforcement learning. arXiv preprint arXiv:2502.12855. 2025.
[19] Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint 2501::2000008858 2025.
[20] Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-compprssibbe chain-of-thougle tuning. arXiv preprint arXiv:2502. 09600, 2025.
[21] Zhiyu Mei, Wei Fu, Kaiwei Li, Guangju Wang, Huanchen Zhang, and Yi Wu. Real: Efficient rihf training of large language models with parameter reallocation. In Proceedings of the Eighth Conference on Machine Learning and Systems, 2025.
[22] Yingqian Min, Zhipeng Chen, et al. Imitate, explore, and self-improve: A reproduction report on slow-thinking reasoning systems. arXiv preprint arXiv:2412.094133 2024.
[23] Yuxiao Qu, Matthew YR Yang. Amrith Setlur, Lewis Tunstall, Edward Emanuel Beeching, Ruslan Salakhutdinov, and Aviral Kumar. Optimizing test-time compute via meta reinforcement fine-tuning. In Forty-second International Conference on Machine Learning (ICML), 2025.
[24] Matthew Renze and Erhan Guven, The benefits of a concise chain of thought on problem- solving in large language models. In 2024 2nd International Conference on Foundation and Large Language Models (FLLM), pages 476-483, IEEE, 2024.
[25] Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03. 030000 2024.
[26] Maohao Shen, Guangtao Zeng, Zhenting Qi, Zhang-Wei Hong, Zhenfang Chen, Wei Lu, Gregory W Wornell, Subhro Das, David Daniel Cox, and Chuang Gan. Satori: Reinforcement learning with chain-of- wol.of-ation--ttte thooght enhances I1m reasoning via autoregressive search. In Forty-second International Conference on Machine Learning (ICML), 2025.
[27] Guangming Sheng. Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279-1297, 2025.
[28] Mingyang Song, Mao Zheng, Zheng Li, Wenjie Yang, Xuan Luo, Yue Pan, and Feng Zhang. Fastcurl: Curriculum reinforcement learning with progressive context extension for efficient training rl ri-like reasoning models. arXiv preprint arXiv:2503. 2025.
[29] Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16411 2025.
[30] Qwen Team. Qwen3 technical report, 2025. URL UtLLtttg::///gtttoo.o/o////////a
[31] Qwen Team. Q2q-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlnn http:://gqeennnniiibbbbte htto//llg////goooo/oo blog//wq-32b/
[32] Luong Trung Xinbo Zhang, Zhamming Jie, Peng Sun, Xiaoran Jin, and Hang Li. Reft: Reason- ing with reinforced fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 7601-7614, 2024.
[33] Songjun Tu, Jiahao Lin, Xiangyu Tian, Qichao Zhang, Linjing Li, Yuqian Fu, Nan Xu, Wei He, Xiangyuan Lan, Dongmei Jiang, et al. Enhancing Ilm reasoning with iterative dpo: A comprehensive empirical investigation. In Proceedings of the 2nd Conference on I Language Modeling (COLM), 2025.
12

---

[34] Songjun Tu, Jingbo Sun, Qichao Zhang, Xiangyuan Lan, and Dongbin Zhao. Online preference- based reinforcement learning with self-augmented feedback from large language model. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, pages 2069-2077, 2025.
[35] Songjun Tu, Qichao Zhang. Jingho Sun, Yuqian Fu, Linjing Li, Xiangyuan Lan, Dongmei Jiang, Yaowei Wang, and Dongbin Zhao. Perception-coosiiteecyy multimodal large language models reasoning via regtion-regularized policy optimization. arXiv preprint arXiv:2509.2185. 2025.
[36] Jikai Wang, Juntao Li, Lijun Wu, and Min Zhang. Efficient reasoning for llms through speculative chain- th-ough. arXiv preprint arXiv:2504.19095, 2025.
[37] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thouuh prompting elicits reasoning in large language models. Advances in neural information processing systems (NeurIPS), 35:24824-24837, 2022.
[38] Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, et al. Light-rl: Curriculum sft, dpo and rl for long cot from scratch and beyond. arXiv preprint arXiv:2503. 10460, 2025.
[39] Tong Wu, Chong Xiang, Jiachen Wang, and Prateek Mittal, Effectively controlling reasoning models through thinking intervention. arXiv preprint arXiv:2503. 24370 2025.
[40] Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for IIm problem-solving. In The Thirteenth International Conference on Learning Representations, 2025.
[41] Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint 2XX.:2501.00680 2025.
[42] Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600 2025.
[43] Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Zheng Lin, Li Cao, and Weiping Wang. Dynamic early exit in reasoning models. arXiv preprint arXiv:2504.1589 2025.
[44] Jingyang Yi and Jiazheng Wang. Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning. arXiv preprint arXivv2504::1:370, 2025.
[45] Bin Yu, Hang Yuan, Yuliang Wei, Bailing Wang, Weizhen Qi, and Kai Chen. Long-short thain-of-thought mixture supervised fine-tuning eliciting efficient reasoning large language models. arXiv preprint arXiv:2505.03444: 2025.
[46] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An spen-source Ilm reinforcement learning system at scale. arXiv preprint ar arXiv:2503.144777 2025.
[47] Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint urXiv:2504.05//6 2025.
[48] Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, and Lin Yan. What's behind ppo's collapse in long-cot? value optimization holds the secret. arXiv preprint arXiv:2503.00 014401 2025.
[49] Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503. 18892, 2025.
[50] Jixiao Zhang and Chunsheng Zuo. Grpo-lead: A difficulty-wware reinforcement learn- ing approach for concise mathematical reasoning in language models. arXiv preprint arXiv:2504.09696 2025.
13

---

[51] Zirui Zhao, Hanze Dong, Amrita Saha, Caiming Xiong, and Doyen Sahoo. Automatic curricu- lum expert iteration for reliable I1m reasoning. In The Thirteenth International Conference on Learning Representations (ICLR), 2024.
14

---

Appendix
A Additional Definition and Prompts
A.1 Definition of No-Thinking
In RI-style models (c.g., DeepScck-R1), Thinking refers to generating explicit, step-by-step reasoning traces enclosed within <think> ... </think>, enabling reflection and backtracking. By contrast, we define the No-Thinking mode [19] as immediately closing the <think> tag without producing any substantive reasoning, e.g.. <think> </think> before moving the final answer. This phe- nomenon, also referred to as Non-Thinking in the Qwen3 Technical Report [30], often emerges under ellipsis prompt, which stochastically toggles the model between reasoning and shorteut modes and thus serves as a lightweight control signal for studying adaptive reasoning behaviors.
A.2 Additional Prompt Variants
Beyond the prompt variants introduced in Section 2, we further explore alternative strategy that explicitly encourages the model to self-select its reasoning behavior. Specifically, we augment the original CoT with Think By Difficulty (TBD) prompt with an additional clause, followed by ellipsis prompt to preserve the optional -hinnnnni behavior. As is shown below, where the red text highlights the added clause:
Let's think step by step and output the final answer within \boxed[). Please decide whether to continue thinking based on the difficulty of the question.
Despite appending the TBD prompt explicitly encourage adaptive thinking, observe ingful emergence of selective thinking behavior. As shown in Figure 9 and Table 5, we plot the no-thinking rate across difficulty levels (on MATH500) and report accuracy and token usage across five benchmarks. Interestingly, the addition of the TBD prompt leads to a slight drop in both accuracy and token consumption. This result suggests that prompting alone without any reinforcement signal is insufficient to reliably induce adaptive thinking behavior in Distill-RI RI models.
binil (Thek by bifficslty Prenat)
Prompt
MATH Minerva Olympiad AIME24
AMC23 AVG
Accuracy (%)
Ellipsis Prompt
78.2
21.9
38.6
25.2
57.2
44.2
024
TBD Prompt
78.0
21.3
37.1
22.5
55.4
42,9
Token Usage
Ellipsis Prompt 4194
4336
7752 6490
13006 11754
7980
7453
0.
TBD Prompt
3893
3122
8796
6811
Difficulty Level
Figure 9: No-Thinking Rate.
Table 5: Comparison Between Ellipsis and TBD Prompts.
B Extended Experimental Results
B.1 Additional Results on Skywork-ORI-Mathhh
To further assess the generality of our method, we apply the AutoThink framework Skywork-ORt Math-7B3, a sfate-of-the-arr 7B model that achieves strong performance on mathematical reasoning tasks. Pretrained and fine-tuned with mleebbased reinforcement learning math and code tasks, this model represents one of the strongest 7B-scale math solvers. As shown in Table 6, the Ellipsis Prompt has limited effect on this highly optimized model, inducing only a marginal proportion of no-thinking responses, indicating reduced prompt sensitivity due to its deterministic reasoning policy.
Despite the limited prompt sensitivity, Stage I training with batch-level contrastive signals effectively captures and amplifies the model's latent no- totthinking behavior, enabling more balanced reasoning
15

---

Table Accuracy and Token Usage Comparison on Skywork PORI-Math-77
Accuracy (%)
Token Usage
Method
MATH Minerva Olympiad AIME24 AMC23 AVG
MATH Minerva Olympiad AIME24 AMC23 AVG
Base Model: Skywork-OR
D-work-ORI-Matt-TT
Standard Prompt
94.0
41.2
62.1
67.1
88.3
70.5
4669
7402
10102
14242
8849
9053
No-Thinking Prompt
85.5
26.1
48.1
45.6
68.2
54.7
1033
775
2982
6416
2402
2722
Ellipsis Prompt
94.0
41.8
61.8
69.1
88.0
70.9
4542
7399
10093
13813
8819
8933
AutoThink- -tugee
92.9
36.9
59.3
65.4
1894
2177
7068
4074
AutoThink Stage2
94.0
40.0
62.3
63.1
88.9
69.7
2298
3247
5437
8091
4521
4719
AutoThink Think-Stage3
93.1
38.8
61.1
62.7
88.2
68.8
1768
2287
4622
7372
3820
3974
BMNAAEAA mRpre Jeggh
400
Figure 10: Training Curves of -orborr-ORI-Matliie on Stagel.
patterns to emerge. Subsequent Stages and progressively refine this behavior. The full AutoThink framework is applied sequentially over three stages, trained for 600, 500, and 30 steps, respectively. Notably, the final stage achieves a nearly 60% reduction in reasoning tokens (from 9053 to 3974), while preserving task accuracy with less than 2% degradation compared the standard prompting baseline. These lightweight training phases are sufficient to induce substantial improvements in efficiency, even on strong pretrained models like Skywork ORI -Math-
We further visualize the training dynamics of Stage in Figure 10, including the proportion of thinking responses, well as the response length and accuracy stratified by thinking versus no-thinking behaviors. At early stages, almost all responses involve explicit reasoning. However, batch-wise balancing gradually promotes the emergence of no-thinking behavior. A clear modality shift occurs between steps 100 and 200, marked by a sharp increase in no-thinking responses, which directly contributes the reduction average response length. To explicitly encourage balanced distribution between thinking and no-thinking responses throughout training, we set the target balance ratio 7 0.5 in each of the three stages. Interestingly, while the accuracy of thinking responses slightly decreases during this phase, the overall accuracy continues to improve. This divergence suggests that the model is learning skip unnecessary reasoming on simpler problems, thereby increasing both efficiency and decision quality through adaptive control over its reasoning mode.
B.2 Additional Prompt Evaluation on Qwen3
We further extend our study to the Qwen3-8B [30] model by applying the proposed ellipsis prompting and adaptive training strategy. Table 7 reports accuracy and average token length across benchmarks. The results show that ellipsis prompting encourages
neglnggiie amount of no-thinking behavior; however, this tendency does not align perfectly with task difficulty (e.g.. AIME problems are significantly harder than MATH500, yet elicit lower thinking rate). Together with observations on Skywork-ORl -atth- (where ellipsis prompting induced only ~0.5% no-thinking behavior), these findings suggest that AutoThink strategy can also induce autonomous reasoning behavior in Qwen3, with ~ 13% occurrence of no- thinking responses under ellipsis prompting.
16

---

Table 7: Results of Qwen3-8B with different prompts. Each cell shows Accuracy (%) Avg. Length.
Qwen3-8B
MATH500
Minerva
Olympiad
AIME24
AMC23
Avg
Standard Prompt
97.0/ 5351
53.3/ 7010
73.5/ 11342
86.7 14690
88,1 10343
79.7 / 9947
Ellipsis Prompt
96.4/ / 5109
4
70.9/ 9891
68.3/ 13349
88.9 9858
74.8 8704
No-Thinking Prompt
84.1 1104
43.2/639
50.8/ 2860
26.3/6518
60.1 6011
52.5 2807
Ellipsis Prompt: Thinking Rate
96.9%
67.5%
89.0%
87.5%
96.2%
87.4%
B.3 Hyperparameter Sensitivity
The three-stage framework is intentionally designed to be modular and interpretable, with each stage serving a distinct and simple role: (i) Stage introduces batch-wise reward balance prevent mode collapse between thinking and no no-thinking behaviors: (ii) Stage 2 focuses purely on reinforcing accuracy within each mode without additional reward shaping: (iii) Stage adds length-aware shaping to encourage brevity for correct responses and elaboration for incorrect ones.
Among these stages, only Stage and Stage 3 involve reward shaping beyond naive correctness. Even in these cases, the formulations remain straightforward and principled. Specifically, Stage balances the modal ratio using linear penalty controlled by hyperparameters 7 and X, which are set to simple default values rather than finely tuned. As illustrated in Figure 3, the resulting reward curve naturally exhibits symmetric form. Stage 3 reuses shaping terms (a, 5) inspired by GRPO-LEAD, again without introducing any ad hoc modifications.
To further examine the robustness of these choices, conduct sensitivity analyses of y, X, and during training, with results summarized below.
Stage 1 Parameters (7, A). Stage 1, we balance the modal ratio between thinking and no- thinking behaviors using linear penalty controlled by y and A. These hyperparameters were not carefully tuned but set commonly used values. To illustrate their effect, Table 8 reports the average thinking rate at steps 100 and 200 during training. Increasing y encourages more thinking trajectories, while larger enforces stricter adherence to the target balance.
Table 8: Thinking rate (%) at checkpoints under different values of and A.
0.5,, 2=2
7=0.2, 2=2
7=0.8, A=2
7=0.5, A=1
7=0.5, A=4
Thinking-Ran Ratenng--ae@@eeee @tee
62.4 Thinking-Rann -ate @ step200
57.8
99.9
71.4
51.7
54.2
51.4
100.0
61.2
48.3
Stage 3 Parameters (a, 5). Stage 3, the shaping terms (a, 9) control the rate of reward decay/growth with respect response length. Table 9 shows response length checkpoints under different a values with 3 fixed at 0.05. Larger a accelerates length decay for correct responses, while smaller a relaxes the penalty.
Table 9: Response length under different a values (3 = 0..5).
a 0.05
a
a 0.1
Cessoons-LLenthhsstee Lessonns-Length@ssttee
4734
6174
3623
4120
6322
2894
Discussion. These results confirm that the shaping functions behave intended: 7/A modulate the balance between modes in Stage 1, and a/3 regulate the brevity of responses in Stage 3. Importantly. the overall training trends remain consistent with the main results, demonstrating the robustness of AutoTTink without extensive hyperparameter tuning. Thus, while the overall pipeline appears multi- stage, each stage was deliberately designed with minimal tuning and clear interpretability. Looking ahead, it may be possible to unify these stages through more holistic reward formulation, enabling the model to learn adaptive reasoning behavior within single-stage process. We leave this promising direction for future work.
17

---

B.4 Training Cost Comparison
We compare the training cost of AutoThink with two baseline methods, normalizing all runs unified batch size of 128. The results are shown in Table 10, AutoThink adopts a 3-stage schedule with increasing context lengths and total of 500 steps, comparable to that of 540 in ThinkPrune. In contrast, ShorterBetter trains in a single stage. While prior methods reduce context length achieve compression, Auto''hink expands it but prunes through shorter response length in Stage 3, resulting in comparable training cost. On H100 clusters with 4 nodes, training the all stages can be completed within one day for 1.5B models, and 2.5 days for 7B.
Table 10: Training Cost of Distill-RI-1150
Method
Steps (Batch Size=128)
Context Length
AutoThink
220 + 220 + 60 = 500
8K/ 16K 24K
ThinkPrane
80 180 + 180 540
4K /2KK
ShorterRetter
Table 11 provide estimated GPU-hour costs for all methods using Distill-RI-1.5B as the base model, AutoThink operates within the same order of compute as concise baselines such as ThinkPrunc and ShorterBetter, yet achieves notably stronger performance. In contrast, DeepScaleR, which primarily aims to maximize performance, requires more than 3x higher compute due its longer context length and increased RL iterations.
Table 11: Estimated GPU-hour cost of training Distill-R1-1.35 with different methods H100.
Method
GPU Hours
Avg ACC/ Length
ThinkPruneiter-2K
<400
49.2/
ShorterBetter
~220
44.7/ 1915
Moiil---BB-AntoTiiin
~7000
51.7/5088
DeepSealeR
~22000
5617/
SerpScaleR-L.B-AAuooTT
~220
5273
C Addressing Potential Challenges
While AutoT demonstrates robust improvements, several open challenges remain in the area of reasoning control. We outline possible directions for addressing these limitations below:
Token-Budget Control. Token-budget constraints have been partially explored in prior works [I, 44], where budget-aware reward functions penalize excessively long completions. Such formulations can be readily integrated with AutoThink in a plug-and-yg and-play manner to enforce global compute budgets and further improve efficiency.
Dataset Noise. The presence of noise in large-scale reasoning datasets can hinder training efficiency. Prior studies [28] suggest that curriculum learning filtering samples by correctness difficulty can improve learning quality. These strategies are orthogonal to our reward design and could be combined with Auto to further enhance robustness.
Reward Hacking. A common issue is reward hacking, where the model continues reasoning after the </think> tag. This can be mitigated by explicitly penalizing reasoning-related patterns outside the <think> span, or by rewarding clean separation between thought and final answer. Both strategies can be incorporated into future iterations of our reward function.
Overall, we view these solutions as complementary and composable with our framework. Future work will explore tighter integration of these mechanisms to provide more comprehensive solution to reasoning control.
D Case Study
Figure 11,12,13 presents some examples comparing four prompting strategies. For easy problems, AutoThink produces correct answers without explicit reasoning, reflecting effective fast thinking. For medium problems, it may activate both reasoning modes, with thinking and no-thinking re- sponses potentially coexisting. For hard problems, the model engages in deeper, slower reasoning, demonstrating iterative understanding and self-verification before arriving the correct solution. These observations demonstrate how AutoThink adapts its reasoning to problem difficulty, balancing efficiency and reliability through dynamic control of reasoning depth.
18

---

Question: What the result when the greatest common factor 6432 and 132 increased by 11?
Standard Prompt, 2125 tokens
No- Nhinking Prompt, 196 tokens
[Understand the Problem, 229 tokens] Okay, so have this muth problem here: by using the Euclidean algorithm... The last non-zero remainder is the GCF.
[Solve the Problem by CoT, 184 tokens] To find Find the GCF of 6432 and 132 GCF=...112
Increase the GCF by 11: 12+11=23.
[Present the Answer, tokens] Final Answer:
[Attempt Solve, 456 tokens] Alright, so let's apply that 6432 and 132. First ... 6432 mod 132 ... 132 mod ... 96 mod 36 ... 36 mod Therefore, the GCF 6432 and 132 is 12.
AutoThink: No- Nhinking Mode, 455 tokens
[Understand the Problem, tokens] Okay, need find can the Euclidean algorithm....
[Reassess the Whole Solution, 1146 tokens] Wait, hold on second GCF = 12 Therefore, the result is 23. Wait, hold on gives 23, But just be absolutely thorough... So, the final answer is 23.
[Solve the Problem by CoT, 345 tokens] Starting with 6432 and 132 ... 6432 mod 132 ... 132 mod ... 96 mod 36 ... 36 mod 24 ... So, the GCF of 6432 and 132
12 11 1
[Finalize and Present the Answer, 294 tokens] To find the result Therefore, the result is 23.
[Present the Answer, 13 tokens] The final result is 23.
Figure 11: Easy Case: AutoTThink solves the problem via no-thinking mode with few tokens.
Question: let 2 v2 (3 3v2)6, and l = - 36, Let w be the result when rotated around by r/4 counter-clockwisc Find w.
Auto Think: Thinking Mode, 2841 tokens
[Understand the Problem, 284 tokens] Okay, need to So, the steps are: 2. 3. Let me write that out in equations.
[Attempt to Solve, 793 tokens] First, ... Now, need to rotate ... Wait, on Now, we need to translate back ... So, -i.
Standard Prompt, 3897 tokens
[Understand the Problem, 186 tokens] Alright, so have this problem Let
recall the exact steps.
[Reassess the Whole Solution, 1275 tokens] Whit, that seems straightforward ... So, z" 4 21. Wall, that contradicts. ... W Wait, so the answer Therefore, final answer 56.
[Attempt Solve, trunslate z so that is at the origin... Okny, next step rotate ... Now, combine these... ... Thus,
tokens] First. think need to
Wait, that's different from ... So, to recapt ...
51.
[Renssess the Whole Solution, 1969 tokens] Wait, hold on ... 5t,
[Finalize and Present the Answer, 489 tokens] Given c= - 31 ... Thus, the complex numher w is: 5t.
5t, But just make sure 6- So, yes ... 56, But just make sure Therefore, all my computations seem consistent.
AhinThink: No Thinking Mode, 1013 tokens
[Finalize and Present the Answer, 542 tokens] Given Thus, the final result 6 Si.
(Understand the Problem, 145 tokens] Okay, so need to point w First, let's write down the given points:... ...
-hinking Prompt,300 tokens
[Solve the Problem by CoT, 836 tokens] rotate the rotation formula So, e v2 V2 Let me compute 2 c first... Now, multiply this by Let me compute this multiplication. Now, add back w = ... 51.
[Solve the Problem by Co 289 tokens] find Ol.
Translate complex plane ...
Rotate by IT/4 w'
Translate back
2i.
[Present the Answer, 32 tokens] Therefore, the result w 6 5i,
[Present the Answer, 19 tokens] Final Answer: 2i.
Figure 12: Medium Case: AutoThink exhibits both thinking and no-thinking modes on the problem.
19

---

Question: In quadrlateralAABCC ungle BAD and angle CDA are trisected as shown. What the degree measure angle AF!??
100
100
No- Thinking Prompt, 360 tokens
[Solve the Problem by CoTT' 344 tokens] To deternine the measure of angle AngleAFD....
Identify the Triangles:... ... Sum of Angles: ... angle ABD 140" Determine Angle AFD However, result
exceeds possible
Reussessing... Totaling
20
Standard Prompt, 13357 tokens
[Present the Answer, tokens] Final Answer: 20".
[Understand the Problem, 1575 tokens] Airight, so have this geometry problem here with guadrilateral ABCD So, it's a trapezoid? Wait Or maybe From the Asymptote code Wait, BC is Wait, the Asymptote code Wait, but angles at C ... But don't know that's directly helpful.
Auto Thinking Mode, 7718 tokens
[Understand the Problem, 589 tokens] Okay, so have this goometry problem here about quadrilateral ABCD... So, AFD the angle at point between points und D.
[Attempt to Solve, 662 tokens] find angle AFD Wait, actually, ... maybe cun the
Wait, but need
[Attempt Solve, 1515 tokens] To find angle AFD. muybe Wait, but Asymptote code just for drawing So, angle BAD is 110 degrees Wail, maybe I'm complicating it... Given that, let me calculate the coordinates. So, theta 80.8 degrees.
Cosines ... we know the coordinates the lengths.
[Explore Other Approaches, 10672 tokens] So, let's compute angle BAD first.. Wait ... Wait maybe we can use Ceva's Theorem Wair, maybe better consider triangle
[Explore Other Approaches, 5072 tokens] Hmm, angle AFD is approximutely 81 degrees But let me check my calculations again Altematively, maybe I made mistake Let me compute it more accurately maybe it's degrees ... Whit, mayhe can consider the entire quadrilateral and angles ... But wait, Asymptote code ... So, perhups angle AFD degrees degrees. Wait, think
angle between AF AD ... don't know angle .. something Wait, triangle ABD Hmm, perhaps can instead consider that angle AFD is 30", But let me think again Wail, perhaps using vectors Wait, but since angle BAD 3x Wait, not sure. Wait... But how does this help us? ... so will go with 30"..
... Wait, angle
degrees ... think
80 degrees answer ... So, answer
[Finalize and Present the Answer, tokens] quadrilateral ABCI) Thus, degree measure angle AFD 30.
[Present the Answer, 542 tokens| To solve the problem the degree measure of angle AFD is 80,
Figure 13: Hard Case: AutoThink solves the problem via thinking mode with repeated verification.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims the abstract and introduction, regarding ellipsis prompts, adaptive reasoning via stage RL, and improved accuracy-effiienne and trade-offs, which consistent with the paper's contributions, analyses, and experimental results (Sections 1, 3, 4).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Limitations are discussed in the last Section. Including Reward Hacking, Uncontrolled Reasoning Budget, and Unfiltered Training Data.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
21

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include formal theoretical results or proofs; the contribution is algorithmic and empirical.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper details training configurations, hyperparameters, model checkpoints, and evaluation settings in Section 4.1 to support reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
22

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes)
Justification: All experiments are conducted on publicly available datasets. Our code and training framework will be released with the submission, ensuring full reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. http/pulllggguieeee Solee//CoSSuummisiiie
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Sections 4. and 4.2 provide full details on datasets, models, training schedules, batch sizes, context lengths, and evaluation protocol.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: While average results over 16 rollouts are reported, statistical significance metrics (e.g., variance or error bars) are not explicitly presented due to computational cost.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
23

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: The compute budget for training is specified in Appendix B. Model types and context lengths are discussed, though detailed hardware specs omitted.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that NeurlPS Code of Ethics https htt////nurrrppppeee http///urrpp..ppppliiiii///////t httpp//bccc/tticcsssieeeeeeeeeee
didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the
Answer: [Yes]
Justification: The work adheres to the NeurIPS Code of Ethics. It uses publicly available datasets and does not involve human subjects or privacy-sensitive data.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
The paper implicitly discusses positive impact through inference efficiency and controllability. While not dedicated section, broader implications are noted in the conclusion and limitation.
Justification:

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymemt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: No new high-risk models or datasets are released at this stage; the work is proof-of of-nccet- for reasoning control in existing models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper uses existing open models (e.g., DeepSeek-R1, DeepScaleR) and datasets under proper citation (Section 4.1 and References), though licenses not explicitly listed.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
25

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcodehcooee.. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [No]
Justification: No new datasets or models are released in the current submission due to anonymity. Upon acceptance, we plan to release well-documented model checkpoints.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The work does not involve any crowdsourcing or human subject experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research does involve human participants.
Guidelines:
26

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: LLMs are the core of our study, and their use described throughout the paper (e.g., Distill-R1, R1,, DeepScaleR). Our method modifies their reasoning policy via reinforcement learning-
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttpa////neurreee /ttpr/ieespp///// cc//oonference///0000 /LLM) for what should or should be described.
27


Reviews:
[{"review_id": "ezdJDy3MoH", "reviewer": "Reviewer_NACH", "summary": "This work tackles the over-thinking problem in R1-style reasoning models by introducing autothink. The author develops a three-stage GRPO-based RL pipeline, to balance the thinking/non-thinking mode, refine the correct behavior, and reduce the redundant reasoning with length-aware rewards. Autothink achieves powerful performance in achieving the balance between the accuracy and the reasoning length.", "questions": "As discussed in weakness,", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n1. This work owns the clear problem motivation and neat prompt insight, which shows that a mere \"...\" can stochastically toggle reasoning, exposing controllability overlooked by prior work.\n2. A well designed multi stage RL, stage-wise reward shaping elegantly prevents mode collapse, boosts accuracy, and then trims length.\n3. The proposed method achieves the best accuracy efficiency trade-off on five standard math sets and delivers significant improvement over baselines.\n4. The plug and play applicability method works on both distilled and RL, suggesting broad utility of this idea.\n\n# Weaknesses\n1. The current evaluation is limited to mathematical reasoning; transfer to commonsense, coding, or general reasoning is untested.\n2. The author discussed that reward - hacking (reasoning after </think>), lack of global token - budget control, and dataset noise remain open issues. Discussing how to alleviate these issues will strengthen this work.\n3. Also, the training cost is only briefly compared; clearer GPU day(hours) totals compared to baselines would aid reproducibility.\n4. The multi-stage will introduce extra efforts for engineering. Is it possible to build a one-stage RL by incorporating the discussed idea into it?", "was_revised": true, "final_justification": "After carefully checking the author-reviewers' discussion, I believe most of my concerns have been resolved. And I will keep my original score.", "rebuttal": "We sincerely thank you for the encouraging and constructive feedback. We are glad that you found the motivation, prompt insight, and stage-wise design of AutoThink to be elegant and effective. We address your concerns below:\n\n---\n\n**1. Generality Beyond Mathematical Reasoning**\n\nWe appreciate the your concern regarding generalization beyond math tasks. In response, we have conducted additional evaluations of our AutoThink models on three non-mathematical benchmarks:\n\n* **GPQA** for scientific multi-hop reasoning,\n* **MMLU** for general multi-task language understanding,\n* **Live-Code-Bench** for code generation, we use the newest 20250727 version.\n\nThe results show that AutoThink-trained models **retain competitive accuracy** while reducing average token usage, suggesting that the learned **adaptive reasoning behaviors generalize beyond math**.\n\n||GPQA|MMLU|LiveCodeBench|Avg|\n|-|-|-|-|-|\n|DeepSeek-R1-Distill-1.5B|35.1/10026|49.5/2727|25.2/13372|36.6/8708|\n|AutoThink-Stage1|31.5/8889|47.7/1190|23.8/5653|34.3/5244|\n|AutoThink-Stage2|37.1/8617|48.8/1743|24.2/9647|36.7/6669|\n|AutoThink-Stage3|35.7/5659|48.8/1300|24.9/9054|36.5/5337|\n\nAlthough these findings are preliminary, they suggest that math-domain training can endow LLMs with transferable control over reasoning depth. In future work, we will explore domain-specific training to further strengthen AutoThink’s adaptability in broader tasks.\n\n---\n\n**2. Reward Hacking, Token Budget, and Dataset Noise**\n\nWe appreciate the your encouragement to further discuss the limitations we acknowledged. We agree that all three are non-trivial and active challenges in the area of reasoning control:\n\n* **Token-budget control** has been partially addressed in prior works [1,2] by designing **budget-aware reward functions** that penalize long completions. Such formulations could be integrated with AutoThink to enforce a global compute constraint in a plug-and-play manner.\n\n* For **dataset noise**, existing works ([3], etc.) shows that **curriculum learning or filtering by answer correctness or difficulty** can help improve learning efficiency. These strategies are orthogonal to our reward design and could further boost training quality when combined.\n\n* As for **reward hacking**, where reasoning continues after the `</think>` tag. we believe this can be mitigated by **explicitly penalizing reasoning-related patterns outside the `<think>` span**, or by **rewarding clean separation between thought and answer**. These ideas can be incorporated into future iterations of our reward function.\n\nOverall, we view these solutions as **complementary and composable** with our framework. We are excited to explore tighter integration of these mechanisms, and incorporate a more detailed discussion of these directions in the camera-ready version to enhance the completeness and forward-looking perspective of the paper.\n\n\n*[1] L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning. COLM 2025.*\n\n*[2] ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning. arXiv:2504.21370.*\n\n*[3] FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models[J]. arXiv:2503.17287.*\n\n---\n\n**3. Training Cost Comparison**\n\nWe thank the reviewer for the suggestion to clarify training efficiency. While we compare training steps in **Appendix B.2**, we now provide estimated **GPU-hour costs** for all methods using Distill-R1-1.5B as the base model:\n\n|Method|GPU Hours|Avg ACC/Token Length|\n|-|-|-|\n|ThinkPrune-iter-2K|\\~400 H100 Hours|49.2/3368|\n|ShorterBetter|\\~200 H100 Hours|44.7/1915|\n|**Distill-1.5B-AutoThink**|\\~700 H100 Hours|51.7/5108|\n|DeepScaleR|\\~2200 H100 Hours|56.7/5817|\n|**DeepScaleR-1.5B-AutoThink**|\\~250 H100 Hours|57.3/5277|\n\nAs shown, **AutoThink operates within the same order of compute as concise baselines like ThinkPrune and ShorterBetter, yet it achieves notably stronger performance.** In contrast, DeepScaleR, a performance-maximizing method, requires **over 3× more compute**, with significantly larger context length and RL iterations.\n\nWe also note that **AutoThink training can be completed within 24 hours** on a 4-node H100 cluster, making it **practical and reproducible** under typical academic compute environments of R1-Style model training. **Our code is fully open-sourced, and all results in the paper are reproducible**, further supporting the accessibility and reliability of our approach.\n\n---\n\n**4. On Single-Stage Design**\n\nWe appreciate your thoughtful question regarding the complexity of our multi-stage RL framework. While AutoThink adopts a stage-wise structure for clarity and training stability, we emphasize that **each stage serves a distinct and interpretable purpose**:\n\n* **Stage 1** balances dual-mode behavior via a simple linear penalty using (γ, λ), which were **not carefully tuned** but chosen as common defaults. The resulting reward curve (Figure 3) is symmetric and robust.\n* **Stage 3** prunes redundant reasoning with length-aware shaping using **default parameters (α, β)** from GRPO-LEAD, also without fine-tuning.\n\nThus, while multi-stage, the design is lightweight and requires **minimal hyperparameter tuning**.\n\nWe agree that consolidating into a single-stage RL framework is a promising direction. One potential solution is to **dynamically switch the reward formulation during training** based on simple training signals:\n\n* When the *thinking rate* stabilizes, automatically transition from Stage 1 to Stage 2.\n* When *accuracy gain plateaus*, switch to Stage 3 to prioritize efficiency.\n\nWe plan to explore such **self-adaptive reward scheduling** mechanisms in future work to reduce engineering complexity while preserving adaptive reasoning capabilities.\n\n---\n\nWe sincerely hope that these clarifications and additional results help address your concerns. If you have any additional feedback or suggestions, we would greatly welcome them and be happy to address them in detail."}, {"review_id": "Ww2SHXAiGa", "reviewer": "Reviewer_9eSD", "summary": "This paper introduces AutoThink, a framework designed to mitigate the over-thinking problem in large reasoning models, where models expend excessive computational resources on simple problems. The authors discover that a minimal \"ellipsis prompt\" (`...`) can stochastically induce R1-style models to either engage in detailed reasoning (\"think\" mode) or provide a direct answer (\"no-think\" mode). To harness this behavior, they propose a three-stage RL process.", "questions": "- How to define Thinking and No-Thinking? I did not find precise definition of this basic concept. Is it a statistic by counting the response length? This is the foundation of the work. Please provide clear definition.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- The discovery of the \"ellipsis prompt\" is a interesting. It is a simple, low-cost method to unlock latent adaptive behavior in existing R1-style models without complex architectural changes\n- The investigated overthinking problem is intime and important.\n- Writing is clear and fluent, easy to follow\n\nWeakness\n- The proposed Elipsis prompt seems as an intermieda point between thinking and no-thinking prompts, and only shows effect on DeepSeek-R1-Distill-Qwen models. The claim that it triggers selective reasoning is not validated on broader models and scenarios.\n- The 3-stage training recipe is similar to DeepScaleR. However, the resulting model starting from DS-distill-Qwen-1.5B underperforms DeepScaleR (51.7% vs 56.7% acc, 5108 vs 5817 length)\n- There are too many design choices in the proposed method, including prompt selection, 3-stage training, hyperparameters in reward design. It is hard to tell which part is essential. The ablation study is simple", "was_revised": true, "final_justification": "My major concern of this paper, the complexity and generalization ability of the proposed method, still remain unsolved. It would be hard for the academic and open-source community to learn from this work given its complixity. Thus, I maintain my boarderline rejection rating.", "rebuttal": "We thanks for your thoughtful feedback and valuable suggestions. We have carefully addressed each concern and added clarifications, analyses, and new results to strengthen our work.\n\n---\n\n**1. Definition of No-Thinking**\n\nThank you for pointing this out. In R1-style models (e.g., DeepSeek-R1), *Thinking* refers to generating a step-by-step reasoning trace within the `<think> {thinking_content} </think>` tag, typically involving continuous reflection and backtracking. This structured reasoning format has been adopted in recent large reasoning models with explicit `<think>`-style reasoning blocks to modularize thought and improve interpretability.\n\nFor the *No-Thinking* mode [1],  the model **immediately closing the `<think>` tag without producing any meaningful intermediate reasoning**, e.g., generating a shallow structure like `<think> </think>` and proceeding directly to the final answer. **It is also called as *Non-Thinking* mode in Qwen3 Technical Report.** This phenomenon often emerges under our **ellipsis prompt**, which stochastically toggles the model between reasoning and shortcut modes. Our work leverages this property as a *control signal* to investigate adaptive reasoning behavior.\n\nWe apologize for not formalizing this definition clearly in the original submission and will include more detailed definitions, examples, and connections to related work in the camera-ready version.\n\n*[1] Reasoning models can be effective without thinking. arXiv:2504.09858.*\n\n---\n\n**2. Validation beyond DeepSeek-R1-Distill-Qwen**\n\nThank you for this valuable comment. We would like to clarify that our method specifically targets at **Large Reasoning Models**. Among open-source models, **DeepSeek-R1-Distill-Qwen** is a representative R1-style distilled model trained from Qwen2.5 by using DeepSeek-R1-671B reasoning data, making it a natural choice for our main experiments.\n\nTo further validate generality, we have already included **DeepScaleR** in our experiments (Table 1), which is built upon DeepSeek-R1-Distill-Qwen-1.5B and strengthened via multi-stage context extension and RL. Even though DeepScaleR has undergone extensive fine-tuning to favor long reasoning chains, we observe that under the **ellipsis prompt**, the average token usage still drops from 5817 to 5511, suggesting that our prompt design still elicits a small amount of no-thinking behavior. Moreover, **AutoThink stages applied on top of DeepScaleR** demonstrate clear ability to further reduce token usage with minimal or no accuracy loss (see Table 1).\n\nIn addition, we conducted further validation on **Skywork-OR1-Math-7B**, a stronger RL-trained R1-style model that has undergone deeper optimization for mathematical and code reasoning. We report below the results on different prompting/training stages:\n\n|Skywork-OR1-Math-7B|Accuracy (%)|Token Usage|\n|-|-|-|\n|Standard prompt|70.5|9053|\n|Ellipsis Prompt|70.9|8933|\n|No-Thinking Prompt|54.7|2722|\n|AutoThink-Stage1|68.2|3966|\n|AutoThink-Stage2|69.7|4719|\n|AutoThink-Stage3|68.8|3974|\n\nAlthough **ellipsis prompting alone induces minimal no-thinking behavior (only ~0.5% occurrence), our method successfully encourages such behavior to emerge through structured reward shaping**, reducing the average token usage from 8933 to 3966 in Stage 1 while maintaining comparable accuracy. During Stage-1 training, we observed a sharp drop in thinking ratio and response length around step 120, indicating a successful emergence of dual-mode balance.\n\nFinally, due to time and compute constraints during the rebuttal phase, we were unable to run the full training pipeline on **Qwen3**. However, we conducted a preliminary evaluation to assess the effectiveness of ellipsis prompting, recording both accuracy and token length:\n\n|Qwen3-8B|MATH500|Minerva|Olympaid|AIME24|AMC23|Avg|\n|-|-|-|-|-|-|-|\n|Standard prompt|97.0/5351|53.3/7010|73.5/11342|86.7/14690|88.1/10343|79.7/9747|\n|Ellipsis Prompt|96.4/5109|49.5/5315|70.9/9891|68.3/13349|88.9/9858|74.8/8704|\n|No-Thinking Prompt|84.1/1104|41.2/639|50.8/2860|26.3/6518|60.1/2913|52.5/2807|\n|Ellipsis Prompt: Thinking Rate|96.9%|67.5%|89.0%|87.5%|96.2%|87.4%|\n\nThe use of the ellipsis prompt can encourage a certain degree of no-thinking behavior, and this behavior does not correlate with task difficulty (AIME problems are significantly harder than those in MATH500, yet elicit a lower thinking rate). **Based on the experimental results from the above Skywork-OR1-Math-7B (about 0.5% occurrence of no-thinking behavior with Ellipsis Prompt), we believe that the AutoThink strategy could also induce autonomous reasoning in Qwen3 (~13% occurrence of no-thinking behavior with Ellipsis Prompt)**.\n\n---\n\n**3. Why AutoThink underperforms DeepScaleR despite similar training recipe?**\n\nThis is a reasonable concern. We emphasize that **DeepScaleR and AutoThink are designed with different optimization objectives**:\n\n* DeepScaleR focuses solely on **maximizing accuracy**, leveraging a deep 3-stage RL training schedule with progressively extended context and a large computational budget (~2200 H100 GPU hours).\n* In contrast, **AutoThink is designed to trade off accuracy and efficiency**, aiming to **reduce unnecessary reasoning** by learning when to think, just costs ~700 H100 GPU hours.\n\nMore importantly, we also **apply AutoThink on top of DeepScaleR**. As shown in Table 1, this leads to:\n\n* A **reduction in average token usage from 5817 to 5277**,\n* While **maintaining comparable accuracy** (from 56.7% → 57.3%).\n\nThis shows that our method can serve as a **post-training compression module**, applicable even to models like DeepScaleR that are already highly optimized. In principle, for excellent performance and efficiency, we can first train a model with the DeepScaleR recipe to maximize performance, and then apply AutoThink to further improve inference-time efficiency without significant performance drop.\n\n---\n\n**4. Concern about too many design choices and simple ablation**\n\nWe appreciate your concern regarding the complexity of our method. We would like to clarify that the **three-stage structure is intentionally modular and interpretable**, with each component driven by distinct and simple design principles:\n\n* **Stage 1** introduces a batch-wise reward balance to prevent mode collapse between thinking and no-thinking behaviors.\n* **Stage 2** focuses purely on reinforcing accuracy within each mode (no additional reward shaping).\n* **Stage 3** adds length-aware shaping to encourage brevity for correct responses and elaboration for incorrect ones.\n\nAmong the three stages, only Stage 1 and Stage 3 involve reward design beyond naive correctness. Even then, the reward formulations are simple and intuitive:\n\n* In **Stage 1**, we balance the modal ratio using a linear penalty with hyperparameters γ and λ. These parameters were not carefully tuned, but simply set to commonly used values. As shown in Figure 3, the reward curve exhibits a natural symmetric form.\n* In **Stage 3**, we reuse default shaping terms (α, β) inspired by GRPO-LEAD, additional any revision.\n\nWe have already provided **ablations in Section 4.3 and Figure 5**, which show:\n\n* Without Stage 1, the model collapses into a single mode (thinking or no-thinking).\n* Skipping Stage 2 results in a clear drop in final accuracy (51.7% → 47.6%).\n\nWe acknowledge the lack of hyperparameter sweeps in the main paper. In additional experiments during the rebuttal phase, we find:\n\n* Increasing **γ** encourages more thinking behavior in Stage 1.\n* Higher **λ** enforces stricter adherence to the target balance.\n* Larger **α/β** in Stage 3 accelerate reward decay/growth in response_length.\n\nTo illustrate these trends, we record the *thinking ratio* measured as the running average over the most recent 20 steps at two key checkpoints (step 100 and step 200) during Stage 1 training:\n\n||γ=0.5, λ=2|γ=0.2, λ=2|γ=0.8, λ=2|γ=0.5, λ=1|γ=0.5, λ=4|\n|-|-|-|-|-|-|\n|Thinking-Rate@step100|62.4|57.8|99.9|71.4|51.7|\n|Thinking-Rate@step200|54.2|51.4|100.0|61.2|48.3|\n\nAdditionally, we record the response lengths in Stage 3 under different values of α, with β fixed at 0.05:\n\n||α=0.05|α=0|α=0.1|\n|-|-|-|-|\n|Response-Length@step100|4734|6174|3623|\n|Response-Length@step200|4120|6322|2894|\n\nThese training trends are consistent with the intended behavior of the shaping functions. We will include these findings in the appendix and release them alongside the camera-ready version.\n\n---\n\nWe sincerely hope that these clarifications and additional results help address your concerns. We would be grateful if you would kindly consider reassessing our work in light of the new evidence presented."}, {"review_id": "6512UnDo9B", "reviewer": "Reviewer_x9j5", "summary": "This paper addresses the over-thinking problem in R1-style large reasoning models (LRMs), where explicit step-by-step reasoning incurs high computational costs for simple tasks. The authors introduce AutoThink, a novel framework that leverages an ellipsis prompt to trigger stochastic thinking/no-thinking modes in LRMs, combined with a three-stage reinforcement learning (RL) framework for adaptive reasoning. The three stages include:\n- Stage 1: Stabilizing dual-mode coexistence to prevent policy collapse.\n- Stage 2: Reinforcing accurate reasoning within both modes.\n- Stage 3: Pruning redundant steps via length-aware rewards.\n\nAutoThink demonstrates superior accuracy-efficiency trade-offs on five mathematical benchmarks compared to prompting and RL-based pruning methods. The study also validates the effectiveness of difficulty-aware reasoning and the necessity of each training stage through ablation studies.", "questions": "While validated on mathematical benchmarks, does AutoThink extend to non-mathematical reasoning tasks?", "limitations": "Please refer to the weakness section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- The progressive reward shaping in AutoThink aligns with intuitive reasoning principles (e.g., balancing thinking modes, reinforcing correctness, pruning redundancy). Extensive experiments and ablation studies (e.g., mode collapse prevention, stage dependency) thoroughly validate the framework’s efficacy.\n- The research tackles a critical limitation of modern LRMs—excessive computational overhead from unnecessary reasoning—making it highly relevant for practical deployment in resource-constrained scenarios.\n### Weaknesses\n- The multi-stage RL pipeline and tailored reward functions are relatively complex.", "was_revised": true, "final_justification": "The authors’ rebuttal provided additional non-mathematical benchmark results (GPQA, MMLU, Live-Code-Bench), which, while preliminary, indicate that AutoThink’s adaptive reasoning control generalizes beyond the math domain. This directly addresses my main question about broader applicability. The explanation regarding the multi-stage RL pipeline clarified that the design is purposeful, minimally tuned, and interpretable, which alleviates concerns about unnecessary complexity. The paper remains technically solid, presents a well-motivated and validated framework, and shows strong empirical gains in accuracy-efficiency trade-offs. Given its relevance, solid evaluation, and potential for broad impact, I maintain the accept rating of 5.", "rebuttal": "We sincerely thank you for the positive and thoughtful evaluation of our work. We're especially grateful for your recognition of AutoThink’s contribution to improving reasoning efficiency and your appreciation of the progressive reward shaping design. We address your concerns below:\n\n---\n\n**1. Generality Beyond Mathematical Reasoning**\n\nWe appreciate the your concern regarding generalization beyond math tasks. In response, we have conducted additional evaluations of our AutoThink models on three non-mathematical benchmarks:\n\n* **GPQA** for scientific multi-hop reasoning,\n* **MMLU** for general multi-task language understanding,\n* **Live-Code-Bench** for code generation, we use the newest 20250727 version.\n\nThe results show that AutoThink-trained models **retain competitive accuracy** while reducing average token usage, suggesting that the learned **adaptive reasoning behaviors generalize beyond math**.\n\n||GPQA|MMLU|LiveCodeBench|Avg|\n|-|-|-|-|-|\n|DeepSeek-R1-Distill-1.5B|35.1/10026|49.5/2727|25.2/13372|36.6/8708|\n|AutoThink-Stage1|31.5/8889|47.7/1190|23.8/5653|34.3/5244|\n|AutoThink-Stage2|37.1/8617|48.8/1743|24.2/9647|36.7/6669|\n|AutoThink-Stage3|35.7/5659|48.8/1300|24.9/9054|36.5/5337|\n\nAlthough these findings are preliminary, they suggest that math-domain training can endow LLMs with transferable control over reasoning depth. In future work, we will explore domain-specific training to further strengthen AutoThink’s adaptability in broader tasks.\n\n---\n\n**2. Complexity of the multi-stage RL**\n\nRegarding your comment on the **complexity of the multi-stage RL pipeline and tailored reward functions**, we would like to emphasize that each component was introduced to serve a clear, intuitive purpose—as you summarized precisely:\n\n* **Stage 1** ensures balanced coexistence between thinking and no-thinking modes via a simple linear penalty mechanism. The associated hyperparameters (γ, λ) were **not carefully tuned**, but set to commonly used values. As shown in **Figure 3**, the reward curve is naturally symmetric and stable.\n* **Stage 3** applies a length-aware shaping strategy using **default (α, β)** values adopted directly from GRPO-LEAD.\n\nThus, while the overall pipeline appears multi-stage, each stage was deliberately designed with **minimal tuning and clear interpretability**.\n\nLooking forward, we believe it is possible to **unify these stages through a more holistic reward formulation**, enabling the model to learn adaptive reasoning behavior in a single-stage process. This is an exciting direction we intend to pursue in future work.\n\n---\n\nIf you have any additional feedback or suggestions, we would greatly welcome them and be happy to address them in detail."}, {"review_id": "rJsgt448jt", "reviewer": "Reviewer_L1QB", "summary": "The paper introduces AutoThink, a novel framework designed to imbue large reasoning models (LRMs) with adaptive thinking capabilities. Addressing the common problem of \"over-thinking\" in CoT-style models, which leads to computational overhead, AutoThink builds upon a unique observation: a simple ellipsis (\"...\") inserted into the prompt of R1-style models can stochastically trigger either a detailed \"thinking\" mode or a concise \"no-thinking\" mode. Leveraging this latent controllability, the authors propose a three-stage reinforcement learning (RL) framework to progressively optimize the model's reasoning policies. Specifically, Stage 1 stabilizes the coexistence of these dual modes, Stage 2 reinforces accurate reasoning within each mode, and Stage 3 prunes redundant reasoning via length-aware rewards. Experimental evaluations on five mathematical benchmarks demonstrate that AutoThink achieves superior accuracy-efficiency trade-offs compared to existing prompting and RL-based pruning methods, notably improving accuracy while significantly reducing token usage, and is seamlessly integrable into various R1-style models.", "questions": "1. I would like to know the relationship between **RL steps and acc** under three different reward functions (Fig. 5), which, when combined with the existing analysis of **RL steps and tokens** (Fig. 5a), could help readers better understand the comprehensive impact of these different reward functions.\n\n2. Why did the training of the `Length-Aware reward function` stop at around step 110 in Fig. 5a? Specifically, I noticed that the token usage increases after approximately step 50. What will happen when its training were continued?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "# Strengths\n1. The finding that a minimalistic, non-semantic token like an ellipsis can stochastically activate distinct reasoning modes in R1-style models is highly original and provides a surprisingly simple yet effective control mechanism.\n2. The three-stage RL design is well-conceived and robust. Each stage systematically addresses specific challenges in developing adaptive reasoning, and the ablation studies effectively validate the necessity and contribution of each phase.\n3.  AutoThink consistently demonstrates favorable accuracy-efficiency trade-offs across a diverse set of mathematical benchmarks and different R1-style models. The substantial reduction in token usage (e.g., 52% on Distill-R1-1.5B) without sacrificing accuracy is a compelling practical advantage. The introduction of the E-F1 metric is also a valuable contribution for evaluating this trade-off.\n\n# Weaknesses\n1. While impressive, the specific behavior triggered by the ellipsis prompt might be highly specific to the architectural nuances and pre-training of R1-style models. It is unclear how transferable this phenomenon is to other LLM architectures or different Chain-of-Thought formats, which might limit the generality of this specific control handle. Further theoretical or empirical investigation into why the ellipsis works could strengthen this aspect.\n2.  This paper introduces several key hyper-parameters, such as $\\gamma$, $\\lambda$ in Stage 1, $\\alpha$, $\\beta$ in Stage 3. However, the authors only state the empirical values without providing any sensitivity analysis for different setting of these parameters. This makes it difficult for readers to understand the impact of different settings.", "was_revised": true, "final_justification": "The detailed experiment in the rebuttal has addressed my main concerns.", "rebuttal": "We sincerely thank you for the encouraging and constructive feedback. We are glad that you found the motivation, prompt insight, and stage-wise design of AutoThink to be elegant and effective. We address your concerns below:\n\n---\n\n**1. On the Generality of the Ellipsis Prompt and Applicability to Other Models**\n\nOur method is specifically designed for Large Reasoning Models (R1-style models), \n\nThank you for this valuable comment. We would like to clarify that our method specifically targets at **Large Reasoning Models**, where *Thinking* refers to generating a step-by-step reasoning trace within the `<think> {thinking_content} </think>` tag, typically involving continuous reflection and backtracking. This structured reasoning format has been adopted in recent large reasoning models with explicit `<think>`-style reasoning blocks to modularize thought and improve interpretability.\n\nAmong open-source models, **DeepSeek-R1-Distill-Qwen** is a representative R1-style distilled model trained from Qwen2.5 by using DeepSeek-R1-671B reasoning data, making it a natural choice for our main experiments.\n\nTo further validate generality, we have already included **DeepScaleR** in our experiments (Table 1), which is built upon DeepSeek-R1-Distill-Qwen-1.5B and strengthened via multi-stage context extension and RL. Even though DeepScaleR has undergone extensive fine-tuning to favor long reasoning chains, we observe that under the **ellipsis prompt**, the average token usage still drops from 5817 to 5511, suggesting that our prompt design still elicits a small amount of no-thinking behavior. Moreover, **AutoThink stages applied on top of DeepScaleR** demonstrate clear ability to further reduce token usage with minimal or no accuracy loss (see Table 1).\n\nIn addition, we conducted further validation on **Skywork-OR1-Math-7B**, a stronger RL-trained R1-style model that has undergone deeper optimization for mathematical and code reasoning. We report below the results on different prompting/training stages:\n\n|Skywork-OR1-Math-7B|Accuracy (%)|Token Usage|\n|-|-|-|\n|Standard prompt|70.5|9053|\n|Ellipsis Prompt|70.9|8933|\n|No-Thinking Prompt|54.7|2722|\n|AutoThink-Stage1|68.2|3966|\n|AutoThink-Stage2|69.7|4719|\n|AutoThink-Stage3|68.8|3974|\n\nAlthough **ellipsis prompting alone induces minimal no-thinking behavior (only ~0.5% occurrence), our method successfully encourages such behavior to emerge through structured reward shaping**, reducing the average token usage from 8933 to 3966 in Stage 1 while maintaining comparable accuracy. During Stage-1 training, we observed a sharp drop in thinking ratio and response length around step 120, indicating a successful emergence of dual-mode balance.\n\nFinally, due to time and compute constraints during the rebuttal phase, we were unable to run the full training pipeline on **Qwen3**. However, we conducted a preliminary evaluation to assess the effectiveness of ellipsis prompting, recording both accuracy and token length:\n\n|Qwen3-8B|MATH500|Minerva|Olympaid|AIME24|AMC23|Avg|\n|-|-|-|-|-|-|-|\n|Standard prompt|97.0/5351|53.3/7010|73.5/11342|86.7/14690|88.1/10343|79.7/9747|\n|Ellipsis Prompt|96.4/5109|49.5/5315|70.9/9891|68.3/13349|88.9/9858|74.8/8704|\n|No-Thinking Prompt|84.1/1104|41.2/639|50.8/2860|26.3/6518|60.1/2913|52.5/2807|\n|Ellipsis Prompt: Thinking Rate|96.9%|67.5%|89.0%|87.5%|96.2%|87.4%|\n\nThe use of the ellipsis prompt can encourage a certain degree of no-thinking behavior, and this behavior does not correlate with task difficulty (AIME problems are significantly harder than those in MATH500, yet elicit a lower thinking rate). **Based on the experimental results from the above Skywork-OR1-Math-7B (about 0.5% occurrence of no-thinking behavior with Ellipsis Prompt), we believe that the AutoThink strategy could also induce autonomous reasoning in Qwen3 (~13% occurrence of no-thinking behavior with Ellipsis Prompt).**\n\n---\n\n**2. Hyperparameter Sensitivity**\n\nWe appreciate your concern regarding the complexity of our method. We would like to clarify that the **three-stage structure is intentionally modular and interpretable**, with each component driven by distinct and simple design principles:\n\n* **Stage 1** introduces a batch-wise reward balance to prevent mode collapse between thinking and no-thinking behaviors.\n* **Stage 2** focuses purely on reinforcing accuracy within each mode (no additional reward shaping).\n* **Stage 3** adds length-aware shaping to encourage brevity for correct responses and elaboration for incorrect ones.\n\nAmong the three stages, only Stage 1 and Stage 3 involve reward design beyond naive correctness. Even then, the reward formulations are simple and intuitive:\n\n* In Stage 1, we balance the modal ratio using a linear penalty with hyperparameters γ and λ. **These parameters were not carefully tuned, but simply set to commonly used values.** As shown in Figure 3, the reward curve exhibits a natural symmetric form.\n* In Stage 3, we reuse default shaping terms (α, β) **inspired by GRPO-LEAD, additional any revision**.\n\nWe have already provided **ablations in Section 4.3 and Figure 5**, which show:\n\n* Without Stage 1, the model collapses into a single mode (thinking or no-thinking).\n* Skipping Stage 2 results in a clear drop in final accuracy (51.7% → 47.6%).\n\n**We acknowledge the lack of hyperparameter sweeps in the main paper. In additional experiments during the rebuttal phase, we find**:\n\n* Increasing **γ** encourages more thinking behavior in Stage 1.\n* Higher **λ** enforces stricter adherence to the target balance.\n* Larger **α/β** in Stage 3 accelerate reward decay/growth in response_length.\n\nTo illustrate these trends, we record the *thinking ratio* measured as the running average over the most recent 20 steps at two key checkpoints (step 100 and step 200) during Stage 1 training:\n\n||γ=0.5, λ=2|γ=0.2, λ=2|γ=0.8, λ=2|γ=0.5, λ=1|γ=0.5, λ=4|\n|-|-|-|-|-|-|\n|Thinking-Rate@step100|62.4|57.8|99.9|71.4|51.7|\n|Thinking-Rate@step200|54.2|51.4|100.0|61.2|48.3|\n\nAdditionally, we record the response lengths in Stage 3 under different values of α, with β fixed at 0.05:\n\n||α=0.05|α=0|α=0.1|\n|-|-|-|-|\n|Response-Length@step100|4734|6174|3623|\n|Response-Length@step200|4120|6322|2894|\n\nThese training trends are consistent with the intended behavior of the shaping functions. We will include these findings in the appendix and release them alongside the camera-ready version.\n\n---\n\n**3. On the Reward Impact and Early Stopping in Figure 5a**\n\nWe appreciate your question regarding the relationship between RL steps and accuracy in Stage 1. First, we would like to clarify that the primary objective of Stage 1 is to stabilize the coexistence of *thinking* and *no-thinking* modes, rather than to directly improve accuracy. The effects of different reward coefficients have already been discussed in detail in the previous section.\n\nIn Figure 5a, we observe that all three reward variants generally show upward trends. Among them, the **naive reward** increases slightly faster than the **batch balance reward**, as it implicitly encourages fully \"thinking\" responses to achieve higher accuracy.\n\nIn contrast, the **length-aware reward** exhibits a non-monotonic pattern: it decreases initially and then gradually increases. This trend aligns closely with the change in response length. Our interpretation is that the length-aware reward encourages shorter, more concise answers. In the early stages of training, **when the model’s overall accuracy is still low, responses with short and no-thinking mode tend to receive higher rewards**, leading the model to collapse into a no-thinking mode. As training progresses, however, **longer no-thinking responses tend to be more accurate, guiding the policy to optimize toward generating longer responses**, resulting in a moderate increase in response length.\n\nWe observed that after step 110, training under the length-aware reward leads to a **stable response length range**. However, since the model remains largely no-thinking single-modal, its accuracy is generally lower than that of models trained with the batch-balance or naive reward.\n\nTo improve clarity, we will include RL step–accuracy curves and more detailed discussions in the camera-ready version, offering a clearer picture of how each reward function influences learning dynamics.\n\n---\n\nWe sincerely hope that these clarifications and additional results help address your concerns. If you have any additional feedback or suggestions, we would greatly welcome them and be happy to address them in detail."}]

Author rebuttal:

Meta-review: Summary of the paper: This paper claims that an ellipsis prompt (“…”) inserted in R1-style models can stochastically toggle between a “think” (CoT) and a “no-think” (direct-answer) mode, providing a simple control signal for adaptive reasoning. Building on this observation, the authors design AutoThink, a three-stage GRPO-based RL pipeline: Stage 1 stabilizes coexistence of the two modes to avoid policy collapse, Stage 2 reinforces correctness inside each mode, and Stage 3 employs length-aware rewards to prune redundant reasoning. Across five math benchmarks, AutoThink yields the best reported accuracy-efficiency trade-off, cutting average response length by up to 52 % while matching or surpassing baselines in accuracy; these gains transfer to both distilled and RL-trained R1-style models. Ablations confirm that every stage is necessary and that the method mitigates mode-collapse, reward hacking, and dataset noise, although global token-budget control and broader-domain transfer remain open questions.

Strengths of the paper: 
1. Simple but interesting discovery: an ellipsis prompt alone can stochastically switch reasoning modes in R1-style models, offering a low-cost, plug-and-play control handle.
2. Straightforward design: the proposed three-stage RL design with progressive reward shaping that systematically prevents collapse, improves accuracy, and trims length; ablations validate each stage.
3. Strong empirical results: achieves sota accuracy-efficiency on five math benchmarks, with substantial token savings and consistent gains across model sizes.
4. Presentation: fluent writing and extensive ablations make the contribution transparent and reproducible.

Weaknesses of the paper: After the rebuttal, I think the authors have addressed most of the concerns raised by three reviewers. Reviewer 9esd still has major concerns during the area chair-reviewers discussion phase:
1. Validity and depth: the proposed methods lack technical and theoretical depth. The ellipsis prompts, reward design in RL and multi-stage pipelines are mostly heuristics.
2. Improvement over existing work: there are already many works exploring controlling response lengths via prompting (cot, pot), RL (L1, ThinkPrune), and multi-stage training (DeepScaleR). 
3. Overdesigned pipeline. AutoThink intruduced a three-staged training pipeline, which is too complicated for the community to apply. Moreover, there are too many hyperparameters to tune under this framework. The training steps across runs also varies, and the author did not propose any method to determine them. From this persipective, currecnt experimental results, which are limited on DeepSeek distilled Qwen models, do not sufficiently support their claims.

Reasons for the decision: Reviewer 9esd raises valid concerns, and I concur with their assessment of the paper’s technical depth and theoretical contribution. It seems to me that the authors addressed the hyper-parameter-tuning issue by adding a focused analysis of design choices along with a concise ablation study. They have also expanded the experimental scope to include Skywork-OR1-Math-7B and Qwen-3-8B. These revisions meaningfully strengthen the work, and—after weighing the recommendations of the other three reviewers—I am inclined to accept the paper.
Decision: Accept (poster)