Controlling Thinking Speed in Reasoning Models
Authors: Zhengkai Lin
Zhihang Fu
Ze Chen
Chao Chen
Liang Xie
Wenxiao Wang
Deng Cai
Zheng Wang
Jieping Ye
Venue: NeurIPS 2025 spotlight ()
Keywords: Large Language Models
Interpretability
Reasoning strategy
Primary area: deep_learning

Abstract: Human cognition is theorized to operate in two modes: fast, intuitive System 1 thinking and slow, deliberate System 2 thinking.
While current Large Reasoning Models (LRMs) excel at System 2 thinking, their inability to perform fast thinking leads to high computational overhead and latency.
In this work, we enable LRMs to approximate human intelligence through dynamic thinking speed adjustment, optimizing accuracy-efficiency trade-offs.
Our approach addresses two key questions: (1) how to control thinking speed in LRMs, and (2) when to adjust it for optimal performance.
For the first question, we identify the steering vector that governs slow-fast thinking transitions in LRMs' representation space.
Using this vector, we achieve the first representation editing-based test-time scaling effect, outperforming existing prompt-based scaling methods.
For the second question, we apply real-time difficulty estimation to signal reasoning segments of varying complexity.
Combining these techniques, we propose the first reasoning strategy that enables fast processing of easy steps and deeper analysis for complex reasoning. 
Without any training or additional cost, our plug-and-play method yields an average +1.3\% accuracy with -8.6\% token usage across leading LRMs and advanced reasoning benchmarks.
All of our algorithms are implemented based on vLLM and are expected to support broader applications and inspire future research.

Controlling Thinking Speed in Reasoning Models
Zhengkai Lin', Zhihang Fu²; Ze Chen', Chao Chen', Liang Xie Wenxiao Wang" Deng Cai', Zheng Wang", Jieping Ye²
State Key Lab of CAD&CG, Zhejiang University, Allibaba Cloud
Coolleee of Computer Science and Technology, Zhejiang University of Technology "School of Software Technology, Zhejiang University
Abstract
Human cognition is theorized to operate in two modes: fast, intuitive System thinking and slow, deliberate System thinking. While current Large Reasoning Models (LRMs) excel at System 2 thinking, their inability to perform fast thinking leads to high computational overhead and latency. In this work, enable LRMs to approximate human intelligence through dynamic thinking speed adjustment, optimizing recuracy-efficience trade-offs. Our approach addresses two key ques- tions: (I) how to control thinking speed in LRMs, and (2) when to adjust it for optimal performance. For the first question, we identify the steering vector that governs slow-fast thinking transitions in LRMs' representation space. Using this vector, we achieve the first representation editing-based test-time scaling effect, outperforming existing prompt- prompt-based based scaling methods. For the second question, we apply real-time difficulty estimation to signal reasoning segments of varying complexity. Combining these techniques, we propose the first reasoning strategy that enables fast processing of easy steps and deeper analysis for complex reasoning. Without any training or additional cost, our plug-in module delivers an average +1.3 1.3% accuracy with -8.6% token usage across leading LRMs and advanced rea- soning benchmarks. All of our algorithms are implemented based on vLLM and are expected to support broader applications and inspire future research
Introduction
Cognitive theory categorizes human thinking into two systems: the fast, intuitive System and the
slower, deliberate System 2 (39 13 Large Language Models (LLMs) exhibit similar specialization.
The chain-of-thougeee (CoT) 26 generated by models like GPT-4o 10 and DeepSeek V3 (6] exhibits fast, intuitive thinking that is ideal for routine tasks but limited in advanced reasoning. In contrast, the slower, more deliberate long thinking utilized by Large Reasoning Models (LRMs), such as DeepSeek-RI 1 7 and OpenAl's ol series [1]. demonstrates superior performance on advanced tasks. However, this advantage comes at the cost of largely increased computational overhead. This contrast motivates our core research question: How can we combine the advantages of both System / and System 2 thinking within one model, thus simultaneously achieving both efficiency and accuracy?
Prior work integrates System and System 2 thinking into one model by fine-tuning on both fast and slow reasoning traces 28 33 Alternatively, we argue that even without training, some LRMs intrinsically possess both slow- and fast-thinking abilities. Through statistical analysis of LRMs' responses, we notice that the slow and fast outputs consistently start with distinct opening words. This built-in feature provides a natural switch for activating different thinking modes within a single LRM. However, achieving human-levee level cognition requires not only the possession of both System and System 2 thinking modes, but also the capacity for dynamic transitions between the 2 modes during thinking. Building on these insights, we refine our core research focus into two key issues:
Conresponding authors. Email: zhihang. www.@aaibbaaiinecoooooo wenxiaowang @ wdw..cu..cn.... Cooe available at: https: //gi thub. com com/ /Dt.aD2I--aa httpl/aat/htiiiiinggee speed- control
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Question: What the value of 1-2+3+4?
a:
find of Eart-Thinsttg
first
Intevention
number: 3 plus 2 3
Nost, add the resutt the therd number 3 sike 5 equel
Fester
Better Response Speed
No.
Stom Thinking
dies, so need to figuroo the waiee of's plas plus pkkss 4 mirrr,, shat doon't seer too bud. bet me just write it down dep byslap make surnl don't make minuas.
Normal
Default
Normal Response
Inference-Timme
Rist tratis (Na thetiigh?? haal, so sovnerhhew have Nest, ineed add to that reult: So, plur 3 Hmm. that stould 6. Let count fligers to chock.
and 21
a dalintuus 3. Dhay
Reading Vector
Intervention
A:
6.66 You, correct...
Slower
Deeper Thought & Better Accuracy
Figure 1: The framework of our thinking speed control method. We utilize the System 1 and System 2 thinking modes of LRMs sample fast- and slow-thought pairs. We then extract the steering vector that governs the transition between these thinking modes from LRMs, which enables us to steer the LRMs toward either fast-thinking for efficiency or slow-thinking for better accuracy.
1. How can we implement transitions between slow- and fast- thinking during LRMs' reasoning?
2. When should we transition to fully utilize the advantages of both thinking modes?
To address the first issue, we identify LRMs' internal representations for different thinking modes using representation engineering techniques [45). As shown in Figure [ the extracted reading vector enables smooth control over the thinking modes, steering LRMs towards either faster thinking (producing concise responses) or slower thinking (generating complex outputs). We term this method "thinking speed control", it governs LRMs' thinking behaviors (i.e., the succinctness of each rea- soning step) and thus determines the efficiency of their derivations of final answers. We demonstrate the effectiveness of our method through promising inference-time scaling effect: (1) responses become more concise under fast-thinking steering while maintaining superior performance compared traditional budget-based methods, and (2) the thoughts become longer and accuracy gradually improves under slow-thinking steering, successfully surpassing both the original performances of LRMs and other inference-ting scaling methods.
For the second issue, ideally, one would expect LRMs to quickly skim over the easy parts of the reasoning and spend more time (tokens) on conquering the difficult parts. Guided by this intuition, first propose a real-time reasoning difficulty estimation method via contrastive decoding (5]. Using this measurement as signal, develop dynamic reasoning strategy that allows us adaptively adjust the thinking speed during inference, accelerating through straightforward segments while slowing down for difficult reasoning. This strategy not only improves the base LRMs' performance in terms of both efficiency and accuracy, but also highlights the potential paths for thinking speed control methods for future LRM reasoning enhancements.
To summarize, our contributions include:
We identify and characterize an intrinsic switching mechanism between slow- and fast- thinking modes in LRMs, revealing their native capacity for speed adaptation (Section (
We develop the first reasoning speed control method for LRMs, which demonstrates a
promising test-time scaling effect that enables either accelerated reasoning speed improved accuracy, depending on user-defined requirements (Section B}
We propose the first dynamic reasoning strategy that adaptively adjusts the thinking speed of LRMs during inference, achieving both efficiency and accuracy (Section
Notably, all our representation editing algorithms and experiments are implemented using the vLLM framework [15]. Benefiting from this foundation and the plug-in design of our method, this inference- time technique can be seamlessly integrated into existing LLM deployment systems. Our method is expected to support broader applications and inspire further research.
2 An intrinsic switch between fast- and slow-thinking
We first demonstrate that some LRMs inherently think in both fast- and slow-thinking modes, and there exists a switch to flip between these modes. We study the thought processes (i.e., the text within i think>> and "</think>"") produced by DeepSeek- DeepSeek-R1-Distill-e -ieesl [7] on the MATH-500 g

---

V.S. Fast- Thinking
DeepSeeke RI Queeill-
DeepSekk-R-- Maeedk-R--Diiiil---o Qwen-22
Slow
100
12000
Nam. Tokens
Pansiji (%%
8000
ااااا Figure 3: Comparison of LRMs' performances with and without leading words restriction. Initiating thought with "To" significantly reduces token usage for LRMs (4.4x reduction) while maintaining comparable performance to regular responses, given the substantial token reduction.
0000
1000
2000
34
M-TH-
GPQA LiveCodeBerch
MATH-500
GPQA LiveCodeBench
benchmark. Specifically, we analyze the leading word frequencies in the top- top-100 shortest responses (average length: 262.5 tokens) and the top-100 100 longest responses (average length: 24,389.2 tokens). As shown in Figure 2] the shortest responses consistently start with leading words like "To" or "First," " while the longest re- sponses begin with "Okay" and "Alright." Given this observa- tion, we hypothesize that different leading words at the begin- ning of LRMss' thought processes might determine the length of their reasoning. To verify this, we use the following prompt template that forces LRMs to start their reasoning with the most common opening words from their shortest responses. We then observe and evaluate how LRMs' thought processes affected by the restriction of the leading words:
Top-100
Top-100 longest responses
shortest responses
40%
First
SKA
44%
Alright
Figure 2: Leading words statistics MATH-500 from responses of DeepSeek-BR DeepSeek-RI-Disill-oee -1-iis-il--------------

> [instruction] < Assistant <hhnkkikknn
Compared to the official recommended template, the only modification we make is appending an additional "To" after the thought- thought-begin symbol. Our experimental results show that this minor change leads to a substantial transformation in the output length of our testing LRMs.
Experimental settings We experiment with widely-used LRMs, namely DeepSeek-R1-Distte Qwen-7B and PeepSeek-RR--isstll---Oee
(7]. We evaluate these models on AIME24 [21],
MATH-500 (8). GPQA Diamond (27 and LivcCodeBench (release __e [12]. These benchmarks cover various reasoning skills across multiple disciplines, including math, biology, physics, chemistry, and coding. For each question, we sample responses from LRMs using both the default chat template (with no leading words restriction) and our testing template. For all models, the maximum generation length set to 32,768 tokens. During sampling. we use a temperature of 0.6, a top-p value of 0.95, and generate 8 responses per query to calculate Pass@ @ (i.e., accuracy).
Results and analysis The quantitative comparison between LRM responses under different leading words shown Figure B] including both the number of tokens per response and the Pass performance the test benchmarks. We notice that: (1) The token consumption of responses starting with "To"" is significantly lower than that of the regular template, achieving 5.4x compression for the 7B model and 3.3x for the 32B model (averaged across benchmarks). (2) Given the substantial token reduction, the shorter responses still preserve 60% (7B) and 68% (32B) of the original model's accuracy on reasoning- ceasoning-intensive intensive tasks, demonstrating their potential for answering simple or routine requests. An illustration of LRM responses under different Icading words is provided in Figure 4] The thought process initiated by "To" exhibits vanilla chain-of-thuuge (CoT) (26] 40 produced by chat models, with direct and linear reasoning style analogous to System thinking. In contrast, the standard LRM output encompasses comprehensive planning and iterative verification, characteristic of System 2 thinking. Based on the succinctness of language and the efficiency in deriving intermediate results and the final answer, we term the first reasoning style as fast-thinking and the latter as slow-thinking. Additional case studies on these two thinking modes are provided in Appendix D. The intrinsic presence of both System and System thinking modes in LRMs motivates our development of a more flexible thinking speed control method in the following section.

---

Output
Question: What is the value of 1+2+3+4
Layer
Fost Fost-Thinie Dast-hhinking
sum
4, adding
numbers: plus 2 equals
Layer n-1
Nert iddd the result to the third number: 3 plus equals 6.
Boo eegg
Finally, add the last number the current total: 5 plus equals 10.
Layer n-2
Slow Thinking
Intervention ht c ht v
Clon, so need figure out the value of plus plus plus 4. Himm,, that dossn't seam too bad. Let me writs it make sure don't make mistake.
Readling Vector
Layer
plus
that's 3. Wait. that right? Yeah, plus 2 definitely Okay so now hove Next, nocd to add that result, So, plus Himm, that should be me count on my fingers to double-dheck. plus i 6. that's cornect...
Figure 4: Illustration of our representation engineering process. We extract the directional vector corresponding to the transition from slow-thinking thinking to fast-thinking in the representation spaces of LRMs by contrasting fast and slow thoughts. During inference, strategically inject this vector manipulate the model's thinking behavior.
Thinking speed control via representation engineering
To resemble human cognitive activity, need more flexible control for transitioning between System I and System thinking. Such nuanced control would enable more options for efficiency-accuracy trade-offss and enhance reasoning depth beyond current LRM capabilities. We term this approach "thinking speed control", as it generally affects the reasoning style of LRMs' thought processes and thus the efficiency in deriving answers. To achieve this, we adopt the representation engincering framework (RopE) 45) to identify the directional vector representing the transition between fast- and slow-thinking modes in LRMs' representation spaces 2 Specifically, our control framework can be decomposed into two key steps (Figure 4): (1) reading the direction pointing to the desired thinking mode the representation space, and (2) controlling the LRM's neural activations during inference.
3.1 Representation reading
RopE posits that abstract cognitive functions are encoded as linear directions in LLMs' representation space. We hypothesize that distinct reasoning styles are also organized along separate directional subspaces. To compute the vector governing the transition from slow-thinking to fast-thinking, we (i) design stimuli elicit each thinking mode, (ii) collect hidden representations, and (iii) compute contrasting vectors and extract the first principal component using Principal Component Analysis.
Step 1: designing stimuli The stimuli should comprise both positive and negative stimuli to elicit opposing behaviors from the LRM. Given an input prompt gi, we choose the fast response as the positive stimulus, denoted as T = (g,, af ). The corresponding negative stimulus is the same prompt paired with the slow response, denoted T (gi, a²). An example pair of fast- and slow-thinking stimuli is shown in Figure 4 Following [45], we retain the initial segment of both fast and slow thinking processes create the input stimuli. More details are provided in Appendix A.1
Step 2: collecting hidden representations We extract the target LRM's hidden states from each stimulus pair. Specifically, given layer and a stimulus pair S, = (T, TT),, we process each input stimulus through the model and collect the hidden states (h h/-) at the final token position of the positive and negative stimuli, respectively.
we employ a fully unsupervised approach to compute the reading vector corresponding to the transitions from fast- to slow-thinking modes across all stimulus pairs. Denote the hidden state pairs obtained in Step 2 as:
Step 3: constructing the PCA model Following 45
{(hi+, h1-, (h2+, , (1nr,h22)) h2-),-. (h hn-))
where layer superscript is omitted for clarity. For half of the hidden state pairs, we compute the difference vector within each pair de -+++ hü+ hę-, For the remaining half of the pairs,

---

compute the reversed difference d 11 litt. We then perform PCA on the combined set of both difference directions {d} + +) u {dj 1d/ The first principal component from this PCA optimally aligns with our desired slsst direction (i.e., v d --++ 0) and contrasts with the reversed direction (i.e., v² a= (d(++>) < 0). This component thus serves as our reading vector of for layer More details can be found in Appendix A.2
3.2 Representation controlling
After obtaining the contrastive direction vector v, we can steer LRM's reasoning styles by injecting this vector into its hidden states during inference. As shown in Figure specifically, for each target intervention layer E we modify the hidden state h' with steering intensity at
h + h² -
(1)
Unless otherwise specified, we apply this intervention during the generation of every token in the response. More details can be found in Appendix A..3 Ideally, an effective control should produce:
Shorter and more concise response when > (enhancing fast-thinking);
More complex and deliberate reasoning when a < 0 (encouraging how-thinking).
The following experimental results demonstrate the success of our controlling method, which exhibits an inspiring scaling effect. As the length of the reasoning increases (i.e., the slower LRMs think), the final performance various reasoning benchmarks improves accordingly.
3.3 Experimental results and analysis
Experimental settings We conduct the experiments on 4 LRMs: CeepSeek-RI-Distiine Reei--iisill--ee--222
QwQ-32B (34] and Qwen3-8B (33). For representation reading, we use only the MATH training set (7.5k math problems) to sample both fast and slow responses from the LRMs. Then utilize the sampled responses to construct stimulus pairs and collect layer-wise hidden states from each model. Finally, we compute the reading vectors for each LRM via PCA.
To test the effect of our reading vectors, apply the control to the LRMs AIME24, MATH-500, GPQA Diamond, and LiveCodeBench, which cover math reasoning, biology, physics, chemistry, and coding. We sweep the stecring intensity in Equation across positive and negative values to evaluate the LRMs' performances when their thought processes are accelerated or decelerated.
For comparison, we utilize the following baselines:
Forcing the LRM early-exit its reasoning by appending the string "Final Answer:" to its current trace to prompt the final answer We align the early exit position the response length of each LRM under different a values for comparison. Thought Extrapolation 25 Extending the reasoning trace by appending the string "Wait" to the LRM's solution, encouraging the model to reflect its current answer. We run this baseline by recursively appending Ix/2x/3x times of "Wait" to the LRMs' last generations.
Budget Forcing 25
43
For all experiments, we use vLLM [15] and set the maximum generation length to 32.768 tokens. To avoid randomness, we generate responses per query, with the temperature set 0.6 and a top-p value of 0.95, as officially recommended [7]. The evaluation code is adapted from Sky-TI [32]. More details can be found in Appendix A.4 We run all the experiments with NVIDIA A.00 80GB GPUs.
Results and analysis A series of responses generated by test model under different steering intensities are shown in Appendix D.2. We observe a clear spectrum of output styles as a changes. When >> 0 increases, the reasoning steps become increasingly intuitive, featuring straightforward step-by-step derivations with structured mathematical expressions. Conversely, as a < 0 decreases, each reasoning step becomes more deliberate and careful, with increasing amount of validations and backtrack strategies. Such reasoning style changes align with our expectations of the thinking speed control method. The quantitative performances shown in Figure exhhibit clear scaling effect:
Note that budget forcing works only for questions whose final answers can be expressed using few tokens, such numbers options. Therefore, this baseline is omitted LiveCodeBench, it is coding benchmark.
5

---

Origind (o
16
12
Budget Forcing
IxWait
2xWait
SKWait
DeepSeek-R -iiss--ll--iiiieteeen
MATH-500
AIME24
GPQA Diamond
LiveCodcBench
86-
360
Pass (%)
47.5
45.9
42.3
45
-01.
37.5
780
35.9
76
32.5
1650
DeepSeek-RI Bereekk-R--iisiiie -isttill-QQen-222
72.5
Pass (%)
70.0
55-
45
6255
60.9
67.5
50
355
45.0
QwQQ32B
Passi (%)
02-
-000
4800
6900
7000
11000 1500e 19300 19300
1900
Ie0en
4.00
1009
10000
5053 1991
7500
Response Length (Avg. Num. Tokens per Response)
Figure 5: Scaling effects thinking speed control. We show the trade-off between response length (x-axis, average token count) and reasoning accuracy (y-axis, Pass 1). Key annotations: (1)"*": Baseline model performance. (2)" Representation control with different steering intensity a (positive to negative). (3)"*", Budget-forced early exiting at varying positions. (4)" Thought extrapolation by appending 1x/2x/3x times of "Wait". Our control method consistently shows superior performance compared to baselines. The table version of the above results is presented in Table A4
all LRMs' performances improve with increased response lengths, ultimately exceeding their original performance through our control method. The results on Qwen3-8B are shown in Figure A3 When accelerating thinking (a 16, 12, 8, 4), we observe clear advantages over the carly-exiting baselines. with averaged Pass@ improvements of ++++1..00 +11.4/11.22 10.7 //.2 respectively. We further demonstrate the superiority of our acceleration method when combined with parallel search in Appendix C Slowing down the thinking speed generally yields higher accuracy than the original responses, with Pass @ improvements of benchmarks. It also clearly outperforms prompt-based thought extrapolation baselines, which can even degrade performance as reflection time increases. We also notice that different LRMs have varying sensitivities to both representation- and prompt-based control (e.g., thought extrapolation). This results in uneven changes in the response lengths of different models as or reflection time varies. We leave the development of more uniform control over the response length for future work.
for a 2// 4/ 6, averaged over all models and
The success of control method stems from two key advantages: (1) token-level granularity and (2) representation--aasee operation. The token-level control enables precise, continuous steering of reasoning styles throughout the generation. By working directly with the internal representations instead of the reasoning text, our approach preserves the natural reasoning flow, better leveraging LRMs' inherent capabilities and thus surpassing numan-crafter prompt design baselines.
4 Adaptive control of thinking speed
Our thinking speed control experiments show that LRMss reasoning processes can either acceler- ated for faster responses or decelerated for deeper thought through constant steering. However, the

---

Okey let me Higsed oue thes penblom sep Ne we home list peuative inspers wit the followere respamees
The suen of Dhe Homm 3..0
arczee mode is Thee more arpeare more breee than any other member the and m other number same trogancy Mt Dot apper:
molian pollive Enoger
Analyze problem
Itnn
boek down.
Recall knowledge
Fist
recall waaa exch kam
Mide The number that appeare moe Irscueetty in the lid Since it's the theluriqge mode appiers mose time than any other number
Molan The uddele value of the wacs It oodcns bron smalles to ligest the lia has an even number of cleners. the medion the average of bon number Honosarr poblan aaso that matan postice snaper the drem srpar in the bet motlan mad sourge
awo
sumbers
everige zo knoger pesiedr
Timofice
Wolr
fur conrent
Wat
Calculation
Yo
Der
me ceetim If theee
unsther
posetlle ber with
Reasonings Rechecks
bode
Figure 6: Visualization of our computed reasoning difficulty in DeenSeek-RI-Distill--veeeee outputs. We highlight tokens with high reasoning difficulty defined in Equation (4) using " T", with darker shades indicating higher values. Tokens with lower reasoning difficulty highlighted using with darker shades indicating lower values.
thinking speed of real humans varies throughout the reasoning process. We quickly skim over the straightforward parts the solution and spend more time deliberating on the challenging parts in the reasoning. To better approximate human-like reasoning, we propose dynamic reasoning strategy that adaptively adjusts the thinking speed within one single reasoning trace to achieve both accuracy and efficiency. As pioneering work, hope our method will inspire future research into inference-time thinking control, leading to enhanced reasoning performances of current LRMs.
4.1 Adaptive speed control via difficulty estimation
To balance between efficiency and accuracy, need to find real-time signal (e.g., "traffic light") during inference that indicates when it is safe to process quickly and when is necessary to slow down and think more carefully. LLMs have been shown to primarily process high-level semantics in later layers, resulting in high logit variations between early and late layer predictions when processing complex information 5 233. Based on this observation,
hypothesize that reasoning involving high difficulty or complex strategies that enhance LRMs'' accuracy, such as reflections and analysis, should also involve high-level semantic processing. Such features would make difficult reasoning segments detectable through logit variation patterns. Given a token sequence (I), ... I;,-1 1 and an N -llayer transformer, we denote the output of the I-th layer as ht. The vocabulary head @(-) computes the next-token distribution:
p( lx < t) = softmax(o(11))
(2)
For defined set of early layers L {1..... .N 1}. we measure the difference between each carly-layer distribution pi pr(-la (+) and the final layer's PM (+12 using Sensen-Shannow divergence:
D (-(xe)), pr(-(==)) = JSD (pN (+((())))))
We the average divergence across early layers quantify the reasoning difficulty at token Øy;
d(x,)= = avg (- (()(()))))) leL
Higher values of d(xz)) should indi- cate greater reasoning difficulty position t. To validate our hypoth- esis, first identify the top 100 tokens with the highest logit varia- tions between the first half of the LRM's layers and the last layer in math reasoning outputs. We cat- egorize these tokens based their most-related reasoning behaviors Table We also visualize the reasoning difficulty distribution 44) in Figure 6 Transitions from low-difficulty (green) to high-difficulty (red) regions
Table I: (Part of) Top 100 tokens with high variations in logits obtained from early and late layers.
Category
Top Tokens
Reflections
Wait, Alternativel. However, mistakes, Equation, equals, multiply, approx, sequentially, What, need, analysis. ...
Calculations
Analysis
using Equation
7

---

Table Results from our adaptive speed control experiments. Our adaptive control method (high- lighted by consistently outperforms the original LRMs in both accuracy and token usage.
Performance (Pass@1 (%)r Num. Tokens L)
Methods
MATH-500
AIME24
ATME25
GPQA Diamond
DeepSsel--K--Distiil---eee-----
92.9 3403,9
52.5 / 12451.2
40.0 13688.9
46.6/ 6189.3
IxWait
92.7 3744.2
52.1 / 12704.0
39.6/ 13866.6
45.9 9376.0
Constant steering
01.4/2784,
50.4 / 10942.7
37.9 / 12315.4
47.8 / 5.16.2
Constant steering -4
94.1 4193.6
55.4 / 15144.8
39.6 16168.9
48.3/
Adaptive control
93.7 3122.8
53.8/
10850.9
42.3 12379.5
48.8 5421.6
DeepSeck- Baiieck-RR--iisee EL-Distill-Qwen-e
94.2 2947.3
69.2/ / 10679.6
51.7 / 12853.1
61.1 / 5493.1
IxWait
94.4 3634.4
68.3 11323.3
51.2 13121.1
61.5/ 6671.6
Constant steering
92.0 2088.7
60.8 8000.1
43.8 9496.0
61.2/ 3793.4
Constant steering -4
95.8 4199.4
73.8 / 14254.7
55.4 16108.7
62.2 7500.3
w. Adaptive control
94.9 / 2732.9
70.7
9322.5
51.9 10542.0
4175/
QwQ-32B
97.4 4305.0
76.7 13627.3
65.8 15852.1
62.7 / 7968.7
1xWait
97.3 4540.5
76.2 / 13868.8
64.6 16010.0
63.8 8369.2
Constant steering
97.2 3921,4
76.2 12191.6
67.5 / 14495.1
63.3 / 7083.8
w Constant steering a = -4
97.4/ 4704.1
78.7 15191.7
71.2/ / 17255.8
64.0 8707.9
Adaptive control
97.4 4133.6
77.8 / 12364.7
67.4 15150.1
64.1 / 7639.1
Qwen3-8B
96.8 5456.2
75.0/ / 14753.6
62.9 / 17797.0
60.1 8378.9
IxWait
9666/5662.0
74.2/ / 14874.7
62.5 / 17909.2
59.8 8489.2
Constant steering
5030.9
73.8 / 14131.4
63.3 / 16695.2
60.3 / 7615.0
Constant steering -4
97.4 5878.3
75.4 / 15910.2
66.2 / 18628.4
61.0/9168.5
Adaptive control
97.1 5170.8 77.5 13629.0
65.4/ 17410.9
61.2 / 7894.0
usually indicate the start of (1) problem analysis, (2) knowledge retrieval, (3) numerical computation, and (4) logical deduction. This pattern closely mirrors human problem-solving processes, making slow-thinking mode when careful processing is required in
effective natural signal for switching the subsequent reasoning. Based on these observations, we develop a aliding-windor algorithm to monitor LRMs' reasoning difficulties and dynamically adjust the steering intensity during inference time. We track the reasoning difficulties of the most recent k tokens in a continuously updated window, denoted as W E R At each generation step, we compare the current token's 's difficulty against a dynamic threshold:
(5) where lw: dw represent the window's 's mean and standard deviation, respectively, and À serves as the outlier detection parameter. When encountering significantly higher difficulty (signaling important reasoning segments), we slow down the thinking speed for deeper analysis (similar to stepping on the brake). Otherwise, progressively increase the steering intensity until we reach the preset upper bound. A detailed pseudocode can be found in Algorithm |
threshold JAW A oow
4.2 Experimental results and analysis
Experimental settings We conduct the experiments on DepsSeek-R--Disilll-eeeee
DeepSeek-
QwQ-32B and Qwen3-8B. We set the sliding window size as k = 8 tokens. The outlier detection threshold is set to 2.0 for the 7B and SB models and 1.5 for the 32B model. The steering intensity is constrained to the range [-4, 4). To decide the range of early layers for logits contrasting, we partition the transformer layers into buckets and select the optimal one based on validation results. Following (5), set 2 buckets [1, 14] and [14, 27] for the 7B model (28 layers),
8

---

2 buckets [1, 18] and [18, 35] for the 8B model (36 layers), and 3 buckets [1, 211, [22, 42] and [43, 63] for the 32B models (64 layers). To determine the optimal bucket for math reasoning, we evaluate model performance across different buckets using the AMC problem set [7 as validation data. The validation results and more details on our parameter settings can found in Appendix For math reasoning tests, we use MATH-500, AIME24 and AIME25 [22] GPQA Diamond is also included
as a generalization test. The vLLM generation parameters remain the same as in Section 3.3
Results and analysis The results from our adaptive speed control experiments shown in Table Compared to each LRM's original performance and the thought extrapolation baseline, our adaptive control method consistently shows advantages in terms of both Pass@1 1 accuracy and token usage (+1.26% accuracy while -8.56% token usage, averaged across all models and benchmarks). For comprehensive comparison, we include constant steering results at boundary intensities a 4 and -4, which collectively validate our approach's 's ability to combine the efficiency of fast thinking with the accuracy of slow reasoning. For straightforward demonstration of our control method's effectiveness, we provide detailed case studies and quantitative analysis in Appendices (eatticessBBBBaaa and B.7 These examples illustrate how our strategic speed adjustments guide LRMs to optimally leverage their reasoning capabilities, yielding more intelligent outputs. The complete ablation studies on the parameter configurations and the effect of our reasoning difficulty measurement are provided in Appendix B
5 Related Works
Large reasoning models Large Language Models (LLMs) like GPT-4o [0] and DeepSeek V3 6 typically employ fast, intuitive, and organized reasoning, which limits their ability to self-criticize and self-correct [9] [4] and their performance on advanced reasoning tasks. To address these limitations, a new series of models, named Large Reasoning Models (LRMs), has emerged, including OpenAl ol series [1I and DeepSeek RI series 77. Compared to LLMs, LRMs employ extended reasoning chains by incorporating self- verification mechanisms and trial-and-error exploration. This dlo-ttiinking paradigm enables LRMs to achieve substantial breakthroughs in complex reasoning tasks.
Reasoning strategy Recent research has developed various inference-time strategies to enhance LRMs' efficiency and effectiveness. One direct way to balance accuracy and efficiency explicitly instruct LRMs to reason within predefined token budgets (35]
255 For example, one can either
force LRMs stop reasoning and directly give the answer at the given budget [2]
43 or ask LRMs to
self-reflect on the original responses for better performance [44 41.. However, studies have pointed
out the insensitivity of current LRMs to time constraints in prompts 31
16.. Our experiments
in Section 3.3 also show the suboptimality of prompt-based methods. Other approaches propose
dynamically y routing inputs based on task difficulty to different models 30 2%
36), which usually
requires the deployment of multiple LRMs. Another line of work attempts to increase the width of the solution search space (i.e., parallel search) (20] 29), including majority voting, self-consistency [37]. and Best-of-N [18), often with the assistance of an external verifier for quality assessment.
Our representation editing-based method maintains full orthogonality to current techniques, which makes our approach particularly suitable for integration with the above methods. An empirical combination of our approach with parallel search is provided in Appendix C We anticipate fruitful combinations of our work with existing techniques in future research.
6 Conclusion
This work rethinks dual-process reasoning (System I vs. System 2) in Large Reasoning Models (LRMs). To approximate human intelligence, we propose an inference-time method that enables dy- namic thinking speed adjustment in LRMs, optimizing aelccinncy-accuree that some LRMs intrinsically support both thinking modes through an easy switching mechanism. We then extract the steering vector from LRMs' representation space that controls the thinking speed, which enables: (1) fast yet accurate responses, or (2) enhanced accuracy through slow reasoning. Finally, we utilize real-time difficulty estimation as the control signal, enabling fast processing of easy steps and deeper analysis on complex reasoning. Combining these techniques, our reasoning strategy shows consistent improvements in both accuracy and token usage across multiple LRMs and
trade-offs. We first reveal

---

advanced benchmarks. We hope this work can provide new insights for future developments in LRM reasoning strategies.
7 Limitation and future work
While study provides valuable insights into the thinking abilities of Large Reasoning Models (LRMs), it also has several limitations. First, our experiments reveal that the fast-thinking abilities of LRMs can be triggered by certain opening words. The underlying reasons for this intriguing phe- nomenon remain unclear. We leave a more systematic and thorough investigation of this phenomenon for future work.
Second, our thinking speed control experiment in Section 3.3 we used the same steering intensities for all LRMs and benchmarks. While this approach generally yields effective results, we also observe that different LRMs have varying sensitivities to the steering vectors, reflected by the uneven changes in the response lengths across different steering intensities. Due to these inconsistencies, model-agnostic approach to determining steering intensities may be needed for more controllable reasoning.
Finally, our adaptive control implementation employs heuristic Biding--window algorithm for steering intensity adjustment. Although this method demonstrates effectiveness across all tested models and benchmarks, we believe there is room for further improvement through more intricate approaches. A promising enhancement would involve developing an integrated framework that directly computes the optimal steering intensities based on real-time difficulty estimates, rather than relying on heuristic adjustments. We believe that such organic coupling of difficulty estimation with speed adjustments could further optimize the efficiency-accurae trade-off, representing valuable direction for future investigation.
8 Societal impact discussion
Our research on thinking speed control in large reasoning models (LRMs) offers several positive societal impacts. As inference-time technique, it empowers users with flexible control over LRMs' reasoning processes, enabling: (1) faster responses for routine queries, and (2) deeper analysis for advanced reasoning tasks when accuracy is prioritized. Our adaptive speed adjustment algorithm further optimizes the efficiency-accuracy tradcoff, surpassing current LRM capabilities. We believe our work would inspire future research the development of more efficient and intelligent AI systems.
We foresee negative societal impacts from our research, enhancing reasoning performance in existing models. All data used in our experiments are sourced from public datasets and contain harmful sensitive content.
work focuses exclusively
10

---

Acknowledgment
This work was supported part by The National Nature Science Foundation of China (Grant No: 62303406, 62273302, 62036009), in part by Zhiyuan Laboratory (NO, ZYL2024022b),
References
[1] Simon A Aytes, Jinheon Back, and Sung Ju Hwang. Sketch-of-thought Efficient Ilm reasoning with adaptive cognitive-inspired sketching. arXiv preprint arXiv:2503.05177 2025.
[2] David L Barack and John W Krakauer. Two views on the cognitive brain. Nature Reviews Neuroscience, 22(6))359-371, 2021.
[3] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint XXiv:21077003374 2021.
[4] Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like llms. arXiv preprint arXiv:2412.21 2024.
[5] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.0388 03883, 2023.
[6] DeepSeek-AI. Deepseek-v3 technical report, 2024. URL URLAttps://axxiv.oo///at
19437
[7] DeepSeek-Al AI. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. URL htttps/:://axxi wttp:/iii..rrg/b//5500000 12948
[8] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021.
[9] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. arXiv preprini 10.00723 2023.
[10] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024.
[11] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed Bl-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameey, et al. Openai ol system card. arXiv preprint 4rXiv:2412.167700 2024.
[12] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Lezama, Koushik Sen, and lon Stoica. Livecodehench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024.
[13] Daniel Kahneman. Thinking, fast and slow. macmillan, 2011.
[14] Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can Ilms actually correct their own mistakes? a critical survey of self-correction of 1lms. Transactions of the Association for Computational Linguistics, 12: 12:1417-1440, 2024.
[15) Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[16] Ayeong Lee, Ethan Che, and Tianyi Peng. How well do lims compress their own chain-of- thought? token complexity approach. arXiv preprint arXiv:2503.0112 2025.
11

---

[17] Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang. Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13:9, 2024.
[18] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20033 2023.
[19] Kaiyuan Liu, Chen Shen, Zhanwei Zhang, Junjie Liu, Xiaosong Yuan, et al. Efficient reasoning through suppression of self-affirmation reflections in large reasoning models. arXiv preprint ar0iv:2506. 12353, 2025.
[20] Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia, Reason- ing models can be effective without thinking, 2025. URL http:::/arxii..org/////////// 09858
[21] MAA. American invitational mathematics examination aime In American Invita- tional Mathematics Examination AIME 2024, February 2024. URL htt.t:tt////// /tttps://maa... org/ math- math- wtt.r///onnameeeaaaiiniiiiiieeee
mattematiltmmtatascc.m.c.aaaaaas www.iceeeemiiiaionnaaaa aime
[22] MAA. American invitational mathematics examination aime In American Invita-
tional Mathematics Examination AIME 2025, February 2025. URL https://maa. org/
math- teappptiiiions/mmmriiccooo wwwpeerriinannninttttioooooo oot.aaaaa-..ataaaaaaaaaaaaaaaaaa aime
[23] Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[24] Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass- editing memory in a transformer. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
[25] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint wrXiv:2501.19393, 2025.
[26] Maxwell Nye, Anders Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Au- gustus Odena. Show your work: Scratchpads for intermediate computation with language mod- els. ArXiv, abs//ll2..../ 12.0011 14, 2021. URL httph:h/tpsappa. /ppi. semanticscholas www.oollarrrrg///oooo 244773644
[27] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google- proof q&a benchmark. In First Conference on Language Modeling, 2024. URL. https //openrevies 1ett/forum2dd-T1675555
[28] DiJia Su, Sainbayar Sukhbaatar, Michael Rabbat, Yuandong Tian, and Qinqing Zheng. Dual- former: Controllable fast and slow thinking by learning with randomized reasoning traces. In The Thirteenth International Conference on Learning Representations, 2024.
[29] Yuan Sui, Yufei He, Tri Cao, Simeng Han, and Bryan Hooi. Meta-reasoner: Dynamic guidance for optimized inference- -imee reasoning in large language models. arXiv preprint 1911 2025.
[30] Guangyan Sun, Mingyu Jin, Zhenting Wang, Cheng-Long Wang, Siqi Ma, Qifan Wang, Tong Geng, Ying Nian Wu, Yongfeng Zhang, and Dongfang Liu. Visual agents as fast and slow thinkers. arXiv preprint
2024.
12

---

[31] Yi Sun, Han Wang, Jiaqiang Li, Jiacheng Liu, Xiangyu Li, Hao Wen, Huiwen Zheng, Yan Liang, Yuanchun Li, and Yunxin Liu. Time up! an empirical study of Ilm reasoning ability under output length constraint. arXiv preprint arXiv:2504. 14350, 2025.
[32] NovaSky Team. Sky-tl: Train your own ol preview model within S450. https://novasky- higithubbioo/psst/kkkttl 2025, Accessed: 2025-01-09.
[33] Qwen Team. Qwen3, April 2025. URL htttt::t/t/gnnnnaa git.uttttttt.... laciio//lloo blog/qwent
[34] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL
nttps://qwenlm.eee www.eeawwn.iiaab....oooooo////ee httpb/////g/gg///2////////
[35] Xinyi Wang, Lucas Caccia, Oleksiy Ostapenko. Xingdi Yuan, William Yang Wang, and Alessandro Sordoni. Guiding language model reasoning with planning tokens. arXiv preprint arXiv:2310.05707 2023.
[36] Xinyuan Wang. Yanchi Liu, Wei Cheng, Xujiang Zhao, Zhengzhang Chen, Wenchao Yu, Yanjie Fu, and Haifeng Chen. Mixllm: Dynamic routing in mixed large language models. arXiv preprint 251v:2252..1882, 2025.
[37] Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprini ar arXiv:2203. /1171, 2022.
[38] Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, al. Thoughts are all over the place: On the underthinking of ol-like llms. arXiv preprint arXiv:2501.18555 2025.
[39] Peter C Wason and J St BT Evans. Dual processes in reasoning? Cognition, 3(2):141-154,
1974.
[40] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting clicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28 December 2022, 2022.
[41] Zhenyu Wu, Qingkai Zeng, Zhihan Zhang, Zhaoxuan Tan, Chao Shen, and Meng Jiang. Large language models can self-correct with key condition verification. arXiv preprint arXiv:2405. 14092, 2024.
[42] Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-man technical report: Toward mathematical expert model via self-improvement. arXiv preprint prXiv:2409.12122 2024.
[43] Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Zheng Lin, Li Cao, and Weiping Wang. Dynamic early exit in reasoning models. arXiv preprint arXiv:2504. 15895, 2025.
[44] Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-ol: Towards open reasoning models for open-ended solutions. arXiv preprint 14144:241..14455, 2024.
[45] Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to transparency. arXiv preprint urXiv:2310.014400 2023.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We list our contributions (i.e., discovering ways to control the thinking speed of Large Reasoning Models (LRMs)) during inference for better accuracy and efficiency) in the abstract and highlight them in the introduction in Section These claims are firmly based experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our work in Section 7
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification:
Our current work does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide comprehensive documentation of our experiment procedures, including: (1) test configurations, (2) fast/Slow-respone stimuli construction, (3) repre-
sentation control implementation, and (4) the pseudocode of the adaptive speed control algorithms with parameter specifications. Detailed descriptions appear in Sections 2 and with supplementary implementation details in Appendices and B
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: The code and data available https://githu: com/D2I ai/ Shinking-sppeee speed-controly Moreover, we believe our descriptions of the experimen- tal methods are sufficiently detailed to facilitate ease of reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
mniissccc/uublic//ee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide comprehensive information for each of our experiments in Sec- tion 2 Section 3.3 and Section 4.2 with supplementary details in Appendices A and
Guidelines:
The answer NA means that the paper does not include experiments. The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: To ensure statistical reliability, we evaluate each test case by sampling minimum of responses per data point and report the Pass metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: As we state in the paper, all of our experiments are conducted on NVIDIA A A100 80GB GPUs.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics Ethics/https: http:///rrripsp.ccuuuut/ttiaiiia
Answer: [Yes]
Justification:
Our research strictly adheres the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the societal impacts of our work in Section g
Guidelines:
The answer NA means that there is no societal impact of the work performed.
17

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work carries no such risks, as we have not released any models, and the data we used are sourced from public datasets and contain no harmful or sensitive content.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The external assets, such datasets and models, are publicly available and have been properly credited or cited in our paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The code and data are available at https://github. www.cmmm/cc///c -ai aii
thinking- speed- soott-c
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in our research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurips: htct:::://eerrpps.ccc//offeeeeee /c/////oeeeeeecc///22////.. for what should or should not be described.
20

---

Appendices
Supplementary materials for Section
22
A.I
Details on stimuli design
22
A.2
Details on PCA model
23
A.3
Details on representation controlling
23
A.4
Details on experimental settings
24
A.5
Results on Qwen3-8B
25
A.6
Effects of extreme values steering intensity
................................
25
Supplementary results for Figure
Supplementary materials for Section
28
B.1
Details on sliding-
Llddngg-windww window based adaptive control algorithm
28
B.2
Ablation study on early layer bucket selection
29
B.3
Ablation study on the effect of reasoning difficulty liffecultybbased adjustments
29
B.4
Ablation study on the outlier detection threshold X
30
B.5
Ablation study on the maximum steering intensity Omax
30
B.6
Case studies on the adaptive control algorithm
30
B.7
Analysis of switching dynamics in the adaptive control algorithm
32
Combining thinking speed control with parallel search
34
D
Example generations
35
D.I Different types of CoT
35
D.2 Same question under different steering intensities
38
21

---

A Supplementary materials for Section 3
A.1 Details stimuli design
we use both fast- and slow-thinking responses as the positive and negative stimulus. The 7.5k questions from the MATH [8] training set are utilized to sample responses of these two modes from the experimental LRMs. We first select those pairs that both end with correct answers. For each response pair, we retain only the initial segment of the thought process to construct the stimuli. We do this by:
As described in Section 3]
1. Extracting the thought process (i.e., the reasoning chain between "<think>" and "</think>") and using double line breaks (i.e." ""\nkn") nn" segment each steps within.
2. For fast-thinking response, we retain only the first 2 steps in its original thought process as the positive stimulus.
3. For compared to its paired positive stimulus.
response, truncate it to the step that has the nearest response length
An example stimulus pair is shown in Figure A1 Following reasoning chain in our stimuli for three key reasons:
45 we retain the initial segment of the
To capture meaningful differences in reasoning conciseness, we avoid using only the first token's representation (e.g., opening words like "To" or "AAright"",, as this would reduce the contrast to mere differences in word embeddings. By including part of the reasoning process, we ensure the resulting steering vector shares dimensional features with the representations of other reasoning tokens in the thought process, which would minimize the unnecessary disruption during inference-time control.
Since our representation control applies to every token during inference, we need to ensure that the extracted direction vector contains semantic features that encourage the model to continue reasoning rather than terminate prematurely. Therefore, we choose to use only the initial part of the thought process instead of the entire process to avoid signals like -fd-uen-e or end-of- of-tohhhttuttt tokens, as the latter might cause the controlled LRM to end the reasoning process prematurely.
To cancel out positional embedding information from the semantics of the representation, we need to ensure that the contrasting stimuli share approximately the same length. Additionally, since fast responses are typically shorter than their slow counterparts, using complete fast responses would require truncating slow responses at comparable lengths, which could potentially cut them during early or mid-reasoning stages. This could introduce noise into the contrasting vectors, as they would capture differences in reasoning stages rather than purely puccinetness-relatee information. These considerations further demonstrate why using entire thought processes for stimulus construction would be suboptimal.
Since the average number of steps in the fast-thinking response is 7.1, we decide to truncate each fast-thinking response after the first 2 steps. We also compare the steering performance by taking representations from different positions in the stimuli, as shown in Table Al The results indicate that using the representation from the last token of the initial segment generally yields the best performance.
Table Al: Steering performance using token representations extracted from different positions in the stimulus. The evaluation is conducted on DeepSeek- -ereckk-R--iistiee -Distitilll -weill using the GPQA Diamond benchmark. Baseline performance (a 0): 46.6% 6189.3 (Pass@) 1 Num. Tokens). Our default configuration
generally achieves the best performance across different steering intensities.
GPQA Diamond Performance (Pass@1 1 (%)t Num. Tokens L)
Representation Position
a 6
a 4
a 4
a 8
End of the initial segment
49.2 8277.0
48.3 7732.3
47.8 5016.2
44.8 / 3712.3
First token
47.9 / 7520.7
47.5 7093.4
46.7 / 5692.9
47.2 / 4871.9
End of the entire thought
48.3 7205.3
47.0/6660.6 6660.6
47.7 / 5955.4
46.1 / 5326.3
22

---

Question
What the greatest integer x such that |6x² 47x + 15| 15| prime?
Fast-Thinking
S1ow- Nlow-Thinking
To determine the greatest integer such that 16x* 47x 1s prime number, first need to analyze the quadratic expression inside absolute value.
problem need find the greatest integer such that the absolute value of
Okay,
6x* 47x + 15 prime number. Hmm, pri pr numbers are numbers greater than positive divisors other than 1 and themselves, So, need to find such that [6x² 47x + 15| prime. And specifically, I'm looking the greatest such integer x.
I'll start by factoring the quadratic equation (6x² 47x 1.). To do this, I'll look two
numbers that multiply 6x15 90 add up to -47. The numbers 45 and -2 fit these
criteria because -45x(22) 90 and 445 +
First, let me consider the quadratic expression 6xr + factored maybe! can find roots to understand its behavior. Maybe factoring will help me see if the quadratic can be broken down into simpler terms, which might make easier analyze.
(-2) 4-7).
quadratic,
Figure Al: An example a positive and negative stimulus pair sampled from the MATH training set.
A.2 Details PCA model
After filtering out incorrect responses sampled from the MATH training set, we used approximately 6k stimulus pairs for representation collection. We randomly chose 4k pairs compute the difference vectors, with half of the pairs to compute directional vectors df - and other half to compute the
-iirsedddireetton vectors d' a-)) We then applied PCA on the combined set {dī ju{a ( to extract the first principal component that best separates these 2 poppostell-directionnn vector groups. Additionally, since the resulting PCA component lacks well-defined direction (i.e., it may assign
positive scores to {d} - and negative scores to {d] + }. or vice versa), we calibrate its direction to ensure final stecring vector consistently points from slow- to fast-thinking representations.
This calibration achieved by computing the mean score across {d] (d.- }. If the average score
positive, we directly use the first principal component as our steering vector; otherwise, we invert it by multiplying by- 1.
The remaining 2k pairs from the MATH set served as a validation set, where we calculated difference vectors both directions to verify whether the principal component could effectively separate these vectors based on their directions. The validation results, shown in Figure A2 confirm both the success of our extraction method and the presence of response succinctmess-relann related information in the models' hidden states.
PCA Validation Accuracy
DeepSeek- R1 A1-Distilll-oe A1-Disiil-Qwea--1
Qwen-7B
DeepSeck-RR BIissk-R--DOA Distil-Qween-22
QwQQ32B
Qwen3-8B
100
100
100
98
Aos (%)
95
Acc (%)
Acc (%)
(%6)
96
94
90
80
80
92
10
20
0
20
40
60
0
20
40
60
0
10
20
30
Layers
Layers
Layers
Layers
Figure A2: The classification accuracy of the PCA model for each LRM on the validation set.
A.3 Details on representation controlling
we evaluate steering intensities a from [16, 12, 4,0, -2, -4, -6]. To decide the range of layers control, we first study the PCA results in Figure A2 and note that the later layers exhibit clearer thinking-mode separation. We hypothesize that higher layers contain stronger encoding of response succinctness, which is also consistent with established findings about
our experiment in Section 3.3
23

---

higher layers processing semantic information 23 24 for each of our testing models by:
Furthermore, we determine the control range
1. Selecting layers with PCA validation accuracies close to or equal to 100%, which primarily focuses the last few layers.
2. Further filtering based on our early observations of their performance on the MATH dataset, mainly determined by the stability of the fluctuations in both response lengths and accuracies under different steering intensities.
Specifically, for:
Biaastee-RR--ssill-------- layer, namely/ E [1].
(28 layers): we control the last 10 layers except for the last
Qwen3-8B (36 layers): we control the last 10 layers except for the last layer, namely E [26,35]:
(64 layers): we control the last 20 layers except for the last layer, namely / [44, 63]
DeepSSedd-K--Dittill--------
QwQQ32B (64 layers): control the last 15 layers except for the last layer, namely E [49, 63).
Each model's final layer is excluded to preserve output distribution stability.
A.4 Details on experimental settings
Baselines To ensure a fair comparison between our control method and baselines, for the budget forcing method, constrain the maximum output length for each question match the average response length achieved under different a > 0 steering intensities for the same question. This align- ment guarantees equivalent token budgets for both approaches the average output length serves as a prior for the difficulty of the problem, allowing the budget forcing method to exit earlier for easy problems and later for difficult problems, which provides more rigorous baseline. Following 43]. after reaching the maximum output length, we append the string "i n' n Final Answer**1, n boxed At this step, we set the maximum generation length to 10 to prevent the LRR from continuing to reason from the early exit position. Given this output restriction, exclude this baseline from coding benchmarks as it may truncate the code generation.
the test benchmarks. Furthermore,
elicit the LRM''s early judgment for the final answer.
For prompt-based thought extrapolation, we first append an additional "Wait" to the LRM's original response and acquire its new final answer after the first recheck. then iteratively append "Wait" two and three times to its last generation to obtain the answer after additional rechecks. The maximum model length is set to 32,768 tokens throughout all regenerations.
Evaluation
For evaluation of each benchmark, we use the following templates from
MATH-500
Reeurr> your final response within (boxedd). Sataataatt><thintttt
[question]
AIME24
[seer\>Return your final response within (boxed [question] <eeiistan|><<thnnkk
GPQA Diamond
24

---


Return your final response within \boxed{) and only include the letter choice (A, B, C, or as your final response. [question]
LiveCodeBench
stdin template:
< <Usee]>>Geeerat executable Python function generated from the given prompt. The function should take stdin as input and print the output. Simply call the function after the definition. [question] < [rrisstan|><thinkk]
Non-stdin template:
< [User|>Generate an executable Python function generated from given prompt. Return the function body without invoking it at the final solution. [question]
< |assistant/><thinn>>>>
A.5
Results on Qwen3-8B
Origina (u
1G
12
-2
Budget Forcing
IxWait
2xWaii
SxWWait
Qwen3-8B
MATH-500
AIME24
GPQA Diamond
LiveCodeBench
07
Pass (%)
30
62
00
60
1500
5090
$500
6930
6500
12000
13000
14000
15000
16000
17000
6400
7200
5600
5800
9000
5659
1200
7800
8100
9000
3900
Response Length (Avg. Num. Tokens per Response)
Figure A3: Scaling effects of thinking speed control on Qwen3-8B. We show the trade-off between response length (x-axis, average token count) and reasoning accuracy (y-axis, Pass
We present the results our thinking speed control experiments on Qwen3-8B Figure A3 and Ta- ble A2 All parameters and generation settings remain consistent with those in Section 3.3 The results align with the scaling effect observed in Figure the LRM's performance improves under slow thinking (a < 0), while fast- fast-thinking steering consistently surpasses madget-forced baselines by a clear margin. These findings demonstrate the generalizability of our control method to the latest generation of LRMs.
A.6 Effects of extreme values of steering intensity
To show the effects of extreme values of steering intensity, we extend our experiments in Figure by scaling up the absolute value of when controlling the thinking speed of DeepSeek-RI-Distill Qwen-7B on AIME24. The results are shown in Table A3 Increasing a leads to progressively shorter responses. And when Similarly, for a < 0, increasing |a:| produces longer responses but also leads to repetition at extreme values. This behavior is expected, a serves hyperparameter and, like many others in machine learning, exhibits an effective operating range beyond which model behavior can become unstable.
becomes too large, observe repetitive generation behavior.
A.7 Supplementary results for Figure
25

---

Table A2: Experimental results of our thinking speed control experiment on Qwen3-8B. The figure version is provided in Figure A3 We compute the average performance difference of each setting compared to the original LRM in the last column (A Avg.)
Performance (Pass@1 (%) Num. Tokens
Methods
MATH-500
AIME24
GPQA Diamond
LiveCodeBench ! Avg.
Qwen3-8B
Original
96.8/ 5456.2
75.0/ 14753.6
60.1 / 8378.9
66.5/8391.1
-/-
6
96.9/6177.0
7588 16830.2
60.6 / 9702.5
69.4/9326.3
+1.09/+1264,
a =
5878.3
75.4 15910.2
61.0 9168.5
69.2 8927.4
+1.15/
4726.2
a =
96.9 5600.9
75.4 15324.4
60.8 8794.2
67.5 8674.3
+0.56/
+353.5
4
5031/
73.8 14131.4
60.3 / 7615.0
61.8/ 7902.1
-1.35/
-575.1
8
96.7 /4792.5
73.3 13432.2
60.3 / 7023.7
7332/4
-2.12/
12
95.7 4562.3
74.2 13236.4
59.3 6401.6 66
/
-2.42/
16
95.8/4384 1384..1
73.3 / 12839.8
59.3 / 6107.3
57.5/
-3112/1755.3
IxWait
96.6 5662.0
74.2 14874.7
59.8 / 8489.2
60.2/8536.4
-1,90/ +145.6
2xWait
96.4 6211.9
74.6/ 15244.9
59.2 8954.0
59.3/ 8896.8
-2.22 +582.0
Waia
96.5 / 6837.7
75.0/ 15509.0
59.7 / /9499.4
58.1 /9655.3
-2.27 +1125.44
Table A3: Performance of ReepSiek-RR--Distlll--wwa.. on AIME24 under extreme steering intensities. The steering effect remains stable and effective across a wide range of a; only at extreme values of |a| does the model begin repetitive generation.
Steering Intensity a
Performance
Pass @1 (%6)
0.0
6.2
41.7
53.3
52.5
53.7
55.4
52.9
39.6
0.8
Tokems
Repetition
1941.2
6232.5
8735.6
12451.2
14364.6
15144.8
15843.4
20241.7
Repetition
26

---

Table A4: Experimental results presented in Section 3.3 The figure version is provided in Figure We compute the average performance difference of each setting compared to the original LRM in the last column (A Avg.)
Performance (Pass@1 @ (%) Num. Tokens ()
Methods
MATH-500
AIME24
GPQA Diamond
LiveCodeBench
Avg.
DeepSeek- DeepSeek-R1-Ditteee R1 R1-Distill-Qwent
Original
940999
52.5 12451.2
46.6/ 6189.3
48.1 8467.3
-/-
94.0/4537..
$2.9 15843.4
49.2 8277.0
49.2 10531.4
+1.32/221695
94.1 4193.6
55.4/ / 15144.8
48.3 7732.3
49.0 9868.7
+1.69 11.99/+60669
93.5 / 3885.1
53.7 / 14364.6
4 6996.6
48.7/ 9152.4
+0.86/ +9718
4
9784/2884.9
50.4/ 10942.7
47.8/ 5016.2
46.0/ / 7152.0
-1.12/ -11440
Q = 8
53.3/ 8735.6
44.8/ 3712.3
4 5857.3
-2.27
2493.44
Q 12
85.5 1803.2
/2.1/ 7824.8
/ 2425.0
40.6
4522.2
-6.87 / -3484.1
a = 16
81.1 / 1410.5
/ 6232.5
38.0/ 1462.1
34.4/
3005.5
-11.2 4600.3
IxWait
92.7 3744.2
52.1 12704.0
45.9/ 9376.0
48.1
8726.6
-0.33 / +1009.8
2xit
92.7 / 5066.2
51.2 14340.9
45.6 / 13847.1
48.1 / 9029.7
-0.63 / +2943.1
3xWait
92.5/74366
50.8 16372.6
43,9 / 18294.7
48.0/
9547.3
-1,23 / +5284.8
DeepSeck-RR
Distill Qwen-32B
Original
94.2/
2947.3
69.2/ 10679.6
61.1/ 5493.1
72.4 6897.1
-/-
6
95.9/
5020.2
71.7 / 15342.5
62.3 / 8501.8
72.6/9153.0
+1.40/ 1
a =
95.8/ 4199.4
73.8
14254.7
62.2/ 7500.3
73.0 8477.1
+1.98 +2103.6
a: = 2
94.9/ 3533.5
68.2/ 11984.2
60.7/ 6428.3
72.6 7615.4
-0.10 +1270.5
a 4
92.0/ 2088.7
60.8/ 8000.1
61.2/ 3793.4
69.8 5412.6
-3.27 16680.6
@ 8
8 1495.8
5 5950.1
58.8/ 2126.4
41.2/4125.1
-6.65 -3779,9
12
/ 1034.5
38.3/ 3893.7
53.2/ 1237.5
61.922830.
-15.5 -4255.3
16
7 720.5
21.7/ 1784.1
46.6/
786.1
54.7 1448.9
-24.9 -5319.4
1xWait
94.4/ 3634.4
68.3 / 11323.3
61.5/ 6671.6
72.6/7880.4
-0,02 / +673.2
2xWait 3xWait
94.9/
8457.8
69.2 14453.3
59.5 9529.3
72.6 7372.3
-0.18
43448.9
95.0 15282.1
69.2 20468.7
57.6 14057.9
72.5 8417.4
-0.65 / +8052.3
QwQ-32B
Original
97.4 4300.0
76.7 13627.3
62.7 / 7968.7
89.2/7190.2
-/-
a
5000.8
78.3 15861.4
63.4 9044.0
90.1 8149.0
+0.89
+1241.0
97.4 4704.1
78.7 15191.7
64.0 8707.9
90.0 7919.0
+1.05/
+857.9
97.4 4438.3
78.7 13896.0
62.9 / 8318.9
89.8/
+0.72/
+268.7
4
97.2 / 3921.4
76.2 12191.6
63.3 / 7083.8
8386/
-0.17
-878.0
8
97.3 37155.
73.3 / 12068.0
62.6 6258.9
58.8/
-1.25/-1294.
12
96.9 3569.00
74.2 11530.1
61.5/ 5629.7
86.4/5523.5
-1.75/-1709.7
α = 16
96.6 3
71.2 11299.9
60.7 5220.5
83.9/5277.1
-.440-116224
IxWait
97.3/4540.5
76.2 13868.8
63.8 18893292
89.2 05.1/5001.
+0.13/ 4297.1
2xWait
97.3 5333.3
75.8 / 14402.5
63.2 8824.2
89.2 39.2/7780 7789.8
-0.13/ +814.7
3xWait
97.3/700 76040
75.8 / 16450.0
62.8 / 9700.2
89.0/ 8093.7
-0,28 / +2190.4
27

---

B Supplementary materials for Section 4
B.1 Details on sliding-window based adaptive control algorithm
Algorithm 1 Sliding-windde based Adaptive Control Algorithm
Require:
N transformer layers { (F) embedding layer E(.), vocabulary head o(-) Landof-sequence token EOS, user prompt p. maximum length max_len
Layers to control L S [1,....,N] N,, early layers L C f1, 1
Reading vectors {v' L L*]
Steering intensities: initial a, min Ommin max Step size window size W, outlier detection threshold X
Initialize:
1: x +-
Input sequence
2: y 0
Output sequence
3: a f
D Current steering intensity
4: window
0
Sliding window of difficulty scores
5: while length(x) < max_len do
6:
12 E(x)
Get embeddings
7:
Hd t 0
>> Initialize early layer states Process through all layers
8:
for i + 1 N do
9:
hi f- F ['
1
P Standard forward pass
10:
if E L then
hi +- hi +
Apply representation control
12:
end if if L then
14:
H t H H (
b> Collect early layer states
end end for
16:
JS DS_iiv f- ComputeISSDivergeneee H ST,,, .h.")
Using Equation
18:
d(x+) + ComputeTokenDiiiicultyy/// Div)
Using Equation
19:
if d(x,) > mean(window) + std(window) then
20:
0s
Slow down for careful reasoning
21:
else
22:
end min(Omaxx Omax
Speed up for efficient reasoning
23:
24:
Update window with d(z,), maintaining size W
25:
It s(h")
D Generate next token Append to input
26:
xU 21
27:
y + yuyu (
28:
if Jr EOS then
29:
break end if
30:
31:
end while
return
We provide a detailed pscudocode of our wwndee-winwwbblleee adaptive control algorithm in Al- gorithm During generation, keep track of the reasoning difficulties of the most recent tokens in the sliding window. In each forward pass, we first steer the representations from the target control layers L using the current intensity (lines 10-12), and then we collect the logit projections from the early layers L (lines 13-15). After computing the current token reasoning difficulty d(x;)) using Equation (4) (lines 17-18), compare
the current sliding window threshold (line 19). If d(2,) is larger than the threshold, we set the next steering intensity to the lower bound amin (similar stepping the brake). Otherwise, we increase the thinking speed by predefined increment size (lines 19-23).
The full hyperparameter settings for the above algorithm are shown in Table B]
28

---

Table Bl: Parameter settings for the adaptive control algorithm.
Values
Parameters
DeepSeek-R1- Distill- Besill-Qwen-7BB
Qwen3-8B
DeepSeek-RI- Distill- Beetll-Qwen-32B
QwQ-32B
Layers control I,
[1, 27]
[26,3]] 35]
[44, 63]
[19, 63)
Early layers L
[1, 14]
1, 18]
[22, 42]
[22, 42)
Initial steering intensity
4
Min. steering intensity Okmin
4
Max. steering intensity C'mxx
Adjustment step size 8
Window size
Outlier detection threshold À
2.0
2.0
1.5
1.5
B.2
Ablation study early layer bucket selection
Table B2: Ablation results of early layer bucket selection. Performances that surpass the original LRMs' are highlighted in bold. We outline the actual parameter settings we use in Table 2 by
Methods
AMC Performance (Pass @ (%)f Num. Tokens J)
81.9
7554.8
Adaptive control with early layers [1, 14
82.8
6414.2
Adaptive control with early layers [14, 27]
82.2 6775.3
Qwen3-8B
89.5
10363.7
Adaptive control with early layers [1, 18)
89.8 9862.6
w. Adaptive control with early layers [18, 35]
89.6 / 9993.7
DeepSeek-Re RI--Distlll---ee--23
88.1 / 6706.4
Adaptive control with early layers [1, 21]
87.0 5516.3
Adaptive control with early layers [22, 42
88.8
5734.4
Adaptive control with early layers [13, 64]
87.1 / 5994.7
QwQ-32B
94.4 / 8735.6
w. Adaptive control with early layers [1, 21]
92.9
8519.1
Adaptive control with early layers [22, 42
94.4/
8413.1
Adaptive control with early layers [43, 64
94.4/
8420.8
To determine the range of early layers for logits contrasting, group transformer layers into buckets and choose the optimal layer bucket based on validation results on the AMC problem set [17]. For DeepSeek-RR R R1-Distill- -1-Distill-Qwwen-71 (28 layers), we 2 buckets: [1, 14] and [14, 27]. For Qwen3-8B (36 layers), we set 2 buckets: [1, 18) and [18, 35]. For DeepSeck-R1 -D-istill- Prsstil-QQeen-311 Qwen-32B and QwQ-32B (64 layers), we
buckets: [1, 21). [22, 42), and [43, 63). For efficiency, we sample logits from every other layer within each bucket. The validation results are shown in Table B2 Based on these results, we decide use the layer bucket [1, 14) for DeepSeek-R R-I--istill--eee
[1, 18) for Qwen3-8B,
and [22, 42] for ReepSeek-R--Dssilll Depisill-Qwen-333 Qwll--QwennnB and QwQ-32B.
B.3 Ablation study on the effect of reasoning difficulty-basse based adjustments
To demonstrate the effect of our reasoning difficulty measurement in our control algorithm, compare with random adjustment baseline. The random baseline sets the steering intensity as at each decoding step a'in or amax based coin flips. Other parameters remain the same as
29

---

Table B3: Comparison of reasoning difficulty- based control method with random control baseline on DeepSeek-Re R1 RR-DDssttii Distill-Qwen= 7B. Performances that surpass the original LRMs' are highlighted in bold. We outline the actual parameter settings we use Table 2 by
Performance (Pass@) I (%)T Num. Tokens {)
Methods
MATH-500
ATME24
GPQA Diamond
DeepSsek-KL-Dittill--eee---
92.9 3403.9
52.5 / 12451.2
46.6 / 6189.3
w. Adaptive control
93.7 3122.8
53.8 / 10850.9
48.8 5421.6
w. Random control
93.4 3862.6
52.5 / 14505.1
47.5 / 7113.2
in Table B1,
The results are shown in Table B3] The reasoning difficulty-bars based method consistently outperforms the random baseline, validating the effectiveness of reasoning difficulty measurement.
B.4 Ablation study on the outlier detection threshold
Table B4: Ablation results of outlier detection threshold A. Performances that surpass the original LRMs are highlighted in bold. We outline the actual parameter settings we use in Table by
Performance (Pass@) (%)f Num. Tokens
Methods
MATH-500
AIME24
Deepseek-KL--sssill--ee---BB
92.9 3403.9
52.5 12451.2
Adaptive control (A 1.0)
93.5 3498.6
52.6 / 12659.6
Adaptive control (A 2.0)
93.7 3122.8
53.8 / 10850.9
w. Adaptive control (A 3.0)
92.9 / 2927.7
51.3/ 11132.6
DeepSeek-R1--istiil--nee
94.2 2947.3
69.2 10679.6
Adaptive control (A 1.0)
94.8 2938.0
69.4 / 9883.6
Adaptive control (A 1.5)
94.9 2732.9
70.7 9322.5
Adaptive control (A 2.0)
94.1 2637.3
64.4 / 8972.9
Adaptive control (A 3.0)
94.3 2478.4
66.1 / 7848.0
We show the results of our adaptive control method under different outlier detection thresholds À in Table B4] We observe a clear trend: as decreases, our algorithm identifies more tokens as difficult, leading to three correlated effects: (1) reduced overall reasoning speed, (2) increased response length, and (3) generally improved accuracy. These results demonstrate how threshold selection directly mediates the ppeed-accuracy tradeoff in our framework.
B.5 Ablation study on the maximum steering intensity Qmaa
The results demonstrate two key trends: (1) response lengths decrease with increasing Omax indicating accelerated overall thinking speed, and (2) accuracy improves when constraining Omaxx confirming the benefits of deeper, slower thinking. These findings collectively validate the effectiveness of our adaptive control approach.
Our ablation study maximum steering intensity O'max is presented in Table B5
B.6
Case studies on the adaptive control algorithm
To provide a more straightforward understanding of the effect of our adaptive control algorithm, we present the following case studies to analyze the impact of our dynamic adjustment. For each case, we first apply our adaptive control algorithm to the DeepSeek-R --Diistill--wen--7 model on the AIME24 benchmark. We then select the top 10 tokens with the strongest reasoning difficulty signals. Among those, we identify the tokens believe to be the "turning points" that are keys to the
30

---

Table B5: Ablation results of maximum steering intensity Omax Performances that surpass the original LRMs' are highlighted in bold. We outline the actual parameter settings we use in Table 2 by
Performance (Pass@) (%) / Num. Tokens
Methods
MATH-500
ATME24
Deppseek-LL-Disill--eeenr
92.9 3403.9
52.5 12451.2
Adaptive control (Armx = 0.0)
93.8 3598.0
54.2 13592.7
Adaptive control (Orax 4.0)
93.7 3122.8
53.8 10850.9
Adaptive control (Orax
92.9 2800.2
48.3 10361.9
Adaptive control (Olmx 12.0)
92.1 2580.0
47.5 9097.2
94.2 2947.3
69.2 10679.6
Adaptive control (O 01.)
94.8 3272.2
69.7 / 11065.6
Adaptive control (Orsax 1.0)
94,9 / 2732.9
70.7 9322.5
W. Adaptive control (Orax 8.0)
94.2 2306.4
65.8/ / 8081.9
Adaptive control (Olmax 12.0)
93.4 2088.5
60.7 7608.6
conrect answers in the solution processes. We compare the original response produced under adaptive adjustment (i.e., deceleration at these tokens) with the response produced without adjustment (iee, maintaining acceleration (a 4)). As shown in Figure B1 made by our control algorithm effectively stimulate the LRM'S s analytical and reflective capabilities, steering the reasoning process toward correct solutions. These strategic pauses in the reasoning flow optimally utilize the LRM's reasoning abilities, thereby preventing incorrect answers or token waste erroneous solution exploration. We believe these characteristies the key elements of adaptive control method in achieving the ideal accuracy-efficieer trade-off.
B2 and B3 the deceleration adjustments
Question
lit positive intagers has the Tollowing propertiss:
The sum of the items the
postivee integer
appear
squares
Model Output
Okay so have this proble where need find the sum of the squares the items a list of positive integers. The list has some spectfic properties: the sum of the items 30, the unizue mode 9, and the median is positive integer that doesn't appear in the list iiseft, Hmm, let break this down stop by stop.
let's recall what each the
number
lagest,
appear
gives
Lot me try to figure out how many numbers are the list Let's denate the longth as m. nce the median the average of two middie numbors, m must be even, as thought before. So is even. Let's try mcG. Then the median is the average the 3rd and 4th numbers.
With Adjustment
Keep Acceleration
Walt, m=s, then the modian is the 3rd number which must be Integer not the list.
Walt, m-5, the median the 3rd number
But the list has five numbers, with mode 9, which must appear more than any other number.
But the median the ard number, so that number must not be the list, which impossible because it's the 3rd number m-5 Invalid.
appears then the other three numbers must be distinct.
Sum so 0*2 c
Remark
In this case, our adaptive control algorithm triggers deceleration at the token "Wait," successfully inducing reflective reasoning in the LRM, Given the problem's expliat the lss containing elements. However, maintaining acceleration at the 'Wait' taken would mislead the LRM into unnecessary case exploration.
the median absent from the the LRM correctly utilizes information eliminate possibllty of
Figure B1: Case study on the effectiveness of our dynamic adjustments LRM reasoning. The token highlighted red and bold identified as having high reasoning difficulty, prompting our deceleration adjustment in the subsequent tokens. We compare the results produced by adaptive control algorithm with those produced by maintaining acceleration.
31

---

Question
chooses A posttive integers. Then Bob lsts finite sets postthv integers with the property that the masmmm element of belongs Bob's list 2024 Find
elaments
Model Output
Alright, so have this problem hare where Alice chooses set positive integars. Then lius firste nonempry sets of positive intagers such that the meximmmm element of in it's given that Boo's lst has 2024 sets, and need find the sum of the elements a.
Himm, okay, let me try to break this down, Firs, let's underttand what tlob doing, He's lobing all finite nonemppt sets 8 where the masimum element of 8 in A. So, for each element in A. Bob can form several sets &l where is the maximum e emmennt
fax element In
eluments sets have their maximum ement? Well, (1,
and other elements less than So, the consider nonemptyy
1J. Howaver,
nonempty
maximum,
[1.2..... 1] added each them.
The number of subsets of 2....... 1) (s - 1). However, since must be nonempty, we need to subtract the empty set, So, the number of such sets with made mam it ( (-ع01] 1)
With Adjustment
Keep Acceleration
Wait, no, hold If nonempty, but contain So, actually, the number sets where maximum is is equal the number subsets of .... - 1J with any number of elements, inclucing the empty set, because even you add the empty set, nonempty set. So, actually 2*(u- 1] sets for a
Wait, let me verify thet, If the maximum, then the set cen be formed by taking any subset (1, 2, ... 1) and adding
(a), which valid
But since 8 must nonempry, subtract for the empty set. So, It'.. (2 13
Remark
In this case, the LRM Initially makes an incorrect assumption, belleving 8 must be the nonempty subsets of (1.2.-,,0 1] while failing to recognize that already element of When adaptive control algorithm triggers deceleration "Wait' token, the LRM successfully corrects error. By contrast, acceleration is mainntinnd at 'Wait, the LRM persists in reasoning with this flawed assumption.
Figure B2: Case study 2 on the effectiveness of our dynamic adjustments to LRM reasoning.
Question
Find the triples nonnegative integers b.c) caninfynng a
0 of + ba
6,000,000.
Model Output
Arriht, so have this probiem where ineed tu find the number of triples nannogative integers (v, 6.4) that satisdy two tiorss (a
300) and (u +
a*2e b-la br + cf c*2b 6,000,000).
Himm, olay, try figure 300)
step First,
the given equations:
(ath
60000.000)
Wail, so the exparaion of (a
100
(c
100) is: (ube
100wh
100mc
1.00e
10,000s
10,0006
10,000.
1,000,000)
But our equation after adding (100*3) 3) is: (abe-
100nb
1000c-
100cn
1,000,000
-2,0000000
1,000,000
-1,000,000)
So, comparing oxaarsion (abe-
100ab
100ac
100be
10,000
10,000
10,00c
1000,000
-1,000,000)
With Adjustment
Keep Acceleration
Walt, but that's the 5 not the right way to go.
earlier equation. Mayoe asproach
Wait, equation adding (1.000,000)
100mb 100bc
1,000,000-
11,0000000
Alternatively, parhaps can let
166). à 100),
Which rewritten as:
c 100). Then, (x y + x 0).
300 0). So,
(abc
100bc 100ca -2,000,000)
Then, the equation (abe 100(ab + bc ca) 2,000,000,, can be rewritten terms of (.))
So, suggests
((- 100 100) 100) -1,000,000)
Remark
this case, the LRM initially selects an incorrect appreach, resulting contradictory condusions. Our adaptive control method addresses this through strategic decalerations: at WWat', which leads the LRM abancon the flawed approach, 'Altemativelly which triggers exploration new stratogy. The following subssitution strategy ultimately leads to the correct solution. By contrast, maintaining acceleration at the first 'Wait' token leoves the LBM trapped its initial, incorrect reasoning path.
Figure B3: Case study 3 on the effectiveness of our dynamic adjustments to LRM reasoning.
B.7 Analysis of switching dynamics in the adaptive control algorithm
To quantify how our adaptive control algorithm influences the reasoning flow of the underlying model, we analyze its transitions between fast- and slow-thinking modes during inference. We use -innseek-RR-Distll-----e--
throughout the experiments.
Switching frequencies We first analyze the model's 's mode switch frequency on math benchmarks, finding averages of 578.6 (AIME24) and 167.2 (MATH-500) switches per question. The higher frequency for AIME24 reflects its greater complexity, requiring more slow-thinking engagement. Next, to better understand the transition dynamics, we examine how switching frequency varies throughout the thought process. We divide the responses into three temporal segments: start (first
32

---

25%), middle (25%-75%), and end (last 25%). We then calculate the average token intervals between these switches (with lower values indicating more frequent switches). The results are presented in Table BB We found that (1) the model most frequently enters slow-thinking mode at the beginning. likely due to initial problem analysis requiring deeper processing: (2) the middle segment also shows high switching frequency, reflecting active reasoning during problem-solving; and (3) switching drops in the final segment, as the model converges on a solution and generates fewer new thoughts.
Table B6: Transition dynamics.
Table B7: Frequent slow-thinking trigger tokens.
Avg. of tokens between switches
Type
Most frequent words
Dataset
Start
Mid
End
Calculation
sqrt, denominator, =,, triangle, Problem, seems, because, find, Wait, Alternatively, no, maybe ...
AIME24
17.8
20.9
54.9
Analysis
MATH-500
14.5
22.5
41.7
Reflection
Relationship between mode transitions and model outputs To analyze the relationship between internal thinking modes and model outputs, we first identify the top-100 tokens that most frequently trigger slow-thinking transitions in Table B7] Our results demonstrate that models tend to switch to slow-thinking for (1) mathematical computations, (2) logical deductions, and (3) triggering certain reflection behaviors. To understand how this control signal influences the generation, we identify the top-5 most frequent tokens immediately after slow-thinking switches:
[" no", maybe", let", perhaps",
but"]
These hesitation markers consistently signal reflections and reconsiderations in models' CoTss
To assess the alignment between internal mode switches and external outputs, measured how often the token "Wait", commonly used as a marker of uncertainty 19]. coincides with actual transitions slow-thinking mode. Surprisingly, on AIME24, the model outputs "Wait" 55.3 times per question on average, but only 1 in 12.2 instances aligns with a true mode switch. On MATH-500, the ratio is 1 in 15.0. These results suggest that (1) the overuse of "Wait" reflects the overthinking behavior 44, and (2) analyses relying on output tokens to infer reasoning states 4 38 may be misleading, given the weak correlation with internal cognitive transitions. We hope these findings will provide additional insights for interpreting LLMss reasoning behaviors and inspire future research in this area.
33

---

C
Combining thinking speed control with parallel search
To further demonstrate the effectiveness and extensibility of our thinking speed control method, we explore one practical application by integrating it with parallel search (20 29 This integration yields non-trivial performance improvements over standard baselines.
Experimental settings We conduct experiments using -er-sSeee-R---isil---------
on both AIME24 and GPQA Diamond. For each question, we sample up to .V 64 responses to enable parallel search. We report Pass @k k B1 which measures the probability of obtaining at least one correct solution among randomly selected completions. To assess the benefit of our control method under breadth-first search regime, we set the steering intensity a 8 //22/1 16 to accelerate the search process and compare against the following baselines:
Vanilla Generation: Default generation from the base model. NoThinking (20]: A concurrent method that improves efficiency by skipping explicit reasoning. Specifically, it pre-fills the thought process with:
Bhhink>\nOkay) I think I have finished thinking. \n</think>.
All methods use identical vLLM generation configurations described in Section 3.3 To ensure fair comparison while keeping the computational costs practical in daily usage, we set the maximum generation length per search to 1000/2000) 3000/000//00//////0 /3000/4000000000000 tokens.
Results and Analysis The results are presented in Figure C4] where the y-axis denotes @ and the x-axis represents the average response length. We observe that the LRM's default generation performs the worst under low token budget settings, while NoTThinking achieves slightly better efficiency. However, both baselines are largely surpassed by acceleration methods across nearly all token budgets and search breadths. These results further demonstrate the superiority of our approach in achieving higher efficiency and greater extensibility, highlighting its potential to support extreme test-time scaling.
16
Nottiakin
0
1=12
e 36
Nothakkng
ATME24
GPQA Diamond Passuia
Pase
Pase@14
P
Pass
10
45.3
Piss0k (%))
42.5
101
00.310
1060 2000 3000 4300
1000 2000 3010
3000
4000
1000
2000
3000
4500
1000
2800
3000
4000
1003
2000
3000 4000
Pass 16
P2
16
PaseCG
70
44
00
53
20
30
10
127
54-
1000
2000
2000
1360
1000
2000
3000
1000
2000 3000 1000
1000 2000 3000
1930
3000
2000
3000
1000
1000
2000 3000 1000
Chrigt Tokens per Respoase
Max Octput Tokens per Respouse
Figure C4: Parallel search results with thinking speed control. The plots show the relationship between response length (x-axis, average token count) and reasoning performance (y-axis, Pass @ under different search space sizes. Key annotations: (1)">": Baseline model performance. (2)* """
Performance of NoThinking (20). (3)" Representation control with different steering intensity a. Our control method consistently outperforms the baselines across different search settings.
34

---

D Example generations
D.1 Different types of CoT
Below, we present question from the MATH training set and three different responses sampled from (1) Owen2.5-MATH--BB-meee
(22, and (2) slow response (i.e., regular response) from Qwen--B.
DeepSeek-RI- -i-tselll- Distill-Qwen-7B 7 and (3) fast response from
Question:
What is the greatest integer x such that 6x 47x + 15 is prime?
Slow response from DeepSeek-RR--istill--eeeae
Okay, so have this problem here: need to find the greatest integer
such that the absolute value of 6r² 47x + 15 is a prime number. Hmm, prime numbers are numbers greater than 1 that have no positive divisors other than 1 and themselves. So, I need to find such that 16x² 47x 15| is prime. specifically, I'm looking for the greatest such integer x.
First, let me consider the quadratic expression 62 47x + 15. Since it's a quadratic, it can be factored or maybe I can find its roots to understand its behavior. Maybe factoring will help me if the quadratic can be broken down me try factoring 6x. 17x + 15. To factor quadratic be c, look for two numbers that multiply to a and add up to b. Here, a = 6, = -47, and = 15. So, x multiply 90 and add up -47.
simpler terma, which might make it easier analyze. the form ar +
6 15 = 90. need two numbers that = x
either both numbers positive. But since they add up to -47, which is negative, they must both be negative. So, looking for negative factors of 90 that add up to -47. Factors of 90: 1 and 90, and 45, 3 and 30, 5 and 18, 6 and 15, 9 and 10. Let try pairs:
Let think:
is positive,
negative both
(-1, -90): Sum -91. Not -47. (-2, -45) Sum
Oh, wait,
that's exactly what need!
So, I can revrite the middle term using these numbers. So, -77 can be written -2x 45z. Therefore, the quadratic becomes:
6z 2x 15x +
Now, let's group the terms:
(6z" 2x) + 4-45x 15)
Factor out the greatest common factor (GCF) from each group: From the first group, Gr² - 22, the GCF is 2x:
2r(3x 1)
From the second group, -55x + 15, the GCF -15: -15(3r- - 1)
So, putting it all together: 15(3x
2r(3x
Now, factor out the common binomial factor (3x - 1):
(3x - (32-1)(2r 15)
So, the quadratic factors as 1) 15). Therefore, 6z -7x 15 (3x- 1)(2r - 115).
Since problem is about absolute value this expression being prime, let's denote N | 1)(2z 15)|. We need N to be a prime number.
Prime numbers have only two positive divisors: 1 and themselves. So, for N to be prime, the product (3r ()(2z 15) must be either prime number or its negative counterpart, since we're taking the absolute value. Therefore, the product must either p or -p, where p is a prime number. Given that (3x (2x 15) is a product of two linear terms in H, for the product to be prime, one of the factors must be +1 and the other must be tp. Because primes have only two distinct positive divisors, and themselves,
35

---

the only way product of two integers is prime is if one of them is or -1, and the other is the prime or its negative.
Therefore, we can set up equations where either 3 #1 or 2r 15 =1, and solve for z. Then, for each solution, check if the other factor is prime number or its negative.
3r -1-1=1 Add to both sides: 3x = 2 Divide by 3: I x= But needs to be an integer, so this solution is invalid.
Let me first solve 3r - =1=1:
Next, solve 3r-1=-1: 3
Add to both sides: 32 Divide by == Check the other factor: 2x - 15 2(0) - -5 So, (3z 1) 15)
(-1)(-15) = 15 But 15 is not a prime number, so this is invalid.
Now, solve 2x-18 15 1: 2x-15 1 Add 15 to both sides: 2r 16 Divide by 2: x = 8 Check the other factor: 3z-1= 1 21-1 23 So, (3x-1) 15)
23 x = 23 23 is prime number, so this is valid. So, x == is a candidate.
Next, solve 2x 15 -1: 2r - 15 -1 Add 15 to both sides: 2x = 14 Divide
by 2: x= 7 Check the other factor: 3x - = 3 21 - 20 So, (3x
prime. So, from the four cases, only = 8 gives prime number. But wait, before I conclude, maybe should consider other possibilities where one of the factors is a prime and the other is -1, but I think I've covered all cases where each factor is either or -1. But just be thorough,
1)(2r 15) 20 (-1) 20 Absolute value is 20, which is
analyze there could be other integer values of x where the product (3r ()(2x 15) is prime, without necessarily one of the factors being 1 or -1. Suppose both 32 - 1 and 2x product is also prime. Well, the only way two integers multiply prime is if is covers all cases.
15 are primes or their negatives, but their
-1 and the other is the prime its negative. So, that
Therefore, the only integer that satisfies the condition is x = 8. But the problem asks for the greatest integer wait, let check higher integers beyond a prime.
maybe 8 is the answer. But see if any of them might give
Let me test x = 9: Compute 6(9)² 47(9) + 15 First, 6+81 486 47+9 423
So, 486 423 15 (486 423) + 15 63 + 15 = 78 1781 78, which is not prime.
10:
6+100 600 47+10 470 600 470
15 130 15 145 144| 145,
which is not prime.
x = 11: 6+121 = 726 47+11 = 517 726 517 + 15 = 209 + 15 = 224 |244| = 224,
not prime.
12:
6+144 864 47+12 564 864 564
15 300 + 15 315 Not prime.
x 13:
6+169 1014 47+13 611 1014 611 15 403 15 418 Not
prime.
x = 7, we already sau it gives 20 in the other factor, which is not prime.
= 6: 6+36 216 47+6 282 216 282 + 15 (-66) 15 = -51 -51 51,
not prime.
x 6+25 150 47*5 235 150 235
(-85) 15 -70 -70 70,
not prime.
= 4: 6+16 96 47#4 = 188 96 188 15 (-92) + 15 = -77
-77 = 77,
not prime.
x 6+9 54 47+3
141 54
15 (-87) 15 -72
-72
72,
not prime.
x = 2: 6+4 = 24 47+2 = 94 24 94 + 15 (-70) + 15 -55 -55 = 55, not
prime.
6+1 47+1
47
15 (-41) 15 -26 -26 26,
prime.
x = 0: 6+0 = 0 47+0 = 0 0000 0 + 15 15 115| = 15, not prime.
= -1: 6+ (-1)2 = 6+1 = 6 4-+(-1) = -47 6 (-47) 15 = 6 + 47 15 68
1681 68, not prime.
x
6+4
(-2( -94 24 (-94) 15
94 15 133 |133|
133, which is... wait, is 133 a prime?
Wait, 133 divided by 7 is 19, because 7+19 = 133. So, 133 is not prime.
x = -3: 6+9 = 54 (-3(-3) = -141 54 (-141) + 15 = 54 141 15 210
|210| 210, not prime.
36

---

-4: 6+16 96 47* 188 96 (-188)
15 96 188 15 300
1
299? Wait, 96 188 is plus is 299.
299 prime? Let'' check: 299 divided by is 23, because 13+23 299.
So, 299 is composite.
= -5: 6+25 150 47*(-5) -255 150
(-235) + 15 150 235 15 400
1400| 400, not prime.
less than 8 either result in composite numbers or negative numbers whose absolute values are composite. So, the only integer that gives prime is 8, as found earlier.
Hmm, во integer values
Wait, but hold on. Let consider if could be such that of the factors is -1, leading the other factor being negative prime. But we already considered that when we set 2x that reaulted in the other factor being 20, which is composite. So, even in that case, it didn't work.
= -1, which gave 7, but
Alternatively, perhaps I should also check if the quadratic could be equal to prime number without one of the factors being or -1, But thought earlier, since the quadratic factors into two linear terms, their product can only be prime if one is 1 or -1. So, I think that covers all possibilities.
Just make quadratic expression again. Since " 47x -7r 15 factors into (3r (3r-11 1)(2x 15), and for the absolute value to be prime, one of these factors must be 1 or -1. Since we've already found that r = gives 23, which is prime, and all other integer values either give composite numbers non-prime absolute values, I think is indeed the greatest integer that satisfies the condition. Therefore, my conclusion that the greatest integer x is 8. **Final Answer** The greatest integer
haven't missed anything, let think about the
Fast response from DeppSea-RR-DDsttil--werrr
To determine the greatest integer such that 16x²
47x + 15 is a prime number, I first need analyze the quadratic expression inside the absolute value.
I'11 start by factoring the quadratic equation 6x 17x + 15. To do this,
I'll look for two numbers that multiply to 6x 15 = 90 and add up to -47. The numbers -45 and -2 fit these criteria because (-45) (-2) 90 and 45 + (-2) = 477
Using these numbers, can rewrite the quadratic as:
6r - - 2x - 15
Next, I'll factor by grouping:
(6x 45x) + (-2x + 15)
(2r 15) 1(2x 15)
(3x [2r 15)
So, the expression becomes:
(3r () 15)
For the product (3x - 1) 15) to be a prime number, one factors must be 1l. I'll consider both possibilities. **Case 1: 3r - 1**
3r 2
2
3
Since must be integer, this doesn't provide valid solution. **Case 2: 3r ++*
3r 0
T 0
37

---

Plugging into (2x 15) gives:
2(0)
(3(0) - --(-15)) ((-1) 15
15 is not a prime number.
**Case 3: 2r 15 = 1**
2r 16
Plugging # into (3r 1) gives:
3(8) 23
(1)) 23
23 is prime number.
**Case 4: 2r 15 1**
2x 14
I = 7
Plugging = 7 into (3r 1) gives:
3(7) = 20
(-20) 20
prime number.
Among the integer values that satisfy the condition, the greatest value is x =.
D.2 Same question under different steering intensities
Below we present an AIME24 question with corresponding responses from DeepSeek-RI-An RI-Distill- Qwen-7B across varying steering intensities.
Question:
Every morning Aya goes for 9-kilometer-long walk and stops coffee shop afterwards. When walks at constant speed s kilometers per hour, the walk takes her 4 hours, including minutes spent in the coffee shop. When she walks and 24 minutes, including minutes spent in the coffee shop. Suppose Aya walks
2 kilometers per hour, the walk takes her 2 hours
+ kilometers per hour. Find the number of minutea the walk takes her, including the minutes spent the coffee shop.
Original response (a = 0):
Alright, so I've got this problem about Aya going for a walk every morning. Let try break it doun step by step. Sometimes these word problems be bit tricky, but taking it piece at time should help.
First, the problem says that Aya goes for a 9-kilometer walk every morning and then stops at a coffee shop. When she walks at a constant speed of s kilometers per hour, the entire trip (walking plus coffee shop) takes her 4 hours. Out of these 4 hours, minutes when she increases her walking speed to spent in the coffee shop. Then, time becomes hours shop. The question is asking, if she walks at 1/2 kilometers per hour, how many minutes does the entire trip take, including the coffee shop time? Okay, let's try make sense of this. The key here is that the time spent in the coffee shop, t, remains constant difference in total time must come from the time she spends walking, That means if her walking time decreases when she walks faster, the total time (valking coffee shop) also decreases.
kilometers per hour, the total minutes, still including minutes the coffee
both scenarios. So, the
38

---

Let denote some variables to make this clearer. Let's let:
walking speed in km/h (first scenario) s walking speed in km/h (second scenario) 8 0.5 walking speed in km/h (third scenario, which is what we need to find) shop d distance walked, which is given as 9 km in both cases Wait, actually, the distance walked is 9 km in all cases, So, the total time hours the first the second case. is in km/h.
minutes spent in the coffee
that's constant. and 2 hours and 24 minutes convert those times to hours because the speed
4 hours is straightforvard, it's 4 hours. 2 hours and 24 minutes: 24 minutes is 24/60 hours, which is 0.4 hours. So, 2.4 hours in total. Now, the total time is the of walking time and coffee shop the first case, the total time is hours, which is equal to (valking time at speed s) t (coffee shop time). Similarly, in the second case, total time is 2.4 hours, which is equal to (walking time at speed 2) t. Since t is the same in both cases, write that out.
So,
set up equations.
First scenario: Total time walking time hours km) (s km/h) t (in hours)
Wait, hold on. is in minutes. Hmm, maybe I should convert hours well to keep the units consistent.
me denote as hours. So, t hours is the time she spends in the coffee shop.
Therefore, in the first scenario: 4 (9 8) In the second scenario: 2.4 (9 (s 2)) So,
have equations:
4 (9 / 8)
2) = (9 (s 2)) +
Since both equations equal to t, I can set them equal to each other: 4 (9 s) 2.4 (9 / (s 2))
me solve for s.
First, subtract 2.4 from both sides:
4 2.4 2.4
(9 / s) (9 / (s 2))
Simplify 4 2.4: that 1.6.
So:
1.6 (9 =)
(9 / (s 2))
Multiply both sides by -1 to make it cleaner:
-1.6 + (9 s) - 9 (s 2)
So,
s)
/ (s
Let me rewrite 1.6 as 8/5 to make calculations easier. 1.6 8/5. So,
(9 s) (8/5) 9 (s 2)
Hmm, so now
equation with s. Let's try solve for s.
First, let me denote equation:
(9/s) - (8/5) = 9/(s + 2)
Let bring all terms to one side: (9/s) (8/5) 9/(s 2) 0
Hmm, maybe it's better But this might get complicated. Alternatively, I can cross-min mooss-multiply. Alternatively, let me let's compute 9/a 9/ (s 2) = 8/5
find a common denominator combine these terms.
Yes, that seems manageable. So,
9/s 9/(s 2) 8/5
Factor out 9:
9 [1/s (/(s + 2)] 8/5
Compute the expression in the brackets:
1/s 1/(8 2) (s
[s(s 2)] [s(s 2)]
Therefore,
9 * [2 / (s 2))] = 8/5
Simplify:
18 [a(s + 2)] 8/5
Cross- cross-mulipllying.
39

---

18
sss 2)
Compute 18
So,
90 (s + 2)
Divide both sides by 2: (a 2)
So,
4s(s 2) 45
Expand the left side:
4s + 85 45
Bring all 1s + 8s 10-45-=0 =
side:
Nov, we have a quadratic equation: 1s² +s =0 Let try to solve this quadratic equation. Quadratic formula:
s= (2c)
Here, = b 8, = -45
Compute discriminant:
b² 4ac 8² 4= (-45) 64 720 784
sqrt (784) 28
So,
-8 28 ] (2
-8 * 28
So, two solutions:
1) [ -8 + 28 8 = 20 8 2.5
2) -8 28 8 - 36 8 -4.5
Since speed cannot be negative, Therefore,
discard -4.5.
2.5 km/h.
So, Aya's walking speed is 2.5 km/h the first scenario. Now, let's find t.
From the first equation: 4 9 /
have s 2.5, so: - / 2.5 +
Compute 9 / 2.5: 9 divided by 2.5 is 3.6. So, Therefore, = 3.6 0.4 hours.
Convert t to minutes: 0.4 hours * 60 minutes/hour 24 minutes. So, t is 24 minutes.
Let verify this with the second scenario make sure.
Second scenario: speed is s
4.5 km/h.
Walking time: 9 4.5 2 hours.
Total time is 2 hours (walking) 24 minutes (coffee shop) hours 24 minutes, which matches the given information. Perfect. So, now, the queation is: Suppose Aya walks number minutes the walk takes her, including the t minutes spent in the coffee shop.
1/2 km/h. Find the
So, s is 2.5 km/h, We need to find the total time when she walks km/h. Again, total time walking time
0.5 km/h.
(coffee shop time)
We know t is 24 minutes, which is 0.4 hours.
Walking time at 3 km/h: 9 / 3 = 3 hours.
Total time: 3 hours + 0.4 hours = 3.4 hours.
Convert 3.4 hours to minutes: 3 hours is 180 minutes, 0.4 hours 24 minutes, so total is
24 204 minutes.
Alternatively, 3.4 hours 60 minutes/hour
204 minutes.
Therefore, the total time is 204 minutes.
Let just recap to make Converted total times hours because apeed is in km/h. Set
I didn't make any mistakes.
up two equations with in houre. 3. Solved for s by equating the two expressions for t. 4. Found 2.5 km/h, then 0.4 hours 24 minutes.
40

---

Calculated walking time + 0.5 km/h, which is 3 hours. 6.
Added coffee shop time: 3 0.4 3.4 hours 204 minutes.
Everything seems to check out. think 204 minutes is the correct ansver. **Final Answer** The total time Aya's walk takes, including the coffee shop, is 204 minutes. </think>
Aya goes for 9-kilometer walk every morning and stops at coffee shop. When she walks
constant speed of kilometers per hour, the walk takes 4 hours including t minutes spent in the coffee shop. When she walks 8 + 2 kilometers per hour, the walk takes hours and minutes, including t minutes spent in the coffee shop. We need to find the number of minutes the walk takes her when she walks at First,
kilometers per hour.
the total times hours:
hours straightforward.
2 hours 24 minutes 2.4 hours.
We set up the equations for the total time: 1. 4= 4 80
2. 2.4 ++2 + 60
By subtracting the second equation from the first, we get:
8
1.6 9
+
2
1.6
s(s + 2)
1.6
8(s 2)
1.6s(s =
1s(s + 2) 45
4s + 8s 45 0
Solving quadratic equation:
+ 8 4 4. (-45)
2.4
8 28
8
2.5 km/h
Finding
2.5 60
4 3.6 +
60
t= 0.4 hours 24 minutes
When walking at + km/h:
Walking time
3 hours
Total time 3 3.4 hours 60 + 204 minutes
The total time Aya's walk takes, including the coffee shop, is 204 minutes.
Response under a 8:
Okay, so I have this problem about Aya's walk and coffee shop visit. Let me try to understand it step by step. Every morning, Aya goes for 9-kilometer walk and then stops at coffee shop. Her walking speed affects the total time, including the time she spends in the coffee shop.
When she walks at a constant speed of kilometers per hour, the entire activity (walking plus coffee shop) takes 4 hours. Out of these 4 hours,
41

---

minutes are spent in the coffee shop. So, the walking time at speed is 4- hours.
Similarly, when she increases her walking speed the total time becomes 2 houre and 24 minutes. Converting hours and minutes to hours, that's 2 24 2.4 hours. So, the walking time at speed + is 2.4 60 hours.
kilometers per hour,
60
+ kilometers per Our goal is to find the total time when she walks hour, including the minutes spent in the coffee shop. First, let's note that the walking distance always 9 kilometers, regardless of her speed. Therefore, we can set tuo equations based on the given information.
When walking apeed s: Walking time hours Total time hours
So, - 60
=
When walking at speed + 2: Walking time 2+2 hours Total time = 2.4
hours - So, x47
8 + 60 = 2.4
a42 + 60 = 2.4 We can subtract the second equation from the first to eliminate 60 Simplify the right side: s+2 = 1.6
Now, we have two equations: 1. + 6i = 2.
y ++2 =
Factor out the 9
( - -
1.6
Compute difference the fractions:
(+42)-s
5+2 s(s+2) s(++2)
Subatitute back: 9 a(++2) = 1.6 Simplify:
2
18 1.6 Multiply both sides by
*(+22)
s(s+2) 2): 18 = 1.6s(s+2) 2) Divide both sides by 1.6:
1s
s( Calculate T.6
18
T6
181.6 = 11.25 So, s(s 2 2== = 11.25
This quadratic equation: 8 11.25 - Because 4uc 4
To solve for 8, the quadratic formula:
(-11.25) 45, so discriminant + 45 49.
Thus: s -2+7
We discard the negative solution because speed cannot be negative: -2-7 = 2.5 km/h
So, Aya walks at 2.5 km/h normally. Now, let's find using of the initial equations. Let's use the first one:
9 9 + to Calculate 255 92.5 3.6 hours
2.
So, 3.6 +
Subtract 3.6: 60 0.4 Multiply both sides by 60:
minutes
Now, we need to find the total time when she walks at 8 + & km/h. Since 8 2.5, then: + + 2.5 + 0.5 3 km/h
Compute the walking time
km/h: = hours
the coffee shop time = minutes: Convert hours to minutes: 3+
180 minutes Total time: 180 + 21 = 204 minutes
Therefore, the total time when Aya walks at 3 km/h is 204 minutes.
**Final Ansver** The total time is 204| minutes. </think>
Aya walks 9 kilometers every morning and stops at a coffee shop for t minutes. When walks km/h, the total time
km/h, the total time is hours. When she
walks at find the total time when she walks at s km/h. First, we set up the equations for the walking times: 1. + + = 2.4
hours and 24 minutes. We need to
2.
Subtracting the second equation from the first:
9
1.6
8 ++2
Simplifying:
1.6
(s + 2)
1.6
s(s + 2)
s(s + 2) 11.25
42

---

Solving the quadratic equation:
+ 11.25
Using the quadratic formula:
7
2.5
2
Substituting 2.5 km/h to find
9
2.5
60
3.6 +
60
0.4
60
1 = 24 minutes
When walking 3 km/h:
9
3 hours
3
Converting to minutes and adding t:
x 60 21 201 minutes
The total time 204 minutes.
Response under a 16:
Okay, so I need to solve this problem about Aya's walk. Let me try to understand it step step. Every morning, Aya goes for 9-kilometer walk and then stops coffee shop. She walks at constant speed of a kilometers per hour, and the entire trip (walking plus coffee shop visit) takes hours, including t minutes spent in the coffee shop.
When she increases her walking speed to
kilometers per hour, the entire trip takes hours and minutes, still including minutes in the coffee shop.
Now, we need to find the total time taken when she walks at s + 1/2 kilometers per hour, including the coffee shop visit.
Let break down the information:
Time spent walking: 9 / a hours Total trip time: hours = 4 * 60 240 minutes Time spent
Walking speed:
km/h Walking distance:
in coffee shop: t minutes
km/h Walking distance: 9 km Time spent walking: 9 (s 2) hours Total trip time: 2 hours 24 minutes (2
2. Walking speed: (s
60) 24 144 minutes Time spent coffee shop: minutes
From this, I can set equations for the total time.
For the first scenario: Total time walking time coffee shop time 240 minutes (9 a) hours 60 minutes/hour
So, 240 (9 / a) 60 Similarly, for the second scenario: Total time walking time coffee shop time 144 minutes = (9 (s + 2)) hours 60 minutes/hour (s + 2)) 60 + t Now, have
So, 144 = (9
equations: 1. 240 (540 s) + 144 (540 / (s
2))
I can subtract the second equation from the first to eliminate t: 240 144 = (540 / s) (540 / (s 2)) 96 = 540 (1/s 1/(s 2)) Simplify the right side: 1/8 (/ 2) = (s
a) / (s(s 2)) = 2
(s 2))
So, 96 540 (2 / (s(s 2)))
(1080) / (s(s 2))
Multiply both sides by ss + 2) 96 s(s 2) = 1080
Divide both sides by 96: a(s 2) 1080 / 96
Calculate 1080 divided 96: 1080 96 11.25
43

---

So, s(s + 2) 11.25
This is quadratic equation: 8² 11.25
Solve for a using the quadratic formula: a [-2 eqrt (4 45)
[-2 # aqrt(49)] 2 [-2 7] /
We discard the negative solution because speed cannot be negative: (6) 2.5 km/h
Now, substitute
2.5 km/h into one the original equations to find t:
Using equation 2: 144 (9 (2.5 + 2)) 60 t 144 (9 4.5) 60 + t
144 - 2 * 60 t 144 = 120 + t t - 24 minutes
Now, need find the total time when she walks speed
1/2 km/h: Walking
0.5 km/h
Walking time
km/h 3 hours
Convert walking time to minutes:
60 180 minutes
Add coffee shop time: 180 + 24 204 minutes
Therefore, the total time taken is 204 minutes.
**Final Answer** The walk takes her 204 minutes. </think>
Aya walks 9 kilometers every morning and stops at coffee shop. When she walks constant apeed of kilometers per hour, the total trip (walking plus coffee shop vieit) takes hours, including minutes apent in the coffee shop. When she increases her walking speed to s + 2 kilometers per hour, the total trip takes 2 hours and 24 minutes, including minutes in the coffee shop.
set up following equations based on the given information:
1. For the first scenario:
9
240
x 60 +t
8
Simplifying, get:
540
240
+
second scenario:
9
144
60 +
+
Simplifying, we get:
540
+
2
Subtracting the second equation from the first to eliminate
96 540
s+2
Simplifying the right side:
2
8
s(8 2)
Thus,
2
96 = 540
8(8 + 2)
1080
96
8(8 1
s(s 1080
Solving the quadratic equation: s(s 11.25
2g 11.25
Using the quadratic formula:
-2 /49 2.5
2
Substituting s 2.5 into the equation to find t:
144 = 540
+
4.5
44

---

144 120 +
f = 24
When Aya walks 8 =33 kilometers per hour:
Walking time
3 hours 180 minutes
Adding the coffee shop
180 + 24 204 minutes
The total time taken is 204 minutes.
Response under a = -4:
Okay, so need to solve this problem about Aya's walk. Let me try to understand it step by step.
First, every morning Aya goes for 9-kilometer-Lowe walk and stops at a coffee shop. So, her total journey includes both walking and time spent in the coffee shop. The problem gives walking speed and the coffee shop. Then, need to find out how long it takes her when she walks 1/2 km/h.
different scenarios about her takes her, including the time spent at the
at speed of Let break down. In the first scenario, she walks km/h and the entire trip (walking plus coffee shop) takes hours. the second scenario, she walks at 2 km/h, and the entire trip takes 2 hours and 24 minutes. I need to find the time when she walks at 8 + 1/2 km/h.
constant speed
both scenarios? The problem says, "including t minutes spent in the coffee shop. So, t is the time she spends in the coffee shop each morning, regardless of her walking speed. That means the difference in total time is entirely due to the difference in walking time.
Wait, the coffee shop time is the
So, me think. Let's denote:
hours. That includes walking time and minutes. Total time in the second case: 2 hours and 24 minutes. That's
Total time the first case:
2 24/60 2.4 hours. So, 2.4 hours. That includes walking time and t minutes well.
Since coffee shop time t is the same in both cases, difference total time must be due to the walking time. So, if I can find the walking time in both cases, I can set up equations and solve for and me denote walking case.
as the first case and in the second So, T1 hours t minutes. Wait, but units are different here. Let me convert everything into hours minutes. Maybe hours would be easier. First, convert minutes hours because the walking speeds me denote t as t/60 hours.
km/h.
So, in the first case: total time hours walking time + t/60. So, T1
= 4 t/60.
In the second case: total time 2.4 hours = walking time t/60. so, T2 t/60.
But also, walking time is equal to distance divided by speed. The distance is km, so walking time T1
hours, and T2 = 9 / (a 2) hours.
So, I can write:
9 8/s0 /
60 ... (1)
9 / (s 2) 2.4
60 ...(2) (2)
So, now I have two equations with two variables: s and t. I can solve these equations to find and t. Let write equation (1) and equation (2): From equation (1))
t / 60 4
45

---

From equation (2)
/ 60 2.4 9 (s 2)
So, since both equal t / 60, I can set them equal:
4 9 a 2.4 9 / (s + 2)
Let write that:
4 9/s 2.4 9/ 2)
me rearrange this equation:
4 2.4 - 9/8 9/(s 2)
So, 1.6 9(1ss (/(s + 2))
Simplify the right side:
1/s 1/(s 2)
(s 2) a
[s(s 2)] [s(s 2)]
So, 1.6 9 (2 / [s(s 2)])
So, 1.6 = 18 / [a(s 2)]
Therefore, s(s 2) 18 / 1.6
Let compute 18 1.6:
divided by 1.6. Let's see, 1.6 times 11 17.6, so 18 1.6 11.25
Wait, 1.6 11 17.6, so 1.6 11.25
Yes, so a(s + 2) 11.25 So,
2² + 2s 11.25
Bring 11.25 to the left:
8² + 2s 11.25 0
So, quadratic equation:
3 + 2s 11.25 0
Let solve for Quadratic equation:
b-b/(b² /2c
Here,
1,b== -11.2
So, discriminant
D 4ac 4 4 (-1.55 = 4 45 49
So, aQrt(D) = So, s [-2 7
Since speed can't be negative, take the positive solution: /
s (2 So, = 2.5 km/h
/ 2 km/h
Now, let's find t. From equation (1): t / 60
/
have e 2.5, so:
t / 60 = 4 / 2.5
Compute 9 / 2.5: 2.5 goes into three times with remaining, so 3.6
So, t / 60 4 3.6 0.4
Therefore, So, t is 24 minutes.
0.4
24 minutes
Let me check with equation (2) to make sure. Equation (2):
60 2.4
(s 2)
2 2.5
4.5 km/h
9 / 4.5
So, t / 60 2.4 = 0.4
So, t = 0.4 60 24 minutes, which matches. Good.
So, now, know 2.5 km/h, 24 minutes.
Now, the question is: Suppose Aya walks at s 1/2 km/h. Find the number minutes the walk takes her, including the minutes spent the coffee shop-
Wait, she's walking km/h.
1/2 km/h. So, her speed is 2.5 + 0.5
So, we need to find the total time when she walks at km/h, which
walking time plus t minutes.
46

---

First, let's compute the walking time km/h. Walking distance So, walking time So, total time is 3 hours 24 minutes.
km, time
hours.
hours, coffee shop time 24 minutes.
Convert 24 minutes to hours: 24 / 60 0.4 hours.
So, total time Convert 3.4 hours
0.4 3.4 hours.
to hours and minutes: 0.4 hours is minutes, so
total time is 3 hours and 24 minutes.
But the question asks for the number of minutes. So, 3 hours is 180 minutes, plus 24 minutes is 204 minutes.
So, total time 204 minutes.
Wait, let me just verify all stepe again to make sure.
First, we had two cases:
1. Speed s = 2.5 km/h, total time 4 hours, including t -24 minutes coffee shop.
3.6 hours, which hours minutes. Which is km 2.5 km/h 3.6 hours. Correct.
Walking time:
24/60 4
Second case: speed s 4.5 km/h, total time hours 24 minutes, which
is 2.4 hours.
Coffee shop time 24 minutes, walking time 2.4 0.4 hours.
Which is km / 4.5 km/h hours. Correct.
So, all steps check out.
Therefore, when she walks at 3 km/h, walking time is 3 hours, coffee shop time is 24 minutes, total time is 3 hours 24 minutes, which is 204 minutes.
So, the answer 204 minutes.
**Final Answer** The walk takea her 204 minutes, including the minutes spent in the coffee shop. </think>
Aya goea for a 9-kilometer--ong walk every morning and stopa at a coffee shop. The problem involves two scenarios with different walking speeds and total times, including the the total time when she walks at + km/h.
coffee shop time minutes. We need to find
First, we convert the total times into hours and set up the equations for walking time and coffee shop time. Let be in hours for easier calculation.
When walking km/h, the total time is hours:
9 = 4 60
b
2. When walking at + km/h, the total time is 2.4 hours:
+
60
By equating the expressions for from both equations, we get:
2.4
Simplifying this, we find:
9
1.6
1
1
1.6 = 9
8 + 2
2
1 9
8(8 + 2)
18
1.6
8(8 + 2)
+ = 11.25
Solving the quadratic equation Next, find
+ 11.25 25
0, get
2.5 km/h.
1
9
=
= 0.4
60
2.5
47

---

1 = 24 minutes
When Aya walks s+3 km/h (which is km/h), the walking time is:
9
3 hours
3
Including the coffee shop time of 24 minutes, the total time is:
hours 24 minutes 3.4 hours 204 minutes
Thus, the number of minutes the walk takes her is 204
48


Reviews:
[{"review_id": "ec99g0UZ2Y", "reviewer": "Reviewer_kUJX", "summary": "This paper characterizes the distinction between slow / fast thinking in LLMs, extracts the steering vector capable of controlling thinking speed, and proposes a dynamic decoding method involving difficulty estimation to improve reasoning accuracy while reducing token usage.", "questions": "The main question is to address weakness 1: \n1. Can you outline the main challenges you faced during this work and what innovations, if any, are employed to overcome these challenges? \n\nHowever, I will most likely not lower my score as long as the following questions are addressed. \n\n2. Can you discuss the claim \"plug-and-play\" and the time cost of obtaining the reasoning strategy for a pre-trained model? \n3. In Step 3 of 3.1 (Line 142), the PCA step of separating hidden state pairs into two halves seem unnecessary. At first glance, the exact same result can be achieved by calculating the first principal component of all $\\{ d_i ^{(- \\rightarrow +)} \\}$'s, at least mathematically. Can you discuss on this, either justify the split or revise this paragraph? \n4. Can you further discuss the use of a sliding-window for difficulty evaluation? \n5. It would be great if you can discuss the effects of extreme values of steering intensity $\\alpha$.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \nThe paper is very well-written, with clear structures, great presentation and thorough justifications. Each step is backed by extensive experimentation. Good job! \n\nWeaknesses: \n1. My main reservation is the significance of this work. While the claims are all clear and significant, it seems to me the methods employed are natural and a little lacking in novelty. \n2. The method is not exactly one-shot: it requires forward passing a dataset of slow vs. fast thinking CoTs to extract the steering vector specific to the model. Even though this does not require additional training, I'm not sure if it qualifies as plug-and-play. \n3. Some minor issues, see Questions.", "was_revised": true, "final_justification": "The authors defended their paper on my major concerns, so I maintain my view that this is a solid paper worthy of acceptance.", "rebuttal": "We are glad that you find our work solid and well-written. We hope our responses below will address your concerns.\n## W1 & Q1\nQ: My main reservation is the significance of this work. Can you outline the main challenges you faced during this work and what innovations are employed to overcome these challenges?\n\n**A**: Thank you for your thoughtful comment. We believe the main challenge—and the key innovation of our method—lies in transforming the _top-down, global, and static_ intervention approaches used in prior works [1,2,3] into a **bottom-up, fine-grained** control mechanism that is both **query-adaptive and reasoning-aware**. \n\nIn our early experiments, we found that traditional fixed intervention methods fall short in capturing the dynamic nature of human-like reasoning (i.e., the fluid transitions between System 1 and System 2 thinking). This stems from a key limitation: most interventions operate along a _single functional axis_, globally inducing or suppressing certain behaviors across the entire generation. They largely ignore the **temporal dimension**, which is the fact that a model’s internal reasoning states evolve over time. To address this, we introduce a second axis of intervention: **time**. This reframes representation intervention as a dynamic, context-aware process.\n\nFor interpretability, our difficulty score metric provides a **novel framework** for analyzing reasoning dynamics. As a reasoning strategy, we extend representation engineering to online, autoregressive generation with **temporal adaptivity**, which opens new possibilities for context-specific alignment and efficiency optimization.\n\n---\n1. Representation Engineering: A Top-Down Approach to AI Transparency\n2. ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models\n3. Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering\n\n## W2 & Q2\nQ: The method is not exactly one-shot. Can you discuss the claim \"plug-and-play\" and the time cost of obtaining the reasoning strategy for a pre-trained model?\n\n**A**: We use \"plug-and-play\" to highlight our method's **seamless** integration into LLM frameworks, offering 3 advantages:\n1. No model structure changes (simple plug-in)\n2. No model constraints (broad generalizability)\n3. No parameter adjustments (zero training cost)\n\nPoint 1 is self-explanatory. For Point 2, we tested across diverse architectures (reasoning/non-reasoning models with varying training approaches). To compute the control vector, we reused slow and fast CoTs generated by DeepSeek-R1-Distill-Qwen-7B for representation extraction on all tested models. The results below show consistent scaling effects, confirming the compatibility of our method.\n\n|Models | AIME 24 ( Pass@1 (%) / Output Length )|  |  |  |  |\n|-|-|-|-|-|-|\n|| $\\alpha=8$|$\\alpha=4$ | $\\alpha=0$ | $\\alpha=-2$ | $\\alpha=-4$ |\n| **Non-reasoning** |  |  |  |  |  |\n| Llama3.1-8B-Instruct | **8.3** / 783.8 | 5.4 / 1293.2 | 6.8 / 3501.2 | 6.7 / 3787.6 | **7.1** / 4748.7 |\n| Qwen2.5-7B-Instruct | 9.2 / 1574.9 | **10.4** / 1841.0 | 9.6  /  1886.7 | **11.7** / 1959.4 | **10.8** / 3177.7 |\n| **Reasoning** |  |  |  |  |  |\n| Llama-3.1-Nemotron-Nano-8B-v1 | 47.1 / 8072.1 | 62.1/ 10719.1 | 62.9 / 11044.5 | **65.0** / 11682.1 | **63.3** / 13414.8 |\n| DeepSeek-R1-Distill-Llama-8B | 35.0 / 10589.4 | 42.1 / 11164.8 | 50.0 / 13398.3 | **53.3** / 14709.1 | **51.7** / 15776.1 |\n| GLM-Z1-9B-0414 | 65.0 / 7870.0 | 67.1 / 8346.8 | 67.5 / 8883.45 | **71.3** / 9585.57 | **72.1** / 10150.1 |\n\n| Models | MATH500 ( Pass@1 (%) / Output Length ) |  |  |  |  |\n|-|-|-|-|-|-|\n|  | $\\alpha=8$ | $\\alpha=4$ | $\\alpha=0$ | $\\alpha=-2$ | $\\alpha=-4$ |\n| **Non-reasoning** |  |  |  |  |  |\n| Qwen2.5-7B-Instruct |72.9 / 534.6|74.0 / 544.2|75.9 / 655.2|**76.2** / 700.3|75.4 / 868.0|\n| Llama3.1-8B-Instruct |**41.6** / 541.8| **42.9** / 599.4 | 41.5 / 702.2|**42.8** / 773.9|39.5 / 815.2|\n| **Reasoning**  |  |  |  |  |  |\n| Llama-3.1-Nemotron-Nano-8B-v1|79.4 / 2117.6 | 87.7 / 3091.64 | 93.7 / 3893.4 | **94.3** / 4118.9 | **94.3** / 4303.09 |\n| DeepSeek-R1-Distill-Llama-8B|75.9 / 2650.7 | 85.1 / 2895.3 | 88.2 / 3694.1 | **89.6** / 4241.2 | **90.2** / 4829.1 |\n| GLM-Z1-9B-0414 | 94.3 / 2215.0| **96.2** / 2439.9 | 96.0 / 2847.1 | **96.5** / 3023.3 | **96.4** / 3265.5 |\n\nRegarding Point 3 and **time costs**, our extraction process consists of three main steps:\n1. Slow and fast CoTs generation (2 hours);\n2. Hidden states computation (**10 minutes**);\n3. PCA computation (**few seconds**). \n\nTimings were measured on an NVIDIA L20 46GB×8 server using DeepSeek-R1-Distill-Qwen-7B and MATH training set (7.5k questions) as stimuli. With reusable stimulus pairs, adaptation to new models takes **minutes** - far faster than training-based methods [1,2].\n\nWe'll open-source pre-computed control vectors for popular models and slow and fast thinking CoTs for representation extraction to enhance plug-and-play usability. We will reconsider the term \"plug-and-play\" in the final version of our paper.\n\n---\n1. CoT-Valve: Length-Compressible Chain-of-Thought Tuning\n2. O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning\n\n## Q3\nQ: In Step 3 of 3.1 (Line 142), the PCA step of separating hidden state pairs into two halves seem unnecessary. The exact same result can be achieved by calculating the first principal component of all $d_i^{(-\\rightarrow+)}$'s.\n\n**A**:  PCA computes principal eigenvectors from the covariance matrix, which captures data dispersion around the mean. Splitting the vector set $\\{d_i^{(-\\rightarrow+)}\\}$ and reversing half **shifts the centroid and changes the dataset’s geometry**, likely resulting in a different covariance matrix and principal components compared to the original $\\{d_i^{(-\\rightarrow+)}\\}$.\n\nAs a simple example, consider four data points: $\\{1, 1, -1, -1\\}$, whose principal direction lies along the x-axis. Flipping the signs of the last two yields $\\{1, 1, 1, 1\\}$, collapsing the variance to zero and eliminating any principal direction.\n\nReturning to our experiment in Section 3, our goal is to extract a direction that captures the transition from slow to fast thinking. While this direction may already align with a principal component of $\\{d_i^{(-\\rightarrow+)}\\}$, by splitting the dataset and reversing half of it, we **amplify the variance** along the slow->fast axis. This introduces a separating hyperplane, making the target direction easier to isolate and more robust.\n\nTo test this, we conduct a controlled experiment using the principal component of $\\{d_i^{(-\\rightarrow+)}\\}$ as a baseline vector. We compare its performance with our method on DeepSeek-R1-Distill-Qwen-7B using AIME24. Results are shown below, where \\(\\alpha\\) is the intervention intensity (as in Figure 5), and each cell reports **Pass@1** (%) ↑ / **Output Length** ↓.\n\n|  | $\\alpha=16$ | $\\alpha=8$ | $\\alpha=4$ | $\\alpha=0$ | $\\alpha=-4$ |\n| --- | --- | --- | --- | --- | --- |\n| **Baseline** | 46.7 / 10447.0 | 52.5 / 11493.8 | 54.6 / 12367.3 | 52.5 / 12451.2 | 52.5 / 14747.4 |\n| **Our Paper** | 41.7 / **6232.5** | **53.3** / **8735.6** | 50.4 / **10942.7** | 52.5 / 12451.2  | **55.4/** 15144.8 |\n\nAs shown above, our control vector consistently achieves more effective length control and boosts accuracy when thoughts are extended, validating the effectiveness of our extraction protocol.\n\n## Q4\nQ: Can you further discuss the use of a sliding-window for difficulty evaluation?\n\n**A**: The sliding-window mechanism detects shifts from straightforward to complex reasoning via sudden increases in the difficulty score (Eq. 4). For such outlier detection, we use a dynamic threshold, defined as the mean plus standard deviation within the window. This method offers two key advantages:\n\n* **Generalizability**: Different models and tasks produce varying difficulty score scales. The adaptive threshold removes the need for manual tuning and works robustly across benchmarks.\n\n* **Accuracy–Efficiency Trade-off**: As shown in Figure 6 and Table 1, hard reasoning often begins with a spike in difficulty followed by sustained high scores. Our method captures this onset, which guarantees necessary pauses to encourage reflection, and avoids over-intervention afterward. Table 2 and Figures B1–B3 validate this balance.\n\nWe provide comprehensive ablation studies in Appendix B to validates the effectiveness of our method. Additionally, we compare our sliding-window approach with a **fixed-threshold baseline**, where the threshold is set to the average difficulty score computed on each test benchmark. The results (shown below) show that our sliding-window method consistently yields a better accuracy–efficiency trade-off.\n\n| **Methods** | AIME24 (Pass@1 (%) / Output Length) | MATH500 (Pass@1 (%) / Output Length) |\n| --- | --- | --- |\n| Original Inference | 52.5 / 12451.2 | 92.9 / 3403.9 |\n| Sliding-window control | **53.8** / **10850.9** | **93.7** / **3122.8** |\n| Fix-threshold based control | 52.1 / 13207.4 | **94.1** / 3811.72 |\n\n## Q5\nQ: It would be great if you can discuss the effects of extreme values of steering intensity $\\alpha$.\n\n**A**: To show the effects of extreme values of steering intensity $\\alpha$, we extend our experiments in Figure 5 by scaling up the absolute value of $\\alpha$ when controlling the thinking speed of DeepSeek-R1-Distill-Qwen-7B on AIME24. The results are shown below:\n|  | $\\alpha=64$ | 32 | 16 | 8 | 0 | -2 | -4 | -6 | -16 | -32 |\n|-|-|-|-|-|-|-|-|-|-|-|\n| **Acc** |0.0|6.2|41.7|53.3|52.5| 53.7 | 55.4 | 52.9 | 39.6 | 0.8 |\n| **Length** |Repetitive generation|1941.2|6232.5|8735.6|12451.2|14364.6|15144.8|15843.4|20241.7|Repetitive generation|\n\nAs shown in the table, increasing $\\alpha>0$ leads to progressively shorter responses. And when $\\alpha$ becomes too large, we observe repetitive generation behavior. Similarly, for $\\alpha<0$, increasing $|\\alpha|$ produces longer responses, eventually causing repetition."}, {"review_id": "YBRNQT2ZJa", "reviewer": "Reviewer_CQee", "summary": "This paper proposes using steering vector to control the reasoning speed of language models. It constructs \"slow\" and \"fast\" reasoning responses by prompting models with specific keywords and then creates representations for reasoning mode switching. Additionally, they introduce a mechanism to dynamically adjust the model's reasoning behavior by estimating question difficulty through token-level logit variation. Experimental results demonstrate that adaptive control via the steering vector can improve performance while reducing the number of tokens used.", "questions": "1. How frequently do models switch between reasoning modes? It would be helpful to include some analysis showing how often the model transitions between fast and slow modes during inference.\n2. Is the steering vector always applied? Are there cases where the steering coefficient $\\alpha$ is zero, preserving the model’s original reasoning behavior?\n3. For Qwen-3 model, how is fast reasoning data constructed? Do you try disabling the think mode when building fast reasoning data?", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper proposes to use the steering vector to change reasoning behaviors, which effectively controls the fast/slow reasoning behaviors of language models.\n2. The research questions addressed are timely and important. Determining when to switch between reasoning modes remains an interesting direction in the community.\n3. The writing is well-organized and clear, making the manuscript easy to follow.\n\nWeaknesses:\n1. The performance gains appear modest. Across different benchmarks and models, the gains are relatively limited, raising concerns about the practical impact of the proposed method.\n2. The method for constructing fast reasoning data is somewhat heuristic. Since keyword-triggered behavior can vary across models, this approach may introduce inconsistencies and limit generalizability.", "was_revised": true, "final_justification": "I have reviewed the authors' response and found that the additional analysis addresses my concerns. Overall, I find this work to be an interesting application of steering vectors for controlling model reasoning behavior. I have increased my score.", "rebuttal": "We are glad that you find our work timely and important. We hope our responses below could address your concerns.\n## W1\nQ: The performance gains appear modest, raising concerns about the practical impact of the proposed method.\n\n**A**: We acknowledge that, as an inference-time method, our performance gains may appear modest compared to optimization-based RL approaches [1,2]. However, we believe the **insights** offered by our work provide meaningful contributions to understanding and improving LLM reasoning. Specifically:\n* We **uncover latent fast-thinking capabilities** in LRMs via prompt-based induction and representation-level control.\n* From an interpretability perspective, our difficulty score and case studies offer a novel lens for **tracking and analyzing** reasoning processes.\n* We highlight a key limitation of existing representation editing techniques[3,4,5], which treat representation control as a static, single-axis intervention problem—typically controlling high-level behaviors (e.g., honesty, helpfulness) globally throughout generations. In contrast, we introduce **temporal adaptivity** as a second control axis, reframing representation intervention as a **dynamic, context-aware process**—a shift that opens new directions for future research.\n\nOur method is highly **extensible** and can be effectively integrated with existing reasoning strategies to achieve **non-trivial improvements**. For example, combining our speed control (Section 3) with parallel search on DeepSeek-R1-Distill-Qwen-32B (using $\\alpha=8$) leads to **significant gains** over regular generation, as shown below.\n\n*   AIME24\n|Methods|Pass@8|Pass@16|Pass@32|Pass@64|\n|-|-|-|-|-|\n|Original ($\\alpha=0$)|62.1|70.3|76.9|83.3|\n|$\\alpha=8$|**69.5**|**75.2**|**79.1**|83.3|\n\n\n*   GPQA Diamond\n|Methods|Pass@8|Pass@16|Pass@32|Pass@64|\n|-|-|-|-|-|\n|Original ($\\alpha=0$)|78.3|82.3|85.3|88.4|\n|$\\alpha=8$|**82.4**|**87.6**|**91.5**|**94.4**|\n---\n1.  O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning\n2.  Efficient RL Training for Reasoning Models via Length-Aware Optimization\n3.  Representation Engineering: A Top-Down Approach to AI Transparency\n4.  ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models\n5.  Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering\n\n## W2\nQ: Since keyword-triggered behavior can vary across models, this approach may introduce inconsistencies and limit generalizability.\n\n**A**: We first investigate the generalizability of keyword-triggered behavior across a diverse set of models using MATH500. We examine the generalizability of keyword-triggered behavior in non-reasoning models (Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct). Treating their original outputs as fast-thinking, we prompt extended reasoning using the trigger word \"Okay\" (inspired by Figure 2). This simple cue fails to elicit longer reasoning, likely due to the lack of specialized CoT training.\n\n|**Model**|**Slow-thinking**(Pass@1 (%) / Output Length)|**Fast-thinking** (Pass@1 (%) / Output Length)|\n|-|-|-|\n|Qwen2.5-7B-Instruct|74.8 / 654.0|75.9 / 655.2|\n|Llama-3.1-8B-Instruct|43.9 / 1188.7|41.5 / 702.2|\n\nFor reasoning models, we include Llama-3.1-Nemotron-Nano-8B-v1, DeepSeek-R1-Distill-Llama-8B and GLM-Z1-9B-0414. We treat their original responses as slow-thinking outputs and use \"To\" (consistent with our main paper) to trigger their fast thinkings. The results shown below validate the **generalizability of keyword-triggered behavior across reasoning models**.\n|**Model**| **Slow-thinking**(Pass@1 (%) / Output Length)|**Fast-thinking** (Pass@1 (%) / Output Length) | **Relative Differences**|\n|-|-|-|-|\n|Llama-3.1-Nemotron-Nano-8B-v1|93.7 / 3893.4|91.6 / 2760.6|-2.2% / -29.1%|\n|DeepSeek-R1-Distill-Llama-8B|88.2 / 3694.1|63.8 / 878.7|-27.7% / -76.2%|\n|GLM-Z1-9B-0414|96.0 / 2847.1|91.7 / 1691.8|-4.5% / -40.6%|\n\nWe show that our method is highly **generalizable across models**. Using **only** fast and slow-thinking data from DeepSeek-R1-Distill-Qwen-7B as stimuli for **all models**, we extract control vectors from each model and evaluate on AIME24 and MATH500. The consistent scaling effects suggest that fast and slow thinking are **intrinsic, shared capabilities of LLMs**. This transferability (1) removes the need for model-specific stimuli generation and (2) avoids inconsistencies stemming from keyword-triggered behaviors.\n\n| Models | AIME 24 ( Pass@1 (%) / Output Length )|  |  |  |  |\n| -|-|-|-| - |-|\n| |$\\alpha=8$| $\\alpha=4$|$\\alpha=0$| $\\alpha=-2$ | $\\alpha=-4$ |\n|**Non-reasoning** | | | | | |\n|Llama3.1-8B-Instruct |**8.3** / 783.8 | 5.4 / 1293.2 | 6.8 / 3501.2 | 6.7 / 3787.6 | **7.1** / 4748.7 |\n|Qwen2.5-7B-Instruct |9.2 / 1574.9 | **10.4** / 1841.0 | 9.6  /  1886.7 | **11.7** / 1959.4 | **10.8** / 3177.7 |\n|**Reasoning**| | | | | |\n| Llama-3.1-Nemotron-Nano-8B-v1 | 47.1 / 8072.1 | 62.1/ 10719.1 | 62.9 / 11044.5 | **65.0** / 11682.1 | **63.3** / 13414.8 |\n| DeepSeek-R1-Distill-Llama-8B | 35.0 / 10589.4 | 42.1 / 11164.8 | 50.0 / 13398.3 | **53.3** / 14709.1 | **51.7** / 15776.1 |\n| GLM-Z1-9B-0414 | 65.0 / 7870.0 | 67.1 / 8346.8 | 67.5 / 8883.45 | **71.3** / 9585.57 | **72.1** / 10150.1 |\n\n| Models | MATH500 ( Pass@1 (%) / Output Length ) |  |  |  |  |\n| --- | --- | --- | --- | --- | --- |\n|  | $\\alpha=8$ | $\\alpha=4$ | $\\alpha=0$ | $\\alpha=-2$ | $\\alpha=-4$ |\n| **Non-reasoning** |  |  |  |  |  |\n| Qwen2.5-7B-Instruct | 72.9 / 534.6 | 74.0 / 544.2 | 75.9 / 655.2 | **76.2** / 700.3 | 75.4 / 868.0 |\n| Llama3.1-8B-Instruct | **41.6** / 541.8 | **42.9** / 599.4 | 41.5 / 702.2 | **42.8** / 773.9 | 39.5 / 815.2 |\n| **Reasoning**  |  |  |  |  |  |\n| Llama-3.1-Nemotron-Nano-8B-v1 | 79.4 / 2117.6 | 87.7 / 3091.64 | 93.7 / 3893.4 | **94.3** / 4118.9 | **94.3** / 4303.09 |\n| DeepSeek-R1-Distill-Llama-8B | 75.9 / 2650.7 | 85.1 / 2895.3 | 88.2 / 3694.1 | **89.6** / 4241.2 | **90.2** / 4829.1 |\n| GLM-Z1-9B-0414 | 94.3 / 2215.0 | **96.2** / 2439.9 | 96.0 / 2847.1 | **96.5** / 3023.3 | **96.4** / 3265.5 |\n\n## Q1\nQ: How frequently do models switch between reasoning modes? It would be helpful to include some analysis.\n\n**A**: Thank you for your insightful suggestion. We analyze reasoning mode switch frequency for DeepSeek-R1-Distill-Qwen-7B on math benchmarks, finding averages of **578.6** (AIME24) and **167.2** (MATH500) switches per question. The higher frequency for AIME24 reflects its greater complexity, requiring more slow-thinking engagement.\n\nNext, to better understand the model's reasoning dynamics, we examine **how switching frequency varies throughout the thought process**. We divide the responses into three temporal segments: start (first 25%), middle (25%-75%), and end (last 25%). We then calculate the average token intervals between these switches (with lower values indicating more frequent switches). The results are presented below:\n| **Dataset** | | | Avg. token interval between switches|\n|-|-|-|-|\n||Start|Mid|End|\n|AIME24|17.8|20.9|54.9|\n|MATH500|14.5|22.5|41.7|\n\nWe found that (1) the model most frequently enters slow-thinking mode at the beginning, likely due to initial problem analysis requiring deeper processing; (2) the middle segment also shows high switching frequency, reflecting active reasoning during problem-solving; and (3) switching drops in the final segment, as the model converges on a solution and generates fewer new thoughts.\n\nFinally, we analyze **the relationship between internal thinking modes and model outputs**. First, we identify the top-100 tokens that most frequently trigger slow-thinking transitions:\n| **Type** | Most Frequent Words |\n|-|-|\n| Calculation | sqrt, denominator, ≈, triangle, product, ... |\n| Analysis | Problem, would, seems, because, find, ... |\n| Reflection | Wait, Alternatively, no, maybe ... |\n\nOur results demonstrate that models tend to switch to slow-thinking for (1) mathematical computations, (2) logical deductions, and (3) triggering certain reflection behaviors. \n\nTo assess **the alignment between internal mode switches and external outputs**, we measured how often the token \"Wait\", commonly used as a marker of uncertainty [1,2], coincides with actual transitions to slow-thinking mode. Surprisingly, on AIME24, the model outputs \"Wait\" 55.3 times per question on average, but only **1 in 12.2** instances aligns with a true mode switch. On MATH500, the ratio is **1 in 15.0**.\n\nThese results suggest that (1) the overuse of \"Wait\" reflects the **overthinking behavior**[1,2], and (2) analyses relying on output tokens to infer reasoning states [1,3] may be **misleading**, given the weak correlation with internal cognitive transitions.\n\nWe hope these findings will provide additional insights for interpreting LLMs' reasoning behaviors and inspire future research in this area.\n\n---\n1. Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs\n2. Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models\n3. Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs\n\n## Q2\nQ: Is the steering vector always applied? Are there cases where the steering coefficient is zero, preserving the model’s original reasoning behavior?\n\n**A**: The steering coefficient can be zero. Our sliding-window control operates in the following logic:\n```\nif current_difficulty_score > current_detection_threshold:\n    current_steering_intensity = minimum_steering_intensity\nelse:\n    current_steering_intensity += acceleration_step_size\n``` \nSince the minimum steering intensity is negative, the intensity will gradually increase (passing through zero) when no high-difficulty signals are detected.\n\n## Q3\nQ: For Qwen-3 model, how is fast reasoning data constructed?\n\n**A**: We followed the same protocol described in Section 2 to generate fast-thinking responses (i.e., prompting the model to begin its reasoning with \"To\"), consistent with all other models in our main experiments."}, {"review_id": "zwr0kWK8ul", "reviewer": "Reviewer_eb6B", "summary": "This work proposes a method to control the thinking speed of large reasoning models while maintaining or even improving accuracy. It identifies differences in leading words between slow and fast thought processes and introduces a thinking speed control method based on learned representation differences between fast and slow response pairs. The method's effectiveness is demonstrated across various datasets and reasoning models. Additionally, an adaptive speed control method is proposed, leveraging problem difficulty estimation, which shows performance improvements over the non-adaptive approach.", "questions": "Could the authors provide additional results on the generality and transferability of both the leading words and the control vector? Furthermore, how do the results in Figure 5 compare to prompt engineering approaches, such as providing different leading words as illustrated in Figure 2? These points are well addressed by the authors in the responses.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The identification of distinctions in leading words between slow and fast thinking processes is interesting. However, it is unclear whether this distinction applies to other reasoning models (e.g., o1) or non-reasoning models, as it might be a side effect of a specific reinforcement learning protocol. The thinking speed control method is intriguing, particularly the existence of a vector in the representation space that adjusts thinking speed. Yet, the generalizability of this vector across other reasoning and non-reasoning models remains uncertain. The work would benefit from deeper investigation into these core phenomena to better understand the proposed approach's broader applicability.", "was_revised": true, "final_justification": "I find this to be a technically solid paper. While I am uncertain about its practical applicability, I am inclined towards acceptance.", "rebuttal": "We are glad that you find our work interesting and our results intriguing. We hope our additional experiments and the responses below could address your concern.\n## W1 & Q1-part1\nQ: Could the authors provide additional results on the generality and transferability of both the leading words and the control vector?\n\n**A**: We appreciate this valuable suggestion. To evaluate the generality of keyword-triggered behaviors and the transferability of our control vector, we conducted experiments across a diverse set of models, including both non-reasoning and reasoning models:\n* For non-reasoning models, we include `Qwen2.5-7B-Instruct` and `Llama3.1-8B-Instruct`, which represents **different architectures and training protocols**.\n * For reasoning models: Since o1 is closed-source with limited API functionality, we instead evaluate the following open-source alternatives: `DeepSeek-R1-Distill-Llama-8B` and `nvidia/Llama-3.1-Nemotron-Nano-8B-v1`. Both models are Llama3.1-8B derivatives but employ **contrasting post-training approaches** (distillation vs. reinforcement learning), enabling direct comparison of RL effects. We additionally evaluate `GLM-Z1-9B-0414`, which features a **distinct architecture** from both Qwen and Llama families and is trained with alternative RL protocols, to further assess cross-architectural transferability. \n\nWe first evaluate the effectiveness of leading words for triggering slow and fast thinking modes across these models using the MATH500 benchmark.\n\nFor non-reasoning models, we treat their original outputs as fast-thinking responses and attempt to induce longer reasoning chains using the most common slow-thinking trigger identified in LRMs (Figure 2), i.e., the word \"Okay\". The results shown below demonstrate that this trigger fails to consistently extend the models' reasoning processes. This outcome is expected, as these non-reasoning models lack specialized long CoT training and have limited exposure to long reasoning data, making their slow thinking capabilities difficult to activate through simple prompting.\n| **Model** | **Slow-thinking**(Pass@1 (%) / Output Length) | **Fast-thinking** (Pass@1 (%) / Output Length)|\n| --- | --- | --- |\n| Qwen2.5-7B-Instruct | 74.8 / 654.0 | 75.9 / 655.2 |\n| Llama-3.1-8B-Instruct | 43.9 / 1188.7 | 41.5 / 702.2 |\n\nFor reasoning models, we treat their original MATH500 responses as slow-thinking outputs and elicit fast-thinking responses using the trigger word \"To\" (consistent with our main paper). The results below demonstrate **consistent keyword-triggered behavior** across diverse LRMs, showing robustness to variations in model architecture and training protocols.\n| **Model** | **Slow-thinking**(Pass@1 (%) / Output Length)|**Fast-thinking** (Pass@1 (%) / Output Length) | **Relative Differences**|\n| --- | --- | --- | --- |\n| Llama-3.1-Nemotron-Nano-8B-v1 | 93.7 / 3893.4 | 91.6 / 2760.6 | -2.2% / -29.1% |\n| DeepSeek-R1-Distill-Llama-8B | 88.2 / 3694.1 | 63.8 / 878.7 | -27.7% / -76.2% |\n| GLM-Z1-9B-0414 | 96.0 / 2847.1 | 91.7 / 1691.8 | -4.5% / -40.6% |\n\nNext, we evaluate the transferability of our control vector extraction method and the generalizability of the resulting thinking speed control across all models. After extracting the control vector from these models using the same method in Section 3, we use the extracted vector to manipulate the models' reasoning behaviors using different steering intensities on AIME24 and MATH500. The results are shown as follows. Despite the differences on the keyword triggering behaviors between non-reasoning and reasoning models, the scaling effects introduced by our intervention methods **generalized across all types of models**, demonstrating the **transferability** of our methods. Notebaly, to extract the models' representations for different thinking modes, we reuse the slow and fast CoTs generated by DeepSeek-R1-Distill-Qwen-7B across all tested models. The effectiveness based on this stimulus sharing further highlights that the ability to think in both modes should be an inherent ability shared across all models, which is more likely unrelated to any specific training protocols or certain model families.\n\n| Models | AIME 24 ( Pass@1 (%) / Output Length )|  |  |  |  |\n| --- | --- | --- | --- | --- | --- |\n|  | $\\alpha=8$ | $\\alpha=4$ | $\\alpha=0$ | $\\alpha=-2$ | $\\alpha=-4$ |\n| **Non-reasoning** |  |  |  |  |  |\n| Llama3.1-8B-Instruct | **8.3** / 783.8 | 5.4 / 1293.2 | 6.8 / 3501.2 | 6.7 / 3787.6 | **7.1** / 4748.7 |\n| Qwen2.5-7B-Instruct | 9.2 / 1574.9 | **10.4** / 1841.0 | 9.6  /  1886.7 | **11.7** / 1959.4 | **10.8** / 3177.7 |\n| **Reasoning** |  |  |  |  |  |\n| Llama-3.1-Nemotron-Nano-8B-v1 | 47.1 / 8072.1 | 62.1/ 10719.1 | 62.9 / 11044.5 | **65.0** / 11682.1 | **63.3** / 13414.8 |\n| DeepSeek-R1-Distill-Llama-8B | 35.0 / 10589.4 | 42.1 / 11164.8 | 50.0 / 13398.3 | **53.3** / 14709.1 | **51.7** / 15776.1 |\n| GLM-Z1-9B-0414 | 65.0 / 7870.0 | 67.1 / 8346.8 | 67.5 / 8883.45 | **71.3** / 9585.57 | **72.1** / 10150.1 |\n\n| Models | MATH500 ( Pass@1 (%) / Output Length ) |  |  |  |  |\n| --- | --- | --- | --- | --- | --- |\n|  | $\\alpha=8$ | $\\alpha=4$ | $\\alpha=0$ | $\\alpha=-2$ | $\\alpha=-4$ |\n| **Non-reasoning** |  |  |  |  |  |\n| Qwen2.5-7B-Instruct | 72.9 / 534.6 | 74.0 / 544.2 | 75.9 / 655.2 | **76.2** / 700.3 | 75.4 / 868.0 |\n| Llama3.1-8B-Instruct | **41.6** / 541.8 | **42.9** / 599.4 | 41.5 / 702.2 | **42.8** / 773.9 | 39.5 / 815.2 |\n| **Reasoning**  |  |  |  |  |  |\n| Llama-3.1-Nemotron-Nano-8B-v1 | 79.4 / 2117.6 | 87.7 / 3091.64 | 93.7 / 3893.4 | **94.3** / 4118.9 | **94.3** / 4303.09 |\n| DeepSeek-R1-Distill-Llama-8B | 75.9 / 2650.7 | 85.1 / 2895.3 | 88.2 / 3694.1 | **89.6** / 4241.2 | **90.2** / 4829.1 |\n| GLM-Z1-9B-0414 | 94.3 / 2215.0 | **96.2** / 2439.9 | 96.0 / 2847.1 | **96.5** / 3023.3 | **96.4** / 3265.5 |\n\nTo summarize, although keyword-triggering behaviors may vary across models, the ability to switch between slow and fast thinking is broadly present **across a wide range of LLMs** and can be effectively manipulated using our intervention methods. We therefore believe our approach has broad applicability. Moreover, we hope that the insights underlying our methods and experimental findings will offer valuable contributions to future research and deepen our understanding of LLMs' internal reasoning processes.\n\n## Q1-part2\nQ: How do the results in Figure 5 compare to prompt engineering approaches, such as providing different leading words as illustrated in Figure 2?\n\n**A**: Thank you for this valuable suggestion. We compared the Figure 5 results against three prompt engineering baselines:\n* Prompt1[1]:\n```\n
[instruction]\\nAnswer after a short amount of thinking. Do not spend excessive time double-checking your work.
<think>\\n\n```\n* Prompt2 (Fast-thinking):\n```\n
[instruction]
<think>\\nTo\n```\n* Prompt3 [2]:\n``` \n
[instruction]
<think>\\nOkay I have finished thinking.\\n</think>\\n\n```\nDue to figure restrictions during this rebuttal period, we present the performance under each prompt strategy alongside the two closest intervention result points from Figure 5. We experiment with DeepSeek-R1-Distill-Qwen-32B and report the results as follow:\n\n*   MATH500\n    \n\n| **Method** | **Length (x-axis)** | **Pass@1 (y-axis)** | **Token Efficiency (Pass@1 / Length, Slope,** %**)** |\n| --- | --- | --- | --- |\n| $\\alpha=0$ | 2947.3 | 94.2 | 3.20 |\n| $\\alpha=4$ | 2088.7 | 92.0 | 4.40 |\n| Prompt 1 | 2079.7 | 91.3 | 4.39 |\n| $\\alpha=8$ | 1495.8 | 87.6 | 5.86 |\n| $\\alpha=12$ | 1034.5 | 81.4 | 7.87 |\n| Prompt 2 | 967.1 | 79.1 | 8.18 |\n| $\\alpha=16$ | 720.5 | 74.3 | 10.31 |\n| Prompt 3 | 641.7 | 79.8 | 12.43 |\n\n\n*   AIME24\n\n| **Method** | **Length (x-axis)** | **Pass@1 (y-axis)** | **Token Efficiency (Pass@1 / Length, Slope,** ‰**)** |\n| --- | --- | --- | --- |\n| $\\alpha=0$ | 10679.6 | 69.2 | 6.47 |\n| Prompt 1 | 9510.7 | 65.4 | 6.87 |\n| $\\alpha=4$ | 8000.1 | 60.8 | 7.60 |\n| $\\alpha=12$ | 3893.7 | 38.3 | 9.84 |\n| Prompt 3 | 3436.4 | 26.2 | 7.62 |\n| Prompt 2 | 3076.3 | 27.1 | 8.81 |\n| $\\alpha=16$ | 1784.1 | 21.7 | 12.16 |\n\nWhen plotted on an x-y plane, the results from these prompt engineering methods **generally fall below the scaling curve** generated by our speed control approach. Moreover, our intervention method offers smoother and more flexible accuracy-efficiency trade-offs, demonstrating clear advantages over traditional prompt engineering techniques.\n\n---\n1. s1: Simple test-time scaling\n2. Reasoning Models Can Be Effective Without Thinking"}, {"review_id": "VJTJiWABVU", "reviewer": "Reviewer_duWd", "summary": "This paper introduces a dynamic thinking speed adjustment method for Large Reasoning Models. By analyzing two types of model outputs, i.e., fast, intuition-based answers and slower, reasoning-based responses, the authors propose summarizing a \"reading vector\" from fast and slow response samples to intervene the model's inference behavior. They first demonstrate that slowing down inference leads to improved reasoning performance. Then, they introduce a heuristic-based dynamic speed control algorithm that enables the model to switch between intuitive and fast reasoning states, achieving comparable results with improved computational efficiency.", "questions": "1. Would it be possible to visualize the $\\alpha$ values as a time series over the course of text generation? This could help assess whether the proposed sliding-window algorithm is performing effectively and provide insights into how the control signal influences generation in real time.\n\n2. What happens if $\\alpha$ becomes too large or too small? How can we ensure that such variations do not disrupt the semantic content of the generation? In other words, is the control direction disentangled from the actual content or the solution being generated?\n\n3. Some prior work introduces special tokens like \"think\" or \"pause\" to encourage slower, more deliberate inference. How does your approach compare to these methods, and are there potential complementarities? (E.g., \"Think before you speak: Training Language Models With Pause Tokens\" @ ICLR 2024)", "limitations": "The authors have adequately discussed the limitations in the appendices.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths:\nThe paper is easy to follow, and it is valuable that it tackles the challenge of controllability across different thinking modes by directly interpreting and intervening in the model's internal architecture. The problem is well-defined within a clear scope, and the experiments demonstrate the effectiveness of the proposed method. The work also raises interesting questions about whether classical tools such as PCA and information theory could further aid in understanding the \"flow\" of large language models.\n\n### Weaknesses:\nThe proposed method, in its spirit,, appears to be closely related to citation [41]. It may be helpful to better motivate this connection by introducing more of the high-level ideas behind [41], rather than focusing only on methodological overlap. In other words, does the theory in [41] help explain or justify the proposed heuristic intervention approach? If so, drawing that connection more clearly could strengthen the argument of this paper. (Please correct me if I’m mistaken.)\n\nAs the authors also acknowledge, the current method is largely heuristic. Some theoretical findings are expected in future research.", "was_revised": true, "final_justification": "I thank the authors for providing the additional experiments. Some more suggestions are raised, and I will maintain my score.", "rebuttal": "We are pleased that you find our work valuable, interesting, and our methods effective. We hope the responses below address your concerns.\n\n## W1\nQ: Does the theory in [41] help explain or justify the proposed heuristic intervention approach?\n\n**A**: We thank the reviewer for the suggestion to clarify the connection between our work and citation [41] (Representation Engineering, RepE). We agree that RepE inspires and provides a theoretical foundation for our method. But we also see our work as a valuable extension of RepE-like framework.\n\nFirst, our core motivation for manipulating LLMs' thinking speed stems from the hypothesis that **reasoning styles** (fast/slow thinking) belong to **high-level, abstract cognitive functions** within LLMs. This hypothesis aligns with the top-down, representation-centered view in RepE to identify such functions, in contrast to bottom-up circuit-level analysis. Moreover, our observations of keyword-triggered behaviors further support that different thinking modes are organized along **directional subspaces** in the model's representation space. This insight justifies our use of PCA to identify steering directions.\n\nWhile our method builds upon RepE's core principles, we significantly extend its framework both **theoretically and functionally**. Our early experiments revealed that static, global interventions could only govern model responses toward either fast (efficient) or slow (accurate) thinking modes. This limitation constrains our goal of enhancing both accuracy and efficiency simultaneously.\n\nWe attribute these constraints to a _fundamental limitation_ in existing RepE-based approaches: their interventions operate along a _single functional_ axis, applying uniform behavior controls across entire generations. To address this, one of our key contributions is the introduction of **adaptive control**, which reframes the traditional intervention paradigm into a **two-dimensional** control task by incorporating the **temporal dimension**. Specifically, we make the intervention reasoning-aware and context-specific by dynamically adjusting the strength of representation control based on the model's evolving internal reasoning state during generation.\n\nThis temporal adaptivity fundamentally distinguishes our work by focusing on **when and how models should think** (dynamic strategy control) rather than _what they should think (static behavior control)_, thereby bridging RepE into the domain of cognitive process control.\n\n\n## W2\nQ: The current method is largely heuristic. Some theoretical findings are expected in future research.\n\n**A**: Though our current methods, such as sliding-window-based adaptive control, remain heuristic, we believe that the core motivation for introducing temporal adaptivity, along with the success of our experimental results and analyses, **provides valuable insights** into the interpretability of LLMs' reasoning flow and may inspire future work to revisit the representation-control framework.\n\n## Q1\nQ: Would it be possible to visualize the $\\alpha$ values as a time series over the course of text generation?\n\n**A**: We provide the following statistical analysis to elucidate our adaptive control method's dynamics. Since the main function of our adaptive control methods is to force LRMs to switch to slow-thinking mode (i.e., set $\\alpha$ to negative values) when encountering difficult reasoning, we specifically study **how the frequencies of these switches evolve over time**. We first divide the model's responses into 3 temporal segments: **start** (first 25%), **middle** (25%-75%), and **end** (last 25%). We measure switching frequency by calculating the average token interval between these switches (where shorter intervals indicate more frequent switching). Below we present results from DeepSeek-R1-Distill-Qwen-7B.\n| **Dataset** | | | Avg. token interval between switches|\n|-|-|-|-|\n||Start|Mid|End|\n|AIME24|17.8|20.9|54.9|\n|MATH500|14.5|22.5|41.7|\nWe found that:\n1. The model tends to switch to slow-thinking mode most frequently at the beginning. This is reasonable, as LRMs typically exhibit complex and diverse reasoning behaviors at this stage, such as problem analysis, knowledge point recall, and solution plan development.\n2. The middle segment shows the second-highest switching frequency, as the model begins problem-solving. This phase often involves intensive reasoning and reflection on trial-and-error attempts.\n3. The switching frequency drastically decreases in the final segment. This is expected, as the model usually converges on its final solution and ceases to generate new thoughts.\nThese patterns demonstrate that our control algorithm produces reasoning dynamics that closely align with human cognitive processes.\n\nTo understand **how this control signal influences the generation**, we identify the top-5 most frequent tokens immediately after slow-thinking switches:\n```\n[\" no\", \" maybe\", \" let\", \" perhaps\", \" but\"]\n```\nThese hesitation markers consistently signal **reflections** and **reconsiderations** in models' CoTs. For quantitative analysis, we include detailed case studies in Figure B1~3, which further validate our method's effectiveness in triggering crucial reasoning behaviors for accurate problem-solving. These findings will be included in our final version of paper.\n## Q2\nQ: What happens if $\\alpha$ becomes too large or too small? How can we ensure that such variations do not disrupt the semantic content of the generation?\n\n**A**: To study the effect of extreme values of steering intensity $\\alpha$, we scale the absolute value of $\\alpha$ when applying our thinking speed control to DeepSeek-R1-Distill-Qwen-7B on AIME24. The results are shown below:\n|  | $\\alpha=64$  | 32 | 16 | 8 | 0 | -2 | -4 | -6 | -16 | -32 |\n|-|-|-|-|-|-|-|-|-|-|-|\n|**Acc**|0.0|6.2|41.7|53.3|52.5|53.7|55.4|52.9|39.6|0.8|\n|**Length**|Repetitive generation|1941.2|6232.5|8735.6|12451.2|14364.6|15144.8|15843.4|20241.7|Repetitive generation|\n\nThe results show that when $|\\alpha|$ becomes excessively large (e.g., $\\alpha=64$ or $-32$), the generation quality degrades significantly, leading to semantic collapse and repetitive outputs.\n\nRegarding the second question, we find that fully disentangling reasoning style from semantic content is challenging. Our experiments in Q1 show that steering with negative $\\alpha$ can induce reflective behavior, as evidenced by increased usage of tokens such as \"no\" and \"but\". However, we believe the underlying mechanism is more complex than simple token-level shifts, with **minimal semantic impact** in most cases. This is evidenced by (1) semantic disruption occurs only with extremely large values of $\\alpha$, and (2) the accuracy-efficiency trade-off remains stable across a wide range of $\\alpha$ values within reasonable bounds.\n\nThe above observations suggest that for reasoning benchmarks, the critical requirement for effective intervention methods is preserving the **semantic correctness** of generated content. To further assess our intervention methods, we compare our thinking-speed vector to 2 baseline vectors:\n* **Hesitation vector**: To test whether our control vector merely increases the likelihood of hesitation tokens, we construct a \"hesitation\" vector using a modified stimulus pair. The negative stimulus remains the same (slow CoT), but the positive stimulus is generated by appending the word \"Wait\" to its paired negative stimulus. The rest of the vector construction process follows our original protocol. Notably, we observe that this vector leads to semantic collapse and repetition at a much lower intensity ( $\\alpha' = 8$ ) .\n* **Correctness vector**: We also explore whether our control direction inadvertently overlaps with the model’s sense of solution correctness. We construct a \"correctness\" vector using paired correct and incorrect solutions from PRM800K as stimuli.\n\nWe measure the cosine similarity between our control vector and these two baselines across the last 10 layers of LRMs. The results below suggest that our vector exhibits only minor overlap with the hesitation vector ($\\approx 10^\\circ$) and remains largely orthogonal to the correctness vector. This further suggests that our control direction is **not trivially token-driven and does not interfere with task correctness**.\n|  |  |  |  |  |  | Layer |  |  |  |  |\n|-|-|-|-|-|-|-|-|-|-|-|\n| **Vectors** |19|20|21|22|23|24|25|26|27|28|\n|Hesitation|81.5|82.5|82.7|79.3|79.4|78.2|79.7|80.9|80.8|85.0|\n| Correctness |94.3|94.4|93.0|90.8|90.5|93.3|92.7|95.5|93.3|96.7|\n\n## Q3\nQ: Some prior work introduces special tokens like \"think\" or \"pause\". How does your approach compare to these methods, and are there potential complementarities?\n\n**A**: We thank the reviewer for pointing out the connection. The mechanisms behind their works and ours are quite different.\n\nThe pause-token methods introduce explicit delays via special tokens, which requires _retraining the model_. They extend the thoughts of models by _explicitly increasing the number of internal hidden states_.\n\nIn contrast, our vector-based intervention method introduces **no architectural or training modifications**. We operate on the LLM's representation space. Our approach is thus **lightweight, inference-time only, and generalizable** across diverse models without retraining.\n\nDespite these differences, we foresee a combination of our methods with these approaches. Since pause tokens offer additional computational bandwidth, our control signal could also be used to govern when and how that bandwidth is used. For models undergoing pause-token training, we can study the mechanisms for the trigger of pause tokens. This would provide more insights for studying the mechanism of reasoning flows."}]

Author rebuttal:

Meta-review: This paper proposes to control the reasoning speed of language models using steering vectors derived from prompting. It also introduces a method to automatically adjust the model's reasoning speed by question difficulty estimate via token-level logit variation.

The idea is novel and interesting, and the experiments are comprehensive. The reviewers raised some questions about the generality and theoretic justifications of the proposed methods. The authors did a good job in clearing those concerns.
Decision: Accept (spotlight)