TrimR: Verifier-based Training-Free Thinking Trimming for Efficient Test-Time Scaling
Authors: Weizhe Lin
Xing Li
Zhiyuan Yang
Xiaojin Fu
Hui-Ling Zhen
Yaoyuan Wang
Xianzhi Yu
Wulong Liu
Xiaosong Li
Mingxuan Yuan
Venue: ICLR 2026 Poster (2026)
Keywords: LLM; Reasoning; Thinking compression; Test-time scaling; Overthinking; Underthinking
Primary area: foundation or frontier models, including LLMs
TL;DR: A verifier-based, training-free, efficient framework to trim reasoning and enhance test-time scaling, explicitly tailored for production-level deployment.

Abstract: Large Reasoning Models (LRMs) demonstrate exceptional capability in tackling complex mathematical, logical, and coding tasks by leveraging extended Chain-of-Thought (CoT) reasoning. Test-time scaling methods—such as prolonging CoT with explicit token-level exploration—can push LRMs’ accuracy boundaries, but they incur significant decoding overhead. A key inefficiency source is LRMs often generate redundant thinking CoTs, which demonstrate clear structured overthinking and underthinking patterns. Inspired by human cognitive reasoning processes and numerical optimization theories, we propose TrimR, a verifier-based, training-free, efficient framework to trim reasoning and enhance test-time scaling, explicitly tailored for production-level deployment. Our method employs a lightweight, pretrained, instruction-tuned verifier to detect and truncate redundant intermediate thoughts of LRMs without any LRM or verifier fine-tuning. We present both the core algorithm and asynchronous online system engineered for high-throughput industrial applications. Empirical evaluations on Ascend NPUs and vLLM show that our framework delivers substantial gains in inference efficiency under large-batch workloads. In particular, on the four MATH500, AIME24/25, and GPQA benchmarks, the reasoning runtime of QwQ-32B, DeepSeek-R1-Distill-Qwen-32B, and Pangu-R-38B is improved by up to 70% with negligible impact on accuracy.

Published as a conference paper at ICLR 2026
TRIMR: VERIFIER-BASED TRAINING-FREE THINKING TRIMMING FOR EFFICIENT TEST-TIME SCALING
Weizhe Lin"*, Xing Li Zhiyuan Yang" Xiioojin Fuz; Hui-Ling Zhen', Yaoyuan Wang', Xianzhi Yu', Wulong Liu², Xiaosong Li', Mingxuan Yuan? Huaaei Advanced Computing and Storage Lab "Huawei Noah's Ark Lab (linweizhed) 1i. li.xing2, yangzhiyuan123, fuxiaojin, zhenhuiling2, wangyaoyuanl, yuxianzhi, liuwaleng, lixiaosong20, yuan.mingxusl @huawei. com
ABSTRACT
Large Reasoning Models (LRMs) demonstrate exceptional capability in tack- ling complex mathematical and logical tasks by leveraging extended Chain-of- Thought (CoT) reasoning. Test-time scaling methods as prolonging CoT with explicit token-level exploration-can push LRMs' accuracy boundaries, but they incur significant decoding overhead. A key inefficiency source is LRMs of- ten generate redundant thinking CoTs which demonstrate clear structured over- thinking and underthinking patterns. Inspired by human cognitive reasoning pro- cesses and numerical optimization theories, we propose TrimR, a verifier-based, training-free efficient framework to trim reasoning and enhance test-time scaling, explicitly tailored for production-level deployment. Our method employs a lightweight, pretrained, Instruction-tuner verifier to detect and truncate redun- dant intermediate thoughts of LRMs without any LRM or verifier fine-tuning. We present both the core algorithm and asynchronous online system engineered for high- -hthuhhhou industrial applications. Empirical evaluations on Ascend NPUs and vLLM show that framework delivers substantial gains in infer- ence efficiency under large-batch batch workloads. In particular, on the four MATH500, AIME24/25, and GPQA benchmarks, the reasoning runtime of QwQ-32B, DeepSeek-R1-Dee DpistillQween-221 and Pangu-R-38B is improved by up to 70 % with negligible impact on accuracy.
Large Reasoning Models
TrimR: Think Trimming
a an Exumple of Overthinking Danection with Varifers
Snarsesocy Thinking Ganeration
Asys, redundent Whisking detection and intervantion
Pt Ensheroe Novtng Cot
23
External Varifir Model Based
Overthinking Trimming
Underthinking Trimming
P2
IntnWoot motal
24 minutes, 200 misutes that arrwene?
4 Training-Free Smaller Verifier Model
WiarWatt 204 cprout Hines Alomativoyy moybe mace midasa inteppoting soad tine?
Answer Evistonos Cheeking Pronpe Pi Chasi Eite Salrenta onstence frre macosico moditi concustoe DonarAies heetation dr Atrndos, and anve 5M07
Solutions Converge
Con't Converge
ineviteennttvelyy maybe I need to present the arneer in hours converted renu 10s go 204 the strens
Guidance Prompts Stop LRMs Thinking ForcafullylGantl
#QUESTION yararinnt #EONTENT
maae/ salrady reoupte for long diess distcon fistt sxsver from the content ahove siheerads
Answer équiralence Checking Prompi Pa
Chesi Vite beo enower praicbed are bpwaay matherubaay
Shabalin 120 & fameliyg & Afrogiy
(eerGiven fya Monaeess.... Iinn (nmber criuuos the 2o
Repetition Truncation
SANSWERY (tAnmanr 7 JANSWERZ wir twe eegi
Thought wio Answer
Thought with Answer
Guidance Prompt to Stop Thinking
Conclusion
Figure 1: The TrimR framework. (1) CoT's are divided into sub-thoughts with reflection tokens in Sec. 3.1 (2a)
Overlunderthinking detection through answer convergence analysis in Sec. 3.2 3.3 (2h) repetition truncation in Sec. 3.4 (3) By simplifying redundant detection as the hinary answer existence and equivalence classification with prompts pl and P2 in Appendix[F] 7B instruction models replace PRMs/ORMs, avoiding instability from full-sequence scoring; (4) The verifier applies P1 to assess conclusion completeness in individual thoughts and p2 to identify overthinking when consecutive thoughts yield identical answers. Early termination is triggered with A Algorithm | 2 (5) Thinking termination prompts for LRMs are generated based on verifier decisions to halt redundant reasoning gently or forcefully in Sec. 3.5
'Equal contribution.

---

Published as a conference paper at ICLR 2026
Coudte Sesurery
tariad
Caned
Rapettion
26.00
PTooughss
#Thoughs
MATH500 Accuracy
(b) AIME24 Accuracy
(c) MATH500 #Thoughts
(d) AIME24 #Thoughts
Figure 2: Correlation between occurrence, accuracy, and the number of generated thoughts/tokens of QwQ- 32B on MATH500 (Hendrycks et al. 202T) and AIME24 (AIME202)). For hard questions, QwQ can spin out up 140 lengthy yet incorrect thoughts indicating need for underthinking trimming. For easier questions, QwQ delivers brief, highly accurate answers yet there's still room to make them even more concise by over- thinking trimming.
INTRODUCTION
Large Reasoning Models (LRMs) such as OpenAl ol (OpenAl 2024), DeepSeek RI (Guo et al. 2025), and Qwen QwQ (Team 2025) achieve expert-level performance on mathematical and sci- entific reasoning tasks by decomposing problems into manageable subproblems, conducting step- by-step analysis, detecting and correcting errors, and exploring alternative solutions. However, this extended "thinking" incurs substantial decoding overhead and inference cost, hindering deployment in resource-consee cooourceconnttrained settings (Guo et al. 2025)
Benchmarking on AIME24 (AIME2024) and MATH500 (Hendrycks al. 2021 reveals two key inefficiencies: overthinking and underthinking (Marjanović et al., 20255. Overthinking mani-
fests redundant verification of already correct often signaled by reflection tokens such as "Wait" or "Alternatively" -hich increases output length without accuracy gains (as in Fig. [ (2) Underthinking occurs on harder problems when the model oscillates among incomplete reasoning chains and fails to converge, producing lengthy yet inaccurate responses (Fig. Representative examples are provided Appendix G
The extended Chain Thought (CoT) reasoning in LRMs poses deployment challenges: decoding lengths vary widely and runtime scales superlinearly with sequence length. Training based methods such as (Yan et al. 2025 Chen et al.) 2024 Munkhbat et al. 2025 Yu et al., 2024) reduce token generation in LRMs but impose heavy training and computational costs on large models and may degrade their general capabilities. In contrast, training- free techniques integrate seamlessly and preserve original behavior: TokenBudget (Han et al.) 2024) dynamically adjusts token budgets via prompting, and Chain of Draft (Xu al., 2025) uses concise instructions to shorten output, yet both require invasive inference-time modifications.
We thus propose TrimR, a verifier-based, training-free approach for online thinking trimming while maintaining reasoning performance. We detect overthinking and underthinking in intermedi- ate thought answers with a lightweight verifier and trigger prompt-based LRM thinking early termination upon redundancy detection. By simply checking answer existence and equivalence in brief thought segments, our method leverages compact verifier models instead of complex Pro- cess Reward Models (PRMs) or Outcome Reward Models (ORMs) (Lightman et al., 2024). The non-invasive early termination preserves original LRM capabilities. Finally, we present an online asynchronous system collaborating LRMs and small verifiers to support industrial-scale workloads. TrimR offers a significant advantage by maintaining the performance and knowledge of LRMss through a targeted intervention strategy that is activated only when redundant reasoning is detected.
Notations. Given an input X, the LRM II generates a response [y1, ./..... 1t1 Vt--1] as the previously generated tokens. Each time ye: px. With slight abuse of notation, let L(XXyeel denote full response including y-t-
IT(X). We denote y-r =
and under- thinking as confidence-three threeshol and diminishing-rette returns processes with mathematical optimiza- tion. People typically stop thinking further after finding answers to simple questions and give up on complex tasks after too many unsuccessful attempts. We model reasoning process as an optimization problem in "language space", where LRMs traverse token trajectories and converge an optimal
Theoretical Foundation. Our method unifies human cognitive
2

---

Published as a conference paper at ICLR 2026
solution. Mirroring numerical optimizers' carly-stopping, we introduce termination criterion that halts reasoning once reasoning converges or marginal gains fall below a preset threshold. Formally, given an imput X with partial response y<t, the reasoning performance of LRM II is denoted as Per [X, K y=)lIT). We derive a compression rule e to determine the stopping time 1' E (0...., ( )- The goal minimize the inference cost of y <e referred to Serfferrrtyyysc.oo <t) without degrad- ing reasoning performance. min <())
Infer Cost(yce)
l' ccy
Per yer 2 Perf(X,
In designing TrimR, we conceptualized LRM reasoning as an iterative optimization-sty search, combining hypothesis generation, verification, and reflective exploration to avoid local optima. Similar to classical optimization practice, reasoning can be terminated when multiple explorations converge to consistent outcomes the model fails to break performance plateau. Overly permis- sive stopping criteria risk premature termination. Based on this insight, TrimR captures a verifier- supported stopping condition that achieves resource reduction while preserving accuracy. In this context, Perf y) denotes the performance a generated reasoning sequence for problem X (e.g., benchmark accuracy), and the compression rule e (TrimR) aims identify the shortest prefix yerr such that reasoning can safely terminate.
Contributions.
We propose a lightweight, training-free method for dynamic thinking trimming in LRMs-mitigating both over- and underthinking-vian a small verifier, enabling efficient
test-time scaling with negligible loss reasoning accuracy.
We develop asynchronous thinking trimming system for industrial-scale deployment, seamlessly integrating with existing inference infrastructures.
Through extensive evaluations and ablations on diverse models (QwQ-32B, Deepseck-RI- Distill-Qwen-33 -2BB and Pangu-R-38B), we demonstrate consistent reductions in reasoning cost and token usage across standard reasoning benchmarks in production settings.
2 RELATED WORKS
This section reviews recent advancements in efficient reasoning techniques for LRMs. For a more detailed discussion, see comprehensive surveys (Sui al., 2025; Qu et al., 2025; Su et al. 2025). The slight difference of the fast and slow thinking models motivates efficient long-to-short reasoning with model merging, which directly fuse weights of models with different thinking patterns using carefully designed algorithmss (Team al.) 2025) [w et al. 2025a). Apart from it, there are three types of commonly used efficient reasoning methods.
Training- based Methods. Approaches in this category modify or fine-tune LRMs to generate con- cise reasoning. Reinforcement learning with length-based rewards prunes verbose chains (Hou et al. Xia et al. 2025 Zhang et al. [025a] Team et al.) 2025 Chen et al., [024 Arora & Zanette Chen et al. |2255 Fang et al. 2025), while supervised fine-tuning teaches models to compress al.] 2025 Yu et al., (024). Latent-space techniques further minimize token usage by operating in a compact semantic embedding space (Hao et al. 2024). Although ef- fective, these methods demand substantial compute, risk task-specific overfitting, and may degrade general-purpose capabilities.
2025
2025
explanations (Munkhbat
Self-Evaluation Methods. These techniques prompt LRMs to assess their own confidence and decide when to stop reasoning. Adaptive schemes ask the model to predict the benefit of restart- ing (Manvi al. 2024) or to estimate certainty at key junctures (Yang al.) 2025). SelfThink (Zeyu et al. 2025) is proposed to facilitate I.RMs' intrinsic task complexity classification capabilities to dynamically switch between fast and slow thinking. ST-BoN leverages embedding-distance metries for carly truncation (Wang al. 2025b). While they avoid external models, the added inference steps may introduce latency. Some contemporaneous works intervene in inference to reduce token usage in reasoning: SpeedAdapt (Lin et al. 2025) adjusts hidden states dynamically to control the reasoning speed; Nhihh (Ma et al. 20255) suppresses reasoning entirely: AlphaOne (Zhang et al. 20256) modulates slow and fast thinking at test time based average thinking length.
Model Collaboration. Hybrid frameworks use auxiliary evaluation models, reward models, or thought proposers guide decoding. Dynasor monitors semantic entropy and ssward-model outputs
3

---

Published as a conference paper at ICLR 2026
for early stopping (Fu et al., 2024 Kuhn et al. 2023). Speculative Rejection uses partial-output scores from a reward model to terminate best-of-N search (Sun et al. 2022)). However, above works heavily rely on the performance of the reward model. Another work (Xi et al. (024) trains critique model which provides step-level feedback to guide the reasoning model on the fly. There are also concurrent works on improve LRM efficiency by generating thoughts with smaller reasoning models for speculative reasoning (Pan et al., 22255 Wang et al.) (025a)). CoThink (Fan et al. 2025) leverages an instruction model to guide reasoning and reduce reasoning steps.
Unlike prior work, our TrimR framework is training-free and non-invasive: a lightweight verifier dynamically detects and helps truncate redundant reasoning. No extra self-evaluation steps are troduced, so our method can be easily integrated into existing inference frameworks.
3
METHOD
We propose efficient, verifier-based, training-free thinking trimming algorithm that dynam- ically prunes redundant CoT generation during online inference. Our algorithm replicates human cognitive mechanisms which utilize internal verifiers to check and stop thinking. We introduce smaller verifiers to detect redundant thinking without fine-tuning verifiers or LRMs. Designed for sndust batch processing, our framework (Fig. []] comprises three modules: overthinking
trimming (Sec. 3.2). underthinking trimming (Sec. (.3). and repetition truncation (Sec. (34).
Guidance prompts for halting reasoning are detailed in Sec. 3.5 The online system is presented in Sec. Sec.B.6
3.1 THOUGHT DETECTION AND ANSWER EXTRACTION
The reasoning thoughts have clear structured patterns and are usually separated with reflection to- kens, such "VninWWaif" "'n' "Bn"", and " "'v' nAlternatively". In addition, LRMs normally gen- erate answers at the end of thoughts and then verify them as in Fig. |
During thinking, we periodically segment the thinking process into sub-thoughts P2.,27...77 rk]:
Think Seg(y [r1, P2..,, Pk]]
Here, separate it by reflection tokens (full list in Appendix E When new reflection tokens are detected asynchronously, we split the whole thought between two consecutive reflection tokens into sentences and concatenate the last several sentences of the current thought as the extracted intermediate thought answers. Formally, The last Nsent sentences of each sub-thought ri are N N. The last several sentences shorter but more
denoted as 81, i.e. 8; Kast_sentencest informative than the whole sub-thought and normally contain answers as demonstrated in Fig. [ We do not further process the answer but only skip the extremely short thoughts, which normally do not contain useful thoughts.
We detect predefined reflection tokens asynchronously so that LRMs continue generating tokens without throughput loss. Since reflection tokens account for only 0.5% of all outputs and thoughts are significantly less than tokens as in Fig. 2 the verifier invoked infrequently and can serve multiple LRMs concurrently as analyzed in Appendix M It makes the overhead of redundancy checks negligible compared to the gains from thinking compression.
3.2 OVERTHINKING TRIMMING
In overthinking scenarios, LRMs typically arrive at correct solutions using only 30 30% of the total generated tokens, yet continue producing redundant reasoning paths or alternative justifications before finalizing an answer (Chen et al. 2024). While this may aid complex or uncertain tasks, burdens simpler ones with uninformative content and higher latency and inference costs without curacy gains. We propose an overthinking detection and compression algorithm that uses lightweight verifier models to prune redundant reasoning while preserving LRM accuracy. It emulates verifier confidence based human thinking termination and convergence-bar based early termination mechanisms of numerical optimizers.
Simple Tasks to Utilize Smaller Models. We simplify overthinking detection as answer existing and equivalence checking, which are simpler binary classification problems than scoring the whole
4

---

Published as a conference paper at ICLR 2026
sequences. By reducing detection checking answer existence and comparing final outputs of consecutive intermediate thought answers, we can deploy compact (7B) verifiers with satisfactory language comprehension and instruction following capabilities without fine-tuning, greatly lowering inference overhead compared to full-sequence reward models (Lightman et al. 2024 Liu et al,, 2025). Moreover, training PRMs/ORMs for accurate scoring is complex and often yields unstable rewards on identical sequences, undermining reliable overthinking detection (Sun et al. 2024).
Two-Stage Verification. To minimize inter- vention from answerless intermediate checks, we introduce two-stage verification. First, we confirm the candidate solution is present in the current reasoning thought. We then ver- ify that intermediate thought answers are se- mantically or logically equivalent, regardless of correctness. Both tasks can be handled by the lightweight verifier Fo:
Algorithm 1:
Overthinking Compression
Input: Iaput X, repeat threshold
Outpot: Generated output Y
stopped + Falae; prev concluded thought None; count +-
+ pn (X):
while & <eos>> do 1RM iteratively
generates yt
// Ferindically segment and check for new concllded thought
F.(((())) defined in Eqn. r(s.)
defined in Egn. 2
y. </think> stopped = True, break; end if a new segment Sq is found and (p (s:)) and stopped Folse then if wrreu_concleded toght & None then if r then
(1) Answer existence checking: The set of thoughts with solutions St is defined as:
S {&i (p)((s)))
(oi((s))) = If pr. "Yes" (si))
se)
count +- count + 1:
(. "No" ((())))
else
1/ reset en mismarch t- 0;
(1)
where pi (8:) = pı (X, sx) (X omitted for sim- plicity) is the verifier prompt for answer exis- tence taking the problem X and thought answer
end
end
count
I/ early-stop after M repeats ye + pnl(X, yet, stop tokens)): stopped + True, break; end
8; as parameters.
(2) Equivalence checking: The equivalence r between two consecutive thoughts in S* is computed as:
pres.concaudeddhoought +- 86:
clse end
+ pn((X.ycct.o)):
F.(p2((,, = -T|pA. (y = "Yes" (2(8, s141)
end return (y
> PF. =No" P2(5) {(+1
(2)
where (2 8,) = (X, X,,,, 81, 83) lence, taking the problem X and two consecutive thought answers in S* that both achieve solutions.
()))( (X omitted for simplicity) is the verifier prompt for answer equiva-
Verifier Prompts. Verifiers take the answer existence prompt Pl and answer equivalence prompt P2 and directly return the binary classification results with the probabilities of "Yes" and "No" tokens. Therefore, only prefilling of verifiers is required. which reduces the computational cost. The two prompts for verifiers (p1 and P2) in Fig. [ with placeholders are available here. Details of placeholders are in Appendix [F. The answer existence checking prompt Pı consists of system prompt for ignoring unimportant formats and phrases, n-shot CoTs with positive and negative examples, the current question, and the intermediate answer. Similarly, the verifier only checks if two consecutive intermediate thought answers are semantically or logically equivalent with the prompt p2.
Verifier Prompt P1 Answer Existence: Check if the following sentences from rea- soning model reach an answer, regardless of formatting phrasing, and solve the question. Return "Yes" if the content linds a solution, oth- erwise 'No". Retur only 'Yes" explanation.
Verifier Prompt Pa Auswer Equivalence
Check if the two answers provided logically mathematically equivalent, regardless of formatting or phrasing. Retum "Yes' if they equal meaning/value and valid solutions the question, other- wise "No". Return only "Yes" or "No" with no explanation. SN-SHOT COTs (details Appendis)
with
Question: SQUESTION
SN-SHOT COTs (details Appendis)
Answer 1: SANSWERI
Question: SQUESTION
Answer SANSWER2
Content: SCONTENT yn. [Yex/No]:
Are they equivalent? [Yes/No]:
5

---

Published as a conference paper at ICLR 2026
Early Termination. We implement early termination through the prompt mechanism introduced in Sec. 3.5 specifically when the model consecutively agrees with previous reasoning steps M times. We only consider thoughts in S to bypass intermediate thoughts lacking definitive solutions. Such a protocol analogously replicates human cognitive pat- terns where reasoning halts upon achieving M 1 consecutive consistent solutions, paralleling the convergence termination criterion in optimization algorithms.
The process is described in Algorithm [T]
Resource Saving. By limiting inputs 200-400 tokens instead of processing full reasoner outputs (8K-128K tokens) as did in PRMs/ORMs (Sun et al., 2024), we drastically cut memory and com- pute overhead, boosting batch throughput and reducing verification latency with shorter verifier prompts. Also, we always reuse the KV cache for the system prompt and question with pre- fix caching, then batch answers in (SANSWER1 in P2). This lets us retain its cache across examples, only the cache for the other placeholder (aaeeer(AANNSS (SANSWER2, used by the first and third answers) needs updating.
reenapinn the second answer to a fixed placeholder
3.3 UNDERTHINKING TRIMMING
In particularly difficult tasks, LRMs repeatedly verify intermediate steps-an an indicator of uncer- tainty that Icads to divergent, redundant reasoning. We find that when the model oscillates between different thought paths, it seldom converges: proposed solutions often fail verification, triggering further exploration that inflates latency (Fig. (6) without boosting accuracy, and excessive diversity can even hinder convergence (Chen et al.) (024).
To mitigate this, we repurpose the verifier's snswerexissenee/conssseen feedback to detect hesitant overthinking. Concretely, if after Rehres% % of the token budget and Netres reasoning rounds, the model has not produced least three consecutive answer-bearing chains that agree on the same solution, we flag the run as underthinking (Appendix B Algorithm 2 At that point, a guidance prompt instructs the model halt further reasoning and summarize existing insights. It mimics human tendency to abandon difficult tasks that exceed its capabillities after multiple failed attempts.
This mechanism relies on two principles: (1) repeated convergence on the same solution indicates that the reasoning process is coherent and sufficiently thorough; and (2) the thresholds Rehree and Nibres must be calibrated afford the model adequate opportunity to explore alternative solution paths before termination. Ablation studies on these thresholds are presented in Appendix
3.4 REPETITION TRUNCATION
Recurrent token loops in LRMs often persist despite probabilistic sampling tweaks (e.g.. high tem- perature), especially when sequence length increases. Robust repetition detection and truncation therefore essential to prevent wasted computation and improve user experience. Our solution uses rolling-hash- detector to identify repeating token-II subsequences real time. By updating hashes incrementally for each new token, avoid recomputing whole-sequence hashes, enabling efficient, on-the-fly repetition checks. This dynamic algorithm is enabled by default unless otherwise specified. The effectiveness of this module is provided in Appendix [|
3.5 GUIDANCE PROMPTS FOR LRMS THINKING COMPRESSION
We devised two gentle and forceful guidance prompts (presented in Appendix F to guide termi- nating the reasoner's thought process. The gentle prompt curbs overthinking by steering the model toward concluding its reasoning with *Fiina Answer* An, whereas the forceful prompt both prevents underthinking and breaks any repetitive loops that occur before the designated "think end" token (e.g. </think>) is emitted. In our experiments, the forceful prompt consistently halted endless generation during the reasoning stage.
3.6 ONLINE SYSTEM DESIGN
Our inference system is built on vLLM (Kwon et al. 2023) (v0.6.0) running on Ascend-9100 64GB NPUs, though it can be integrated equally well with other inference frameworks (e.g. SGLang (Zheng et al.] 2024h) or hardware platforms (e.g. GPUs). As shown Fig. B it com- prises two tightly coupled elements: the inference engine itself, which orchestrates decoding via vLLM, and the Time Thinking Trimming System (T4S), which performs real-time reasoning compression in parallel. T4S is structured around Reasoning Verifier-ree responsiblee for detok-

---

Published as a conference paper at ICLR 2026
Overthininguuddee Delection Thread
Duickenizaion
Chain
(Chack Solutin
Data Tramamniie
Peparilin Diteciion
Chreck Coneasteni Solition
Verifier
Payss Cuttanl Data Heaee
Ope nas Diflvved Thaaars
Vertter Model
Found
Foternll Lighweight Vedieer
Engine
Message Controller
Mensaga
Oct
Sampling Controller
poue aargo Seq 2: yes. considennt willllt then 11 is the rackus of the first cirdin (ckoseet to B), which tergent both All BC tin mark lireedy thoughe liit The
Seqq 1: So aserta conmaitnnt lein Towe the attonse Whll clack anethar manbar.
Seg::
Figure 3: System design for the Test-Time Thinking Trimming System (T4S). The figure shows three se- quences that are flagged as (I) overthinking; (2) underthinking; and (3) repetitive generation. The inference engine streams updates through the Message Controller into the external T4S process, which issues asyn- chronous API calls lightweight verifier.
enizing output into discrete reasoning chains, detecting overthinking underthinking, and using an external small verifier model to confirm solution validity-a Message Controller that exchanges operational data (prompt and generated token IDs) with the verifier configurable intervals (per Naend tokens generated), and Sampling Controller that adaptively modifies output logits to en- force specific token generations.
4 EXPERIMENTS
4.1 EXPERIMENT SETUP
Benchmarks. This work focuses on mathematical and scientific problems: Mathematical bench-
marks: MATH500 (Hendryeks et al.] (021), AIME24 (AIME2024). AIME25 (AIME2025) and
Scientific benchmarks: GPQA Diamond (Rein et al. 2024). Details are available Appendix H We additionally include results on LiveCodeBench (Jain et al. (024) for dense and MoE models. As the primary emphasis of this work is on mathematical and scientific reasoning, code tasks are included for supplementary reference in Appendix O
Metries. Apart from accuracy (Acc.), we mainly care about the efficiency metrics. Runtime denotes the total wall-clock time to process all requests in each dataset. TPR is the average Time Per Request, while TPR-T90 is the TPR of the fastest/top 90% requests. #Tokens(M) is the number of generated tokens in millions. Runtime represents the total waiting time of requests, while lower TPR and TPR-T90 indicate better single user experience and higher Queries-per-Second (QPS).
Configurations. All the experiments conducted on servers with 8 Ascend 910B-64GB NPUs and 192-cores Kunpeng- 9200 CPUs with 1.5TB memory. All dataset requests (e.g., the 500 ques- tions in MATH500 are submitted to vLLM concurrently, and we record key metries such as total wall-clock time and per-request latency. We benchmark two open-source models (QwQ-32B and RI-Distill---en--------- 32((RIQ332B) alongside the closed-source Pangu-R- R-3gu using fixed input/output lengths of 2K/30K tokens. For readers' interest, the results on Pangu Pro MoE (Tang) el al. 2025), publicly released after the date of this study, are presented in Appendix N Although extending outputs to 1228 tokens yields marginal gains, such settings are impractical for produc- tion, so we cap the output at 30K. M=2, 550 The verifier is Pangu-7B 7B copen-sourceds (Pangu (Table
Team 2025)
(by default) and Qwen2.5-7BB IB-trrcutc (Qwen Team 2024)
4.2
MAIN RESULTS
Table shows that introducing TrimR delivers consistent and substantial efficiency gains across all three models and four benchmarks (up to 70% runtime reduction), with accuracy largely unaf- fected (less than 1.7% drop). Specifically, runtime is reduced by 16-39% for QwQ-32B (e.g., from 4,413s to 1818 on MATH500, -2.33%) 19-39% for --aguu-R--B(-333 an impressive 53-70% for RIQ-32B (-67.0% on MATH500; -70.0% on GPQA Diamond). Similar reductions are seen in TPR ge.g.. RI R-322B'ss TPR on AIME24 drops from 3,717.68 to 1,433.9s, -61.4%). Token usage also drops by 8-46% overall, with R R1Q-32B showing the largest reduction (from 2.447M to 1.320M tokens, -6..1% 1 on GPQA Diamond). Despite these gains, accuracy is preserved or even improved. QwQ-32B gains on MATH500 (+1.2%) and AIME25 (+0.8%), while R1Q-32B improves 2.0-13.2% on three benchmarks. For readers' interest, we discussed the reasons
on MATH500), and
Sinne AIME has only 30 questions, we replicate it eightfold ensure the engine receives enough requests.

---

Published as a conference paper at ICLR 2026
Table Performance comparison of QwQ-32B, Pangu-R-336 and DeepSeek-R1 -iiis------ittl-WW -wlll--QeeenBBB the MATH500, AIME24, AIME25, GPQA Diamond benchmarks. Relative improvements are highlighted green, and regressions in red.
Model
Runtime(s)
TPR(s)
(PR-)9(())
Acc.
#Tokens(M)
MATH500
QwQ 32B
4413
593.1
439.7
95.6%
2.278
TrimR
3118
499.7
377.4
96.8%
1.953
DeepSeek-HR Baeck--R--iiiiiee H--iitill-Qwennt -wenn3228
76412
733.1
278.8
92.4%
2.219
Trimn
2511 .0tu
315.8 260rs
218.0 21ass
94.4% zum
1,330 JUUS
Pangu-R-38B w/ TrimR
3665
447.4
300.8
95.6%
1.912
2426 53.89
367.6
264.6
94.4%
1.551
AIME24
QwQ 32B
6992
2437.6
2138.6
76.6%
3. 3.189
w/ TrimR
4255 -5.09
1572.6 33.5%
1431,6 3808
76.6% 4x
2.444 -2339
DeepScek-RIe Di-ies--R--sssii---eeeee
10299
3717.6
3156.0
66.6%%
3.252
w/ Trimk
4799 -3.05
1433.9 41.45
1228.7 41.5
70.0% 125
2.096 -5.85
Pangu- R- R-388 TrimR
6164
1912.3
1639.4
78.3%
2,466
3848
1299.4
1154.8
2008
76.6%
2,006
AIME25
QwQ 32B
7436
2771.5
2513.8
60.0%
3.426
w/ Trimn
6215 -6-4
2302.5 16.9%
2032.4 -922
60.8% bas
3,070 -m04s
DeepSeek-RR ---eckk------illlll-- Distill-Qwen Qwen-32B
13055
5474.3
4861.7
47.9%
3,932
TrimR
6169 -1.7%
1897.0
1549.6
56.3%
2.434
Pangu-R338B w/ TrimR
9216
3053.4
2723.6
57.5%
3.117
5591 19.05
1958.6 ox.
1731.6 -66-3
57.5% burs
2.470 2018
GPQA Diamond
QwQ 32B
4406
1302.6
1115.0
66.0%
1,572
TrimR
3198
1170.7
1025.5
65.2%
1.438
DeepSeek- R1 HI-Distill-Queer Qwen-32B
11366
3568.0
2786.2
45.4%
2.447
TrimR
3411 -mors
902.0 -127
720.4 -4.19
58.6% 13.29
1,320 -6.3
Pangu-R-38B w/ Trimk
3120
994.7
866.8
59.1%5
1.378
2516 19:15
901.2 9:9
788.9 4m
60.1
102
1,273
.748
behind such improvements in Appendix K Minor regressions are all under 2%, a reasonable tradeoff' for significant runtime reductions.
In T we include concurrent basclines (Certain-
Table Comparing TrimR with baseline meth- ods. Acc.: accuracy change relative to respective baseline; #Tok.: token usage reduction.
dex (Fu al. 2024). CoThink (Fan al.] 2025). SpeedAdapt (Lin et al. 2025); preprints at the time of submission, with some released after our initial draft). we find that TrimR consistently delivers substan- token savings while maintaining accuracy across datasets. For example, on R RIQ-32B with AIME24, TrimR reduces token usage by 35.6% with stable accuracy, whereas CoThink achieves only 12.5% reduction but suffers a sharp 13.3% accuracy drop. SpeedAdapt, which adjusts hidden states control reasoning speed, maintains accuracy reasonably well but yields only 4.0-9.3% token savings on QwQ-32B. Certaindex introduces extra inference cost yet still underperforms TrimR in token reduction. Over- all, TrimR strikes more favorable balance between efficiency and reliability relative to the baseline methods.
MATHS50
AIME24
Aec.
#Tok.
QwQ-32B
CoThink
-3.0%
-19.1
3.3%
-1622%
SpeedAdapt
0.0%
.4.0%
.9.3%
Trimk (ours)
1.2%
-14.3%
0.0%
-23.3%
DeepSeek--R --istiiD DDiistil-Qwen--3 Qwen-32R
Certaindex
-4.0%
-4.0%
-15.0%
CoThink
-2.0%
-36.6%
-13,3%
-12.5%
SpeedAdaapt
0.7%
-7.3%
1.5%
-12.7%
TrimR (ours)
2.4%
-40.1% %
3.3%
.35.6%
4.3 ANALYSIS
Effects of Trimming Methods. Ablation results in demonstattt that combining overthink- ing and underthinking trimming achieves the greatest efficiency gains with minimal accuracy trade- offs. For QwQ-32B on MATH500, overthinking trimming alone reduces TPR by 12.0% and tokens by 10.6% while improving accuracy by 1.2%, whereas underthinking trimming yields smaller gains (TPR: -4.8%, tokens: -3.3%) with a minor 0.6% accuracy drop. Their combination maintains the 96.8% accuracy while achieving TPR and token reductions of 15.8% and 14.3%, respectively.
For Pangu-R-38B3 the combined approach reduces TPR by 17.8% and tokens by 18.9% on MATH500 with negligible 1.2% accuracy loss. R1Q-32B shows stronger gains: combined
8

---

Published as a conference paper at ICLR 2026
Table 3: Ablation analysis of overthinking and underthinking trimming, showing that both methods markedly reduce TPR and token usage without compromising reasoning accuracy.
MATH500
AIME24
TPR #Tokens
Accuracy
TPR
#Tokens
Accuracy
Pangu-R-38B
95.6%
78.3%
w/ overthinking trinminng
-6.6%
-16.1%
95.4%
-3.22%
-5.7%
76.6%
w/ underthinking trimuinn
-10.5%
-8.3%
95.8%
-26 0%
-13.7%
75.4%
both
-17.8%
-18.9%
94.4%
132.1%
11.6%
76.6%
QwQ 32B
95.6%
76.6%
overthinking trinming
-12.0%
-10.6%
96.8%
-13.1%
-12.4%
76.6%
w/ underthinking trinuming
-4.8%
-3.3%
95.0%
-2700%
-16.1%
76.3%
w/ both
-15.8%
-14.3%
96.8%
-35.5%
-23.3%
76.6%
DeepSeck-RI-Die RR-----iiilll-eee------
90.4%
60.0%
w/ overthinking trinunnng
-3.5%
-28. 1%
91. 1.6%
-49.0%
-22.3%
63.3%
w/ underthinking trimming
-11.4%
-25.0%
92.8%
-57.1%
-29.8%
63.8%
w/ both
-56.9%
-40.1
92.4%
-61.4%
-35.6%
63.3%
trim slashes TPR by 56.9% and tokens by 40.1% while boosting accuracy from 90.4% to 92.4%.
Model-specific patterns emerge: QwQ-32B exhibits lower redundancy (overthinking TPR reduc- tion: 12.0% vs. 43.5% for RIQ-32B), Conversely, RIQ-32B's high token usage (2.447 vs. 1.572 for QwQ-32B on GPQA) reflects frequent veff-verification, which dynamic trimming mitigates. Varia- tions in trimming efficacy across models and benchmarks (e.g., QwQ-32B: MATH500 TPR-18 -120%% AIME24 -3.11% underscore the need to apply both strategies to optimize efficiency across
tasks.
Effects Distribution. Fig. depicts the empirical token-count distri- butions for our reasoning tasks before and after applying TrimR to QwQ- 32B. In the original (untrimmed) setting, approximately 64% of problem instances fell within the lowest bin (0-5K tokens), with the remainder spread across higher token ranges (5-32K tokens). After trimming, this proportion rises to nearly 70%, and the frequency of "long-com "long-context" in- stances (2 10K tokens) drops by over 25%. In particular, the heaviest tail (20-32K tokens) is reduced by more than two-thirds, from roughly 6% of cases down under 2%. This pronounced leftward shift in the dis- tribution demonstrates that TrimR effectively prunes superfluous context, lowering the average token footprint per query.
Olipkel SHD
390
300-
252
Sample
200
1:9
102
"iken Ranges
Figure 4: Token distribu- tions with/without TrimR.
Verifier Accuracy. To assess chain-level consistency, we used Pangu- R-38B to generate full reasoning traces for all MATH500 questions, split them into T4S-defined chains, and manually annotated answer consistency for 684 randomly sampled adjacent chain pairs. Table Verification accuracy, and downstream performance of Pangu- and Qwen2.5-77 7--7-Inttue verifiers on the MATH500 dataset. Verifier Acc. (%) denotes the fraction of correctly judged chain pairs; The rest are downstream performance on MATH500: MATH500 Acc.(% 3) is downstream task accuracy.
Verifier
Verifier (cc.(%)
Runtime
TPR
MATH500 Acc.(%) #Tokens(M)
Pangu-7B
87.87
3,665
447.4
95.6
1.912
w/ in context examples
85.67
3,894
455.1
2.032
Qwen2.5-7B. I--stunttt w/o in context examples
86.70
3,722
459.2
95.0
1.982
83.48
3,938
474.2
94.8
2.103
As shown in Table 4
Pangu-7B outperforms 1een2..--B-Instruue in annotation accuracy (87.87%
vs. 86. 86.70%) speeds up thinking trimming (3,665s vs. 3,722s), and reduces total tokens (1.912M vs. 1.982M). Downstream accuracy on MATH500 is essentially unchanged (95.6% vs. 95.0%) ), showing that occasional consistency errors have negligible effect. Ómitting in-context demonstrations in the verifier's prompt (in-context examples shown in Appendix F) slightly increases runtime and token use of the LRMs when the verifiers are applicd to the downstream task (Pangu-7B: +229s, +0.1 120M; Qwen: +216s, +0.121M). Limited by space, the full results using Qwen verifier are presented in Appendix P Overall, the choice of verifier model has negligible effects on the overall performance.
TrimR in Test-time Scaling with BoN. Beyond sequential token extension, additional test-time scaling approaches to improve LRMs accuracy include BoN sampling (Lightman et al. 2024).
Monte Carlo Tree Search (MCTS) (Wu et al. 2025b), and beam search (Lightman al.
2024

---

Published as a conference paper at ICLR 2026
Integrating TrimR with BoN (N=8), evidenced in Table 10 Appendix demonstrates significant reductions in token consumption (-13.8-16.2%) and runtime duration (up to 23.3%) while main- taining performance parity (-3.3% AIME24). These results highlight TrimR's s broad applicability across diverse test-time scaling frameworks.
5 CONCLUSION
This work introduces TrimR, a training-free, verifier-based framework that dynamically trims rea- soning in Large Reasoning Models (LRMs) eliminate redundant thinking. By leveraging lightweight pre-trained verifier to truncate unnecessary intermediate Steps, TrimR significantly improves inference efficiency without compromising accuracy. Empirical results on MATH500, AIME24/25, and GPQA benchmarks demonstrate up to 70% reduction in runtime across models, particularly in large-batch industrial settings. We also present T4S. TrimR's online deployment sys- tem integrated with Ascend NPUs/VLL.M, highlighting TrimR's scalability for high-throughput de- ployments. By balancing computational efficiency and reasoning rigor, TrimR offers cost-effective solution for real-world LRM applications, advancing the viability of production-level Al reasoning systems.
REPRODUCIBILITY STATEMENT
The following materials may help reproduce this work.
Sec. details the proposed method;
Sec. 3.6 discusses the system design of the proposed method;
Algorithms
and describe the underthinkingooeerhhiknngg trimming procedures;
Appendix offers the full prompts used verifiers;
Appendix
H describes the experimental setup detail;
This study mainly evaluates on open-soureed LRMs (QwQ-32B, DeepSeek- ---Seck--I--istill.
Qwen-32B, and Pangu-Pro-Mof. (Appendix (Qwen2.5-7B-Instte, and Pangu-7B).
N) and open-sourced verifier models
10

---

Published as a conference paper at ICLR 2026
REFERENCES
AIME2024. American invitational mathematics examination (aime), 2024.
AIME2025. American invitational mathematics examination (aime), 2025.
Daman Arora and Andrea Zanette. Training language models to reason efficiently, 2025. URL https://arxiv. org/abs/2502..44463 2025.
Qiguang Chen, Dengyun Peng. Jinhao Liu, HuiKang Su, Jiannan Guan, Libo Qin, and Wanxiang Che. Aware first, think less: Dynamic boundary self-awareness drives extreme reasoning effi ciency in large language models. arXiv preprint arXiv:2508. /582, 2025.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like llms. arXiv preprint urXiv:2412.21188 2024.
Siqi Fan, Peng Han, Shuo Shang, Yequan Wang, and Aixin Sun. Cothink: Token-efficient reasoning via instruct models guiding reasoning models. arXiv preprint arXiv:2505.22017 2025.
Gongfan Fang, Xinyin Ma, and Xinchao Wang. Thinkless: Llm learns when to think. arXiv preprint wrXiv92505.1337. 2025.
Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Aurick Qiao, and Hao Zhang- Efficiently serving IIm reasoning programs with certaindex. arXiv preprint arXiv:4412.20993, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint arXiv:2501. 2025.
Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen, Token- budget-aware IIm reasoning. arXiv preprint arXiv:24/2.18547 2024.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in continuous latent space. arXiv preprint arXiv:2412.06769, 2024.
Dan Hendrycks. Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2 103.038744 2021.
Bairu Hou, Yang Zhang, Jiabao Ji, Yujian Liu, Kaizhi Qian, Jacob Andreas, and Shiyu Chang. Thinkprune: Pruning long chain-of-though of Ilms via reinforcement learning. arXiv preprint urXiv:2504.012900 2025.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and lon Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024.
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation natural language generation. arXiv preprint arXiv:2302. 096644 2023.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Hunter Lightman, Vincet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2024.
Zhengkai Lin, Zhihang Fu, Ze Chen, Chao Chen, Liang Xie, Wenxiao Wang, Deng Cai, Zheng Wang, and Jieping Ye. Controlling thinking speed in reasoning models. arXiv preprint 230002507,03304,, 2025.
11

---

Published as a conference paper at ICLR 2026
Zijun Liu, Peiyi Wang, Rumxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint arXiv:2504.02499 2025.
Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint (rxiv:550400985: 2025.
Rohin Manvi, Anikait Singh, and Stefano Ermon. Adaptive inference-time compute: Llms can predict they can do better, even mid-generation. arXiv preprint arXiv:2410.02725, 2024.
Sara Vera Marjanović, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mehar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer. Xing Han Lü, et al. Deepseek-rl thoughtology: Let's; think; about IIm reasoning. arXiv preprini arXiv:2504.077 1288 2025.
Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se- Young Yun. Self- training elicits concise reasoning in large language models. arXiv preprint arXiv:2502.20122, 2025.
OpenAl. Openai ol system card, December 2024. URL https::/openai..cm//// openai -ol -ssstssssssee card/
Rui Pan, Yinwei Dai, Zhihao Zhang, Gabriele Oliaro, Zhihao Jia, and Ravi Netravali. Specrea- son: Fast and accurate inference-time compute via speculative reasoning. arXiv preprint vxXiv:2504.0789/ 2025.
Pangu Team. opennangg--mmbblldeee----ooooo 2025. URL http::://ai.gitcooo com
sssendd-trbbe//pppppaau-uu embeeddd---eddd-d-e--e...e 7bbmodde Ascend-native open-source reposi- tory for the openPangu Embedded 7B model.
Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang. Junxian He, et al. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond. arXiv preprint arXiv:2503.2/6644 2025.
Qwen Team. Qwen2.5: A party of foundation models, September 2024.
qwwellm.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel Bowman. Gpqa: graduate-level google-proof q&a bench- mark. In First Conference on Language Modeling, 2024.
Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and over- thinking: An empirical study of reasoning length and correctness in llms. arXiv preprint arXiv:2505.00127, 2025.
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint 26311:25003.6419,, 2025.
Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang. Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection. arXiv preprint arXiv:2410.20296 20020000, 2024.
Yehui Tang, Xiaosong Li, Fangcheng Liu, Wei Guo. Hang Zhou, Yaoyuan Wang, Kai Han, Xianzhi Yu, Jinpeng Li, Hui Zang, Fei Mi, Xiaojun Meng, Zhicheng Liu, Hanting Chen, Binfan Zheng, Can Chen, Youliang Yan, Ruiming Tang, Peifeng Qin, Xinghao Chen, Dacheng Tao, and Yunhe Wang. Pangu pro moe: Mixture grouped experts for efficient sparsity. 2025. URL https:
7/arxiv. org. abs//bba/25000 21411
Kimi Team, A Du, B Gao, B Xing, C Jiang, C Chen, Ci, Xiao, C Du, C Liao, et al. Kimi k1. 5: Scaling reinforcement learning with Ilms, 2025. URL https://arxiv arg/abs/2501./2599, 2025.
Qwen Team. Qwq 32b: Embracing the power of reinforcement learning, March 2025. URL https:///veee thub. ot/bll/g/////ooo/oooooooooooooot holloo/g/www/323 32b/
12

---

Published as a conference paper at ICLR 2026
Jikai Wang, Juntao Li, Lijun Wu, and Min Zhang. Efficient reasoning for Ilms through speculative thonnh.. arXiv preprint arXiv:2504. 19095, 2025a.
Yiming Wang, Pei Zhang, Siyuan Huang, Baosong Yang, Zhuosheng Zhang, Fei Huang, and Rui Wang. Sampling-efficient test-time scaling: Self-estimating the best-of-n sampling early de- coding. arXiv preprini arXiv:2503.01422 2025b.
Han Wu, Yuxuan Yao, Shuqi Liu, Zehua Liu, Xiaojin Fu, Xiongwei Han, Xing Li, Hui-Ling Zhen, Tao Zhong, and Mingxuan Yuan. Unlocking efficient long-to-short Ilm reasoning with model 2025a.
merging, arXiv preprint
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for Ilm problem-solving. In The Thirteenth International Conference Learning Representations, 2025b.
Zhiheng Xi, Dingwen Yang. Jixuan Huang, Jiafu Tang, Guanyu Li, Yiwen Ding, Wei Hc, Boyang Hong, Shihan Do, Wenyu Zhan, et al. Enhancing Ilm reasoning via critique models with test-time and training-time supervision. arXiv preprint arXiv:24/ 1,659, 2024.
Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms. arXiv preprint arXiv:2502. 12067, 2025.
Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengeheng He. Chain of draft: Thinking faster by writing less. arXiv preprini 250iv:2502. 18600, 2025.
Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. Inftythink: Breaking the length limits of long-contex. reasoning in large language models. arXiv preprint arXiv:2503.0660 06692,, 2025.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Zheng Lin, Li Cao, and Weip- ing Wang. Dynamic early exit in reasoning models. arXiv preprint 25X4:22504.1895, 2025.
Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system 2 into system 1. arXiv preprint arXiv:2407.0660 2024.
XING Zeyu, Xing Li, Huiling Zhen, Xianzhi Yu, Mingxuan Yuan, and Sinno Jialin Pan. Large reasoning models know how to think efficiently. In ES-FoMo III: 3rd Workshop Efficient Systems for Foundation Models, 2025.
Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. Lightthinker: Thinking step-by-step compression. arXiv preprint arXiv:2502.15589, 2025a.
Junyu Zhang, Runpei Dong, Han Wang, Xuying Ning, Haoran Geng, Peihao Li, Xialin He, Yutong Bai, Jitendra Malik, Saurabh Gupta, et al. Alphaone: Reasoning models thinking slow and fast at test time. arXiv preprint arXiv:2505.2444 24863, 2025b.
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, lon Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs. Advances in Neural Information Processing Systems, 37: 62557-62583, 2024.
13

---

Published as a conference paper at ICLR 2026
A USE OF LARGE LANGUAGE MODELS
We employed LLMs for proofreading the manuscript, while ensuring that all substantive points and contributions remain entirely original.
B ALGORITHM OF UNDERTHINKING TRIM
We provide the formulation of underthinking trim in Algorithm 2 The underthinking detection depends on the result of overthinking detection. If sequence can not converge to solution within the given budget Rthres and Nhhres, We too_tokens (Sec. 3.5
stop further thinking.
Algorithm 2: Underthinking Trimming
Input: Input X, undertinking threshold Rekeres Nihres
Output:
Generated ouiput Y
stopped <- False;
< (X));
while y: <<es>> do
iterately generates Be while loop
1f Check i. this sequence is flagged as overthinking
False indicates that there chains that agree on the same sclution is ss.orethhiniing t- check Algorithm P current state;
is.merthinking
consacutive anaver-bearing
thoughts Ien(hhénk Seg(Lshs
if yss </think>> then
stopped True, break end if sssoonnrthn Polsecand' Rehres M and num_thoughts Nehres and stopped False then + pm pn((X, Yet. stop.tokene)), stopped = True, break;
else
pn(( vet,
return (
C MORE ANALYSIS OF THINKING LENGTH AND ACCURACY
The distribution of decoding length occurrence and corresponding accuracy of Deepseek-RI- Soapsseek-RI-isttlll Qwen-32B (R1Q-32B) the MATH500 and AIME24 datasets available in Fig.
5a
5b
The
correlation the number thoughts and generated tokens are available Fig. 5c
5d
The Deepseek-BP - - DDistill-Qwen3311 model is weaker than QwQ-32B and tends generate long but wrong responses, which underthinking. The accuracy R1Q-32B quickly decreases from above 80%~~90% to 0% as visualized in Fig. 5a 5b In contrast, QwQ-32B still is able solve some complex questions as in Fig. 2a 2b addition, the repetition occurs more frequently, resulting in non-stopping 32K tokens. The number of thoughts of R RIQ-32B linearly correlates with the number of tokens, similar to QwQ-32B 32B in Fig. 2
Courte Bearery
Gred
Canedt
Zoon
Uncertanan
Oestriking
Counts
Cos
Underdenking CER
(a) MATH500 Accuracy
(b) AIME24 Accuracy
(c) MATH500 #Thoughts
(d) AIME24 #Thoughts
Figure Histogram of occurrence and accuracy of decoding length and scatter plot of the number of thoughts generated tokens of of Deepseek-RI-Distill-oe
(R1Q-32B) on MATH500 and AIME24 datasets
D DECODING THROUGHPUT/LLTENCYY OVER DECODING LENGTH
Deploying LRMs in large-scale production environments presents substantial challenges for im- proving reasoning efficiency. First, effective methods mitigate redundant reasoning in LRMSs
14

---

Published as a conference paper at ICLR 2026
2500
130
2500
10699 33599 Oeooedng Lengh Toheer)
1958e
Figure 6: Throughput reduction due to increasing length of Q-QB32B. The one-step attention complexity is linear to the sequence length or KV cache length, inference with long decoding length long input length has the same issue. For example, for the 32K decoding step, the latency of attention same the first decoding step with 32K input length, which significantly higher than the attention latency with shorter length.
critical, such inefficiencies significantly hinder the performance of inference systems in produc- tion. Since the generation latency of LLMs typically increases linearly with decoding length (as shown in Figure reducing unnecessary token generation reduction relative to the proportion of tokens saved. This, in turn, enhances the efficiency and seal- ability of test-time compute. Second, proposed solutions must be compatible with state-of-the-an inference infrastructures designed for large-scale deployment, such as vLLM and SGLang.
yield super-lineat lineer gains in runtime
E FULL LIST OF REFLECTION TOKENS
We utilize the following markers as reflection tokens to partition model reasoning into sub- thoughts: "But", "aii",""""", "ooweverr""""", ",mm""mmm,, nLnt me verify this", and "(n" nlltematiiill."" We do not utilize those without "nn'n" such as "but" and "But" as reflec- tion tokens to reduce the number answer existence checking with verifiers, because many are in the internal step checking before approaching answers. "nnkn" nn" is a strong structural separator for different thoughts Deepseek and Qwen QwQ-32B.
These reflection tokens were chosen by prompting the LRMs with small set of manually-craffie questions and then capturing the outstanding reflection phrases. Some of the questions are presented as follows:
Emma walks into her kitchen in the morning and finds the floor wet and several small puddles near the sink. She notices the faucet handle turned on, but water is flowing.
What most likely happened, and what should Emma do next?
An A/B test shows a +2.1% uplift with p=0.049 on day 7, but on day 14 the effect shrinks +0.7% with p=0.18. Provide at least two explanations and the next diagnostic checks. A classifier has 98% accuracy dataset where 97% of samples are negative. Why might this be misleading? Propose better metrics and quick verification.
(Continue on the next page)
15

---

Published as a conference paper at ICLR 2026
F DETAILED PROMPTS
Default Verifier Prompt for Answer Existence pt
Check the following seniences from reasoning model reach a cunclusion, regarilles of formalting or phrasing, and solve the question. Retur "Yes" if the content finds solution, otherwise "No'. Return only 'Yes' 'Na" with explanation.
Example I:
Question: 3-+33
Content: The answer 5, should return Yes.
Example
Question: +-3
Content: think should 5, but You should return Yos.
Example
Question: 2 + ++3=7 =
Content: Wait, think made mistake.
You should return Yes.
Example
Question: [(x) 85-2, what value of f(-2) (- f(-1) [00? Express
fraction.
Content: + and
Yos, 4 4. So. f(-2) f(-1) + f(0)
You should return Yes.
Example
Question: ((x) Bam2 what value of () f(-1) (0)? Express your
fraction.
Content: Since denominators are 3, we can add numerators: + = 14. Therefore, sum +
You should return Yes.
Example
Question:
fraction. Content: Wait, another thought: When adding fractions, is 14 the correct sam? Let's compute decimal to cross-verify. 14 divided approximately 4.666...
[() Bx J. what value of (- (- [0)? Express
You should return No.
Question: SQUESTION
Content: SCONTENT [YewNa]:
Verifier Prompt for Answer Consistency P2
Check the two unswers provided are logically or mathematically equivalent, regandless of formatting phrasing. Return "Yes" if they are equal meaning/value valid solution the question, otherwise 'No'. only 'Yes 'No' with explanation.
Example 1:
Question: =?
Answerl:
Answer2: the answer seoms to be five. Are they equivalent? [Yes/No]: Yes
Example Question: Define
P
R2
and
q
33
Find way to write
(1 3)
in terms of and
(p ()-1) 1) (q 1)
Therefore, the original double sum is equal to Answer 2: Given that terms are docreasing and positive, convergence seems plausible. However, since p a 0.4429 our partial sum about 0,3513, it's possible that the analytical result correct. Therefore, unless there's mistake in the analytical steps, the unswer should be q. Are they equivalent? [Yes/No]: Yes
Question: SQUESTION
Answer 1: SANSWERI
Answer 2: SANSWER2
Are they equivalent? [Yes/No]:
16

---

Published as a conference paper at ICLR 2026
Guidance Prompt Stop Thinking
Gentle Prompl (Overthinking Trim):
\ole I think already thought for long time, and should be able to the simple finel answer aftez **kinal Answezv Aa with \boxedf) and end thinking now! **Pinal Answare*(pp
Forceful Prompt (Underthinking Repetition Truncation):
\n/e I think already thought for long Lime, and should be able to deduce the finel
the content </think>\a/p
G OVERTHINKIIN//NDDETTIINNNN EXAMPLES
The following is an example overthinking in QwQ-32B. The question comes from AIME24. During the thinking process, reflection tokens like "Wait"""t" "Vi'nBuu" and "vrlnAllerraavivll""" appear times. After the reflection tokens, the content verifies the answer "204" which first appears in the first third of the entire reasoning process.
An example of overthinking phenomenon (QwQ-32B)
Every morning Aya goes 9-Kilometer-long walk and stops at coffee shop afterwards. When she walks at a constant speed of kilometers per hour, the walk takes her 4 hours, including minutes spent in the coffee shop. When she walks s kilometers hour, the walk takes her hours and 24 minutes, including minutes spent in the coffee shop. Suppose Aya walks her, including the minutes spent in coffee
Question:
kilometers
per hour. Find the number minutes the walk shop.
Answer: Okay, so have this problem about Aya's morning walk and coffee shop visit. Let me try parse it step by step ...... Convert that to minutes: 3460 +24= 180+24=204 minutes. Hmm, hut let confirm again. Wnin in the first scenario ...... So is that the answer? Waiit which is 204 minutes. nn't let me double-check total time? (na,WWait ...... which is 204
so 204 is correct. Hmm. Al-
ternatively, maybe made mistake in interpreting minutes. L'lteeeattaiellly ...... So that's consistent. \n nAlternatively. maybe there's error in calculation Aaii, hold on, let me re-derive that step minutes. So I think that's correct. An In'ennttrllyiellyy maybe the question is in minutes per kilometer or something else?
Adding 24 minutes gives 204
nllerraaiiie maybe need to present the answer in hours converted minutes? ...... 204 the answer. Waiit, but let think again. Let compute every- thing step by step once more to be thorough ...... Yes, so that's 's exactly 180 +24=204, Alright, so n.l.errraattiiiii Alltennativlly maybe the problem expects the answer in hours
I think that's 's solid ..... An
in Iteemtttivlll. maybe made mistake the quadratic equation ..... \n Alternatiivl, maybe should check with another approach
A Therefore, I think 204 is the correct answer. Maybe
I overcomplicating, but better safe than sorry. I'll go with 204, **Final *nswer* 204 </think>
(Continue on the next page)
17

---

Published as a conference paper at ICLR 2026
The example of underthinking in QwQ-32B is present below. The question taken from AIME24 is difficult and the LRM doesn't 't reach the correct answer. The reasoning contents switch between different approaches frequently and can't 't' provide an answer confidently. In fact, the figure 37/128 is made up without reasoning and the answer "475" has been checked to be wrong in previous thinking.
An example of underthinking phenomenon (QwQ-32B)
Each vertex of regular octagon is independently colored either red blue with equal probability. The probability that the octagon can then rotated so that of blue vertices end up at positions where there were originally red vertices is m where mn and v are relatively prime positive integors. What is m + n?
Question:
Answer: Okay, I need to find the probability that...... An Iteemnativlly using Burnside's 's lemma An Wait, Burnside's 's lemma counts the number of orbits, but I'm not sure. Aa nnltermativllyy let me think of follows ...... Thhrrffee,,,,, this the same as before. Perhaps the expected value approach
nBut expectation of the union is tricky, but maybe we can approximate it ....... (n iltterttttavvvllyy since B is randomly chosen, the events for different elements are not inleetteraaiiielyy since the problem is from an
independent, but perhaps can compute it Olympiad, the answer might be 37/128 or something ...... \n' ........ttaltt answer is 37/1287 nWiit let think differently equal to the sum over all non-zero rotations of ...... AntnBut without knowing the overlaps. it's hard This is getting too involved. Given that this is Olympiad problem, perhaps there's smarter way ...... An' nBut without the second term, can't compute it. An nAliernativel, perhaps assume the number
maybe the correct 'n' nnAlemaativlyy perhaps the number is
that the events are independent... ...... Wait, this is confusing. (ultermatiive of subsets... ...... suggests that this count is low. Alternntttyyvlllyy perhaps the correct 37/128? An'nWait
An)nBut the problem says "can be rotated that all blue vertices end up An'nn Therefoe, perhaps the count indeed 219, giving 475, Given that this a competition problem, and 475 is plausible answer (as 219+256=475), think this is the intended answer. [nnnThereoore, I will go with 475. **Final *nswer** 475
originally red vertices"
</think>
H DETAILED EXPERIMENT SETUP
Benchmark description: MATH500 (Hendrycks et al., 2021) comprises 500 problems spanning arithmetic, algebra, geometry, and calculus with varying difficulty levels. It evaluaaes models' ability in complex mathematical formalism, equation solving, and reasoning. AIME24 (AIME2024) sists of Olympiad-style problems assessing logical deduction and advanced problem-solving skills. AIME25 (AIME2025) features updated problems from the same competition as AIME24. GPQA Diamond (Rein et al. 20224) is a challenging dataset containing 198 multiple-choice questions, writ- ten by domain experts in biology, physics, and chemistry.
All the experiments are conducted on servers with 8 Ascend 910B-64GB NPUs and 192-cores Kunpeng-920 CPUs with 1.5TB memory. During decoding, TorchAir (Torch Ascend Intermedi- ate Representation) captures the computation graph to accelerate kernel dispatch and alleviate host- bound bottlenecks. The maximum number of concurrent decoding batches is set to 128. We config- ure vLLM to pre-allocate eight scheduler steps, thereby reducing scheduling overhead. All dataset requests (e.g., the 500 questions in MATH500) are submitted to vLLM concurrently, and we record key metries such as total wall-clock time and per-request latency. We employ the Qwen2.5 math evaluation tool to score the solutions (Yang et al.] 2024) and apply postprocessing to ensure that formatting quirks (e.g., spacing, notation style) don't penalize valid solutions.
In the BoN experiments, we use Pangu-ORM (close-source) as the Outcome Reward Model to select the best solution from the generated N solutions. The ratio of LRM and ORM is 1:1 in our experiments, although in production this ratio can be much higher.
I EFFECTIVENESS OF REPETITION TRUNCATION
As shown in Table 5 applying repetition truncation to DeepSeck-RI-Distill--eeeee
yields sub-
stantial efficiency gains without sacrificing- and even slightly improving accuracy: Enabling trun- cation reduces total runtime by 22.0%, cuts TPR by 50. 10%, and decreases token consumption by
18

---

Published as a conference paper at ICLR 2026
Table 5: Effects of Repetition Truncation five GPQA Diamond runs.
Model
Rumtime (H:M.S)
Accuracy
#Tokens
Detected Repetitions
RIQ-32B
4:02:51
5355.66 3568.02 02
0.444 0.454
3.09M
with repetition truncation
3:09:26
2.45M
29 our of 198
20.7%, while delivering 1% accuracy gain. Further analysis of truncated outputs confirms that most early termination occur during later stages of reasoning, where the model becomes stuck in particular attention patterns and fails to generate diverse contents. We also found that the guidance prompt effectively mitigates infinite repetitive outputs by steering the language model toward con- textually relevant generation grounded in prior analysis. This module thus serves as an effective, low-overhead component of the T4S, streamlining inference and enhancing answer clarity,
UNDERTHINKING THRESHOLDS
We chose our initial configurations for Retres and Nitres by prompting the model with a small suite of several challenging calculus problems that not within the test sets of the benchmark datasets. We inspected model traces: on these problems, Pangu-R-388 typically exhibited underthinking after ~55 tokens, whereas successful solutions consistently converged within 10K tokens and 20 rounds. Guided by these observations, we fixed a uniform Retres = 50% and Nehres = 20 for all benchmarks and models, under which performance remained strong.
One of the questions that can easily trigger underthinking is:
SN
Σ
In
1
x)
N1nn N.
Tasks:
1. Prove that SN admits full asymptotic expansion as N via the Euler- Maclaurin formula; compute explicitly the first four terms (including the constant term).
2. Identify the Bernoulli-number contributions and state rigorous remainder bound in terms of (m) Il on [0, 1].
3. Discuss numerical stability: which truncation order minimizes the actual error for moderately large N (e.g., N = 109_1
3.0
#Tokens (M)
Tokens Accuracy
2.8
Accuracy
78
2.8
78
*Tokens (M)
76.6%
76.6%
77.1%
76.6
16.6%76
76
Accuracy (%)
76.6%
76.6%
76
Accuracy (%)
Tokenss
2.6
75.3%
75.3%
75.3%
# 2.478.3%
2,404
2.444
2.515
2.513
74
2.4
2.396
2,406
2.444
2,464
2,45
74
280
72
72
2.2
70
70
30
40
50
60
70
Roves (%)
Noves
(a) Tokens and Acc. vs. Ribres-
(b) Tokens and Acc. vs. Nihres.
Figure Effect of token- and round-budget thresholds token and accuracy (QwQ-32B AIME24). Although performance largely insensitive to Ritres and Novezs accuracy degrades when Risves 40%.
After setting the hyperparameters used for deriving the main results, change these values and observe the performance as an ablation study. Figure ( accuracy and token usage against the token threshold Pihres and round threshold Nehres Raising Rihres from 30% to 40% increases
19

---

Published as a conference paper at ICLR 2026
curacy from 73.3% to 76.6% while tokens rise modestly (2.28 M to 2.40 M). Beyond 50%. accuracy plateaus but token count continues to grow. Likewise, increasing Nhre from 10 to 20 rounds boosts accuracy to 76.6% (2.44 M tokens) with clear gains thereafter. Though lowering the threshold to Rehres = 40% yields greater token savings without sacrificing accuracy, our default operating point set by inspection, though not optimal, already offers reasonably good trade-off between token efficiency and accuracy.
K DISCUSSION ON PERFORMANCE CHANGES
We performed case-by-case review of instances in which correctness shifted-either positively or negatively-afee after applying TrimR.
In those cases where accuracy improved, two phenomena were at work. First, on some occasions (approximately 30%), the model would descend into repetitive or lengthy generation and exhaust its token budget without ever producing a final answer; TrimR's early-exit mechanism enables the model to stop and emit its best partial solution instead, turning non-answers into correct outputs. Second, we observed that very long chains of thought can degrade model performance, as global attention becomes less effective correct solution midway through its reasoning but then later contradicted itself; by detecting conver- gence and triggering timely exit, TrimR preserves that correct intermediate answer.
extended sequences. In such samples, the model arrived at
On the other hand, a small fraction of samples saw a slight drop in accuracy. In these cases, the verifier identified premature convergence on an incorrect hypothesis often when the model re- peatedly reaffirmed the same wrong answer-thereby cutting off subsequent reflections that might have corrected the eтror. While such instances are uncommon, they highlight trade-off inher- ent in any early-exit strategy between minimizing wasted computation and allowing extra time for late-stage corrections. Overall, however, the net effect of Trimk remains strongly positive, yielding significant token savings with only minimal impact on accuracy.
L EXTENDED ANALYSIS ON DISTRIBUTION
180
300
original distribution
original distribution
160
distribution w/ TrimR
distribution w/ TrimR unchanged samples
unchanged samples
250
140
overthinking samples
overthinking samples
underthinking samples
underthinking samples
120
200
#Samples
#Samples
100
150
80
60
100
40
50
20
0
5000 10000 15000 20000 25000 30000
100
150
200
#okens
Rounds
(a) The distribution of number of tokens with and without TrimR
(b) The distribution of number of rounds with and without TrimR
Figure 8: The distribution of tokens and reasoning rounds with and without TrimR (QwQ-32B on MATH500).
The original distribution is indicated by the black curve, while the unchanged, overthinking, and underthinking samples shown stacked bars. After applying dynamic think trim, both the total tokens and number of reasoning rounds are substantially reduced compared to original distribution.
20

---

Published as a conference paper at ICLR 2026
As shown in Figure Sa dynamic thinking trim produces marked leftward shift in the token-usse usage distribution. In the uncompressed model, the "knee" lies around 3,500 tokens, with a substantial tail beyond 10,000 tokens. After truncation, over 80% of samples require fewer than 6,000 tokens, a roughly 30% increase. Also, the heavy tails (> 15,000 tokens) are nearly eliminated. Decompos- ing the stacked bars reveals that unchanged samples remain tightly clustered in the low-token bins (54,000 tokens), overthinking samples are effectively truncated into the lower-range bins (0-8, ,000 tokens), and underthinking samples (<5% of cases) occupy lengths that are modest relative the original tail.
parallel effect appears in the reasoning- trivial fraction extends beyond 50 rounds (with outliers over 200), whereas after trimming over 85% of samples complete within 30 rounds-an an increase of nearly 20 percentage points the <200-round regime. Overthinking cases shift from the heavy tail into the 0-30-round interval, while underthink- ing cases, though rare, have been truncated to mid-range (50-120 rounds).
distribution (Figure (b). Prior to trimming, a non-
Taken together, these results show that TrimR preserves valid reasoning, curtails redundant over- thinking, and minimally affects cases needing additional confirmation markedly improving inference efficiency in both token and round dimensions.
M ONE VERIFIER CAN SERVE MULTIPLE LRMS
A single verifier model can simultaneously support multiple LRM instances. Runtime performance data indicates that when serving a single LRM instance, the verifier's computational workload re- mains within manageable thresholds. Each LRM instance generates an average request rate of requests per second, while the verifier demonstrates an average processing capacity of approxi- mately 128 requests per second establishing LRM-to-verifeer request ratio of 14:1 under ideal conditions. However, production systems adopt a conservative 8:1 deployment ratio to maintain operational safety margins.
When accounting for verifier infrastructure costs, TPR (Time Per Request) improvements must be adjusted by a cost-efficiency factor: 14x8 14x8=1 99.1% (8 Ascend 910-64GB per LRM instance, Ascend 910-64GB per verifier instance). This calculation demonstrates that the performance gains per computational instance remain effectively preserved (9).1%) despite the additional verification overhead. is noteworthy that enhancements in token utilization and user-perceived TPR cannot be further optimized by scaling the number of computing instances within the cluster. In contrast, TrimR demonstrates significant performance gains by reducing user waiting times, achieving mea- surable improvements in latency reduction.
N PERFORMANCE ON PANGU PRO MOE
We present the results on an open-sourced Pangu model, Pangu Pro MoE, released after the date of this study, in Table 6 The results show that TrimR can readily reduce the token usage with very minor or no performance regression.
21

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison of Pangu-Pro-Mer the MATH500, AIME24, AIME25, and GPQA Diamond benchmarks. Relative improvements are highlighted green, and regressions red.
Model
Runtime(s)
TPR(s)
TPR-T90(s)
Acc.
#Tokens(M)
MATH500
Pangu Pro MoE w/ TrimR
6520
805.4
569.2
95.2%
16618
4900
662.2
488.3
95.6%
1.383
AIME24
Pangu Pro MoE w/ TrimR
3047
1263.9
1096.3
80.0%
0.269
2231 26.3%
1111.0 4218
989.1 9988
76.7% 3.3%
0.231 -14.28
AIME25
Pangu Pro MoE w/ TrimR
3273
1361.7
1194,4
70.0%
0.290
2256
1225.8
1112.3
66.7%
0.256.1.79
GPQA Diamond
Pangu Pro MoF
7729
1068.0
927.3
76.6%
1.347
w/ TrimR
6415 -0.0%
924.3 .3.5%
878.8 45.3%
76.0% 4.5%
1.146 .4.5%
O RESULTS ON LIVECODEBENCH
Table 7 shows that, similar to the results on Table Performance and token usage reduction on LiveCodeBench after applying TrimR. Acc.: accuracy change; #Tok.: token usage change.
mathematical and scientific reasoning, TrimR can readily reduce token usage by around 10% while preserving the performance of models.
Beyond mathematical and scientific reasoning, the results on LiveCodeBench (Table 7 further demonstrate the robustness of TrimR. TrimR achieves nearly 10% token reduction with accu- racy drops consistently below 1%. This trade-off
Model
Acc.
#Tok.
QwQ-32B
-0.6%
-9.4%
Pangu Pro MoE
-0.8%
- -10.2%
indicates that the method effectively removes redundant exploration while retaining the essential steps needed for correctness. Moreover, its effectiveness across both dense (QwQ-32B) and MoE (Pangu-Pro-MoO)) architectures suggests that TrimR is largely architecture-agnosiic.
P PERFORMANCE OF QWEN 7B AS VERIFIER
The performance of QwQ-32B (LRM) and Qwen2.5-7B-Insttee (verifier) with and without TrimR across benchmarks is presented in Table g negligible effects on TrimR's effectiveness.
The results show that the choice of verifier model has
Table 8: Performance of QwQ-32B (LRM) and Qwen2.5-7B 1-- InIstruct (verifier) with and without TrimR across henchmarks. Relative changes are shown in parentheses.
Dataset
Model
Runtime (s)
TPR (s)
TPR-T90 (s)
Acc.
MATH500
QwQ 32B
4413
593.1
439.7
95.6%
MATH500
QwQ 32B w/ TrimR
3169
(-28.2%)
507.1 (-14.5%)
380.3 (-13.5%)
96.6% (+1.0%)
AIME25
QwQ 32B
7436
2771.5
2513.8
60.0%
AIMB25
QwQ 32B w/ TrimR
6321
(-15.0%)
2342.2 (- 15.5%)
2061.3 18.0%)
60.8% (+0.8%)
GPQA Diamond
QwQ 32B
4406
1302.6
1115.0
66.0%
GPQA Diamond
Qw 32B w/ TrimR
3305 (-25.0%)
1185.4 (-9.0%)
(-7.0%)
66.0% (0%)
TIMING OF EARLY STOPPING
We summarize an additional study examining verifier behavior under multi-step checking.
We analyze the trigger timing of early-stop signals by comparing them against human-annotater gold labels. Gold stopping points are defined as three consecutive reasoning segments that lead semantically equivalent answers. Using both Qwen-based and Pangu-hased verifiers, early-stop
22

---

Published as a conference paper at ICLR 2026
signals were triggered average 1.44 and 1.37 reasoning turns later than the ground-truth optimal stopping point, respectively. Importantly, only 0.5% (Qwen) and 0% (Pangu) of early-stop signals were activated earlier than the annotttdd gold labels.
These observations suggest that although per-step verifier accuracy is not perfect, premature stop- ping events extremely rare. In over 99% of cases, early-stop signals do not trigger earlier than the gold stopping points, indicating that TrimR remains robust under imperfect single-step verification. Additional discussion and visualizations will be included in the camera-ready version.
R
VERIFIER MODEL BIAS AND GENERALIZATION
We further analyze the impact of verifier model characteristics detection behavior and overall system performance. The study focuses on two aspects: verifier scale and cross- pross-model generaliza- tion.
To evaluate the effect of model scale, we experimented with larger verifier models, including Qwen 32B and Pangu 38B. Compared with smaller variants, these larger models produced an approximate 1.5% improvement in detection accuracy. This observation indicates that stronger verifier models can enhance verification reliability, while introducing a predictable increase in computational over- head during the verification stage.
We additionally examine cross-model consistency by comparing multiple 7B-scale verifiers from different model families (Qwen, Pangu, and Llama). Across these variants, detection accuracy re- mained stable within the 85-90% range. No systematic performance degradation was observed when switching between verifier architectures.
Overall, these results suggest that verifier capability primarily affects the accuracy-efficiene efficiency trade- off, while the trimming framework itself exhibits stable behavior across different verifier model families.
CONVERGENCE COUNTER DYNAMICS AND SENSITIVITY TO M
We analyze the behavior of the convergence counter used in TrimR and evaluate the impact of al- ternative counter update rules, In particular, we compare the default reset-based logic against non-resetting variant. Under the non-resetting strategy, early-stop signals were triggered more fre- quently, yielding an approximate 4% reduction in token usage. However, this efficiency gain was accompanied by notable accuracy degradation (AIME24/25: -6.6%, MATH500: -2.1%). These results indicate that reset-based counter dynamics provide more reliable convergence behavior, pre- venting premature termination under transient reasoning fluctuations.
We further conduct an ablation study over the termination threshold parameter M E (1,2,3). Ta- ble 9 summarizes the relative changes in accuracy and token consumption.
Table 9: QwQ-32B
Effect of M on Performance (relative changes in Accuracy / #Tokens)
Setting
MATH500
AIME24
GPQA Diamond
M
3% / -6%
6.6% / -8%
-2.1% / -9%
M 2 (Baseline)
0% 0%
0% 00%
0% / 0%
M =
+ 48%
+7.7%
+0.2% +5.4%
The results show that M = significantly harms accuracy, suggesting insufficient allowance for self-correction. Increasing the threshold to . = 3 produces only marginal accuracy improvements while incurring a noticeable efficiency penalty. The default choice M = 2 consistently provides the best balance between accuracy and computational efficiency.
T TEST-TIME SCALING WITH BoN
Beyond sequential token extension, additional test-time scaling approaches to improve LRMs accu- racy include BoN sampling (Lightman et al.] 2024). Monte Carlo Tree Search (MCTS) (Wu et al. 20255), and beam search (Lightman et al. 2024). Integrating TrimR with BoN (N=8), evidenced
23

---

Published as a conference paper at ICLR 2026
in Table 10 demonstrates significant reductions in token consumption (-13.8-16.2%) and runtime duration (up to 23.3%) while maintaining performance parity (-3.3% % on AIME24). Table 10: Integrating TrimR with of-t-of-N (BoN) (Pangu-R, R- 38B) yields comparable efficiency improvements and token reduction while preserving the accuracy- peruraccperfo trade-offs.
MATH500
AIME24
Runtime
TPS
Acc.(%)
#Tokens(M)
Runtime
TPS
Aoc(%)
#Tokens(M)
BoN
19,423
97.4
13.055
35.084
1.628
86.6
18.882
w/ TrimR
16,171
562
97.2
11,251
26,911
1,281
83.3
15.822
U LIMITATIONS
In oder to improve the overthinking and underthinking detection accuracy, introduce an addi- tional small 7B verifier model. may require additional hardware for verifiers during deployment, which may increase the complexity of the inference system. However, our algorithm and online system design can dramatically reduce the LRM inference cost. In addition, one verifier can serve multiple LRMs as discussed in Sec. M so the additional deployment complexity for small veri- fiers is acceptable compared with the runtime reduction of LRMs. In addition, verifiers can also be deployed in the same hardware with LRMs and they can be collaborated in the way like serving smaller speculative models and larger generation models in speculative decoding.
TrimR currently relies on curated set of reflection tokens perform reasoning segmentation. While such lexical markers are widely adopted in recent LRM designs, future models may exhibit weaker or entirely different structural cues. Although segmentation can be re-adapted via corpus analysis, sampling- based updates, or learned segmentation modules, the robustness of this compo- nent may vary across model paradigms. Empirically, manual annotation sampled traces suggests high segmentation accuracy under current models; however, adapting to models without explicit reasoning markers remains an open direction for future work.
V BOARDER IMPACTS
Large Reasoning Models are widely used provide more guidance with internal thinking process and improve user experience. Our work improves the efficiency of LRMs, which is helpful to reduce the cost and carbon footprint of LRMs. This approach can enhance the accessibility of LRMs for broader population while minimizing environmental impact.
24


Reviews:
[{"review_id": "u7YniRFnkM", "reviewer": "Reviewer_Px7t", "summary": "This paper addresses inefficiency in Large Reasoning Models (LRMs) that generate extended Chain-of-Thought reasoning. The authors identify two redundancy patterns: overthinking (redundant verification of correct solutions) and underthinking (oscillation without convergence on hard problems). They propose TrimR, a training-free framework that uses a lightweight 7B verifier model to detect redundancy by checking answer existence and equivalence in intermediate reasoning segments. When redundancy is detected, the system uses prompt-based early termination. The approach achieves up to 70% runtime reduction with negligible accuracy impact across mathematical and scientific reasoning benchmarks (MATH500, AIME24/25, GPQA) on models including QwQ-32B, DeepSeek-R1-Distill-Qwen-32B, and Pangu-R-38B. The paper includes both the core algorithm and an asynchronous online system designed for production deployment.", "questions": "How does the verifier's 87-88% accuracy propagate through multiple sequential verification checks during long reasoning chains? Have you analyzed the cumulative error rate and specifically quantified how often false-positive convergence detection (claiming convergence on wrong answers) occurs versus beneficial early stopping?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper makes several strong contributions. First, it provides thorough empirical analysis of redundancy patterns in LRMs with concrete statistics demonstrating overthinking and underthinking phenomena. Second, the approach is highly pragmatic—it's training-free, non-invasive to base models, and integrates seamlessly with existing inference frameworks. The use of lightweight 7B verifiers with simple binary classification tasks (answer existence and equivalence) is more practical than training complex reward models. The two-stage verification design cleverly minimizes false positives by first checking for answer existence before equivalence. Third, the experimental evaluation is comprehensive, covering multiple models and benchmarks with detailed ablation studies, consistently showing substantial efficiency gains (16-70% runtime reduction) with minimal accuracy loss (<2% in most cases). Finally, the system design addresses real deployment concerns including asynchronous operation, verifier-to-LRM ratios (8:1), and integration with production inference systems like vLLM.", "weaknesses": "Several limitations merit attention. First, the method introduces infrastructure complexity by requiring a separate 7B verifier model, though one verifier can serve multiple LRMs. Second, the verifier accuracy is only 87-88%, and while claimed to have \"negligible effect,\" there's limited analysis of how verification errors compound across multiple sequential checks or lead to premature incorrect convergence. Third, the hyperparameters (M=2, Rthres=50%, Nthres=20) appear determined by informal inspection rather than systematic optimization. Fourth, the approach shows largest gains on weaker models (R1Q-32B: 67% reduction) that exhibit more redundancy, raising questions about whether it primarily benefits suboptimal models. Fifth, evaluation beyond mathematical/scientific reasoning is limited, with only brief code generation results. Finally, the theoretical framing as an optimization problem is somewhat superficial and doesn't strongly guide algorithm design—the connection between the formal optimization objective and the actual detection heuristics is loose.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7yk0WsHAYC", "reviewer": "Reviewer_uF9K", "summary": "This paper proposes TrimR, a training-free framework that enhances the inference efficiency of Large Reasoning Models (LRMs) by dynamically managing their reasoning process during test time. TrimR identifies and trims redundant reasoning (overthinking) and halts unproductive reasoning (underthinking) through a lightweight external verifier that checks answer existence and equivalence within intermediate reasoning segments. When redundant patterns are detected, the system adaptively terminates generation to reduce unnecessary computation. Experiments across multiple reasoning benchmarks demonstrate token reductions.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper tackles the inefficiency of overthinking and underthinking in Large Reasoning Models (LRMs), which is a practically important issue for real-world deployment. The explicit handling of both forms of reasoning redundancy is well-motivated.\n- The proposed method is prompt-based and requires no additional training or fine-tuning, making it straightforward to integrate into existing reasoning frameworks.\n- Experiments across multiple LRMs and reasoning benchmarks demonstrate reductions in token usage with negligible loss in accuracy.", "weaknesses": "- The verifier may inherit biases from its own training data, which could influence the trimming behavior. It remains unclear whether the proposed method generalizes when switching to verifiers of different models or sizes. The authors should clarify whether prompt engineering is needed for different verifier models. And whether larger size verifier models could lead to better performance.\n- The evaluation omits recent closed-source reasoning models such as Gemini 2.5-Pro and GPT-5. \n- It is unclear whether the R-threshold for early stopping is task-specific, dataset-specific and model-specific. A more systematic analysis of how this threshold influences the accuracy–efficiency trade-off across different scenarios would strengthen the paper.\n- If a fixed “thinking budget” is specified in the initial prompt, how does it affect the reasoning behavior? The paper should analyze the performance difference compared to TrimR’s R-threshold.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2DRFhhXYfy", "reviewer": "Reviewer_Wv7h", "summary": "This paper introduces TrimR, a novel, training-free framework designed to improve the inference efficiency of LRMs at test-time. The core idea is to use a lightweight, pre-trained verifier model to dynamically detect and prune redundant reasoning steps, categorized as \"overthinking,\" \"underthinking,\" and \"repetition\"—in the CoT process. By halting generation once a stable solution is reached or when the model enters a non-productive loop, TrimR aims to significantly reduce runtime and token usage with minimal impact on task accuracy. The authors conduct extensive experiments on mathematical and scientific reasoning benchmarks (MATH, AIME, GPQA) using several state-of-the-art LRMs, demonstrating substantial efficiency gains. They also present T4S, an online system designed for production-level deployment.", "questions": "1. Equations (existence/equivalence) omit tie-breaking and calibration details for binary decisions. please specify the exact scoring rule (e.g., logit for “Yes” token vs. sum of subword probabilities), thresholding, and how vocabulary/tokenization differences between Pangu-7B and Qwen2.5-7B-Instruct affect decision comparability\n\n2. Section 3.3 states underthinking is flagged when there are not at least three consecutive agreeing, answer-bearing chains within budgets, but Appendix Algorithm 2 only checks $ t \\ge R_{thres}\\cdot M $ and $ {num\\_thoughts}\\ge N_{thres} $ without any consensus logic; please reconcile this inconsistency and provide the correct, consensus-aware pseudocode and results.\n\n3. For Eqn. 2, define the precise notion of “logical or mathematical equivalence” operationalized by the verifier: is identity on normalized numeric answers sufficient, or must the steps be semantically equivalent; a formal criterion would reduce ambiguity and enable reproducibility across verifiers.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1.  The \"training-free\" nature of TrimR is its most compelling feature. Unlike methods that require costly fine-tuning or reinforcement learning, TrimR is non-invasive and preserves the LRM's original capabilities. This makes it immediately applicable to a wide range of pretrained models, a major advantage for real-world deployment. The use of a small, external verifier to classify answer existence and equivalence is a simple yet elegant approach that sidesteps the complexity and instability of training full-fledged Process or Outcome Reward Models (PRMs/ORMs)\n\n2. The evaluation across multiple powerful LRMs (QwQ-32B, DeepSeek-R1-Distill-Qwen-32B, Pangu-R-38B) and challenging, well-regarded benchmarks demonstrates the method's robustness. The inclusion of results on MoE models (Appendix N) further strengthens claims of architectural agnosticism.\n\n3.  The authors rightly focus on metrics beyond simple accuracy, such as wall-clock runtime, Time Per Request (TPR), and total token consumption under a batched inference setting. These are the metrics that matter for production systems, and the reported gains (e.g., up to 70% runtime reduction) are impressive.​\n\n4. The detailed description of the T4S online system (Sec 3.6, Fig 3) is a valuable contribution. Showing how TrimR integrates with modern inference frameworks like vLLM on industrial hardware (Ascend NPUs) moves the work from a purely academic concept to a practical engineering solution. This level of detail greatly enhances the paper's reproducibility and impact.​", "weaknesses": "1. The paper introduces a \"Theoretical Foundation\" (Section 1, page 2) by framing the reasoning process as a formal optimization problem:\n$$ \\min_{c} Infer_Cost(y_t) \\quad s.t. \\quad Perf(X, y_t) \\ge Perf(X, y_{t^*}) - \\epsilon $$\nThis formulation is presented as the mathematical underpinning of the method.\nThis formalization is entirely disconnected from the proposed algorithms. The paper never defines the performance function `Perf(X, y_t)`, the tolerance `\\epsilon`, or shows how the compression rule `c` is derived from this objective. The actual method does not solve or approximate this optimization problem. Instead, it employs a set of disconnected, heuristic rules based on binary classifications from a verifier. The \"foundation\" is an analogy, not a mathematical basis, which is a significant overstatement of the method. \n\n2. The set of thoughts containing a solution is defined as `S = {s_i | F_v(p1, s_i) > 1}`. Since `F_v` is an indicator function, its output is {0, 1}, making the condition `> 1` impossible. This is likely a typo for `= 1`. More critically, the verifier for answer existence (`p1`) only checks if a thought contains a concluding statement, not if the reasoning leading to it is sound. A thought can contain a numerically correct but logically flawed answer, which TrimR would incorrectly identify as a valid convergence point.\n\n3. The algorithm's counter for equivalent thoughts resets to zero upon any mismatch (`else reset on any mismatch count = 0`). This makes the termination condition exceptionally brittle. A valid reasoning sequence like `A -> A -> B -> A -> A` (where `A` is a correct answer and `B` is a brief, self-corrected error) would have its convergence counter reset by `B`, failing to trigger an early stop. This behavior is not robust to the noisy, non-linear nature of LRM reasoning. Furthermore, the choice of `M=2` (terminating after only two equivalent thoughts) is aggressive and its sensitivity is not analyzed. \n\n4. Section 3.3 states that underthinking is flagged if the model fails to produce \"at least **three consecutive** answer-bearing chains that agree\". However, **Algorithm 2** in Appendix B contains no logic to check this condition. The algorithm merely checks if the token budget (`t > R_thres * M`) and thought count (`num_thoughts > N_thres`) have been exceeded, without any reference to the history of answer agreements. This is a direct contradiction. The implemented algorithm is a simple budget check, not the convergence failure check described in the main text.\n\n5. The entire framework is built on two key heuristics whose robustness is assumed rather than proven. The method's ability to segment reasoning chains depends entirely on a manually curated list of \"reflection tokens\" (e.g., `,...`, `Hmm, ...` in Appendix E). This approach is inherently model-specific and fragile. There is no guarantee that future or different LRMs will use these exact lexical cues. The paper provides no quantitative analysis of the segmentation quality (e.g., precision/recall of thought boundaries) or how the system would perform if a model reasoned without these explicit markers.\n\n6. Section 3.1 states that the \"intermediate thought answer\" is extracted by taking the last `N_sent` sentences of a thought segment. Section 4.1 reveals this parameter is set to `N_sent=50`. An \"answer\" that is 50 sentences long is not a brief segment; it is likely the entire thought. \nThis contradicts the paper's claims of using a lightweight verifier on \"shorter but more informative\" segments to save computation. This parameter choice is counter-intuitive and lacks any justification, suggesting either a typo or a fundamental flaw in the description of the answer extraction process\n\n7. The paper repeatedly describes TrimR as \"non-invasive\", while the verifier operates externally, the \"forceful prompt\" (`I think I already thought for a long time... think...`) is an explicit and invasive intervention. It injects a specific string into the generation context to hijack the model's output flow and force it to emit the `</think>` token. This directly manipulates the generative process and contradicts the claim of being a purely non-invasive method that simply \"halts\" generation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2NDW7osfYh", "reviewer": "Reviewer_CC2y", "summary": "The manuscript proposes TrimR, a training-free, verifier-based termination rule to shorten the “thinking” phase of large reasoning models without materially hurting accuracy. The verifier is a small model that checks for (i) the presence of a candidate answer and (ii) stabilization or equivalence across successive partial answers; when either signal indicates convergence or diminishing returns, TrimR stops the chain. The paper claims token savings and latency gains, and sketches an online serving setup that sparsifies verifier calls alongside a high-throughput LLM stack. While the idea is intuitive and pleasantly simple, the experimental scope and missing head-to-head baselines make the novelty feel narrower than it could be, especially given very recent work on token-budget prompting, minimal-draft reasoning, certainty-based early exit, and BoN accelerators.", "questions": "1. How does the verifier behave relative to certainty-style indices under identical serving conditions? A paired experiment replacing TrimR’s verifier with Certaindex inside the same stack (same batching, same schedulers) would clarify whether the proposed proxy is actually superior at the iso-accuracy frontier. Please report false-stop rates and area-under-compute-curve for both. \n\n2. Can the authors provide a frontier plot comparing TrimR against TALE and Chain-of-Draft on AIME24/25, MATH500, and GPQA-Diamond, where x-axis is tokens per solved instance and y-axis is accuracy? This would isolate whether TrimR’s how-to-stop signal gives better trade-offs than how-to-speak methods. \n\n3. Could the paper include a BoN study where TrimR replaces Speculative Rejection’s early abort, using the same reward model and candidate pool? If TrimR’s structure-aware convergence check is stronger, it should win on time-to-target-reward at equal sample budgets. \n\n4. Because over- and under-thinking are now measured with concrete diagnostics, can the authors evaluate TrimR on an underthinking-sensitive suite and report thought-switching metrics and depth utilization? A small ablation where reflection tokens are suppressed or noisy would further test robustness of the equivalence check. \n\n5. The serving section would be more convincing with a cross-platform replication. Please report throughput and latency on both vLLM and SGLang, and make QPS vs. latency curves available under identical prompts and batch sizes.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "From an originality viewpoint, using a binary, training-free verifier to detect convergence rather than to score full solution traces is a clean reframing. Compared with certainty indices or reward-model scoring, the authors’ proxy has the advantage of being cheap and model-agnostic, and it aligns with the community’s turn toward adaptive test-time compute. This direction is timely because recent methods like TALE (token-budget-aware prompting) and Chain-of-Draft (short, dense drafts instead of verbose CoT) show that much of today’s long chains are unnecessary; TrimR adds a different lever by detecting when to stop rather than prescribing how to speak.\n\nQuality-wise, the paper’s engineering instinct to make the verifier calls sparse and to integrate with production-oriented stacks is sound. If realized properly, the approach is compatible with high-throughput serving systems such as vLLM (PagedAttention) and SGLang (RadixAttention, structured decoding), where shaving thinking tokens often translates directly into more QPS under the same memory budget.\n\nThe central mechanism is easy to follow and does not rely on delicate finetuning or proprietary signals. Given the rising concern about over- and under-thinking in o1/R1-style models, a simple stop-when-stable rule is appealing to practitioners who need something they can deploy tomorrow. In significance, a strong result here could matter beyond math: certainty-style early-exit (Certaindex) and BoN accelerators (Speculative Rejection) are quickly becoming standard ingredients in serving stacks; a verifier that outperforms them at the same accuracy would be valuable.", "weaknesses": "The experimental design underplays novelty by omitting the baselines that readers will expect. At minimum, the paper needs controlled, iso-setup comparisons against TALE/Token-Budget prompting and Chain-of-Draft to show that a convergence-verifier adds value on top of short-draft prompting. Without those, improvements could be attributed to any reasonable length control, not specifically to the verifier. Please add head-to-head results on AIME24/25 and MATH500 at matched accuracy deltas (±1%) and plot a token-efficiency frontier.\n\nEarly-exit and serving claims are not persuasive without Certaindex as a first-class baseline. Certaindex already signals when further compute is unlikely to help and reports real serving wins; readers will ask whether TrimR’s binary verifier is actually better at minimizing false stops and at maintaining throughput under identical batching/scheduler policies. A direct comparison (same hardware, same scheduler, same models) is necessary.\n\nFor BoN or multi-sample regimes, Speculative Rejection is designed to abort low-reward trajectories efficiently. If TrimR is claimed to be more compute-efficient than BoN accelerators, it should be compared inside the same BoN pipeline and reward model, with time-to-win and win-rate controlled. Otherwise the reader cannot separate the benefit of what is scored (reward-model vs. answer-equivalence) from the benefit of simply stopping earlier.\n\nVerifier-design ablations are thin. Given the rich recent literature on PRMs and generative verifiers, two yes/no checks feel brittle without stress tests. The paper should show that equivalence detection is robust under paraphrase and reflection-token noise, and report mis-fire rates versus PRM/GenVerifier scoring when chains include small algebraic deviations or reordered steps. Right now, the evidence is not enough to argue for stability across models and prompting styles.\n\nThe benchmark mix is too math-centric and potentially contaminated. GPQA-Diamond is a good inclusion; however, several recent works show that AIME24 is contaminated for some model families and advocate live or freshly released streams. The paper would be much stronger with OlympiadBench for very long traces, MMLU-Pro for general reasoning, and MathArena to mitigate contamination concerns; a light GSM8K check is also useful to demonstrate that stopping early does not hurt easy-regime accuracy.\n\nFinally, the claims about mitigating both overthinking and underthinking are only partially evidenced. Underthinking has recently been operationalized and measured; TrimR should adopt or adapt those metrics, and show improved depth-when-needed rather than only shorter chains on average. Otherwise, the method risks favoring brevity in cases where more thought is necessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)