Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Authors: Yang Sui
Yu-Neng Chuang
Guanchu Wang
Jiamu Zhang
Tianyi Zhang
Jiayi Yuan
Hongyi Liu
Andrew Wen
Shaochen Zhong
Na Zou
Hanjie Chen
Xia Hu
Venue: Accepted by TMLR ()

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as OpenAI o1 and DeepSeek-R1, have further improved performance in System-2 reasoning domains like mathematics and programming by harnessing supervised fine-tuning (SFT) and reinforcement learning (RL) to enhance Chain-of-Thought (CoT) reasoning. However, while longer CoT reasoning sequences improve performance, they also introduce significant computational overhead due to lengthy and redundant outputs, known as the ''overthinking phenomenon''.
Efficient Reasoning, which seeks to optimize reasoning length while preserving reasoning capabilities, offers practical benefits such as faster processing times, lower energy consumption, and improved responsiveness, especially valuable for reasoning-intensive applications. Despite its potential, efficient reasoning remains in the early stages of research. 
In this paper, we provide the first structured survey to systematically investigate and explore the current progress toward achieving efficient reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we categorize existing works into several key directions: (1) model-based efficient reasoning, which considers optimizing full-length reasoning models into more concise reasoning models or directly training efficient reasoning models; (2) reasoning output-based efficient reasoning, which aims to dynamically reduce reasoning steps and length during inference; (3) input prompts-based efficient reasoning, which seeks to enhance reasoning efficiency based on input prompt properties such as difficulty or length control.  Additionally, we introduce the use of efficient data for training reasoning models, explore the reasoning capabilities of small language models, and discuss evaluation methods and benchmarking.

Published in Transactions on Machine Learning Research (08/2025)
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Yang Sui"
Yu-Neng Chuang Guanchu Wang! Jiamu Zhangl
Tianyi Zhang Jiayi Yuank
Hongyi Liu'
Andrew Wen
Shaochen Zhong'
Na Zou2 Hanjie Chen'
Xia Hu'
RRice University
2 University of Houston
Reviewed on OpenReview: https: // openreview. net/ forum? id=
Abstract
Large Language Models (LLMs) have demonstrated remarkable capabilities in complex lasks. Recent advancements in Large Reasoning Models (LRMs), such as OpenAl ol and DeepSeek-RL. have further improved performance in System-2 reasoning domains like math- ematics and programming by harnessing supervised fine-tuning (SFT) and reinforcement learning (RL) to enhance Chain-of- Thought (CoT) reasoning. However, while longer CoT reasoning sequences improve performance, they also introduce significant computational overhead due to lengthy and redundant outputs, known as the "overthinking phenomenon".
Efficient Reasoning, which sceks to optimize reasoning length and computation during inference of reasoning models while preserving reasoning capabilities, offers practical bene- fits such as faster processing times, lower energy consumption, and improved responsiveness, intensive applications. Despite its potential, efficient rea- soning remains in the carly stages of rescarch.
especially valuable for reatenveg-intensive
In this paper, we provide the first structured survey to systematically investigate and ex- plore the current progress toward achieving efficient reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we categorize existing works into several key di- rections: (1) model-based efficient reasoning, which considers optimizing full-length reasoning models into more concise reasoning models directly training efficient. reason- ing models; (2) reasoning output-based efficient reasoning, which aims to dynami- cally reduce reasoning steps and length during inference; (9) input prompts-bass based ef- ficient reasoning, which seeks to enhance reasoning efficiency based on input prompt propertics such as difficulty or length control. Additionally, we introduce the use of effi- cient data for training reasoning models, explore the reasoning capabilities of small lan- guage models, and discuss evaluation methods and benchmarking. We maintain public repository to continuously track and update the latest research in this promising area: -ewriiiinnt-assnnnnn--iaeee
https://githu/ com wwt//coppsess/////ccccccccocssee
1 Introduction
Large Language Models (LLMs) have emerged as exceptionally powerful AI tools, demonstrating advanced capabilities in natural language understanding and complex reasoning. Recently, the rise of reasoning- focused LLMs, also referred to as reasoning-capable models or Large Reasoning Models (LRMs) (Xu et al., 2025a) such as OpenAl ol (OpenAT, 2024) and DeepSeek-R1 (Guo el al., 2025), has significantly improved performance in System-2 reasoning domains (Li et al., 2025d), particularly in challenging mathematics (Cobbe al., 2021; Hendrycks et al., 2021) and programming tasks (Codeforees, 2025; Chen al., 2021). Evolving from foundational pretrained models (e.g., LLaMA (Touvron et al., 2023; Grattafiori et al., 2024)) trained with next-token prediction (Devlin et al., 2019), these models typically leverage Chain-of-Thought

---

Published in Transactions on Machine Learning Research (08/2025)
Reinforcement Learning (RL)
Reasoning Models: OpenAl o1, DeepSeek-RA. R11..
This Survey
looe
Efficient Reasoning:
Efficient Reasoning
Supervised Fine-Tuning (SFT)
Base Model
Steps
Input Prompts
+ Correct Answer
Models
Reasoning Outputs
Figure The pipeline of developing efficient reasoning for LLMs. reasoning model can be trained on the base model using SFT, RL, a combination of both. While reasoning models demonstrate strong rea- soning capabilities, they often suffer from the "overthinking phenomenon", generating unnecessarily lengthy reasoning steps. To improve efficiency, various methods can be applied to reduce redundant steps while maintaining accuracy during post-training based on reasoning model, or to fine-tune non-rcasoning models to incorporate efficient reasoning capabilities. This approach enables the model lo answer questions with concise and effective reasoning steps. In this paper, we explore the latest progress in efficient reasoning for LLMs, aiming provide insights that can guide futuure rescarch and the development of reasoning-driven applications across various domains.
(CoT) (Wei et al., 2022) reasoning chains to generate explicit, step-by-step reasoning sequences before arriving at a final answer, significantly improving their effectiveness reasoning-inlensiee tasks.
Such reasoning abilities in LLMs are typically developed through supervised fine-tuning (SFT) and rein- forcement learning (RL), which promote itcrative and systematic problem-solving abilitics. For instance, DeepSeek-R1 (Guo al., 2025) undergoes multiple rounds of SFT and RL training, emphasizing structured thinking templates and rule-based reward mechanisms. In particular, the rulo-based rewards provides precise and explicit feedback signals during training, effectively enhancing the general reasoning capabilities beyond the pretrained LLM.
However, while long CoT reasoning significantly boosts accuracy, step-by-stop thinking mechanisms also lead to lengthy output responses, resulting substantial computational overhead and increased reasoning time. For instance, the "overthinking problem' arises when answering a simple question (Chen et al., 2024c) like, "what is the answer of 2 plus 3 Some reasoning models, especially smaller ones, can generate reasoning sequences spanning thousands of tokens. This verbosity significantly increases both inference costs and latency, limiting the practical application of reasoning models in computation-seesiiivi real-world scenarios. such as real-time autonomous driving systems, interactive conversational assistants, precision robotic control tasks, and large-scale online search engines.
Efficient reasoning, particularly the reduction reasoning length and computation during inference of rea- soning modcls as shown in Figure 1, provides dircct cost reduction and improved feasibility for real-world deployments. Recently, numerous studies (Luo et al., 2025; Yeo al., 2025; Han al., 2024; Ma al., 2025b; Hao et al., 2024) have explored ways to develop more concise reasoning paths, making efficient reasoning rapidly evolving rescarch area.
In this paper, we present the first structured survey systematically exploring the progress in efficient. rea- soning for LLMs. As illustrated in Figure 2, we categorize existing work into three key directions: (1) Model-based efficient reasoning, which focuses on optimizing full-length reasoning models into more con- cise variants or directly training efficient reasoning models. (2) Reasoning output-based efficient reasoning. which dynamically reduces reasoning steps and length during inference. (3) Input prompls- based efficient
2

---

Published in Transactions on Machine Learning Research (08/2025)
reasoning, which enhances reasoning efficiency based on input properties such as difficulty length control. Unlike model compression techniques such quantization (Xiao et al., 2023; Frantar et al., 2023; Lin et al., 2024) or KV cache compression (Zhang et al., 2023; Liu et al., 2024b; Shi et al., 2024; Yuan et al., 2024), which focus on reducing model size for lightweight inference, efficient reasoning in LLMs emphasizes smart and concise reasoning by optimizing the length of generated reasoning sequences and reducing unnecessary thinking steps.
Overall, we provide summary of the current key approaches to efficient reasoning, organizing them into the following categories:
Reinforcement Learning with Length-Base Based Reward Design (Section 3.1)
Supervised Fine- Fine-Tuning with Variable-Length CoT Data (Section 3.2)
Compressing Reasoning Steps into Fewer Latent Representations (Section 4.1)
Dynamic Reasoning Paradigms During Inference (Section 4.2)
Crompt-Gudee) Efficient. Reasoning (Section 5.1)
Routing Prompts to Optimize Reasoning Efficiency (Section 5.2)
Furthermore, beyond these approaches, we also include efficient. LLM techniques towards reducing the com- putation related to the reasoning models, including efficient data for reasoning model training, and model compression methods for reasoning models. We explore other relevant topics, including:
Training Reasoning Models with Efficient Data (Section 6.1)
Reasoning Abilities of Small Language Models and Model Compression (Section 6.2)
Evaluation and Benchmarking of Efficient Reasoning Models (Section 7)
The overall taxonomy of existing literature related efficient reasoning for LLMs and efficient techniques related to large reasoning models provided in Figure 3.
2 Background: Long CoT Reasoning Models and the Overthinking Phenomenon
2.1
ofaiight-TToougt (CoT) Reasoning
Chain-of-Thought (CoT) reasoning (Wei et al., 2022) is a key approach that has been purposefully introduced in LLMs to enhance their reasoning capabilities. In this setting, models are typically prompted to generate a structured reasoning chain before arriving at a final answer. Techniques in this domain have been shown improve overall accuracy (Wei al., 2022) since a higher-quality generation context often leads more consistent and reliable final results. Several notable CoT variants have been developed: Self-Consistency CoT (Wang et al., 2023) replaces the standard greedy decoding approach by sampling diverse reasoning paths (ToT)
and selecting the most consistent answer through marginalization and aggregation. prompting (Yao et al., 2023) further structures the rcasoning process as tree with backtracking, significantly improving elficiency in solving parallelizable subtasks. Graph-of-Thooghh (GoT) prompting (Besta et al., 2024) extends this concept by structuring thoughts into a graph, allowing iterative refinement of individual reasoning steps. While many COT variants exist, they generally involve different prompting techniques guide the behavior of models, sometimes incorporating controller- like mechanisms to manage thought progression and usage.
3

---

Published in Transactions on Machine Learning Research (08/2025)
Input Prompts
Length Reward
Input Prompts
Latent Reasoning
Input Prompts
Length Prompts
Prompt
Answer
Original Questions
Latent Reasoning Steps
Let's use less than 10 tokens."
Model
Model
Model
Variable-Length CoT Reasoning Data
Dynamic Reasoning
Routing by Difficutty
Questions
Draft
Reasoning Output
Reasoning Output
Reasoning
Output
Strong Model
Figure 2: Overview of efficient. reasoning methods, which can be summarized model-oriented (Left: I, II) and reasoning output-oriented (Middle: III, IV), and input prompts-oriented (Right: V, VI) methods. Specifically, (I) Reinforcement Learning with Length Reward Design (Section 3.1); (II) Supervised Fine- Tuning with Fariable-Lengte CoT Data (Section 3.2); (III) Compressing Reasoning Steps into Fewer Latent Representation (Section 4. 4.1); (TV) Dynamic Reasoning Paradigm during Inference (Section 4.2); (V) Prompt-
guided Efficient Reasoning (Section 5.1); (VI) Routing Prompts to Optimize Reasoning Efficiency (Section 5.2);
2.2 The Mechanism Behind Large Reasoning Models
Multi-step reasoning refers to the ability of LLMs ability to generate structured reasoning steps before committing to final answer. This capability is particularly beneficial for logic-intensive tasks such mathematics and programming. More broadly, reasoning-capalle models are often favored by human users over their non-reasoning counterparts, as evidenced by rankings in the Chatbot Arena LLM Leaderboard.
Recent reasoning models, such as DeepSeek-RI (Guo al., 2025) and OpenAI ol (Luo al., 2025), are known or believed to have internalized reasoning behaviors, reducing reliance on explicit test-time augmentations. These models generate detailed CoT reasoning by iteratively producing intermediate steps and refining solutions scquentially until reaching final answer. Unlike traditional CoT approaches, which rely prompting, these reasoning models internalize their reasoning capability through extensive training.
The OpenAl ol model is speculated to employ a tree-based search approach, such as Monte Carlo Tree Search (MCTS) (Kocsis & Szepesvári, 2006; Coulom, 2006), combined with Process Reward Model (PRM) explore reasoning paths and determine optimal solutions through guided simulations.? DeepSeek-R1, on the other hand, explicitly learns its reasoning capability through supervised fine-tuning and reinforcement learning, with particular emphasis rule-based rewards for math and coding (asks. These models are trained to generate reasoning steps in predefined format before arriving at their final answers.
2.3
The Overthinking Problem in Long CoT Reasoning Models
The "overthinking phenomenon" (Team et al., 2025; Chen et al., 2024c) in long CoT reasoning models refers to situations where LLMs generate excessively detailed or unnecessarily elaborate reasoning steps, ultimately reducing their problem-solving efficiency. In particular, many modern reasoning models, especially those with smaller parameter scales, tend to produce verbose reasoning or redundant. intermediate steps, making them unable to provide answers within the user-defined token budget. In worse cases, excessive reasoning steps introduce errors or obscure logical clarity, leading to incorrect answers.
A community-driven evaluation of leading LLMs and AI chatbots: https: https://lmaan ///////maanaaaaa/eeee//e//aaaaea ai at/Pleaderboard. 2There official confirmation regarding OpenAl ol 's training details and mechanisms. However, sources such https:// interconneccs...... as/p/ as/p/openais- al using- search- was- a-yayap https://wwws //upt/////oyuuuuubee..m con/uatchhyy-6PE.ekk.t. dis- cuss these speculations in detail and are recommended for interested readers.
4

---

Published in Transactions on Machine Learning Research (08/2025)
eg. Kimi (Team 2025]; O1-Pruner al., 2025]; (Aggarwal Welleck,
Oplimization Length
2025); Training (Arora & Zanette, 2025); Demystifying (Yeo et al,, 2025); DAST (Shen et al, 2025b); MRT (Qu al., 2025];
Reward
Model-based Effecient
Reasoning
SFT with
Distilling 2-1 (Yu al., 2024); C3cT (Kang 2024); TokenSkip (Xia et al., 2025]; CoT-Valve (Ma et al., 2025b); Self Training (Munkbbat et al., 2025]; Learn to Skip (Liu es al., 2024a]: Badget (Han al., 2024); Stepwise Cui et al. (2025):
Variahle- Length CoT
Latent Repre-
egg. Coconut (Hao al., 2024): CODI (Shen et al., 2025c]; CCcT (Cheng & Van Durme,
sentatiinn
2021): Heima (Shen al., 2025a); Token Assoried (Su es al,, 2025); Loop (Saunshi et al,, 2025); SoftCoT (Xu et al., 2025c):
Compression
Ressoninng Output-based
Ellicient
Reasoning
Speculative Rejection (Sun al., 2024); Campliin-Efflllest TTS (Wang al., 2025); DPTS (Ding et al., 2025); Certaindex (Fu sh,, 2024); Fest MCTS (Li et al,, 2025a); ST-BoN (Wang et al., 2025); More Less (Wu et al., 2025); RSD (Lino et al., 202na);;
Dynamic Reasoning Paradigm
LigheThinker (Zhang at., 2025); INFTYTHINK 2026); RASC (Wan et al., 2024); Adaption Reasoning (Yu et al., 2025); AdaptivoStep (Liu el al, 2025c);
ah,, 2025); SCoT (Xiang al.,
Prompt-
egg. Token-Budget (Han al., 2024); Complexity (Loe et al., 2025); Concise Chaln-of-Thowgge (CCoT) (Renze & Guven, 2024): MARP (Chon al,, 2024n);
(Xu al., 2025b);
Dacommnn
Guided Efficient Ressoning
Inpot Prompts- based Efficient Ressoning
Routing Question Attributes
egg. Claude (Anthropic, 2023); SuT (Ayles al., 2025); SelFRREF (Chuang al, 2025c): Confident (Chuang et al., 2025b); RouteLLM (Ong et al., 2025);
Less Training Data
egg. LIMO (Ye el al,, 2025); a1 (Muemnigholl el al., 2025); S'R (Ma el al., 2025a);;
Light-R1 (Wen et al., 2025);
Efficient Data and Models
e.g. Struggle(Li et al., 2025c); Strong Verifeess (Srivastava et al., 2025);
Pruming
TinyR1-S2B-Preee (Sum 2025); Mixed Distillation (Chenglin 2024);
Quantization Distillation
Counterfactual Distillation (Feng et al., 2024); Foedback-Driven Distillation (Zhu et al., 2024); SKInteru (Liso et al., 2025b); Adaptive Thinking (Chen et al., 2024b); PRR (Zhao et al., 2024):
Benchmark Insights
Evaluation & Benehmarks
e.g.
(Liu al., 2025b): Sys2Bench (Parashar al., 2025); Danger (Cuadron et al., 2025); Inference-time Computation (Liu et al., 2025a); Impact (Jin et al., 2024);
Taxonomy of existing litcrature on efficient reasoning for LLMs and efficient techniques related to large reasoning models.
Figure 3:
Figure 4 illustrates an example of overthinking, Even though the model arrives at the correct answer early in its reasoning process, continues generating unnecessary intermediate steps, leading to inefficiencies. Given the substantial resource costs associated with LLM inference (e.g., OpenAT ol costs $60 per 1M generated tokens), such bchavior highly undesirable. Morcover, the problem becomes even worse if longer reasoning leads wrong answers. In contrast, efficient reasoning models would use fewer reasoning steps to obtain correct answers while reducing inference costs.
Addressing this challenge is particularly difficult because the pretraining recipes for reasoning-capabe models oflen explicitly encourage generating extended reasoning steps to improve accuracy. For example, DeepSeek- Ri-Zero, a more or less a development prototype DeepSeek-R1, exhibits a direct correlation between increased training duration with longer response lengths and improved benchmark performance (Guo et al., 2025). These trends are often viewed as proxies for successful reasoning training. Consequently, improving inference efficiency requires working against certain pretraining objectives, making it. a non-trivial challenge.
5

---

Published in Transactions on Machine Learning Research (08/2025)
Which larger, 0.9 0.11?
Which larger, 0.9 or 0.11?
Okay, so need figure which number bigger between 0.9 and 0.11. Hmm, Let's Both decimals, but they have different numbers of digits after the decimal point. Muybe should sturt by understanding what each decimul represents. First, 0.9. That's nine tenths, right? Because the first digit after the decimal is the tenths place. Now,
Okay, so need figure out which number larger between 0.9 and 0.11. Hmm, let me think. Both decimals. right? Maybe should compare them digit by digit. Let me recall how decimal comparison works. First, I remember that
Wait, but 0.9 has only one digit after the decimal, while 0.11 has two. Maybe should write them with... Now, ...
Wait, but let me check another way to be sure... Another method could be converting them to pereentages....
But just to make sure huven't made mistake, let me visualize on number line...
wait, me double-check. Maybe I made mistake here.
me think again. Alternatively,
there any other way to verify this? Maybe subtraction....
Altematively, .. I'm confident answer 0.9.
Another way could be...
0.2
Final Answer: The larger number 0.9.
Overthinking (602 words, 42 seconds)
Overthinking (616 words, 19 seconds)
DeepSeek-R1
QwQ-32B
Figure 4: An example of the "overthinking phenomenon": when asked "Which is larger, 0.9 or 0.119", the reasoning model takes an unnecessarily long time (e.g,, 19 seconds for QwQ-32B (Team) and 42 seconds for DeepSeek-Ri (Guo al., 2025)) to arrive at the correct answer. This example was tested March 2025.
This paper aims to systematically summarize various approaches and methodologies toward achieving the challenging yet valuable goal of developing reasoning models with high efficiency and strong rcasoning capa- bilities.
3
Model-based Efficient Reasoning
From the model perspective, these works focus fine-tuning LLMs to improve their intrinsic ability to reason concisely and efficiently.
3.1 RL with Length Reward Design
Most reasoning models are trained using RL-based methods (e.g., DeepSeek-R1 (Guo el al., 2025), DeepSeek- R1-Zero (Guo al., 2025), OpenAI ol (OpenAI, 2024), Be-Q-32B-Previee (Team)) which focus on the accuracy reward and format rewards (Guo et al., 2025). To enhance reasoning-length efficiency, some studies propose integrating length reward into the RL framework, which effectively shortens the reasoning process (as shown in Table 5). In principle, the length reward assigns higher scores to short, correct answers while penalizing lengthy or incorrect ones, thereby optimizing the length of the reasoning path.
The key question is: How to formulate the length reward in RL?
CoT Reasoning Response
Format Reward Accuracy Reward
RL
+1
LLMs/
-1
+1
1 +1 -1
Efficient Reasoning Model
Reasoning Model
Length Reward
Other Reward Functions
Figure 5: Illustration of the method for RL fine-tuning with length reward designs. In principle, the length reward assigns higher rewards to short, correct answers and penalizes lengthy or wrong answers to achieve efficient reasoning LLMs.

---

Published in Transactions on Machine Learning Research (08/2025)
Table 1: Comparison of different length reward-based RL methods. I(.) denotes the way of calculating the prediction length. r0/ră' denotes reward (correct/wrong) for L())=0. ri./rf. Reward (correct/ (correctwrrong) wrong) for L()) = mmax(*) (+) with hyperparameter a. re is the exceed length penalty. yoT represents the ground truth
answer, Mpred is the prediction, and Vref is the reference prediction from reference model of input data of dataset D. Tref is the policy of reference model and To is the policy of targetod-to-train LLMs.
Method
RL
Length Coustraint Reward
Data
Model
GSM8K
O1-Pruner (laoo al, 2025)
PPO
B
B Jed M1
GanKao
Macco-ol-22 Qu(Q-3255 Prrviiee
MATH--500
16 0.5 (f) (61(1 cool (6lbyan))
correct,
MATII-500
AIME 2021
LL1MA-8.1-BB
Denastafying (Yeo et al., 2025)
PPO
16 0.5 (f2 -6 pool Fi.
wrong if Elygont) Linoss
TherreQA
Qwent 5-7B-Math
MMLJU-Pro-Ik
AMC
GPQA
Zuaoe CONCAT (z, "Think for N tokens.".. ")
LAST
Li (Aggareal & Welleck, 2025)
GRPO
MMLU
DeepSook- Ri-t-t-ilDD--i---- Soalll--eee--bba 1.5B
riy. Mor. L(yer)) = M3ppnd 2CT) a- Ligrr) Miigna)
MATH-500
A1ME-2022
Olympiaci-Hench
DAST (Shom al,, 20256)
SimPO
Trained with constructed length preferencee data
MATH-500
DorpSook-Rt -tn-Sook-----aatillo.o DittillQurnn88: Dastsel--R--istl---eeeee
AIME-2024
GSMSK
DeepSeek R1 Distil Qwen L5B DeepS ReegSiek-R--isiill-ee Distiill--een-1
Training (Arora & Zamette, 2025)
PG
En-D [Aape perr((1 (fCeee))))))
MATH--500
AIME-2024
Table 2: Comparison of different policy optimization methods in CoT length controls. RG represents the reward model. Tref is the policy of reference model and To is the policy of target-to-train LLMs. y is target reward margin term for SimPO and 3 is hyperparameter to amply the preference to winning than losing. À is a clipping-related hyper-parammteer with KL-divergence ID [xxl--I-)- clip(-) is a gradient clipping function with hyperparameter e to control the gradient updating progress of GRPO. The yo is for winning responses, and yt is for losing responses of the given input data 2t, where some with G on superscript denote the outputs of different sampled groups
Method
Optimization Objective
Policy Gradient (PG)
Ex,
Vo log
PPO (Schulman et al., 2017)
E
min
Ta( lits)
Re, clip
mo(selar)) Figeat Fieer (Dclire) 1 - 1 1+c) Ri)
Th (Del
SimPO (Meng et al., 2024)
E
log
log ne (vv"
log tooo (x)) (2))-7)
GRPO (Shao et al., 2024)
E
min
Ro (ax) Te (o. (1c))
No, clip
>a(u (a)) To, (ufll1))
1 1 1+5) R&))
ADxx [ [red]
Existing works leverage traditional RL optimization techniques combined with explicit length-based re- ward to control the length of CoT reasoning are shown in Figure 5. The detailed length rewards are shown in Table 1, and the details of different reinforcement learning optimization objectives, accompanied with length rewards, are shown in Table 2. The work (Arora & Zanette, 2025) proposes utilizing length-based rewards conditioned on correctness, where shorter correct answers receive higher rewards. They then apply traditional policy gradient methods guided by this reward scheme to encourage I.I.Ms to produce concise reasoning steps. Expanding from the policy gradient, the following discussed work is primarily built upon proximal policy optimization (PPO) (Schulman et al., 2017) with CoT length penalty. Demystifying (Yeo et al., 2025) presents empirical findings from RL experiments examining how reasoning capability is in- fluenced by length. They demonstrate that RL does not consistently or reliably increase the length and complexity of CoT reasoning, emphasizing the necessity of controlling CoT length growth to ensure stable performance. To mitigate these issues, they proposed a Cosine Reward based on a Dirichlet function of a concise reward formula (Loshchilov & Hutter, 2016) and the proposod "cxcoed length penalty" scores. Due to the performance impact of CoT length, Kimi k1.5 (Team et al., 2025) incorporates a length penalty into its policy optimization (a variant of online policy mirror decent. (Tomar et al., 2020)) to improve long CoT acti-
7

---

Published in Transactions on Machine Learning Research (08/2025)
vations and facilitate effective model merging. Besides optimizing with length penalty reward, L1 (Aggarwal & Welleck, 2025) modify the training data with the designated length constraint instruction (i.e. Think for N tokens) before launching the policy optimization with pre-trained reasoning LLMs. O1-Pruner (Luo et al., 2025) introduces the bength--arroniziin
Reward, combined with a PPO-style loss, to optimize reasoning LLMs by effectively shortening the CoT length. Specifically, the Suggth-Harmoniiing Reward is computed based on the ratio of CoT lengths between the reference model output and the predicted results. Additionally, this reward incorporates accuracy-based constraints comparing predictions to the reference model outputs, ensuring that shortening the reasoning process does not degrade task performance. Without relying reference model, DAST (Shen et al., 2025b) employs SimPO (Meng et al., 2024) to fine-tune reasoning LLMs using a constructed length-ppeference dataset. This dataset is generated based on a self-defined token-length budget measurement Luudget, defined as a linear combination of the average loken length of correct responses and the maximum allowed generation length.
These Rased methods enable the mitigation of overthinking reasoning-capable LLMs, where overthink- ing refers to unnecessarily extended reasoning processes, leading to longer inference times and exceeding com- putational budgets. By achieving nearly lossless alignment with the original reasoning capabilities of LLMs, these budget-efficient RL strategics democratize the deployment of reasoning LLMs in resourco-constraneed scenarios.
3.2 SFT with Variable-Length CoT Data
Fine-tuning LLMs with variable-length CoT data is an effective way to improve the efficiency of reasoning. As shown in Figure 6, this series of works typically involves: (1) Constructing variable-length CoT reasoning datasets via various methods, and (2) Applying SFT with collected data reasoning models to enable LLMs to learn compact reasoning chains that encapsulate effective knowledge. Note that this method is not limited to RI-trained reasoning models; it can also directly enhance reasoning models by injecting elficient reasoning capabilities, similar to those used in distilled reasoning models.(e.g, DeepSeek- R.-Distill- Qwen (Guo et al., 2025)).
The key question is: How to collect variable-length CoT reasoning data, especially for short CoT data?
Variable-Length CoT Reasoning Data
Long CoT Data
SFT
p
LLMs/ Reasoning Model
LLMs
Efficient Reasoning Model
Short CoT Data
Figure 6: Illustration of methods for utilizing SFT with variable-longth CoT reasoning datascts.
3.2.1 Constructing Variable-Length CoT Reasoning Datasets
Variable-length CoT reasoning datasets refer to datasets of long/short reasoning steps that could guide LLMs to achieve correct answers. Existing works typically gather long CoT data by prompting pre-trained reasoning models with questions. Based on the long CoT data, the key challenge is: How to collect short CoT data? Overall, variable-length CoT reasoning datasets can be created via either post-reasoning or during-reasoning. We list some detailed approaches in Table 3.
Post-rcasoning CoT Compression. This approach collects short CoT data by reducing redundant rea- soning steps after full-length reasoning, either by heuristic criterion or LLMs (Yu et al., 2024; Kang et al., 2024; Xia et al., 2025). Specifically, Yu et al. (2024) uses reasoning-capable LLMs to generate the reasoning
8

---

Published in Transactions on Machine Learning Research (08/2025)
Table 3: Comparison of various approaches that utilize SFT with variable-length CoT reasoning datasets.
Method
Source Data
Reasoning Pruning
SFT
LLMs
Sampling N reasoning then select the shortest one
GSM8K
Llama-3.2- (a-ma-3.--(1BB333
Seli-iriinnng (Munkhbat et al,, 2025)
Standard
MATH
Llamm-3.1-8B
GSMSK
Skip tokens according to semantic importance
LLaMA-3.1--8 InB--uttt
TokenSkip (Xia et al., 2025)
Standard
MATH
Qwen2 5-Insrret
GSM8K
GPT-4 compressor to make concise reasoming
MathQQA
C36T (Kang et al., 2024)
Standard
Llama 2-cnm-2222---7B11113
ECQA
StrategyQA
Distilling-11 (Yu et al., 2024)
OASST2
Removing rensoning
Standard
Llana- 2-70-----70Blut
GSM8K
Persuing an optimal token budget for LLMs to complete the remsnnig
Tolen-Budget (Han al., 2024)
GSMSK-Z
Standard
Llama-3.1-44 1-8--Insstett
MathBench
QwQ-33BB- 220--rreeeww
Merging paraneterrs of non-russoning and long reasoning L.LSs
DeepSeck ftl-Distil-Llamma-e
GSM8K PRMS00K
VolvValve (Ma et al., 2025h)
Progressive
LIaMA-3.1-88
L.2MA-3.2-II Qwen32B-Inntruuc
Analog of Algebra Multi-digit Addition
Stage 1: Manually skipping
Standard & Progressive
Llama-2-7B
LearnSkip (Liu et al., 2024n)
Stage 2:
Prompting LLMs for shorter reasoning
Phi-3-mini (3.8B)
Directional Reasoning
and answers. After generating full-length CoT data, they discard the reasoning process, only using the questions and answers to distill system-1 LLMs. Another work C3oT improves the reasoning efficiency by compressing the reasoning process (Kang et al., 2024). The long CoT reasoning steps were generated by explicitly prompting LLMs. Then, it employs GPT-4 as a compressor to reduce the length of the reasoning process while ensuring the compressed reasoning retains all key information and removes redundant words. In addition, TokenSkip reduce the reasoning steps driven by interpretation (Xia et al., 2025). Tt estimates the semantic importance of each reasoning part to the final answer and reduces the reasoning tokens. The important parts preserve the key reasoning steps that could improve the accuracy of the final answer. The advantage of post-reasoning CoT compression is that it can achieve a higher reduction rate of the reasoning steps, which advances more efficient reasoning.
Obtaining Compressed CoT Data during Reasoning. This approach collects short CoT data by prompting LLMs to generate short reasoning steps during inference and reasoning (Lin ct al., 2024a; Munkhbat et al., 2025; Han el al., 2024; Ma et al., 2025b). Specifically, Liu et al. (2024a) proposes a human-like step-skipping method for generating shorter reasoning steps. In the first stage, based on the original training datasets, they manually create solutions by skipping steps, either guided by human exper- tise or by randomly merging removing steps. Further, these concise data are labeled with prompts such as "Solve it in steps.". After SFT, the model is able to generate shorter reasoning paths. In the second stage, they prompt this model to solve problems by intrinsically skipping or compressing steps during rea- soning. The generated concise reasoning steps with questions and answers are collected as datasets, which are then used in SFT to make LLMs solve problems with fewer steps. Moreover, Token- Buddet (Han et al., 2024) has an important insight: an optimal token budget helps LLMs actively follow the token constraint to complete the reasoning process. Motivated by this insight, it proposes binary search-based method to achieve the optimal token budgets, and follow these budgets to generate short reasoning steps. In addition, based based method to improve reasoning efficiency. Specifically, it examines the distribution of reasoning lengths and finds that shorter solutions appear more frequently than the typical reasoning length. Driven by this finding, it proposes a Best-of-N (BoN) Sampling at test time, which generates N paths of reasoning and selects the shortest one. These short reasoning paths are collected as the datasct. Finally, Calve (Ma et al., 2025b) controls the reasoning length by mix-up the parameters of long reasoning and non-reasoning LLMs for generating variable-length reasoning steps. They also release their mixed dataset, such MixChain-Z-GSMMKK and MixChain-C-LLIM.. The advantage of
Munkhbat et al. (2025) proposes sampling-
9

---

Published in Transactions on Machine Learning Research (08/2025)
COT compression during reasoning is that the naturally generated reasoning steps align with the intrinsic knowledge of LLMs, which advances more effective learning of L.L.s.
3.2.2 Fine- Tuning Approaches
Fine-Tuning
After collecting variable-length CoT data, existing works fine-tune LLMs to achieve efficient reasoning in several ways, which include standard fine-tuning (e.g., parameter-efficient fine-tuning such as LoRA (Hu et al., 2022) or full fine-tuning) and progressive fine-tuning.
Standard Fine-tuning. Most of the work adopts standard methods to fine-tune LLMs (Liu al., 2024a; Munkhbat al., 2025; Yu et al., 2024; Kang et al., 2024; Xia et al., 2025; Han et al., 2024). Specifically, these approaches adopt LoRA (Hu et al., 2022) or full fine-tuning (Kang et al., 2024) to minimize the perplexity loss function or DPO loss function (Han et al., 2024) on the reasoning-effccien datasets. The LoRA enables LLMs to adapt to short reasoning steps with less than 1% of the parameters tuned. In addition, (Liu et al., 2024a) observed the growing reasoning efficiency can generalize to out-of-domains beyond the collected datasets.
Progressive Fine-tuning. Progressive fine-tuning aims to smoothly reduce the reasoning steps during fine-tuning (Ma et al., 2025b; Liu et al., 2024a). One way is to progressively reduce the reasoning steps of data during fine-tuning LLMs, as employed in (Liu et al., 2024a). Another effective way is to progressively adjust the generation of reasoning steps, as proposed by CoT-Valve (Ma et al., 2025b). Specifically, it first learns LoRA adaptor A0N and Ot,, where LLMs with A0N have reasoning steps, and that with A0,. have long reasoning. Then, it mix-up A0x and Aftong by aAOx + A1 to generate dataset reasoning with variable length. Here 0 < < 1 controls the parameter to shift from AdN to A0L, controlling the reasoning length generated by LLMs. Finally, it fine-tunes LLMs on the generated data while progressively reducing a from to O. In this way, reasoning efficiency is progressively improved during fine-tuning.
4 Reasoning Output-based Efficient Reasoning
From the perspective of reasoning steps in the output, these works focus on modifying the output paradigm to enhance the ability of LLMs to reason concisely and efficiently.
4.1 Compressing Reasoning Steps into Fewer Latent Representation
Although standard CoT methods improve LLM performance by explicitly writing reasoning steps, recent work (Deng al., 2024) has shown that simply adding intermediate "thinking" tokens, or even meaningless filler (e.g., ".....") (Pfau al., 2024), can also increase performance. Geiping et al. (2025) scales up deeper reasoning through recurrent expansions in the hidden space rather than verbose text. These findings highlight that the benefit often lies in more hidden computation rather than purely textual decompositions. Building on the insight that latent reasoning can allow LLMs to reason more efficiently and flexibly, with fewer for no) explicit textual intermediate steps, several new methods focus on compressing or replacing explicit CoT with more compact latent representations.
The key question is: How to compress reasoning steps into latent space?
In general, these methods can be categorized into two types: training LLMs to inference using latent repre- sentations or using an auxiliary model. A visualized comparison of some of these approaches is presented in Figure 7.
Training LLMs to Leverage Latent Representations. Among the first explorations, Coconut (Chain of Contimuous Thought) (Hao et al., 2024) treats the final-layer hidden states of an LLM as "continuous thought" to replace traditional discrete tokens. It then reuscs these hidden states as the next input em- beddings. Trained step by step, Cocomut gradually adds these latent CoT tokens. The results suggest that compressing tokens into latent representations improves both accuracy and efficiency by reducing the number
10

---

Published in Transactions on Machine Learning Research (08/2025)
Large Language Model +
Gradually Training LLM A
Self-distillation LLM
Coconut
CODI
CoT
Prompt
Large Language Model
Intermediate Reasoning Step
cCOT
Decode
Answer
Assistant LLM
Projection
Latent/Soft Thoughts
CCOT
SoftCoT
Figure Comparison of mothods of compressing reasoning steps into fewer latent representations.
of intermediate "thinking" tokens. CODT (Shen et al., 2025c) leverages a. different training process compared to Coconut, which learns the continuous latent CoT via self-distillation In CODI, the model serves both teacher and student, jointly learning explicit and implicit CoT while aligning hidden activations on the to- ken, generating the final answer. This self-distillation process enables LLMs to perform reasoning internally without generating explicit CoT tokens. Similarly, CCOT (Cheng & Van Durme, 2024) condenses long CoT reasoning into short contentful and conlinuous contemplation lokens. First, it precomputes the full CoT for a query and selects the most important hidden states as gold standard for compression. The CCOT module (a LoRA) is trained to predict these key tokens. Then, the DECODE module (another LoRA) is trained on the query plus compressed tokens. During inference, CCOT generates compressed tokens, which DECODE uses to produce concise reasoning steps. Another type of work, summarization-uaee dynamic reasoning, as mentioned in Section 4.2 explores compressing and summarizing reasoning steps in discrete space during inference, which is similar to the introduction of "contemplation token".
Another work, Heima (Shen et al., 2025a), inspired by Cocomut (Hao et al., 2024), brings latent reasoning into Multimodal Large Langue Models (MLLMs). Instead of always using full, lengthy reasoning explanations, Heima replaces each stage of detailed reasoming with a single "thinking token". With this change, the training data is updated. Instead of long textual explanations, each reasoning stage is just one of these thinking tokens. Then, they continue fine-tuning the model to achieve efficient reasoning. Token Assorted (Su et al., 2025) adopts a hybrid approach. During training, part of the CoT is replaced by discrete latent tokens learned via VQ-VAE (Van Den Oord et al., 2017), and then the LLM is trained with a partial and high-level abstract of the reasoning steps. The authors show that mixing text tokens with latent tokens can facilitate training and inference by representing some reasoning steps in a compact latent form. Other than explicitly compressing the discrete tokens into latent space, Saunshi et al. (2025) demonstrates that looping k-layer transformer times can emulate the performance of a k--laaer model. This looping mechanism effectively increases the depth of the model depth without adding paramcters, cnabling itcrative reasoning processes within the latent space. The study reveals that looped models implicilly generale latent thoughts, allowing them to simulate multiple steps of CoT reasoning through successive loops.
Training Auxiliary Modules while Keeping LLMs Frozen. While most methods for continuous- space reasoning fine-tune the pre-trained LLM, SoftCoT (Xu et al., 2025c) keeps the underlying LLM frozen. A lightweight auxiliary model generates instance-specific soft thought tokens projected into the embedding space of the frozen LLM. Experiments show that SoftCoT consistently boosts performance, demonstrating the viability of augmenting LLMs with external latent reasoning tokens.
These methods hint a broader move toward latent reasoning, where critical thinking occurs in compressed, non-textual forms. Such approaches can unlock improved speed, adaptive inference, parallel backtracking, and new ways to interpret partially reveal the model reasoning. As L.MSs grow larger and tasks become
11

---

Published in Transactions on Machine Learning Research (08/2025)
more complex, balancing thorough reasoning with computational efficiency is greatly beneficial from these flexible and compact latent CoT paradigms.
4.2 Dynamic Reasoning Paradigm during Inference
Existing works focus on modifying the reasoning paradigm for more efficient inference. The key during inference choosing the proper criterion to guide the reasoning strategy. Current training-free approaches explore dynamic reasoning using various criteria, such as reward-guided, confidence-based. and consistency- based selective reasoning. Additionally, summarization- baaeerrrzaatooo-aaedddd dynamic reasoning method intrinsically integrates the output summarization paradigm of LLMs during training.
The key question is: Which criterion to guide the inference? What is the appropriate efficient inference paradigm?
4.2.1 Dynamic Reasoning via Explicit Criteria
Train-time scaling with RL (Guo et al., 2025) can significantly enhance the reasoning ability of LLMs. How- ever, it requires substantial computational resources to scale up the model training, making it prohibitively expensive (Guo et al., 2025). As an alternative, researchers have explored test-time reasoning, also known as test-time scaling (Snell et al., 2024). Instead of relying on training to learn CoT reasoning steps, test-time scaling leverages various inference strategies that allow models to "think longer and broader" on complex problems. This approach consistently improves performance challenging math and code problems that require reasoning by increasing the computational resources allocated during inference (Snell et al., 2024; Beeching et al.).
Test-time scaling utilizes various inference strategies to generate longer and higher-quality CoT responses. There are several ways to scale up the inference. (1) Best-of-N sampling (Sun et al., 2024; Wang et al., 2025) involves generating multiple responses for a given prompt, expanding the search space identify better solutions. After generation, the best response is selected using either majority voting, where the most frequently occurring response is chosen; or by reward model, which evaluates response quality based pre-defined criteria. This method has been shown to significantly enhance the reasoning capabilities of LLMs (Beeching et al.). (2) based searching (Ding et al., 2025; Fu et al., 2024; Beeching et al.), which differs from Best-of-N by structuring generation into multiple steps. Instead of generating an entire response in one pass, beam search selects the most promising intermediate outputs with process reward model (Uesato el al., 2022) at each step, while discarding less the optimal ones. This enables a more fine- grained optimization of both response generation and evaluation. (3) Monte Carlo Tree Search (MCTS) (Li et al., 2025a), where multiple solution paths explored in parallel. MCTS generates partial responses along different branches of a solution (ree, evaluates them, and back-propagates reward values to earlier nodes. The model then selects the branch with the highest cumulative reward, ensuring a more refined selection process compared to traditional beam search.
Although test-time scaling can significantly reduce train-time scaling up overhead (Beeching et al.), the large number of generated responses still makes inference computationally expensive. To address this, recent works have been exploring methods optimize test-time scaling.
Reward-guided Efficient Reasoning. Speculative Rejection (Sun et al., 2024) is an efficient inference- time reasoning algorithm that optimizes Best-of-N (BoN) decoding by dynamically reducing computational overhcad (as shown in Figure 8, left). It generates multiple responses until memory limits are nearly reached, then discards low-quality oulpuls based on evaluation by a reward model. This adaptive filtering substantially heward-Guided reduces inference costs compared to vanilla BoN, On the other hand, Reward- Guided Speculative Decoding (RSD) (Liao et al., 2025a) enhances the efficiency of speculative decoding specifically for multi-step reasoning tasks. Unlike traditional speculative decoding methods, which strictly require exact token matching between the draft model and target model, RSD leverages a Process Reward Model (PRM) to dynamically evaluate
12

---

Published in Transactions on Machine Learning Research (08/2025)
Table 4: Comparison of different methods of dynamic reasoning paradigm of test time compute during
inference.
Category
Method
Training- free?
Baseline and Its Draw- backs
Method Description
Reward-
Speculative Rejec-
Yes
Best-of-N (BoN) De- coding: underutilizes GPU memory computational resources during the early stages, leading lower final rewards.
Starts BoN with large initial batch size and rejects unpromis- ing sequences periodically, efficiently achieving higher rewards.
guided
tion 2021)
(Sun et ali
Efficient
Reasoning
Reward-Guided Speculative Decod- ing (RSD) (Lian
Yrs
Speculative Decading: strictly enforces unbiased- ness, discarding useful intermediate outputs leading to computational inefficiency:
A speculative decoding method that leverages a reward model (PRM) selectively accept high-quality out- puts from a lightweight draf: model, reducing computation while preserv- ing accuracy.
et al., 2025a)
Confidence/ Certainty- based Adap- tive Heason-
Dynamic Parallel Tree Search (Ding et al., 2025)
Yes
Treo-of- Teoo-gf-Thoughts:: diffi- cult parallelize due to fre- quent switching rensoning focus, and inefficient because redundant exploration suboptimal solutions
Dynamically parallelizes node expan- sion through adaptive batching and implements a searchanndcraasstion mechanism (including Early Stop and Deep Seek) to prune unpromis- ing paths early.
ing
Dynasor (Certaiudex- based based Scheduling) et al., 2024)
Yes
Serving systems with uniform resource allo- cation: allocate compute uniformly. leading to inef- ficient resourec usage uamet latency targets
Dynamically allocates compute reasoning queries based on Certain- der, statistical measure of ing progress, to maximize accuracy within resource constraints.
FastMCTS (Ti et al., Yrs 2025a)
Rejection Sampling: inef- ficient, discards intermediate stops, and fails to balance problem difficulty
An MCTS--ispired sampling strat- egy that efficiently generstes high- quality multi-step reasoning data, providing step-level evaluation sig- nals and balanced sampling across prohlem difficulties.
Length-filten Vote Yes (Wu al., 2025)
Majority Voting: ignores reasoning quality, includes suboptimal CoT lengths, and suffers from noisy pre- dictions
voting strategy that selects an- with the optimal CoT length by filerring out exerssively short or long reasoning paths.
Consistency-
Self-Truncatinn
Yes
Best-of-N Sampling: fully generates samples re- lies on costly reward models
Estimates the most promising sam- ple early via interual embedding sistency, truncating inferior samples prematurely.
based Selective
Best-of-N (ST-BoN) (Wang et al., 2025)
Reasoning
Summarization based Dy- namic Rea- soning
TigbeThinker (Zhang 2025)
Chain-of-Thought Thought (CoT): high memory and computa- tional overbead due the generation of excessive number tokens
Trains LLMs learn when compress intermediate thoughts. condensing long reasoning chains sparse-
et al.,
into gist tokens, and patterned attention mask during in- ference to enhance computationsl ef- ficiency.
InftyThink al., 2025)
(Yan No
Monolithic Reasoning: reasoning output is verbose, and can quickly excord the context window limit of the LLM, resulting in poor performance
An iterative reasoning paradigm that interleaves reasoning steps with in- termedlate summarization, enabling unbounded reasoning depth without architectural modifications.
intermediate outputs from the smaller, more efficient draft model. Outputs with high reward scores are directly accepted, while those with lower scores are further refined by a larger, more capable target model.
Adaptive Reasoning. Dynamic Parallel Tree Search (DPTS) (Ding et al., 2025) optimizes treo-based reasoning in LLMs by addressing two main inefficiencies by introducing: (1) Parallelism Streamline optimizes memory and compute by storing only incremental KV cache updates and dynamically adjusting the number of extended nodes based on available GPU memory, (2) Search and Transition Mechanism balances exploration and exploitation using confidencc-based criteria. Overall, dur- ing inference, the system cuts off uncertain paths save time. FastMCTS (Li al., 2025a) is another confidence-based method that aims to optimize multi-step reasoning data synthesis. Traditional rejection
Caafidencee/Ceraaitttyy
13

---

Published in Transactions on Machine Learning Research (08/2025)
3rd
Dropped somple
S2
Prompts
Reward model
2nd
Prompt
S3
Drapped sample
4nd
Drapped sumple
S4
Dropped sample
Early generatton
Rank and early stop
Continuous generation
Early generanton
Latent embedding
Pairwise consistency
Continuous generation
Speculative Rejection
Seff-Truncation Best-of-N (ST-BoN)
Figure 8: Examples of efficient Best-of-N sampling methods. (Left) Speculative Rejection (Sun al., 2024) reward model estimate partial generation quality. It then early stops the sampled sequence with lower scores. (Right) ST--Bot (Wang al., 2025) evaluates the latent embedding of the early generation. The latent embedding of cach thinking path will be used to calculate pairwise consistency between other tokens. The sequence with the highest consistency is more likely to arrive at the correct answer.
sampling generates multiple candidate responses independently, selecting only the correct ones, but it is often inefficient and struggles with imbalanced sampling. Inspired by MCTS, FastMCTS prioritizes high- confidence traces for deep reasoning. Additionally, it adjusts tree expansion based problem complexity, improving both efficiency and reasoning diversity. Another line of research leverages certainty or uncertainty measures to guide adaptive reasoning. Certaindex (Fu et al., 2024), a certainty metric, quantifies the confi- dence of LLMs throughout reasoning using semantic entropy, reward model scores, or a combination both. A higher certaindex indicates that further reasoning steps are unlikely to change the final answer, allowing early termination to free resources for more challenging queries. Dynasor, an inference system built on this principle, optimizes compute scheduling by dynamically tracking reasoning progress instead of allocating resources uniformly. Length-filtered Vote (Wu et al., 2025) is another work that leverages uncertainty to im- prove CoT reasoning. The study finds that longer reasoning chains do not always improve accuracy; instead, performance initially improves but eventually declines due to error accumulation. The authors provide a mathematical analysis proving the existence of an optimal CoT length, determined by model capability and task difficulty. To exploit this, they propose Length-filtered filtered Voting, length-aware majority voting method that groups answers by CoT length and selects the most reliable group based prediction uncertainty.
Consistency-based Selective Reasoning. Trunncctnn Best-of-N (ST-BoN) (Wang et al., 2025) en- hances BoN sampling efficiency by introducing early termination (as shown in Figure 8, right), similar to Speculative Rejection (Sun et al., 2024). However, unlike Speculative Rejection using reward models, ST- BoN leverages consistency as the metric to mcasure the importance. Specifically, it leverages the consistency of latent embeddings to evaluate response quality. The core insight is that "the closer a sample is to oth- ers, the more likely its path will lead to the correct answer". Then, ST-BoN selects the most consistent Chain-of Emindddnn (CoE) to others and regards it as the optimal sample.
4.2.2 Summarization based Dynamic Reasoning
Some existing methods choose to optimize reasoning efficiency by training LLMs to summorize intermedi- ate thinking steps. LightThinker (Zhang al., 2025) proposes to train LLMs to learn when and how compress intermediate reasoning steps. Instead of storing long thought chains, LightThinker compresses verbose reasoning into compact "gist tokens" to reduce memory and computational costs. Implementing this summarization paradigm requires sparse-patterned attention mask, ensuring the model focuses only es- sential compressed representations. InftyThink (Yan et al., 2025) introduces an iterative reasoning method that enables essentially infinite reasoning chains while maintaining strong accuracy without surpassing the context window limit. It achieves this by iteratively generating a thought, summarizing it, and discard- ing previous thoughts and summarics, retaining only the most recent summary. Additionally, InftyThink provides technique for converting existing reasoning datasets into an iterative format for training models under this paradigm.
14

---

Published in Transactions on Machine Learning Research (08/2025)
Table 5: summary of prompts used with reasoning models generate concise reasoning outputs. For further details, refer to Section 5.1.
Method
Prompt
TALE-EP (Han et al., 2024)
Budget Estimation: (...) Task: Analyze the given question and estimate the minimum number of tokens required to generate a complete and accurate response. Please give the response by strictly following this format: [[bud- get]], for example, Budget: [[12]]-
Token-budget-aweee CoT: Please answer the above question. Let's think step by step and use less than TookenBBudget> tokens.
CoD (Xu et al., 2025b)
Think step by step, but only keep minimumn draft for each thinking step, with 5 words at most. Return the answer at the end of the response after separator ###0.
CCoT (Renze & Guven, 2024)
concise.
Token Complexity (Lee et al., 2025)
BulletPoints (...) only bullet points.
OnlyNumbers (--.) only use numbers or equations.
NoSpaces (...) do not use any spaces or line breaks.
NoProperGrammar (...) do not use proper grammar.
AbbreviateWords (...) abbreviate words much possible.
WordLimit(k) (...) use at most words. & [1...., 100}
CharLimit(k) (...) use at most h letters. & E {1....,,500 500}
TokenLimit(k) (...) use at most & tokens. & E (1,...,,000 5003
StepLimit (...)
most l steps. E (1.
ChineseCoT (...) Respond Chinese
ChineseCoT(k) (...) Use at most l Chinese characters. & E (1,...,500}
5 Input Prasets-based Efficient Reasoning
From the perspective of input prompts and questions, these works focus on enforcing length constraints or routing LLMs based on the characteristics of input prompts to enable concise and efficient reasoning.
5.1
Prompt-guided Efficient Reasoning
Prompt-guided efficient reasoning explicitly instructs LLMs to generate fewer reasoning steps, can be a straightforward and highly effective method for improving the efficiency of reasoning models. As shown in Table 5, different methods propose different prompts to ensure concise reasoning outputs from the model.
The key question is: Which prompts can accurately control the reasoning length of LLMs?
Enforcing Concise Reasoning via Varying Prompts. Token-Budget (Han et al., 2024) proposes ting token budget in prompts to reduce unnecessary reasoning tokens. To optimize efficiency while preserv- ing accuracy, TALE-EP (Han et al., 2024) introduces training-free, zero-shot method for budget estimation. TALE-EP first estimates reasonable token budget by prompting the LLM itsclf. It then incorporates this estimate into a prompt that specifies the loken constraint, guiding the LLM to generate more token- efficient yet accurate response. This work also categorized in Section 3.2 with further SET. CoD (Xu et al., 2025b) observes that LIMS often generate excessively verbose reasoning steps, whereas humans typ- ically record only the most essential insights. To enhance reasoning efficiency, they propose Chain-of -haff- prompting. Similar to CoT prompting, CoD encourages step-by-step reasoning but introduces policies to
15

---

Published in Transactions on Machine Learning Research (08/2025)
limit verbosity. For instance, their prompt instructs: " "Think step by step, but only keep minimum draft for each thinking step, with at most five words." 'Thyy find that this approach preserves the necessary interme- diate steps while maintaining accuracy, significantly reducing token usage. Lee et al. (2025) systematically studics the relationship between reasoning length and model accuracy across various prompts with explicit compression instructions (e.g., "use 10 words or less"). Their analysis reveals a universal trade-off ween reasoning length and accuracy, showing that different prompt-basod compression strategies align on the same cccuracy-compprssion curve. They hypothesize that each task has an intrinsie token complexity, the mini- mum number of tokens required for successful wroblem-solving By computing coormationntheorete limits trade-off, they found that existing prompt-based compression methods fall far
on the short of these limits, indicating significant room for improvement. Renze & Guven (2024) introduced Concise Chain-of- Thhuughh (CCoT) prompting, a technique that prompts LLMs to perform step-by-step reasoning while explicitly instructing them to "be concise." " MARP (Chen et al., 2024a) introduces modifying prompts to limit single-step computations, effectively refining the reasoning boundary. Further, they increase the per-step computation and decrease global planning steps.
Finc-tuning after Prompting. As noted in Section 3, some approaches collect short CoT data using prompl-based methods, then apply SFT to develop efficient reasoning model (Han et al., 2024). Beyond performing direct prompt-based reasoning, these fine-tuned models often deliver more promising performance when tackling complex reasoning challenges.
5.2 Prompts Seributee-riiven Reasoning Routing
User-provided prompts range from easy to difficult tasks. Routing strategies for efficient reasoning dynamically determine how language models handle queries based on their complexity and uncertainty. Ideally, reasoning models can automatically assign simpler queries to faster but less reasoning-onpuble LLMs, while directing more complicated queries to slower but stronger reasoning LLMs.
The key question is: What criterion should be used to determine the attributes (e.g.; difficully) of prompts?
Unknown Criteria. Authropic releases Claude 3.7 Sonnet (Anthropic, 2023), notable for being the first hybrid reasoning model. Claude 3.7 Somnet was developed through RL, enabling it to allocate more time to complex reasoning tasks that require deeper analysis, ultimately producing better results. The model offers two response modes: quick answers or step-by-step thinking. Users can leverage APT to manage the amount of time the model spends thinking. Although the specifics of the routing criterion remain unclear, Claude 3.7 Sonnet represents the first hybrid reasoning model, setting foundation for subsequent routing-basct large reasoning models.
Training Classifier. RouteLLM (Ong et al., 2024) trains a query router to dispatch incoming queries to suitable LLMs based on complexity. The authors utilize a substantial amount of preference data col- lected from Chatbot Arena as training data, enabling effective routing decisions for question-answering and reasoning tasks. Consequently, simpler queries are directed to low-latency LLMs, while complex queries are assigned to higher-latency, more powerful LLMs, significantly accelerating overall reasoning efficiency. Shetch-of-Thhught (SoT) (Aytes et al., 2025) leverages routing and prompting to minimize token usage during reasoning. A lightweight DistilBERT-base router dynamically selects the most suitable paradigm based on the characteristics of the questions. Inspired by cognitive science, SoT employs three distinct paradigms: Conceptual Chaining, which connects ideas with minimal verbalization; Chunked Symbolism, which structures mathematical reasoning into concise symbolic representations; and Expert Lezicons, which adopts domain-specific shorthand used by experts.
Uncertainty. Besides relying on additional routers, Self-Ref (Chuang et al., 2025c) enables LLMs to autonomously decide when to route by extracting intrinsic uncortainty scores as self-routing indicators. Specifically, they fine-tune uncertainty-speeialieed tokens within the LLMs to align uncertainty predictions
16

---

Published in Transactions on Machine Learning Research (08/2025)
with prediction correctness in both question-answeringg and reasoning tasks. This ensures that only uncertain or incorrect outpuis trigger routing to more capable LLMs, which decreases the latency of LLM inference. Confident (Chuang et al., 2025a) aims to provide calibrated data for predicting and initializing routing strategics in both LLM question-answering and reasoning tasks without requiring access to user queries. This approach enables more efficient and reliable decision-making in determining whether an LLM should confidently generate an answer or escort the query a stronger model, ultimately improving reasoning efficiency from a query-level perspective in online LLM service scenarios.
6 Reasoning Abilities via Efficient Training Data and Model Compression
6.1 Training Reasoning Models with Less Data
Improving the efficiency of reasoning models requires optimizing not just the model architecture but also the data used for training. Recent work has shown that carefully selecting, structuring, and leveraging training data can significantly reduce data requirements while maintaining or improving reasoning performance. Although all approaches locus on ellicient data selection, they vary in defining and utilizing efficiency.
The key question is: How to construct less but high-quality training data? ?
Minimal but High-Impact Data Selection. LIMO (Ye et al., 2025) challenges the conventional belief that complex reasoning lasks require exlensive training data. They introduce LIMO, a framework that elicits sophisticated reasoning abilities using minimal but precisely curated examples. By choosing high-quality questions based on Level of difficulty, Generality, and Knowledge Diversity and high-quality solutions based Optimal Structural Organization, Effective Cognitive Scaffolding, and Rigorous Verification, with only 817 carefully selected training samples, LIMO can outperform previous models that utilized over 100,000 examples. s1 (Muennighoff et al., 2025) focuses on enhancing reasoning performance by controlling test-time computational resources. They curate a compact dataset based on Qualily, Difficulty and Diversily, s1K, comprising 1,000 high-quality questions paired with reasoning traces. Through supervised fine-tuning on this dataset and implementing "budget forcing", which regulates the reasoning duration during inference, s1-32B exceeds OpenAl ol-preview on MATH and AIME24, demonsttating that strategic test time scaling can effectively enhance reasoning capabilities without extensive training data.
Self- Verification as a Data-Efficient Training Signal. S (Ma et al., 2025a) infuse LLMs with self- verification and self-correction abilities through RL. Initially, models are fine-tuned on a curated dataset to establish these capabilities. Subsequently, RL both at the outcome level and the process level employed to enhance these skills further. With only 3,100 initialization samples, their finc-tuned models consistently improve the performance reasoning tasks among all base models. S²R fine-tuned Owen2.5-Math- can outperform models trained on comparable amounts of long CoT distilled data on the MATH500 and GSMSK.
6.2 Reasoning Capabilities of Small Language Models via Distillation and Model Compression
LLMs have demonstrated remarkable reasoning capabilities across various complex tasks, benefiting from their extensive training diverse datasets. However, their substantial computational and memory de- mands pose challenges for deployment in resource-constrained environments, such as edge devices, mobile applications, and real-time systems. In scenarios where efficiency, cost, or latency is a primary concern, Small Language Models (SLMs) offer a viable alternative. The ability SLMs to retain strong reasoning capabilitics while operating under strict resource constraints crucial for expanding the accessibility and practicality of Al-powered reasoning systems. To achieve this, two main categories of approach are explored: Distillation and Model Compression.
17

---

Published in Transactions on Machine Learning Research (08/2025)
The key question is: How do small language models perform on reasoning tasks? What impact does model compression (e.g., quantization) have on their reasoning abil- ities?
Distillation. Distillation is a crucial technique for transferring the reasoning capabilities of LLMs to SLMs while maintaining efficiency. However, Li et al. (2025c) finds a phenomenon named Small Model Learnability Gap, which highlights the challenges of distilling complex reasoning processes from large model to small model, showing that SLMs struggle to emulate the reasoning depth of their larger counterparts. To address this, various approaches have been proposed. Both Li et al. (2025c) and Chenglin al. (2024) explored mixed distillation, with Li et al. (2025c) blending long and short CoT reasoning examples, while Chenglin et al. (2024) combined CoT and PoT (Program of Thought) to improve the effectiveness of knowledge distillation from LLMs to SLMs on specific tasks. In comparison, Feng et al. (2024) introduced counterfactual distillation, augmenting the training set by masking causal features in the original question, prompting the LLM to complete the masked text, and generating multi-view CoT (positive and negative views) of each data for enhancing the effectiveness of knowledge distillation. In addition, Zhu al. (2024) developed a feedback-driven distillation technique that iteratively refines distillation datasets. They first prompt LLM to generale an initial distillation datasel, then expand it by creating diverse and complex questions from existing ones, and finally, this enriched dataset is used to fine-tune SLMs. Another strategy, proposed by Zhao et al. (2024), incorporates probing and retrieval mechanisms into the distillation pipeline. It trains two complementary distilled SLMs, a probing model and a reasoning model, where the probing model retrieves relevant knowledge, which the reasoning model then uses to construct a step-by-step rationale for the answer. Chen et al. (2024b) introduced adaptive thinking during distillation, allowing the models dynamically adjust reasoning strategies based the complexity of the task. Furthermore, Liao el al. (2025b) proposed SKIntern, framework that internalizes symbolic knowledge into SLM to improve CoT reasoning quality and efficioncy, while Zhang et al. (2024) introduces SCORE, pipcline that generates self-corrcetion data from SLMs and fine-tunes the model to function as a self-correcting reasoner. These diverse distillation techniques demonstrate that efficiently transferring reasoning capabilities from LLMs to SL.Ss requires not only reducing the model size but also carefully and strategically structuring the knowledge transfer process to preserve logical depth and generalization.
Pruning and Quantization. Beyond directly distilling knowledge from LLMs to SLMs, an alternative approach involves compressing an LLM into an SLM using techniques such as quantization and pruming. Srivastava et al. (2025) conducted a comprehensive study analyzing the impact of various model compression techniques on reasoning ability. Their findings reveal that quontization, which reduces model precision to lower-bit representations, preserves reasoning performance remarkably well, allowing SLMs to maintain logical coherence and problcm-solving capabilitics while significantly reducing memory and computational costs.
In contrast, pruning, which removes specific weights neurons in the model based their importance, leads to severe degradation in reasoning quality, disrupting the model's ability to follow multi-step logical processes. This suggests that compression-based approaches are more effective than training SLMs from scratch, as they allow models to retain reasoning structures inherited from LLMs. However, a critical challenge remains: SLMs often struggle with the instruction following, indicating that compression alone is insufficient. Additional fine-tuning or adaptation methods may be required to align compressed models with user intent and ensure they can effectively interpret and execute complex reasoning tasks.
7 Evaluation and Benchmark
Recent research has introduced innovative benchmarks and evaluation frameworks to systematically assess the reasoning capabilitics of LLMs. As LIMs continue to advance in their ability to perform complex rea- soning tasks, the need for rigorous, standardized evaluation metrics and frameworks has become increasingly important.
18

---

Published in Transactions on Machine Learning Research (08/2025)
Evaluating Overthinking. Cuadron et al. (2025) introduces framework to systematically analyze the 'overthinking" in I.MMs, where models favor extended internal reasoning over necessary environmental in- teractions. By examining 4,018 trajectories in agentic tasks, the study identified patterns such as Amalysis Paralysis, Rogue Actions, and Premature Disengagement. Cuadron et al. (2025) also proposed a novel "overthinking score" and showed a strong correlation bet ween higher scores and decreased task performance. Mitigation strategies such as selecting solutions with lower overthinking scores can improve performance by 30% and at the same time reduce computational overhead by 43%.
Effect of Long CoT Reasoning. Yoo et al. (2025) provides a comprchensive analysis of the mechanism underlying long CoT reasoning. In addition to presenting several key insights, they propose a reward design to enhance the stability of reasoning ability during training and reduce the CoT length, which is also shown in Section 3.1. Jin et al. (2024) reveals strong relationship between the length of the reasoning chain and the effectiveness of model outputs. Models tend to perform better with extended reasoning steps, suggesting the CoT length is more crucial than accuracy for elfective problem-solving.
8 Applications and Discussion
8.1 Applications
Autonomic Driving. Efficient reasoning LLMs are able to greatly improve autonomic driving (Cui et al., 2024; Xing et al., 2025) by helping them understand large amounts of sensor data in a human-like way. They make the cars better at making decisions, so the vehicles can plan for difficult driving situations and react quickly when unexpected events occur. By combining information from cameras, LiDAR, radar, and other sensors, these models help cars drive more safely, choose better routes, and assess risks as they happen. Moreover, because they can explain why they make certain decisions, both passengers and regulators feel more confident in the technology, and the cars can interact more smoothly with smart road systems.
Embodied AI. Efficient reasoning LLMs make embodied AT (Duan el al., 2022) much smarter by helping robots and smart devices understand and react to the world around them. These models process lots of data from cameras, sensors, and other inputs a way that resembles human thinking. This deep understanding means that a robot can quickly decide the best way to move, handle unexpected changes, and interact safely with people. For example, in busy factory or home setting, robot using these models can navigate obstacles, adjust to situations, and even explain its actions in simple terms. Altogether, efficient reasoning LLMs boost the reliability, safety, and usefulness of embodied AI systems in daily environments.
Hcalthcare. Efficient reasoning LLMs would improve healthcare (He et al., 2023) by helping doctors and researchers work with large amounts of medical data more easily. They can quickly analyze patient records, test results, and medical research to spot important trends and patterns that might be hard to see otherwise. This support can lead to faster and more accurate diagnoses, better trcatment recommendations, and fewer mistakes. In addition, these models can break down complex medical information into plain language, making it easier for both medical professionals and patients to understand. Generally, efficient reasoning LLMs make healthcare processes smoother and more reliable, leading to better care and outcomes for patients.
8.2 Discussion and Future Directions
Improving Reasoning Ability. From another perspective on efficiency, improving reasoning performance is an important topic (Chen et al., 2025; Sui et al., 2025). To prioritize promising avenues by discarding ineffective strategies early, evaluating reasoning progress and selecting the optimal strategy. In cach round, the LLM produces new reasoning step, and the mela-reasoner evaluates its out and generates a progress report, the mela-reasoner uses contextual multi-arm bandit to choose the best guidance strategy for the reasoning step. ITT (Chen et al., 2025) treats each transformer layer as a step in an internal thinking process. By dynamically allocating extra processing to difficult tokens through adaptive routing, ITT enables smaller language models to achieve performance comparable to larger models while using fewer training resources.
(Sui et al., 2025) leverages contextual multi-armed bandits for
19

---

Published in Transactions on Machine Learning Research (08/2025)
Safety of Efficient Reasoning. Safety and efficiency in LLMs often pull in opposite directions, as op- timizing one always leads to the performance degradation of the other. When enhancing safety, such as filtering harmful content, mitigating adversarial attacks, and enabling self-correction, the reasoning model typically requires additional computational resources and longer reasoning sequences, leading to increased inference costs and slower response times. Conversely, prioritizing elficiency by minimizing token usage and computational overhead may reduce the reasoning ability to self-reflect, verify its outputs, or defend against adversarial manipulations. This trade-off reflects the well-known principle that there is no "free lunch", making it crucial to strike careful balance between safety and efficiency. Kuo et al. (2025) investigates the robustness of safety checks in large CoT reasoning models, revealing severe security flaws in commer- cial systems. They introduce the malicious-eduuctoor benchmark and demonstrate that with their hijacking Chain-of-Thought (H-CoT) atlack, models can drastically reduce their refusal rales, leading to the gener- ation of harmful content. Li et al. (2025b) investigate the safety of long reasoning models. It is observed that while longer outputs enable self-correction and enhance safety, some attack stratogics exploit extended generations. They propose a dynamic output length control via an RL-based method to maintain both rea- soning quality and security. Balancing safety and efficiency in long reasoning models remains challenging yet crucial area of investigation.
RL vs. SFT, which is better? When comparing RL (Section 3.1) and SFT (Section 3.2) for creating efficient reasoning language models, the answer is unclear as each method has its own strengths. RT allows a model to learn by trial and error, rewarding it for satisfactory decisions, which can assist it find creative ways to solve problems in new situations. However, this approach can sometimes be unpredictable and require a lot of training. On the other hand, SFT teaches the model using carefully chosen efficient CoT examples constructed by either humans modcls, lcading to more consistent behavior and casier control. Yet, SFT might struggle when faced with challenges that are not covered in its training data. In practice, combining both methods might be a promising direction and potentially works best because it harnesses the creativity of RL and the reliability of SFT, resulting in a model that is both adaptable and stable.
Evaluation and Benchmark. While previous works have introduced certain evaluation frameworks and benchmarks for efficient reasoning, they still lack a comprehensive and standardized leaderboard for fair comparison. Several challenges hinder advancement in this area: First, different studies often utilize varying model architectures, making direct comparison difficult. Second, the evaluation tasks and datasets used are not consistent across works. Third, disparities in training resources further complicate benchmarking. Therefore, there is a pressing need to establish a comprehensive evaluation framework and public boards that can facilitate transparent, reproducible, and holistic assessment of efficient reasoning methods.
Multimodal Efficient Reasoning. Building on the advances in reasoning with L.LMs, recent efforts have extended reasoning techniques multimodal large language models (MLLMs) (Huang et al., 2025) for tackling visually grounded and complex tasks. As computational demands increase with multimodal data, there is growing need for concise and efficient reasoning within MLLMs. However, optimizing reasoning length and computation in multimodal settings introduces unique challenges, such as modality alignment and mooss- information fusion. Addressing these issues and developing effective strategies for efficient multimodal reasoning represents a promising and impactful research direction.
Comparisons of Efficient Reasoning Methods. Efficient reasoning methods can be divided into three main categories: model-based, outpul-based, and input-based approaches. Specifically, the model-based category includes RT-based and SFT-based methods. RL--based approaches often suffer from instability and are expensive to train, while SFT-baased methods are generally more stable but require additional computa- tional resources and costs for dataset construction. Output-based methods encompass both latent space and dynamic inference-time computation techniques. The key distinction here is that dynamic inference-time computation is typically training-free frree However, these methods are not primarily designed to reduce reason- ing length, but rather to decrease inference-time computational requirements or implement paradigms like Best-of-N reasoning. Input- based methods include prommp and query-basee based strategies. Input-promp methods are advantageous due to their simplicity. Adding budget-controlling prompt can effectively achieve concise reasoning, Query-based routing, on the other hand, involves integrating specific modules to evaluate the question difficulty, which can be implemented in both training and training free settings.
20

---

Published in Transactions on Machine Learning Research (08/2025)
Reasoning Length vs. Usefulness and Interpretability. Reducing reasoning length may improve computational efficiency, but it also raises questions about the usefulness and interpretability of the result- ing outputs. Shorter reasoning traces can accelerate inference and reduce resource consumption; however, if reasoning steps are overly condensed, they may omit important intermediate justifications, potentially reducing the transparency and worthners of the decisions of models. More systematic investigation is needed to understand how compression impacts the practical utility and clarity of model outputs.
Faithfulness of Efficient Reasoning. Efficient reasoning methods should ideally retain interpretable and logically consistent intermediate steps, ensuring that outputs remain meaningful. Nonetheless, there are observed cases where LLMs generate correct final answers even if some intermediate steps are meaning- less. Ensuring that reduced-length reasoning remains both accurate and interpretable is an open research challenge.
8.3 Broader Impacts
Efficient reasoning models hold promise for making advanced LLMss more accessible and sustainable, reducing computational costs and energy consumption. However, as these models become more widely adopted, it is crucial to ensure their transparency, fairness, and robustness, especially in high-stakes applications.
9 Conclusion
This paper provides the first structured survey of efficient reasoning in LLMs, categorizing existing ap- proaches into three arcas: model-based, reasoning output-based, and input prompts-based methods. Addi- tionally, it discusses efficient data utilization, reasoning capabilities of smaller models, evaluation techniques, and benchmarking, accompanied by a continuously updated public repository to support future research. Crucially, efficient reasoning approaches offer significant practical benefits across various domains: reducing computational costs in healtheare diagnostics, enhancing real-time decision-making and safety in autonomous driving, boosting the reliability and usefulness of embodied AT systems, and enabling quicker, more prof- itable responses in financial algorithmic trading and risk assessment. These advancements highlight the broad economic and societal value of efficient reasoning in LLMs.
References
Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint rxXi:22533.44999 2025. 5, 7,
Anthropic. Claude 3.7 sonnet, 2023. URL https://ww.antthoopic..co
anthropic. 3ow.//eeesclladd--------00- http:////////////oooooo/ooo///aa 3-7-sonnet. Ac-
cessed: March 10, 2025. 5, 16
Daman Arora and Andrea Zanette. Training language models to reason efficiently. arXiv:2502.04460 2025. 5,
arXiv preprint
Simon A Ayles, Jinheon Back, and Sung Ju Hwang Sketch--ofthoughht Elficient 1lm reasoning with adaptive cognitive-inspired sketching. arXiv preprint orXin:2503.08 05100011799 2025. 5, 16
Edward Beeching, Lewis Tunstall, and Sasha Rush. Scaling test-time compute with open models. URL https://huggingfent co/
teest-time-comppute
Macicj Besta, Nils Blach, Ales Kubicck, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehunaun, Hubert Niewiadomski,, Piotr Nyezyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 38, pp. 17682 17690, 2024. 3
Mark Chen, Jerry Tworck, Hoewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained code. Xiu preprint ar X=Xi:22007.0374, 2021.
21

---

Published in Transactions on Machine Learning Research (08/2025)
Qiguang Chen, Libo Qin, Jiaqi Wang, Jingxuan Zhou, and Wanxiang Che. Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of-thought. Advances in Neural Information Processing Systems, 37:54872 54904, 2024a. 5, 16
Xiaoshu Chen, Sihang Zhou, Ke Liang, and Xinwang Liu. Distilling reasoning ability from large language models with adaptive thinking. arXiv preprint arXi::204.000
2024b. 5, 18
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3-? on the overthinking of ol-like Ilms. arXiv preprinl ar Xrx:2244 21187 2024c. 2, 4
Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, and Haifeng Wang. Inner thinking transformer: Leveraging dynamic depth scaling to foster adaptive internal thinking. arXie preprint or Niv:25002 13842, 2025. 19
Jesfrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412 13171, 2024. 5, 11
Li Chenglin, Qianglong Chen, Liangyue Li, Caiyu Wang, Feng Tao, Yicheng Li, Zulong Chen, and Yin Zhang. Mixed distillation helps smaller language models reason better. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 1673 1690, 2024. 5, 18
Yu-Neng Chuang, Leisheng Yu, Guanchu Wang, Lizhe Zhang, Zirui Liu, Xuanting Cai, Yang Sui, Vladimir Braverman, and Xia Hu. Confident or seek stronger: Exploring uncertainty- on-device Ilm routing from benchmarking to generalization. arXiv preprint arXiv:2502.04433 0428, 2025a. 17
Yu-Neng Chuang, Leisheng Yu, Guanchu Wang. Lizhe Zhang, Zirui Liu, Xuanting Cai, Yang Sui, Vladimir Braverman, and Xia Hu. Confident or seek stronger: Exploring uncertainty-based on-device 1lm routing from benchmarking to generalization, 2025b. URL https: //////aaee /tt////xxvv////////////////.atvv org/ /250..04228. 5
Yu-Neng Chuang, Helen Zhou, Prathusha Kameswara Sarma, Parikshit Gopalan, John Boccio, Sara Bolouki, and Xia. Hu. Learning route 1lms with confidence tokens, 2025c. URL https://arxiv orr// 13284. 5, 16
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchrr Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14 14168, 2021. 1
Codeforces. Codeforces competitive programming platform, 2025. URL https://codeforros.comm Ac- cessed: 2025-03-18.
Rémi Coulom. Efficient selectivity and backup operators in monte-carlo tree search. Tn Internalional con- ference on computers and games, pp. 72 83. Springer, 2006.
Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, Aditya Desai, Ton Stoica, Ana Klimovic, Graham Neubig, and Joseph E. Gonzalez. The danger of overthinking: Examining the reasoning- action dilemma in agentic tasks, 2025. URL
http::///xxxvvvvoeee abs 220.. 08235. 5, 19
Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei-Da Da Liao, et al. A survey on multimodal large language models for autonomous driving. In Proceedings of the TEEE/CVT Winter Conference on Applications of Computer Vision, pp. 19
2024.
Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, et al. Stepwise perplexity-guided refinement for efficient chain-of-thought reasoning in large language models. arXiv preprint arXiv:2502. 132600 2025. 5
Yuntian Deng, Yejin Choi, and Stuart Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step. ar Xie preprint or or 14898, 2024.
10
22

---

Published in Transactions on Machine Learning Research (08/2025)
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171-4186, 2019. 1
Yifu Ding, Wentao Jiang, Shunyu Liu, Yongcheng Jing, Jinyang Guo, Yingjie Wang, Jing Zhang, Zengmao Wang, Ziwei Liu, Bo Du, et al. Dynamic parallel tree search for efficient IIm reasoning. arXie preprint arXiv:2502. 16235, 2025. 5, 12, 13
Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research lasks. IEEE Transaclions on Emerging Topics in Compulalional Intelligence, 6(2): 230 244, 2022. 19
Tao Feng, Yicheng Li, Li Chenglin, Hao Chen, Fei Yu, and Yin Zhang. Teaching small language models reasoning through counterfactual distillation. In Proceedings of the 2024 Conference Empirical Melhods in Natural Language Processing, pp. 5831-5842, 2024. 5, 18
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quan- tization for genorative pro-trainod transformers. In The Eleventh International Conference on Learning Representations. OpenReview, 2023.
Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Aurick Qiao, and Hao Zhang. Efficiently serving Ilm reasoning programs with cortaindex. arXiv preprint 1rXiv:2412.200 2000999, 2024. 5, 12, 13, 14
Jonas Geiping, Scan McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach. arXiv preprint or arXi::2502.05110 771 2025. 10
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letaan, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arxiv preprint arX.0:2407.21788 2024. 1
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Rumxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint 220442501.1144,, 2025. 2, 4, 5, 6, 8, 12
Tingxu Han, Chunrong Fang, Shiyu Zhao, Shiqing Ma, Zhenyu Chen, and Zhenting Wang. Token- budget- aware Imm reasoning. arXiv preprint arxiv:24 18.55777, 2024. 2, 5, 9, 10, 15, 16
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint 2rX.:2241200666, 2024. 2, 5, 10, 11
Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. arXiv preprint XrXiv:2310.00 00666994, 2023. 19
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In Thirty-fifth Con- ference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021. 1
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 10
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXio preprint arXiv:2503.06740 2025. 20
23

---

Published in Transactions on Machine Learning Research (08/2025)
Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. The impact of reasoning step length on large language models. Xiv preprint arXin:2/01.04000 2024.
5, 19
Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3ot: Generating shorter Chain-of-thoughn without compromising effectiveness. arXiv preprint 1rxxx:2412.11644 2024. 5, 8, 9, 10
Levente Kocsis and Csaba Szepesvári. Bandit based monte-carlo planning. In European conference on machine learning, pp. 282-293. Springer, 2006. 4
Martin Kuo, Jianyi Zhang, Aolin Ding, Qinsi Wang, Louis DiValentin, Yujia Bao, Wei Wei, Da-Cheng Juan, Hai Li, and Yiran Chen. H-cot: Hijacking the chain-of-thoughi safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-rl, and gemini 2.0 flash thinking, arXiv preprint arXiv:2502. 12893, 2025. 20
Ayeong Lee, Ethan Che, and Tianyi Peng. How well do Ilms compress their own chain-of-thought? token complexity approach. arXiv preprint arXi::2503.0111 01141, 2025. 5, 15, 16
Poiji Li, Kai Lv, Yunfan Shao, Yichuan Ma, Linyang Li, Xiaoqing Zheng, Xipeng Qiu, and Qipeng Guo. Pastmets: A simple sampling strategy for data synthesis. arxio preprint or 111:2502.14476 2025a. 5, 12, 13
Xuying Li, Zhuo Li, Yuji Kosuga, and Victor Bian. Output length effect on deepseek-r1's safety in forced thinking, arXiv preprint ar Xrx:::2503.0 01923, 2025b. 20
Yuetai Li, Xiang Yue, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubra- manian, and Radha Poovendran. Small models struggle to learn from strong reasoners. arXie preprint arXiv:2502. 12143, 2025c. 5, 18
Zhong-Zii Zhi Li, Duzhen Zhang, Ming-Liame Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system 1 system 2: A survey of reasoning large language models. arXiv preprint arXiv:2502.191 17419, 2025d.
Baohao Liao, Yuhui Xu, Hanze Dong, Jumnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. Soward-guides speculative decoding for efficient Ilm reasoning. arXiv preprint (xXv:2551111322 2025a. 5, 12, 13
Huanxuan Liao, Shizhu He, Yupu Hao, Xiang Li, Yuanzhe Zhang, Jun Zhao, and Kang Liu. Skinterm: Internalizing symbolic knowledge for distilling better cot capabilities into small language models. In Proceedings of the 31st International Conference on Computational Linguistics, pp. 3203 3221, 2025b. 5,
18
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-aware weight quantization for on-device Ilm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87-100, 2024. 3
Fan Liu, Wenshuo Chao, Naiqiang Tan, and Hao Liu. Bag of tricks for inferenco-time computation of Ilm reasoning. arXio preprint arXiv:2502.00 07191, 2025a. 5
Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1b 1lm surpass 405b IIm? rethinking compute-optimal test-time scaling, 2025b. URL https::/ //axxiv.
org/abs/2502.061
Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang, Can language models learn to skip steps? arXiv preprint arXiv:2411.01885 2024a. 5, 9, 10
Yuliang Liu, Junjie Lu, Zhaoling Chen, Chaofeng Qu, Jason Klein Liu, Chonghan Liu, Zefan Cai, Yunhui Xia, Li Zhao, Jiang Bian, al. Adaptivestep: Automatically dividing reasoning step through model confidence. arXiv preprint Xiv:2500 13943, 2025c. 5
24

---

Published in Transactions on Machine Learning Research (08/2025)
Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. Kivi: A tuning-free asymmetric 2bit quantization for kv cache. arv preprint or (0.:2402.2250 2024b. 3
Tya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.0300 2016. 7
Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwci Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. Ol-prumer: Sangthhharmoniiing fine-tuning for ol-like reasoning pruning. arXiv preprinl arXiv:2501.155%) 2025. 2, 4, 5, 7,8
Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, and Jia Li. Srr: Teaching Ilms to self-verify and self-correct via reinforcement learning, 2025a. URL https: //arxiv. org/ abs/2522. 12853. 5, 17
Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-
compressible chain-of-thought tuning. orXiv preprint (exii:2502.0960.0 2025b. 2, 5, 9, 10
Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with reference-free reward. Advances in Neural Information Processing Systems, 37:124198 1242335, 2024. 7, 8
Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Fei-Fei, Hannanch Hajishirzi, Luke Zettle- moyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL http:p://axxvv.oo///a//
19393. 5, 17
Tergel Munkhbat, Namgyu Ho, Seohyun Kim, Yongjin Yang, Yujin Kim, and Se- elicits concise reasoning in large language models. arXiv preprint arXin:200..00 201222221222 22, 2025. 5, 9, 10
Se- Young Yun. Self-training
Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route 1mms with preference data. arXiv preprint arXiv:2406.1880 2024. 16
Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Walcod Kadous, and Ton Stoica. Routellm: Learning to route 1lms with preference data, 2025. URL https: wlarxiv.org/ab////20 18665. 5
OpenAT. Learning to reason with 1lms.
/twp////e/////eiiiioooorrroooeee 2024. Accessed: 15 March 2025. 1, 6
Shubham Parashar, Blake Olson, Sambhav Khurana, Eric Li, Hongyi Ling, James Caverloc, and Shuiwang Ji. Inference-time computations for Ilm reasoning and planning: A benchmark and insights, 2025. URL
https://arxiv. attps://arriiv.orrab////// w2tp.///..rr.aabbb/.....2 org/abs/2200..22211. 12521. 5
Jacob Pfau, William Merrill, and Samuel R Bowman. Let's think dot by dot: Hidden computation in transformer language models. arXiv preprint 2rX4::2444115558 2024. 10
Yuxiao Qu, Matthew YR Yang, Amrith Setlur, Lewis Tunstall, Edward Emanuel Beeching, Ruslan Salakhut- dinov, and Aviral Kumar. Optimizing test-time compute via meta reinforcement fine-tuning, ar preprint arXiv:2503.0700 2025. 5
Matthew Renze and Erhan Guven. The benefits a concise chain of thought on problem-soovingg in large lan- guage models. In 2024 2nd International Conference on Foundation and Large Language Models (FLLMM), pp. 476 483. IEEE, 2024. 5, 15, 16
Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J Reddi. Reasoning with latent thoughts: On the power of looped transformers. arXiv preprint arxi::2502. 17416, 2025. 5, 11
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov, Proximal policy optimiza- tion algorithms. ar Xiv preprint Xiv Xix:1707.0000 707.06347, 2017.
25

---

Published in Transactions on Machine Learning Research (08/2025)
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXim:2402.03333 2024. 7
Xuan Shon, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. Efficient reasoning with hidden thinking. Xie preprint urXiv:2501.1922 2025a. 5, 11
Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang. Kai Wang, and Shiguo Lian. Dast: Dificulty-adaptive slow-thinking for large reasoning models. aiv preprint arXiv:2503.044 04472, 2025b. 5, 7,
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan Hc. Codi: Compressing chain- of-thought into continuous space via self-distillation. arXiv preprint vrXim2252.21074, 2025c. 5, 11
Luohe Shi, Hongyi Zhang, Yao Yao, Zuchao Li, and Hai Zhao. Keep the cost down: A review on methods to optimize I1m's kv-cache consumption. arXiv preprint 0rXtv:2407.80003 2024.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprinl arXio:2408. 03314, 2024. 12
Gaurav Srivastava, Shuxiang Cao, and Xuan Wang. Towards reasoning ability of small language models. arXiv preprint Xrxi:2200 1 11699, 2025. 5, 18
DiJia Su, Hanlin Zhu, Yingchen Xu, Jiantao Jiao, Yuandong Tian, and Qinqing Zheng, Token assorted: Mixing latent and text tokens for improved language model reasoning. arXiv preprint orXiv:2502.08275, 2025. 5, 11
Yuan Sui, Yufei He, Tri Cao, Simeng Han, and Bryan Hooi. Measoeeaso Dynamic guidance for optimized inferenco-time reasoning in large language models. arXiv preprint arXin:2502. 19918, 2025. 19
Hanshi Sun, Momin Haider, Ruiqi Zhang, Huitao Yang, Jiahao Qiu, Ming Yin, Mengdi Wang, Peter Bartlett, and Andrea Zanette. Fast best-of-n decoding via speculative rejection. arXiv preprint arXiv:2410. 10. 10.200900 2024. 5, 12, 13, 14
Lin Sun, Guangxiang Zhao, Xiaoqi Jian, Yuhan Wu, Weihong Lin, Yongfu Zhu, Linglin Zhang, Jinzhu Wu, Junfeng Ran, Sai-er Hu, et al. (onrr--22b-prvieewe Boosting accuracy with branch-merge distillation. arXiv preprint ar arXiv:2503.044 04872, 2025.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with 1lms. arXiv preprint XrXix:2501. 12599, 2025. 4, 5.
Qwen Team. Qwq-32b-previeww ptapittss//qqeeln.iittttttt ho/blog/////oo/o /blpg///q-3222/ hwrq-32bprevview/... Accessed: 15 March 2025.
Manan Tomar, Lior Shani, Yonathan Efroni, and Mohammad Ghavanzadeh. Mirror descent policy opti- mization. arXiv preprint arXiv:2005.0953 2020. 7
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. orXiv preprint ar Xiv:22302. 1100022119771 13971, 2023.
Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint 2rxiv:2211.14 14275, 2022. 12
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017. 11
26

---

Published in Transactions on Machine Learning Research (08/2025)
Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Dynamic self-consistency; Leveraging reasoning paths for efficient Ilm sampling. arXiv preprint ar arXin:2408. 17017, 2024. 5
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. Tn The Eleventh International Conference on Learning Representations, 2023. 3
Yiming Wang, Pei Zhang, Siyuan Huang, Baosong Yang, Zhuosheng Zhang, Fei Huang, and Rui Wang. Sampling-efficiens test-time scaling: Self-estimating the best-of-n sampling in early decoding. arXiv preprint ar crXtt:2503.01423 2025. 5, 12, 13, 14
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. thouugtt prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24535 2022. 2,
Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, et al. Light-rl: Curriculum sft, dpo and rl for long cot from scratch and beyond. arXiv preprinl Xrx::22503. 10460, 2025.
Yuyang Wu, Yifci Wang, Tianqi Du, Stefanic Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thought length in llms. arXie preprint or Xiv:2500.. 07266. 2025. 5, 13, 14
Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of- thought compression in llss. arXiv preprinl ar Xiv:2502. 1 12067, 2025. 5, 8, 9, 10
Kun Xiang, Zhili Liu, Zihao Jiang, Yunshuang Nie, Kaixin Cai, Yiyang Yin, Runhui Huang, Haoxiang Fan, Hanhui Li, Weiran Huang, et al. Can atomic step decomposition enhance the self-structured reasoning of multimodal large models? aiviv preprint arXiv:2503. 06252, 2025. 5
Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference Machine Learning, pp. 189887-30098 PMLR, 2023. 3
Shuo Xing, Chengyuan Qian, Yuping Wang, Hongyuan Hua, Kexin Tian, Yang Zhou, and Zhengzhong Tu. Openemma: Open-source multimodal model for end-to-end autonomous driving. In Proceedings of the Winter Conference Applications of Computer Vision, pp.
2025. 19
Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.0668 2025a. 1
Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft:
Thinking faster by writing less.
arXiv preprint Xix::2200.. 18600, 2025b. 5, 15
Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. Softcot: Soft thain-of-thought for efficient reasoning with lims. arXiv preprint arXiv:2502.1213. 2025c. 5, 11
Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yucting Zhuang. In- ftythink: Breaking the length limits of long- congecontext reasoning in large language models. arXie preprint arXiv:2503.0666 2025. 5, 13, 14
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809 11822, 2023. 3
Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijic Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025. URL
5, 17
Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long of reasoning in 1ms.. Xie preprint or 0333:2250..33333, 2025. 2, 5, 7, 19
27

---

Published in Transactions on Machine Learning Research (08/2025)
Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. Distilling system into system 1. arXiv preprint arXiv:247706000 2024. 5, 8, 9, 10
Zishun Yu, Tengyu Xu, Di Jin, Karthik Abinav Sankararaman, Yun He, Wenxuan Zhou, Zhouhao Zeng, Fryk Helenowski, Chen Zhu, Sinong Wang, et al. Think smarter not harder: Adaptive reasoning with inference aware optimization. arXiv preprint arXiv:2501.1799
2025. 5
Jiayi Yuan, Hongyi Liu, Shaochen Zhong, Yu-Neng Chuang, Songchen Li, Guanchu Wang, Duy Le, Hongye Jin, Vipin Chaudhary, Zhaozhuo Xu, Zirui Liu, and Xia Hu. KV cache compression, but what must we give in return? comprehensive benchmark of long context capable approaches. In Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics, November 2024.
3
Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujic Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. Lightthinker: Thinking step-by-step compression. arXiv preprinl 155529002.1589, 2025.
5, 13, 14
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. Small language models need strong verifiers self-correct reasoning. arXin preprint arXiv:244,111440, 2024. 18
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36:34661 -447100, 2023. 3
Yichun Zhao, Shuheng Zhou, and Huijia Zhu. Probe then retrieve and reason: Distilling probing and reason- ing capabilities into smaller language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC- COLICCGIING 2024), pp. 13026-13032,
2024. 5, 18
Xunyu Zhu, Jian Li, Can Ma, and Weiping Wang. Improving mathematical reasoning capabilities of small language models via feedback deriveck-driven distillation, arXiv preprint 11.:22411.16000 2024. 5, 18
28


Reviews:
[{"review_id": "eSHPhfWzB8", "reviewer": "Reviewer_iELw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1. The paper is easy to read and follow. \n2. The paper provides a comprehensive taxonomy and summary of methods developed for efficient LLM reasoning. In particular, the tables for each category of methods list the critical components with clarity, including datasets, core idea, and base models. \n3. The paper offers great timeliness and breadth. It discusses recent developments across a broad range of aspects for efficient reasoning with LLMs. \n4. Many of the graphical illustrations of the key methods discussed are quite helpful for the readers to understand. \n\n**Weaknesses**\n\n1. This survey has provided a detailed summarization and categorization of the existing literature, but their comparison, limitations, and potential future research directions are not discussed. Therefore, although the survey paper serves as a good starting point for other researchers to browse through a wide range of recent related works with an organized hierarchy, it lacks in-depth critical analysis and a potential roadmap for future research in this area. Therefore, it only partially meets the TMLR survey criteria that “draw new connections, highlight trends, and suggest new problems in an area”. \n2. The mathematical notations are not clearly described, especially in Tables 1 and 2. To name a few: $L_{\\mathrm{max}}, y_{\\mathrm{ref}},  \\pi_{\\theta}$. This can potentially confuse the readers, especially for those who are new to the area or have different prior understandings of the notations. In addition, Table 2 was neither referenced nor explained in the main text. \n3. “Efficient Reasoning” seems to be defined as “optimizing for the length of the generated responses while preserving the performance” (Abstract). However, this token-length efficiency is only one aspect of efficiency during inference. I suggest the authors carefully revise the scope of “efficient reasoning”. In particular, \n    1. Under the length-based definition for efficient reasoning, some surveyed works do not appear strictly relevant. For Section 4.2, the length-based definition for efficiency does not strictly apply. Techniques like speculative decoding or reward-guided reasoning do not necessarily reduce length. It’s even harder to parse the meaning of “length” for pruning the paths for Best-of-N and MCTS. For these works, the efficiency seems to be better defined with respect to the amount of compute/FLOPs or time used during inference. \n    2. Similarly, Section 5.2 describes routing the queries, and Section 6 discusses data selection, model distillation, and quantization, which are not necessarily related to length-based reasoning efficiency. \n4. In Section 7, “Inference-time Computation.” does not appear to be related to efficient reasoning.\n5. Many of the works are only on arXiv without going through peer review, so it is unclear whether the reliability and reproducibility of these works have been verified by the authors.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper surveys the recent advances in developing LLMs with more efficient reasoning behavior. The survey groups these methods into three categories according to their working mechanisms, and summarizes the contribution of these methods. In addition, the survey also discussed the practical benefits of these methods in real-world applications.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nxzXUPj0RV", "reviewer": "Reviewer_NPn7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. Nicely written survey timely with the recent bloom of reasoning models. The inefficiency of current reasoning models is a pressing issue in LLM research.\n2. The general coverage of the survey is rather extensive, including both model-based and reasoning output-based methods. There are also additional discussions on efficient training, model compression and distillation. Some comments on specific categorization and coverage issues are below in the weaknesses.\n3. There are easy-to-understand figures, and useful summaries in the form of Tables, which are essential for surveys.\n4. Overall, this survey should provide extensive yet concise summaries of this specific research direction to researchers.\n\nWeaknesses:\n\n1. The efficiency in reasoning is defined as methods that “optimize reasoning length”. However, it is unclear from the abstract and introduction that does it only encompasses methods that reduce the length of the reasoning model output/completion, or also extends to methods that make reasoning (especially during inference scaling) more efficient at test time (e.g., more efficient search) as later seen in Section 4.2. Thus, the survey paper does not have a comprehensive and clear definition of “efficient reasoning” in the abstract and Introduction.\n2. Relatedly, I suggest the authors also explicitly discuss other aspects of efficiency (e.g., memory efficiency like GPU usage, and architectural efficiency like MoE architectures) for reasoning models for a complete discussion. Otherwise, the authors can also state clearly that the paper only focuses on efficiency in terms of reasoning length.\n3. The whole survey paper focuses on methodologies to reduce reasoning lengths, but there is no discussion on the implications of applying such methods. For example, is there any study on the length of reasoning v.s. usefulness of the result? Are outputs still interpretable? Do the reasoning steps/traces still make sense? Furthermore, there are observations that LLMs sometimes can still produce the correct final answer even when some intermediate steps are wrong. There is also no discussion on whether the evaluation will still be useful in such cases.\n4. The survey could be more complete by including some datasets that are collected or released for training with an efficient reasoning objective, if any of them are available.\n5. It would be good to include a section on future directions and outlook in the conclusion.\n6. Section 5.1 is awkwardly categorized in the survey. Firstly, prompt guidance is directly related to performing length control on the outputs during inference, which is Section 4. Secondly, such methods are also used in Section 3 for creating training data for SFT. Lastly, it seems to be closely related to Section 6.1 too for creating efficient training data.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents a structured survey focused specially on efficient reasoning with large language models (LLMs). The issue of “overthinking” is commonly observed in reasoning models recently developed, and this survey summarizes the current approaches for efficient reasoning into meaning categories. Specifically, the key directions are (1) model-based, (2) output-based, and (3) input prompt-based efficient reasoning. Overall, this survey paper is a timely contribution to the community - the researchers can quickly grasp the current state of research for improving the efficiency of reasoning LLMs with the aid of this survey paper.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. (Weaknesses 1 and 2): A major change required will be clearly defining “efficient reasoning” and hence the scope of the survey.\n2. (Weakness 3): The survey could potentially be improved by including the implications of the efficient reasoning methods discussed. Also, this might also motivate a discussion on potential broader impacts (either societal or ethical).\n3. (Weaknesses 4 and 5): See suggestions above in the weaknesses for improving the completeness of the survey.\n\nMinor changes:\n1. There are nicely drawn figures in the paper, e.g., Figures 1, 2, 3, 5. However, they are not referred to anywhere in the text of the main paper. It should be incorporated into the text to aid understanding of the paper.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QJiksQKgrB", "reviewer": "Reviewer_4Fai", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n- The discussion on the various approaches to addressing the overthinking problem is quite comprehensive, with detailed figures and tables to enhance comparison.\n- The paper is decently easy to follow and generally suitable for readers to understand the landscape of addressing overthinking in reasoning models.\n- The authors appropriately discuss the evaluation and implication of application aspects of reasoning.\n\n**Weaknesses:**\n- The section on efficient data and models does not seem to be well aligned with the central theme of the paper as it concerns more about efficient training, rather than curbing the overthinking problem.\n- A comparison of the pros and cons between the different categories could be helpful. Currently, the connection between different sections seems weak.\n- Figure 1 is a bit misleading. While the survey discusses the different training techniques to improve the reasoning of LLM, Figure 1 shows the positioning of this survey as somewhere after the training stage.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors survey the current approaches concerning the problem of overthinking in large reasoning models: reasoning often introduces significant computational overhead due to lengthy and redundant outputs. The authors group recent efforts curbing this problem in four different categories, depending on whether the approach targets the input prompt, the model, or the reasoning output. \n\nAdditionally, the authors discuss existing benchmarks that assess the reasoning capability of LLMs and the implications of reasoning models on various application domains. The authors also touch on areas related to reasoning for future research.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Discuss more specifically how current works on efficient training data and model compression help alleviate the problem of overthinking.\n2. Modify Figure 1 to make it consistent with the claims of the survey (see weakness).\n3. Include more discussion comparing and contrasting the different approaches of addressing the overthinking problem.", "broader_impact_concerns": "I do not find any concerning ethical implication of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)