TokenSqueeze: Performance-Preserving Compression for Reasoning LLMs
Authors: Yuxiang Zhang
Zhengxu Yu
Weihang Pan
Zhongming Jin
Qiang Fu
Deng Cai
Binbin Lin
Jieping Ye
Venue: NeurIPS 2025 poster ()
Keywords: Efficient Reasoning
Chain-of-Thought
Large Language Models
Primary area: deep_learning

Abstract: Emerging reasoning LLMs such as OpenAI-o1 and DeepSeek-R1 have achieved strong performance on complex reasoning tasks by generating long chain-of-thought (CoT) traces. However, these long CoTs result in increased token usage, leading to higher inference latency and memory consumption. As a result, balancing accuracy and reasoning efficiency has become essential for deploying reasoning LLMs in practical applications. Existing long-to-short (Long2Short) methods aim to reduce inference length but often sacrifice accuracy, revealing a need for an approach that maintains performance while lowering token costs. To address this efficiency-accuracy tradeoff, we propose TokenSqueeze, a novel Long2Short method that condenses reasoning paths while preserving performance and relying exclusively on self-generated data. First, to prevent performance degradation caused by excessive compression of reasoning depth, we propose to select self-generated samples whose reasoning depth is adaptively matched to the complexity of the problem. To further optimize the linguistic expression without altering the underlying reasoning paths, we introduce a distribution-aligned linguistic refinement method that enhances the clarity and conciseness of the reasoning path while preserving its logical integrity. Comprehensive experimental results demonstrated the effectiveness of TokenSqueeze in reducing token usage while maintaining accuracy. Notably, DeepSeek‑R1‑Distill‑Qwen‑7B fine-tuned by using our proposed method achieved a 50\% average token reduction while preserving accuracy on the MATH500 benchmark. TokenSqueeze exclusively utilizes the model's self-generated data, enabling efficient and high-fidelity reasoning without relying on manually curated short-answer datasets across diverse applications. Our code is available at \url{https://github.com/zhangyx1122/TokenSqueeze}.

T Performance-Preesrviig Compression for Reasoning LLMs
Yuxiang Zhang'; Zhengxu Yu Weihang Pan', Zhongming Jin
Qiang Fut, Deng Cail, Binbin Lin Jieping Y
State Key Lab of CAD&CG, Zhejiang University
School of Software Technology, Zhejiang University
Alibaba Cloud "Zhiyuan Research Institute
Abstract
Emerging reasoning LLMs such as OpenAl-ol and DeepSeek-RI have achieved strong performance on complex reasoning tasks by generating long chain-of- thought (CoT) traces. However, these long CoTs result in increased token us- age, leading to higher inference latency and memory consumption. As result, balancing accuracy and reasoning efficiency has become essential for deploying reasoning LLMs in practical applications. Existing long-to-short (Long2Short) methods aim reduce inference length but often sacrifice accuracy, revealing a need for an approach that maintains performance while lowering token costs. To address this efficiency-accumee Long2Short method that condenses reasoning paths while preserving performance and relying exclusively self-generated data. First, to prevent performance degradation caused by excessive compression of reasoning depth, we propose to select self- liff-aenretedd samples whose reasoning depth is adaptively matched to the complexity of the problem. To further optimize the linguistic expression without altering the underlying reasoning paths, introduce distribution-aligana tic refinement method that enhances the clarity and conciseness of the reasoning path while preserving its logical integrity. Comprehensive experimental results demonstrated the effectiveness of TokenSqueeze in reducing token usage while -i-isssi---iiittte-eeeeeeee fine-tuned by using our proposed method achieved a 50% average token reduction while preserving accuracy on the MATH500 benchmark. TokenSqueeze exclusively utilizes the model's self-generated data, enabling efficient and high-fidelity reasoning without relying on manually curated short-answer answer datasets across diverse applications. Our code is available at https:: // //gi thub. com/ hhtp//gyx1122///oosssssuaueeeee
tradeoff, we propose TokenSqueeze, a novel
linguis-
maintaining accuracy. Notably,
Introduction
Large Language Models (LLMs) have revolutionized artificial intelligence with their exceptional performance complex reasoning tasks, such as mathematical problem-solving and algorithmic pro- gramming [3 40
38 39]. Recent advancements in this field primarily stem from enhancing models' ability to use extended chain-off ofain-of-thhuuht (CoT) reasoning in reinforcement learning based self-play training [3I) seen in OpenAl-ol [14] and DeepSeek-RI [II]. These models leverage multi-step reasoning to simulate human cognitive strategies, including hypothesis generation, iterative
18
15
TO
refinement, and self-correction 9
"Equal contribution.
i Corresponning author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Although long CoT enables deep reasoning abilities, it introduces issues like increased inference latency and memory usage, making models impractical for time-sensitive or resstree-constrained applications 220 g Meanwhile, this has also led to an "overthinking" phenomenon, where models produce redundant reasoning steps that do not contribute meaningful value in simpler tasks, thereby impeding their effectiveness in real-world applications [36 17
34 This is particularly evident in LLM-based agent systems, where the multi-turn trial-and-crror nature necessitates swift and concise reasoning for effective interactions and timely decision-making.
28 7
Some existing methods aim improve efficiency through inference- compression using coompt-based approaches or modified decoding strategies to shorten outputs. However, these techniques often face performance limitations, as the underlying model remains unchanged. In contrast, most current train-time strategies [3 29 promote shorter responses by incorporating penalty terms into either the reward or objective functions during training. Although effective in reducing output length, these approaches often compromise essential reasoning steps, leading to significant accuracy declines a phenomenon known as the reasoning oversimplification dilemma.
In contrast, we argue that the concise and efficient short answers we pursue are a matter of expression preference. Our experimental results demonstrate that when the reasoning length exceeds a certain token threshold, the correlation between token number and model performance significantly weakens. Hence, the Long2Short problem can be framed as a preference learning task that teaches the model to answer in a succinct tone while preserving its reasoning depth. Furthermore, we argue that maintaining adaptive reasoning depth, tailored each problem's 's complexity. is key preserving model performance.
In this study, investigate the Long2Short problem from two perspectives: the creation of effective long and short response pairs, and the formulation of preference learning objectives. To address the Long2Short problem, we introduce TokenSqueeze, novel training-time preference learning method that enhances reasoning efficiency without the need for external teacher models or additional annotations. Instead, our approach leverages self-generated reasoning data meticulously construct long and short preference pairs, ensuring scalability and resource efficiency. To generate differentiated preference pairs, we introduce data construction methodology that employs two key techniques: (1) adaptive reasoning depth selection, which modulates reasoning depth based on problem complexity to ensure essential steps are retained; and (2) intra-step refinement, which rewrites individual reasoning steps to enhance information density while preserving their meaning. Additionally, to optimize these compact and coherent reasoning traces, we incorporate ength-awwat signals into a meference-basee training objective, thereby promoting responses that are both concise and logically sound.
Our approach strikes balance between efficiency and accuracy, showing that high-quality reasoning can be achieved without relying on handcrafted short-answer datasets. The key contributions of this work are as follows:
We propose method to generate high-quality long-short reasoning pairs by combining adap- tive depth selection with intra-step linguistic refinement, without relying on external models or annotations.
We introduce length-aware preference objective that explicitly reinforces the model's preference for concise reasoning.
Extensive experiments show that our method significantly improves reasoning efficiency while maintaining model performance, demonstrating its effectiveness, and broad applicability across reasoning tasks.
2
Related Work
2.1
Online Reinforcement Learning with Length Penalty
Recent advances in reasoning efficiency have explored online reinforcement learning methods like Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO) [25] 26 These approaches modify the reward function RL training to reward concise and correct responses while penalizing verbose or incorrect ones, promoting reasoning compression. Notable implementations include Kimi-k1.5 (RL) (29), which integrates a length penalty into its framework, LI [2]. which leverages reinforcement learning to satisfy prompt-specified length constraints during
37.

---

training, and O1-Pruner [2I], which introduces length- henethhirmmnizing reward mechanism based on the length ratio between reference and predicted seuin-of-thougee sequences.
While effective, online RI. methods are computationally expensive due to the need for response sampling each training step. This increases overhead and limits their scalability for large model training. Moreover, large language models require many samples to generate efficient reasoning traces, further raising the computational burden.
2.2 Offiine Data-Driven Optimization Algorithms
In contrast to online methods, offline optimization approaches rely on existing response datasets, avoiding repeated sampling during training and offering substantial computational savings. They typically follow two-stage process: first constructing concise reasoning datasets, then training the model via supervised fine-tuning or direct preference optimization [24).
Several studies have focused on constructing training datasets by identifying the shortest correct responses from base model outputs. For instance, Kimi-k1.5 (DPO) (29 and Sky-T1-Flash [30] construct training sets by selecting minimal-length correct samples from response pools, while Self- Trainingg [22] employs few-shot prompting to guide models toward generating shorter answers before selection. Token- Budget [2] adopts prompt-search strategy to generate maximally concise reasoning samples, and DAST 27 implements length-aware reward shaping to select preferred training samples. These methods reduce token usage by selecting the shortest correct responses, but this often leads to overly aggressive compression that removes essential reasoning steps and harms accuracy. In addition, relying solely on unaltered model outputs may miss more concise yet high-quality traces due to sampling variability. TokenSqueeze addresses both issues by combining adaptive depth selection with trace refinement to improve training quality.
Alternative methods improve the quality of training data by rephrasing -odee-ggeerated responses, which are then used as supervision in the training process. These methods avoid the need for extensive sampling through two primary strategies: prompt- based rephrasing with external LLMs, as demonstrated by C3oT [66], which employs GPT-4 [T] for response refinement, and rule- content pruning exemplified by TokenSkip (33, which uses importance-based token elimination, along with Learn-to-Skip [9], which integrates step merging and skipping mechanisms. While efficient, these refinement methods can inadvertently remove important context or break coherence, hurting reasoning quality. TokenSqueeze addresses this trade-off by rewriting reasoning steps into denser forms under KL constraints, preserving information without external models.
3 Methodology
TokenSqueeze optimizes reasoning efficiency with a three-stage training method that balances logical accuracy and linguistic conciseness. First, adaptively filter self-generated reasoning traces to maintain the appropriate depth (Section Next, we refine individual reasoning steps to compress them while preserving meaning (Section 3.2). Finally, we optimize the model using composite objective that promotes both correctness and brevity (Section 3.3
3.1 Adaptive Reasoning Depth Selection
Our data construction pipeline starts by generating diverse reasoning traces through self-sampling from the base language model. Unlike existing methods that simply select the shortest correct responses 229 30 27 we introduce adaptive reasoning depth selection method that ensures the appropriate trace length. Maintaining an appropriate reasoning depth is crucial for achieving high accuracy, especially optimal range of chain lengths for high accuracy, this range shifts toward longer chains as problems become more difficult 322.
complex problems. Previous studies have shown that while there is an
We formalize the selection process using a dynamic quantile mechanism. Let a denote a tunable hyperparameter, and define y n the fraction of correct responses among N total responses, with e representing the number of correct samples. The adaptive quantile computed as g = a (1 p): enabling the preferred chain length vary naturally with problem difficulty.
3

---

1. Adaptive Reasoning Depth Selection
Intra-Step Refinement
top a1
Resample multiple
Select shortest variant
shortest
variants
satisfying KL constraint
Correct Responses
Chosen
Step
Resamples
Refined Step
Problem
Step
Resamples
Refined Step 2
longer than chosen
Wrong Responses
Intra-Step Refinement
Pairs
Proforances Pairs
Refined Preference Pairs
Composite Objective with Length Regularization
Refined
Curusi anp
loger
loog To(Uwlas) Toad (enllalle)
molal (s) (alllll)))
log
((tre)
[(sx)
Proferance
Varying
Chosen
Rejected
Lengthe
Σ tos
COLL 3(maw)
Creaal BCuvus + (1-1)) Cn)
Figure I: Overview of the TokenSqueeze. It first selects self-generated responses based on adaptive reasoning depth, then rewrites them under KL constraints to improve information density, and finally trains the model with composite objective that promotes accuracy and brevity.
For each problem, we first sort all correct reasoning traces by token length, forming an ordered set = (TI, 72,-. .................... Te) where Ti is the shortest and T. the longest. The selection threshold index is determined by = [q c), and the subset {π.....: -R is chosen as the set of preferred positive examples. Each selected trace 7; is further paired with longer incorrect responses to construct contrastive preference samples, with up to M preference pairs generated per problem to maintain data diversity, where we set M 64 in our experiments.
This adaptive selection has two key advantages: (1) For easier problems with higher correctness rates, it favors shorter reasoning chains, and (2) for harder problems with lower correctness rates, it retains longer chains to capture critical logical steps. Overall, the resulting dataset ensures balanced coverage across different difficulty levels while optimizing reasoning efficiency.
3.2 Intra-Step Linguistic Refinement via Distributional Alignment
Building the preference pair construction from Section 3.1 further improve the conciseness of training samples through a systematic intra-step linguistic refinement method. While existing methods typically use rule-based truncation LLM-based rewriting to compress reasoning traces, they often fail to guarantee logical integrity and can introduce information loss. To address these issues, propose intra-step linguistic refinement method that minimizes information loss while effectively reducing length.
Given generated reasoning trace A = (p, ....... 8x), consisting of prompt and sequence of reasoning steps [ we aim to compress each step while preserving informational content.
For each reasoning step 8;, we first resample set of K candidate rewrites {s{ Jk=I conditioned on preceding context Ac; = (p. 81,.... 8i-1) 15-1), where we set K 64. Among these candidates, we
select the one that minimizes token length while satisfying KI. divergence constraint to preserve downstream semantics. The optimization for each step can be formulated as:
min (
((s
subject to
DKL (Po(-l,, (ec.sp <E,
where E(x() represents the token length of the candidate, and & >> 0 is tunable threshold that controls the level information preservation. Here, Po denotes the probability distribution over possible future responses, conditioned on the model parameters 0.

---

Question
conference All-Star Gome, eight All-Stors will taking questions two Yonkees teammetes insist on sitting tegether, many ways can eight Sll-Stars seated row?
three
determine the Original COT
Rewrite COT
ways the eight All-Sttrs can be sitting fogether,
determine the number ways the eight All-Sttars can be each o single have three blocks: one each team (Cubs, Sox Yorkees). The rumber of ways to arronge these three blocks is 31 6.
sitting tegether, "block."
each teom as single
"block"
First, we colculate the number of weys to orrange the three blocks (Cubs, Red Sox, Yarkees). This is permutation three distinct blocks, which is 3
Next, within each block, the team members can be orranged themselves.
Next, within each block, the feom members can be amranee among thenselies. The Cubs can be arranged ways, the Red Sox
The Cubs members, they arranged in 31
and the Yonkees 21 ways Multiplying these.
ways.
gives the rumber 432.
have members, they arronged
orrongements:
ways.
The Yankees have menbers, so they can be arvanged 2 ways.
To find the total number of orrongements, we multiply the number of ways anrange the blocks by the rumber of ways to orronge the members within each block (for the blocks) (Cubs) (Red Sox) (Yonkses)-
Figure 2: Example of reasoning trace before and after intra-step linguistic refinement. The refined version achieves higher information density while preserving logical integrity.
To estimate the KL divergence across full distributions, we apply a local token window approximation, with the detailed derivation provided in the Appendix:
min(T,L)
full
DKL (Qo(: p. p, 1183,44--1 11 (p, 82n, (1)-1)).
Here, Diull corresponds the KI. divergence term defined in Equation 1 The index ij denotes the j-th token in the consecutive reasoning steps following the i-th step 8;- In particular, l corresponds to the first token of step 84+1 If the tokens within step 8 insufficient fill the token window, the sequence continues by accumulating tokens from subsequent reasoning steps until the window reaches its length limit the response ends. Qe represents the conditional probability distribution over the vocabulary at position j. given all preceding tokens. T denotes the total number of tokens in the following reasoning segment, and 512 the fixed window size used to maintain computational tractability.
More examples of refined reasoning traces provided in the appendix for reference. Our refinement method offers three key advantages over previous methods: (1) it ensures information preservation through explicit KI. divergence constraints, (2) it enables adaptive compression by adjusting the ratio based on local context importance, and (3) it is model-agnostic, avoiding reliance on external LLMs. Together, these features allow TokenSqueeze to achieve efficient and accurate reasoning trace compression. Illustrative examples of refined reasoning traces, demonstrating improved clarity and information density, are provided in the appendix.
3.3 Composite Optimization Objective
The final component of TokenSqueeze jointly optimizes the language model for reasoning fidelity and conciseness through composite preference-basee objective. Building upon standard Direct Preference Optimization (DPO), we introduce an adaptive length-aware margin encourage efficient reasoning traces without sacrificing logical correctness.
We define the underlying reward maximization
max Ex.yy (ro(x,)) - DKr. (TTo| - X log {(y)], 39
where E(y) denotes the token length of response y, À controls the strength of length regularization, and ( regulates the divergence penalty from the reference policy Nee-
5

---

Transforming this reward into preference loss yields the DPO-L (Direct Preference Optimization with Length-aware) objective:
logor or (log Tyer Yuo (2)
(T
log To yyt (x) (3) (3) (x)
((xi) ((ym.)
DPO-L -Fa (w
A Alog
Tret
where (a, yw 31) denotes the input along with the preferred (shorter, correct) and rejected (longer or incorrect) responses. The additional logarithmic term log({(w) (())))(())))) adaptively scales the margin based on the relative length difference, strengthening preference signals for pairs exhibiting greater compression gains while preserving standard behavior for comparable-lengto pairs.
To further stabilize training and prevent reward collapse on preferred responses during optimization, incorporate supervised fine-tuning positive examples, resulting in the final composite objective:
Toral = T/CDPOL + (1 - B))
log no (y I, 2,,
(5)
where we set 1) 0.5 to balance the two components.
This composite training formulation maintains the theoretical guarantees DPO while explicitly promoting compact, high- fidelity reasoning traces. Combined with adaptive sample selection and intra-step refinement, completes the TokenSqueeze method for efficient reasoning optimization.
4 Experiments
4.1
Experimental Setup
4.1.1
Training Contigurations
We train DeepSeek-RI -Iii---il--iiiiiie-ee----- and - the TokenSqueeze method described in 3.. 3.2 and 3.3 All models are optimized with a
[1] models using
learning rate of 5 10-6 and batch size of 128, using the Adam optimizer. Training is conducted using the Py Torch framework on computing nodes equipped with 8xNVIDIA Tesla A100 GPUs. Additional training details are provided in the appendix.
4.1.2 Datasets and Evaluation Metries
We evaluate model performance on four benchmark datasets: AIME24, MATHS00 [3), AIME25, and LiveCodeBench 23). As comparisons, include several recent strong methods such DAST 27
TrainEff [4) Notably, we also compared with a reproduced version of Kimi-kl.5 (DPO) following the experimental setup described in its official release, since the original implementation not released and the reported results are not based on Qwen-2.5 7B/1.5B model. In our result tables, "baseline" refers to the unaugmented original model prior to applying any enhancements or the TokenSqueeze method.
29
We report all results as the average of 16 independent runs to reduce variance and ensure robustness. sampling temperature of 0.6 is used consistently across AIME24, MATH500, and AIME25 to ensure fair comparison. For LiveCodeBench. 2025.01.31, following the setup used by DeepSeck-RR [T]. This time window is chosen because contains fewer examples affected by data leakage, making it more reliable benchmark for real-world performance.
evaluate problems dated from 2024.08.01 01 to
To evaluate model performance, we use four key metrics: Answer Accuracy, the percentage of correctly solved problems; Average Length Correct Responses (Len-T), the average number of tokens in correct answers; Average Length of All Responses (Len-A), the average number of tokens across generated responses; and Area Under the Curve (AUC), which measures performance under a 32K token budget by computing the area under the accuracy-toke usage curve.

---

Table 1: Performance comparison of different training methods across two model sizes. TokenSqueeze consistently improves token efficiency while maintaining or improving accuracy, achieving higher AUC scores compared to baselines. Bolded metrics the table indicate our method.
DeepSeck-RI RieiasRR-DDDiiiii-- Diettll--Qee-7
DeepSeek-HH -I--i-t-D--------- --ietill--wee-1.58
Dataset
Method
Ace (%)
Len-T
Len-A
AUC (%) Ace (%)
Len-T
Leu-A
AUC (%)
Baseline
55.5
7543
13337
16906
22.5
Kimi-k1 (neproducd)
5249 (-30,4%)
9221 (-30.9%)
5034 (:31.79)
12159 (-28.
AIME24
DAST 27
53.3
6339 (-16.0%)
TrainFmi g
56.0
10766 (- (-19.2%)
31.7
9399 (-44.4%)
TokenSqueeze
57.5
5157 (-31.6%
9189 (-31.1 %))
48.5
33.3
5841 (-20.8%
10731 (-36.5%)
27.4
Baseline
92.8
3638
4190
83.6
83.9
3637
5412
76.1
Kimi-k1.5 (reproduced)
68.2
1698
(-53.3%)
2298 (-45.2%%
83.7
80.8
1870
(-48.69)
3029 (-44.0%)
76.2
MATH500
DAST
(-23.0%)
TrainEMi
92.3
3259 (-22.2%) 2%))
82.7
2818 (-47.95)
TokenSqueeze
92.4
1773 (-51.3%)
2045 (-51.2%%
87.5
83.2
2005 (-44.9%)
2750 (-49.2%)
78.1
Baseline
39.2
6646
14372
31.2
24.4
5818
16070
20.1
AIME25
Kimi-k1.5 (reproduced)
38.1
4337 (-34.7%)
10575 (-26.498)
33.1
23.3
4528 (-22.298)
13991 (-12.9%%
20.1
DAST'
39.1
4602 (-30.8%)
TukenSqueeze
4711 (-29.1%)
10550 (-26.6%)
(612.%19))%)
(-30.5%)
Baseline
31.3
3961
20690
27.5
13.4
2629
26513
12.3
LiveCode Bench
Kimi-k1.5 (eprroducdd)
3256
(-17.8%)
19242
(-20.69)
25318
12.6
DAST'
29.7
3494
(-1188%)
TukenSqueeze
35,0
3200 (-19.2%)
15635 (-24.4%)
31.6
16.7
2587 (-1.6%)
24842 (-6.3%)
15.4
Reproduced from the official open-source model, as the original paper did not report this result.
AIME24: Acc. vs. Output Length
MATH500: Acc. vs. Output Length
DeepSeek-R] Distiii-----Q
0.6
TökenSqueeze
0.8
0.5
Accuracy
Accuracy
0.3
0.4
Max AAccuracy; +43.1% at 1k tokens
0.7
Max &Acouracy. +15.5% 3k
DeepSeek-RR Disi------illl---e--00 TokenSqueeze
0.0
0.0
2500
5000
7500 10000 12500 15000
2500
10000 12500 15000 Output Length (tokens)
5000
7500
Output Length (tokens)
Figure 3: "Quenn outperforms the hase model across token budgets on AIME24 and MATH500.
with up to 15.5% higher accuracy on AIME24 (3K tokens) and 43.1%% % on MATH500 (IK tokens).
4.2 Evaluation on General Reasoning Benchmarks
Table summarizes the performance of TokenSqueeze compared to both base models and recent strong methods across four benchmark datasets and two model scales. Overall, TokenSqueeze consistently improves accuracy while significantly reducing token consumption, leading to better token efficiency and higher AUC scores.
On the mathematical reasoning datasets AIME24, MATH500, and AIME25, TokenSquecze com- presses token usage significantly without sacrificing accuracy. In MATH500, it reduces the average length of correct reasoning traces by nearly half, while maintaining accuracy close to the base model. Similarly, on AIME24 and AIME25, it not only compresses the output effectively but also improves matches the base model's 'ss accuracy, outperforming methods like Kimi-k1.5 (DPO) 22 and DAST (27 in both efficiency and effectiveness. Notably. TokenSqueeze demonstrates stronger compression performance on the 7B model compared the 1.5B model across most metries.
Beyond mathematical reasoning, TokenSqueeze also excels in programming tasks, demonstrating its ability to generalize. On LiveCodeBench, which includes roal-world coding challenges, Token-
7

---

Squeeze significantly improves accuracy and reduces output length, showing its robustness and adaptability across domains.
In terms of AUC, which measures accuracy under token budget constraints, TokenSqueeze consistently improves performance across all datasets, proving it to be an effective solution for enhancing reasoning quality while reducing computational costs. Figure 3 further illustrates the efficiency gains: under the same token budget, TokenSqueeze achieves up 15.5% higher accuracy on AIME24 at 3k tokens and 43.1% 1% higher accuracy on MATH500 at Ik tokens compared to the base model. These results highlight TokenSqueeze success in balancing reasoning quality and efficiency, addressing the long-standing trade-off between accuracy and computational cost in LLM tasks.
4.3 Ablation Study
4.3.1 Impact of Adaptive Reasoning Depth Selection
We first evaluate the contribution of TokenSqueeze's 's adaptive selection mechanism, which adjusts the threshold for selecting appropriate reasoning depths based on problem complexity. As shown in T compare four configurations for selecting training pairs from self self-generated responses: Shortest, Q-FIX, Q-DYN (w/ extra pos), and Q-DYN.
All configurations use the same preference-base based learning method, where correct response is selected as the positive sample and paired with longer incorrect responses. In the Shortest baseline, only the shortest correct trace is selected. Q-FIX selects correct responses at a fixed quantile of the length ranking, without adapting to problem difficulty. Q-DYN introduces an adaptive threshold, q = x (1 p). where p is the correctness rate and o = 0.2, allowing the reasoning depth to vary with task complexity. Q-DYN (w/ extra pos) extends Q-DYN by including correct traces longer than 1.5x the positive sample length- as in the Kimi-k1.5 (DPO) [29] setting- as additional negative samples.
Compared to Shortest, Q-FIX achieves higher accuracy with only moderate increase in re- sponse length, suggesting that avoiding exces- sive compression beneficial. Q-DYN (w/ extra pos) further reduces response length, but at the cost of accuracy. We believe this accuracy drop is due to the inclusion of correct traces nega- tive samples, which weakens the model's ability to distinguish correct from incorrect reasoning and encourages excessive compression during training. In contrast, Q-DYN achieves the high- est accuracy on both datasets while maintaining competitive length, supporting our hypothesis that adapting reasoning depth to task complexity is crucial for balancing efficiency and correct- ness [32]
Accuracy & AUC vs Hyperparameter a on Math500 0.95
Accuracy
0.875
Accuracy
0.850
0.85
0.825
0.2
0.4
0. c.8
1.0
Hyperparameter a
Figure 4: Effect of the a parameter in adaptive quantile selection. Moderate values (e.g., a 0.2) provide the best balance between accuracy and token efficiency, while extreme values degrade performance.
To further explore the impact of the parameter, we conduct a comprehensive analysis and visual- ize the results in Figure 4 We find that = 0.2 strikes the best balance between accuracy and token efficiency across both datasets. Extreme values, such = 0 or 1, degrade performance due to either excessive compression or overly long reasoning chains that introduce noise and irrelevant logic.
4.3.2 Impact of Intra-Step Linguistic Refinement
Building on the preference data from Section 3.1 we now evaluate the effectiveness of intra-step linguistic refinement in improving reasoning efficiency. We compare four rewriting methods: (1) TokenSqueeze without refinement, (2) prompt-based rewriting using GPT-4o-mini, (3) the TokenSkip method from prior work, and (4) our proposed refinement method. As shown in Table 3 our method achieves higher accuracy on both AIME24 and MATH500, performing similarly to the No Refinement baseline. Notably, both GPT-4o-mini and TokenSkip are less effective at reducing response length,
8

---

Table 2: Ablation results comparing different preference data configurations. Among all variants, Q-DYN, the method adopted by TokenSqueeze, achieves the best overall performance.
AIME24
MATHS00
Method
Acc (%)
Len
AUC (%)
Acc (%)
Len
AUC (%)
Shortest
53.3
5960
43.7
90.8
1926
85.5
Q-FIX
55.0
6126
44.8
92.2
2054
86.5
Q-DYN (w/ extra pos)
52.3
5666
43.3
90.8
1742
86.0
Q-DYN
57.3
6190
46.5
92.8
2180
86.7
Table 3: TokenSqueeze achieves the highest accuracy by reducing both the number and length of
reasoning steps, demonstrating its effectiveness under compression and outperforming other methods.
AIME24
MATH500
Method
Acc (%)
AUC (%)
Steps StepLen
Acc (%)
Len
AUC (%)
Steps StepLen
Baseline
55.5
7543
41.6
267.0
28.1
92.8
3638
83.6
100.2
34.8
No Refinement
57.3
6190
46.5
198.5
29.1
92.8
2180
86.7
55.0
35.1
4o-mini Rewrite
39.1
6596
31.3
79.0
3333
56.4
TokenSkip Rewrite
54.4
6378
43.8
84.3
2686
64.1
TokenSqueeze
57.5
5157
48.5
194.4
26.3
92.4
1773
87.5
57.5
30.6
likely due to diminished reasoning ability. This confirms that refinement method preserves essential information while shortening responses, offering a better balance between conciseness and correctness.
To better understand how compression achieved, we break down the total response length into two components: the number of reasoning steps and the average tokens per step. The method without refinement significantly reduces the number of steps but slightly increases the average length per step, suggesting that compression primarily occurs along the reasoning depth axis. In contrast, our full method achieves further reduction by shortening individual steps, highlighting the effectiveness of linguistic-level refinement.
4.3.3 Impact of Composite Optimization Objectives
We further evaluate the effectiveness of TokenSqueeze's multi-objective training strategy by compar- ing four optimization variants, all trained on the same dataset: (1) Direct Preference Optimization (DPO), (2) supervised fine-tuning (SFT) selected samples only, (3) training with combined DPO and SFT loss, and (4) our full method, combining SFT loss with length-regularizee DPO loss.
As shown in Table 4 analysis reveals that pure DPO training, while effective reducing reasoning length, significantly degrades reasoning accuracy. Without safeguards, the reward for preferred samples diminishes time, leading unstable, low-quality outputs. In contrast, SFT training on positive samples maintains reasoning accuracy but achieves only modest reductions in token usage, it lacks mechanisms to penalize verbosity.
Combining DPO loss and SFT loss partially addresses these issues, recovering much the accuracy loss while enabling more effective compression. In this setup, the SFT loss helps stabilize training, while the DPO loss encourages concise outputs. However, further analysis suggests that there is still room for improvement, particularly in terms of better leveraging the datasct's compression potential. Our complete approach, which integrates SFT loss with a length-regullaized DPO loss, delivers the strongest performance by balancing accuracy with controlled output length. By explicitly incorporating length signals into the optimization objective, TokenSqueeze effectively fine-tur mee-uune the trade-off between brevity and logical completeness.
These findings underscore the importance of Tokensqueeze's integrated training strategy. The SFT loss provides stability during training, the DPO loss optimizes the model using preference signals, and the length regularization component reinforces the model's 's tendency to generate more concise responses.

---

Table 4: Comparison of training objectives using the same data. Our method combining SFT and length-regularized DPO achieves the best balance of accuracy and compression.
AIME24
MATHS00
Method
Acc (%)
Len
AUC (%)
Acc (%)
Len
AUC (%)
DPO
48.3
4300
42.0
91.6
1974
86.1
SFT
56.0
5734
46.3
91.8
2271
85.5
DPO+SFT
57.0
5420
47.3
92.6
1865
87.4
TokenSqueeze
$7.5
5157
48.5
92.4
1773
87.5
5 Limitation
While TokenSqueeze achieves substantial efficiency gains without compromising reasoning accuracy, several limitations remain that we aim to address in future work.
Heuristic Hyperparameter Selection. Some hyperparameters- most notably the KL. threshold used during intra-step rewriting- determined heuristically based on limited preliminary experiments rather than systematic tuning. Although we conducted multiple experiments with varying hyperparameter configurations to thoroughly validate the effectiveness of our method, did not have sufficient time or computational resources to perform exhaustive hyperparameter optimization. We have not yet explored the sensitivity mutual interactions of these parameters in depth. In particular, e governs the trade-off between semantic fidelity and linguistic brevity: setting it too low can restrict compression excessively, while setting it too high may cause semantic drift. more principled study of this balance, combined with adaptive or data-driven tuning strategies, could improve robustness across tasks and model scales. In future work, we plan develop an adaptive mechanism that automatically adjusts e based on context difficulty and local token divergence.
Offline-only Setting. Our current framework operates entirely under oflline preference optimiza- tion paradigm. where preference pairs are pre-generated and the model is trained without real-time interaction with the environment. This design simplifies implementation and ensures training stability, but it limits adaptivity the model cannot continuously refine its reasoning strategy based on new feedback task distribution shifts. In the future, we plan extend TokenSqueeze into an online reinforcement learning setting, where preference generation, policy updates, and reward estimation are jointly optimized. Such an extension could enable continual self-imprrvement and stronger alignment between reasoning efficiency and task accuracy.
6 Conclusion
In this paper, we introduce TokenSqueeze, a method for compressing reasoning traces in large language models using only self-generated training data, without manual annotations teacher models. TokenSqueeze addresses the reasoning oversimplification dilemma by combining adaptive reasoning depth selection, intra-step linguistic refinement, and length-aware preference optimization to maintain task performance. Extensive experiments on reasoning benchmarks, including math and code tasks, show that TokenSqueeze significantly reduces token usage while preserving improving accuracy, offering a practical path toward efficient, high-fidelity reasoning for real-world deployment.
Acknowledgements
This work was supported in part by the Key R&D Program of Zhejiang Province (No. 2025C01212) and in part by the Yongjiang Talent Introduction Programme (No. 2022A-240-G).
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Tlge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 23xi:22333308774,, 2023.
[2] Pranjal Aggarwal and Sean Welleck. LI: Controlling how long reasoning model thinks with reinforcement learning, arXiv preprint arXiv:2503.0469) 2025.
[3] Janice Ahn, Rishu Verma, Renze Lou, Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. arXiv preprint arX1v:2402.00157 2024.
[4] Daman Arora and Andrea Zanette. Training language models to reason efficiently, arXiv preprint arXiv:2502.04466; 2025.
[5] Simon A Aytes, Jinheon Back, and Sung Ju Hwang. Sketch-of-thought Efficient Ilm reasoning with adaptive cognitive-inspired sketching. arXiv preprint arXiv:2503.05179, 2025.
[6] Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like Ilms. arXiv preprint 2rXiv:2412.21187 2024.
[7] Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, et al. The danger of overthinking: Examining the reasoning- dilemma in agentic tasks. arXiv preprint arXiv:2502.08235, 2025.
[8] Tianyu Ding, Tianyi Chen, Haidong Zhu, Jiachen Jiang, Yiqi Zhong, Jinxin Zhou, Guangzhi Wang. Zhihui Zhu, Ilya Zharkov, and Luming Liang. The efficiency spectrum of large language models: An algorithmic survey. arXiv preprint 1rXiv:2312.00677 2023.
[9] Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint 2xXi:2503300300 2025.
[10] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. In International Conference on Machine Learning, pages 10764-10799. PMLR, 2023.
[11] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 2025.
[12] Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. foken-budget- Ilm reasoning. arXiv preprint arXiv:2412.1855/1 2024.
[13] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Bric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurLPS, 2021.
[14] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Camnyy, et al. Openai ol system card. arXiv preprint arXiv:2412. 2024.
[15] Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint arXi::2406.005/5. 2024.
[16] Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3ot: Generating shorter chain-of- thought without compromising effectiveness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24312-24320, 2025.
[17] Ayeong Lee, Ethan Che, and Tianyi Peng. How well do lims compress their own chain-of- thought? token complexity approach. arXiv preprint arXiv:2503.0112 2025.
11

---

[18] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards. Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever. and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
[19]
Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. Can language models learn to skip steps? arXiv preprint vrXiv:241100185. 2024.
[20] Yue Liu, Jiaying Wu, Yufei He, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, Zhiqi Huang, and Bryan Hooi. Efficient inference for large reasoning models: A survey. arXiv preprint arXiv:2503. 23077, 2025.
[21] Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. 01-pruner: pruning. arXiv preprint arXiv:2501.12570, 2025.
fine-tuning for ol-like reasoning
[22] Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang, Yujin Kim, and Se- Young Yun. Self-training elicits concise reasoning in large language models. arXiv preprint
urXiv:2502.20122, 2025.
[23] Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida Wang Armando Solar- Koushik Sen Ion Stoica Naman Jain, King Han. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint, 2024.
[24] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728- 53741, 2023.
[25] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1007.06377, 2017.
[26] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprini arXiv:2402.03330 2024.
[27] Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang. Kai Wang, and Shiguo Lian. Dast: Difficulty-dappiin slow- slow-thinking for large reasoning models. arXiv preprint arXiv:2503.044:: 0444722 2025.
[28] Yang Sui, Yu-Neng Chuang. Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503. 16419, 2025.
[29] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao. Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with 1lms. arXiv preprint arXiv:2501.12599 2025.
[30] NovaSky Team. Think less, achieve more: Cut reasoning costs by 50 http://novasky- 2025. Accessed: 2025-01-23.
ai wiggihubiiooosss/rcuccevvettttti
[31] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
[32] Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding tfain-of-thought length in llss. arXiv preprint arXiv2250..07266, 2025.
[33] Heming Xia, Yonggi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable thain-of-thought compression in llms. arXiv preprint arXiv:2502. 12067, 2025.
[34] Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran, and prompting for large language models. In NeurIPS 2023 Workshop on Backdoors in Deep Learning The Good, the Bad, and the Ugly, 2024.
Bo Li. Badchain: Backdoor chain-of-
[35] Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint 18000 2025.
12

---

[36] Junjie Yang, Ke Lin, and Xing Yu. Think when you need: Self-adaptive chain-of Shrin-of-thought learning. arXiv preprint arXiv:2504.05234, 2025.
[37] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source IIm reinforcement learning system at scale. arXiv preprint arXiv:2503. 2025.
[38] Zhanwei Zhang, Kaiyuan Liu, Junjie Liu, Wenxiao Wang, Binbin Lin, Liang Xie, Chen Shen, and Deng Cai. Geocad: Local geometry-controllable cad generation. arXiv preprint arXiv:2506. 10337, 2025.
[39] Zhanwei Zhang, Shizhao Sun, Wenxiao Wang, Deng Cai, and Jiang Bian. FlexCAD: Unified and versatile controllable CAD generation with fine-tuned large language models. In The Thirteenth International Conference on Learning Representations, 2025.
[40] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al. Least-to-most prompting enables complex reasoning in large language models, arXiv preprint arXiv:2205. 10625, 2022.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims the abstract and introduction are clearly stated and well-supported by the methods and experimental results presented in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [NA]
The proposed method demonstrates broad applicability across multiple rea- soning tasks and model sizes. As a result, we do not include dedicated discussion of limitations.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limmtations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [NA]
Justification: The paper focuses on empirical methodology and practical effectiveness rather than formal theoretical analysis.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The paper details model configurations, training setups, data construction methods, evaluation protocols, and baselines, enabling faithful reproduction of the main results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided not.
If the contribution a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
Depending the contribution, reproducibility
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (c.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code and data generation pipeline will be released upon acceptance, and all experimental settings, datasets, and training configurations are deseribed in detail to support full reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage release of code and data, we understand that this might not be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ooooosol for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and basclines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized ver- sions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The paper clearly outlines the training framework, base models, batch size, and key optimization parameters, demonstrating careful experimental design and reproducibility consideration.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: For all experiments, results are averaged over 16 independent runs reduce variance and ensure robustness of the reported metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper specifies the type of compute used (8xNVIDIA Tesla A100 GPUs), batch size, learning rate, and training framework, which are sufficient to reproduce the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't t make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http://////eeeepssc////l//////ia
Answer: [Yes]
Justification: The research uses only self- geffreeee data without involving human subjects, personal data, external annotations, and aligns with principles of transparency, safety, and fairness.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
17

---

Answer: [NA]
Justification:
This paper presents a method for improving reasoning efficiency language models, which does raise any direct societal concerns or cause potential social harm.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of Technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not involve any models datasets that pose high risk for misuse; all data is self-generated by open-source models using publicly available datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All external codebases, datasets, and models used in our experiments (e.g., MATH500, AIME24/25, LiveCodeBench, DeepSeek-R11 are properly cited with associated versions and URLs, and their usage complies with publicly stated licenses terms of use.
Guidelines:
18

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged reach out the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The paper introduces TokenSqueeze and clearly states that the code will be relcased upon acceptance, with placcholder URL provided for reference.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodll as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve any crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
19

---

Justification:
This paper does not involve any human subjects, user studies, or crowdsoureed
data.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: Our method is fundamentally a training algorithm for large language models.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy what should should not be described.
hwtp::pssscccccoooocccncccceeee 2025 /LLM) for
20

---

A Hyperparameter Settings and Implementation Details
A.1 Data Construction and Compression Settings
We generate self-sampled reasoning traces from the base model using a sampling temperature of 0.9 to promote output diversity, During the intra-step linguistic refnement stage, each reasoning step is resampled temperature 1.0 produce 64 candidate rewrites. From these, the shortest variant is selected, provided it satisfies a KL divergence constraint of 0.005, which we determined through empirical tuning. This ensures that downstream semantics are preserved while improving overall conciseness.
Throughout data collection and training, adopt consistent prompting format that explicitly separates the model's 's reasoning trace from its final answer. The prompt template is shown below:
A conversation between User and Assistant. The user asks question, and the Assistant solves it. The assistant first thinks about the reasoning process in the mind and then provides the user with answer. The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tage, respectively, i.e., <think> reasoning process here </think> <answer> **Final </answer>
</ee (Utaer>>(quee I>quession). Asssistant|>>
To construct preference pairs, refined traces selected using adaptive depth selection strategy. The selection threshold dynamically determined by the formula q = a(1 p), where p is the correctness rate and is set to 0.2. This ensures that reasoning depth adapts to problem difficulty, balancing informativeness and brevity.
A.2 Training Configurations
We tinettun both DeepSeek-R1- -i--DRstill--wnn--- and 1.5B using full full-parameter training. Our implementation builds the DPO pipeline from the LLaMAPactory framework, with the following modifications:
Addition of combined training objective using both supervised fine-tuning (SFT) loss and DPO loss, weighted equally at 0.5 each;
Integration of a length-aware DPO objective (DPO-L) with the length penalty coefficient = 1, to explicitly promote concise outputs.
We set the learning rate to 5 10 use a batch size of 128, and configure the maximum context length to 9000 tokens to fully capture long prompt-response pairs. Training conducted on 8XNVIDIA A100 GPUs and completes within approximately one day.
A.3 Evaluation Protocols
We evaluate our models on AIME24, MATH500, AIME25, and LiveCodeBench. For AIME24, MATH500, and AIME25, we decoding temperature of 0.6 and average results over 16 indepen- dent runs. all cases, the maximum token generation limit is set to 32,768.
For LiveCodeBench, we follow the evaluation setup from DeepSeek-RI and restrict evaluation to problems released between August 1, 2024 and January 31, 2025 to minimize data leakage. We use decoding temperature of 0.2 and again average over 16 runs ensure robustness.
B Proof from Equation (1) to Equation (2)
Notation and Setup. We restate the two KL divergence expressions used in the main text:
(Ri(Po(: p.)) Pa(: p, s())
(I)
21

---

(in(F.L)
Dill
Σ
DKL(Qul pp, (15-1) Qo(. p, D,84i 8/, f1:--1))
(2)
These quantities measure how the model's continuation distribution changes after rewriting a reason- ing step 8,.
We the following notation:
pt input prompt.
syt the i-th reasoning step.
Sgy the reasoning trace to and including step i.
d the rewritten version of step 8t.
ty: the j-th token in the continuation sequence after or s/-
11:2t a continuation of length T tokens.
The probability distributions are:
(- [-): token-level next-token distribution,
Pa I-) sequence-level distribution over (1.7.
For brevity, define:
A(ter) == Po(tu:r p. 8gi),
B(trr) Po(tu:r p, (,,)c;,)))
Step 1: Sequence-Level KL Expansion. By definition,
(AIlB) = EA log A(ti:r) B(t
(6)
Expanding both and autoregressively using Q#:
A(ta:r) = II
Co(t, systtt B(trr) II
Qolty p. p.see t/15111).
j=1
j=1
Substituting yields:
T
Qolt, p. scnttg))))) Qolty p, Sei o (15--)
(A = EigTAA
Σ
log
E E [KL (Qel- p, p. p. Scc s (,t))=
(7)
Thus, the sequence-level KL is the expected of per-token conditional KL divergences.
Step 2: Monte Carlo Approximation. practice, the full expectation over all token prefixes (1:j-1 is intractable. We approximate it using single sampled trajectory firr ~ A:
D
DRL (Qol pp Dlll p, 8 (1g))))))
(8)
1=1
22

---

Step 3: Window Truncation. To further reduce computation, we truncate the summation to a fixed window size L (e.g.. 512 tokens);
min(T.r)
DKL
Σ
(Rt( (1..sh (15)))))0))))00 I,, 8ei, 8y ())-
This yields Equation (2) in the main text. The approximation incurs a residual term
RL = Σ Bas (olDxL(-I1)) 0,
jzL
which remains small in practice since the influence of rewriting 8; decays over long continuations.
C Refined Reasoning Trace Examples
To complement our main results, we provide several illustrative examples of refined reasoning traces in this appendix. These examples demonstrate how TokenSqueeze compresses reasoning steps while preserving core logic and improving clarity.
Question
There ars has $0%t prethiees 5:t for wh equation $35x 22tteee hos rational solutione What is the product those values $:$?
Original COT
Rewrite COT
let's tockie prob pesifive inteyees V 1/ for which the quadraticc couttio V Bxx22 12x O how ranional soluriane We need to find the preduct of those tae voluss of o 1) First. I remeeber that for a qusdratic equction Vt ax"x bx c 0 have national selutions discrtsinnn nust perfect
question dbout finding two
find equation V 5xx IDx c 0 has rational solutions Then, I have find the preduct of those has volues of v)
soditive integers
which
First. remember that for euadnetic equetion 1 22 bi e- 1) have not
disoniibust intger, allawing the
discrkiiomm
perfect square discrisieant
perfect
national bessuse noots to siepiity to So, applying this to our couttion V. 5x 11x UJ coloulate the discrimirant
rost perfact
not enal.
fractions.
St. applying this our eqastion 5w IIk c le calcukate the discrinnnon:
Detto todot todat 121 20c
Aeeit (11)22 Vedort Vodot on 20c X)
For the equation to have netional colitions, M 121 20c 1) must he perfact square
For the equation to have rot rational solutions, 121 20c 1) must be a perfect squere.
donote perfect oquane as M where AC positive intaper.
the Therefore,
positivu integer.
[21 20c k²2
ies- 20c kh2
1]
Resrranging this equation to sobe for Mr
Reamnnging equetion
30c 121112112211
20c 121 K72 c- Minoot221 k'2)120)
<a >>fred[21- k 222]20)
1]
V)
Siece V[ must be posituus integer, M K22 Ummeet divisble 20, and rosalt must be positive injeeer Therefore, V/121 be
Since must positive integer, 121 must diviiblee by result
Thercfore, matpple of 20, and M mun grester than The 1(K22 121 V)
((k.) take volues fron ug 10
Vi 1) con take veluee from up to 10 Let's fot the perfect squores for 3 to Mike 1010 10
Now, let's list ponnill viluer of M.k and check whan V[ 12121- -112 1) divisble
by 20
1:
<cc=
20.
I Iqued 1/
99
1) 112 not disstble 20.
2= Vpuod 4 43
ttkc 4: 121-11100
16:
100
1-- 108 is not divisble by 20
x= Spood 9 V>
bxk 121 2
1) 96 la not dinsible by 20.
lqued
85
diisible by 20.
x-6 A Squed
11k k
-64-5 1) 57
divestle 20. 40 1) M.C. 40 2
x- à kouod 45
1[k= 121
x- laued
10. kik
10: 121 100
21 21 not divibbic by 20
k- iqued
5a. the valid values = 1) sre and The product these two volues
lendldige")
Mtnines 2
1)
Now. we'l calcalate V2123 K22 fer each V/ and check rasutt
1boved[12)
120 V.
Vecd
intagee
-Miks Ve (111-41111)
4:7
20:
"Mat" integer
-.k+ 3 1/ 121
9-
1) 112
20 B6
"Nar* integer.
M 121
105
10305
30--5 5.25 V. "Not" inteper
-MK-5 V.
-5-
4.6 "Nast integer.
integer "Net"
121
49 72 7272 20
-1cke V- 121
64- 57
V.57 20 2.85 () "Nar integer
-Mk- 9 12 121
88 40.) V.40
20- 2.1 V.c 2 har on integer
1Kk M: Y 12)- 100 21 21 20:1.00 V. "Nar"" inneger
MLC
are positive integers
of these teo values
Atines 12
(30xeo[32)
Figure 5: Reasoning trace comparison for quadratic equation problem before and after refinement.
23

---

Question
find fou--lsthrr worde begit
How many of letters seniefy property?
Original COT
Rewrite COT
First, de ne the sumber of choices for the first and last lenters noe they must be the some. There are 26 letters in the English appisbet, so there are 26 passible choices
First, determine the rumber of cheices for the first and lost lntters since they must the vere, There one letters the English aphaabst so thene are de Next, choices. can any letter. To find the total sumber four letter words multiph the number of choices for each pasttion 25 (for the first and lest lettend) 26 (for the secend letter) 26 (for third letter).
shird of 26 lathers, so there one 26 chaioes for the second letter and cho oei
these
the middle hao letters,
passible cho choices since they
for the rrind letter.
To firs the tontl in numbr of foundetter comblstt one rultiply the rumber cheices cach position
first and lest lette-s) malt ppied by 26 (for
17,576
the total number of combinstions To deterinne the number of four-lether worde that beg ond and with the some
+ 26 26 17,576
To detertine the number of four-letter worde that beghh and and with the some.
lether, an I folow these shape:
letter a break down the probien step by step
*Choossin the First and Last Letter*
"Choosing
and Lost Lefterts
ore
English olphcbet.
Engl
first
some, "126 choicesth for
Since the first and lurt lethers be the some, we have 2*26 choiceart for
this letter.
this letter.
Chooosnng the Middle Twr Lettergs
2. C*hhoosing the Second and Third Letters
The second lenter can be any of the 26 letters
second thind letters can be any inttersin in phobet
be
letters.
*CClooultingg Tetal Continctionss*
Per the third letter, there 1226 choices
Mully the number choices for each position:
3. Calculatingg the Tatal Number Continctionss* Aultiply the number of choices for each position
TaaxttTatal combination)) + 26 Viimes 26 Xtines 26 17,1575
1)
tneett lettorl) 26 Mextt (socund letnerl) Viines Veext! 26"3
(third stimes treint
1boxed (876)
Calculating
Atinns 26 Mrires 17.576
"FFinl Atdeer Shoveo[17 (976)
Figure 6: Reasoning trace comparison for constrained letter-counting problem before and after refinement.
Question
The point $0$ is the conter the ords cieoueerrbed about $ Sitriongle ABCS with $ Flangl SOC: 120" and Shongle 4GB 1407Dorcci es shown What is the desrce mossure 5 S'angleABB ABCS? pair ABC.C drow(Crakett..o) Jodiinnidd 2/g koers (0.00 Aef 16, 123 caps 12y B-3319.7) droats B cycle (nswidthii) 71t (nee1"1140 (harej5 jo.0).w. (510)" (teeo) (0,0 drow(C-00.0)-- droxtA--00 by AAS" A.5Ws, ins JANE) ibbll 2.9E)
Original COT
Rewrite COT
the point crgle BDC 180 deyrees, and ongle AOS is 140 degrees I need to find the messure of ongle ABC triongle ABC.
Okey, croumecribed eround tmongls AlC They
this
this
O
circl
cincsmenberd cround trongle ABC They twe angles paiet O argle POC is 120 degrees, and orgle 4GB is 540 degress I need find the mezsure of ongle ABC.
Himm, me recall my gesmetry coneapts. Since Oi the direumcenter, the angle I remember that the angle at the center (ikk ongle ADB) twice the mexsure of the inseribed angle
Hrm, resal gesmetry concapte Since O the orcuncenter: ono
ongles
ongle
sides of the center (like ongle ADB) is haice the nessure of the insoribed ongin (lke ongle
trlongle through the
through the centrol ongles remember the
(Mlic angle ACB) aubteededd by the sume orc.
ACB) aucitended the some ane.
So, ongle a0ls
degrese which should be twiee onyle ACk That moons arghs dag
Sa, angle is U40 deynees,
should twise angle ACB meard angic degrese
the incoribed stgle subtended the same ard BC would be half of that whtch is 50 dogrees. St ongle BAC is 60 degness. Now, find engle ABC, which paint
Sivilarly BDC
Simpleriy Stpulerly BCC is 120 dugress, ongle BAC should be holf of that which
60 degrees
Now, the sun of the ongles in any trizngla is 180 dagrnes So if have ong' BAC degraes, the ongle ABC must be mimu
can use
that the
60 degreas and angle AcB
sun
in tringgl 180 degrees aiready ongles AGB (70 cogrees)
minus whi
degress.
BAC (60 degroes). Wait me make sure dich mor the angles. The control angla cornaaendd to oe the bed angle so thik that part cornect Angle ADS over are AB, to the incriied engle over are All is ongle ACi which 70 degrnes
70 which
70, Angle BOC L20, zo angle BAC is 60. Then yes, angle ABC must 50. thin that's it
ler deuble check Argle ADB 140 angle
The messaae of angle ABC determined by calculating the inscr angles cor rereppoddingg to the centrol ongles and then sing the triangl argln sun. cormeganns inserited ongln Usingg the tricrgle suni <hBC + 150 70 60 30"
Similarly, angle BOC over BC, as inseribed angle over ore angle which 50
Angle
Thes, Yeuh, that The momore of ongle ABC is coleureted by finding the inscrieed ongins cornesponding to the given centrol ongles and then, using the triangle ongle sum formula
nevoring angle.
BDC
180 70 60- 60
right.
eeplnol Aneaerth
(boxeed(50)
Angle Boc (1800) cornesponds 6C-
onyle BAC, .BAC. 1201
2. Angle 4DB (140") cornesy rnds inscrited ongle AGB, so cACB 140 2 70:
Using triangle angle sun formuis CABC 180 <BAC- LAA3 50:
*Fimal *n 1bones(30)
Figure 7: Reasoning trace comparison for a circle geometry problem before and after refinement.
24


Reviews:
[{"review_id": "AgO3Az9Gsg", "reviewer": "Reviewer_JKMo", "summary": "This paper proposes *TokenSqueeze*, a method to compress the reasoning traces by choosing rewrites that would minimize the length of while subject to a KL-divergence constraint of the token distributions of the next reasoning steps. It then uses the chosen shorter reasoning trace to train the model using a modified version of the DPO algorithm that also incorporates the length preference. Experiments are conducted on AIME, MATH, LiveCodeBench datasets and the proposed methods are compared with very recent baselines such as DAST and TrainEffi. Results show that the proposed TokenSqueeze method outperforms previous methods by reducing more tokens while mostly preserving the reasoning accuracy.", "questions": "Q1: Why DAST and TrainEffi are not compared for AIME25 and LiveCodeBench? Is it because the originally reported numbers do not contain these two datasets? Is it possible to reproduce their results on these new datasets?  \nQ2: Right now we have a two step process, in each of which a KL-divergence is used. First it's used to get the \"good\" sequences for training, then it appear again in the DPO objective. Have you thought about potentially combining them in a single objective that can be jointly optimized?   \nQ3: When you measure the intra-step probability distribution change, there is an approximation using the accumulative token probability distribution. For these tokens, are you using the generated next steps from the original (probably lengthy) solution? If that is the case, this approximation seems to be a bit limited, as what you want is probably a *sequence-level* KL? i.e., $\\forall y,  D_{KL}(Q_\\theta(y|s_i) || Q_\\theta(y|s_i')) < \\epsilon$? Of course this would be intractable to approximate, but I'm curious if you have been thinking about doing this on the sequence level as well.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:  \nS1: The proposed method, i.e., using the probability distribution difference of **next reasoning steps** to measure information loss, is quite interesting and intuitive.   \nS2: The paper is very well-written and easy to follow in general. Specifically, I found the math formulation discussed in 3.2 quite clear and sound at the same time. All the intuitions are well-explained and the presentation of the experiment settings and results is quite clear in delivering the main messages. The discussion of related work is also quite comprehensive, covering both online and offline methods for training the models to generate more concise reasoning traces.  \nS3: The experiments are also well designed to showcase the effectiveness of the TokenSqueeze method. I especially like using AUC as an evaluation metric, which is perfect to measure the performance with a clear trade-off (i.e., tokens vs. accuracy). I also found the ablation studies presented in section 4.3 quite insightful.  \n\n**Weaknesses**:  \nW1: The evaluations are only conducted on two small models within the same model family (i.e., `DeepSeek-R1-Distill-Qwen-1.5B/7B `), which makes it a bit unclear how much the findings would transfer to reasoning models in general. While I do understand that the training of large reasoning models (e.g., R1) is prohibitively expensive, but I think training on other model series (especially given the new smaller reasoning models came out in the past few months) would significantly help strengthening the claims;   \nW2: Having a separate section to discussion the limitations and future work would be great;  \nW3: There are a couple of minor points that I hope the authors can clarify, which I put under the \"questions\" section.", "was_revised": true, "final_justification": "There was a bit of additional confusion during the rebuttal phase but the authors clarified it nicely.\n\nTherefore I maintain my original assessment of this work and think it should be accepted.", "rebuttal": "We sincerely thank you for your valuable feedback and positive evaluation of our work. We have carefully addressed each of your suggestions and questions in the responses below.\n\n**Response to Weakness 1:** To further demonstrate the effectiveness of our model, we conducted experiments using Phi4-Reasoning-14B. The results are presented below:\n\n| Model               | **Dataset** | **Accuracy (%)** | **AUC**  | **Avg. Length** |\n| ------------------- | ----------- | ---------------- | -------- | --------------- |\n| Phi4-Reasoning-14B  | AIME24      | 72.1             | 55.1     | 7768            |\n| TokenSqueeze (Phi4) | AIME24      | **73.8**         | **59.6** | **6326**        |\n| Phi4-Reasoning-14B  | MATH500     | 93.8             | 72.0     | 7644            |\n| TokenSqueeze (Phi4) | MATH500     | **96.0**         | **87.5** | **2921**        |\n\nIt is important to note that, due to time constraints, we reducing K in intra-step linguistic refinement from 64 to 16. We expect that further hyperparameter tuning would yield even better performance.\n\n\n\n**Response to Weakness 2:** We will include a dedicated section on limitations and future work in the camera-ready version, if the paper is accepted. Some of the limitations and future works we plan to include are listed below:\n\n\t1. Heuristic Hyperparameter Selection: Certain hyperparameters (e.g., the KL threshold $\\varepsilon$ used in intra-step rewriting) were selected heuristically based on preliminary experiments. We did not conduct a systematic analysis. In future work, we plan to investigate how varying $\\varepsilon$ affects the trade-off between semantic fidelity and brevity.\n\t2. Offline-only Setting: The current pipeline is restricted to offline preference optimisation. We plan to extend it to online reinforcement learning to enable tighter integration between data generation and model updates, potentially improving the trade-off between reasoning efficiency and accuracy.\n\n\n\n**Response to Question 1:** At the time of submission, DAST and TrainEffi had not provided results on the AIME25 and LiveCodeBench datasets in their papers, and neither released training code or model checkpoints. However, in June, DAST made its model publicly available. We evaluated their model on AIME25 and LiveCodeBench, and report the results below:\n\n| Model        | Dataset       | Accuracy (%) | AUC      | Avg. Length |\n| ------------ | ------------- | ------------ | -------- | ----------- |\n| DAST         | AIME25        | 39.1         | 33.6     | **4602**    |\n| TokenSqueeze | AIME25        | **39.8**     | **34.1** | 4711        |\n| DAST         | LiveCodeBench | 29.7         | 26.6     | 3494        |\n| TokenSqueeze | LiveCodeBench | **35.0**     | **31.6** | **3200**    |\n\nOur model outperforms DAST in both accuracy and AUC on AIME25 and LiveCodeBench, with especially notable gains on LiveCodeBench, demonstrating stronger generalization.\n\n\n\n**Response to Question 2:** The two KL terms serve different purposes. In the intra-step rewriting, the KL divergence aims to ensure that the semantic information of the data is preserved before and after rewriting. In contrast, the KL divergence in the training objective is used to constrain the model from deviating too far from the original model (to prevent training collapse). Given the multi-stage training process (data collection → data rewriting → model training), it is currently challenging to combine these two KL terms into a single joint objective. However, for online reinforcement learning methods such as PPO and GRPO, data collection and model training occur simultaneously during the RL process, and we believe there is potential to use a unified optimization objective. We will continue to explore this alignment in future work.\n\n\n\n**Response to Question 3:** In Equation $2$:\n$$\nD_{\\mathrm{KL}}^{\\text{full}} \n\\approx \n\\sum_{j=1}^{\\min(T,L)} \nD_{\\mathrm{KL}}\\left( \nQ_{\\theta}(\\cdot \\mid p, s_{\\leq i}, t_{1:j-1}) \\,\\|\\, Q_{\\theta}(\\cdot \\mid p, s_{<i}, s_i', t_{1:j-1}) \n\\right),\n$$\n\nWe clarify that $t_j$ refers to the $j$-th token in the consecutive reasoning steps following the $i$-th step $s_i$. Specifically, $t_1$ corresponds to the first token of $s_{i+1}$. **If the tokens in $s_{i+1}$ are insufficient to fill the window, we continue accumulating tokens from subsequent steps until the window is filled or the response ends.**\n\nIn fact, KL divergence at the token level can be used as an **unbiased approximation for sequence-level KL divergence**. In Equation 1, we use sequence-level KL divergence, while in Equation 2, we approximate it using token-level KL divergence. Below is a proof that shows how we transition from sequence-level to token-level KL divergence.\n\n**Proof (from Equation (1) to Equation (2))**\n\n---\n\n**Notation and Setup**\n\nWe begin by stating the two key KL divergence expressions from the paper:\n\n- Equation (1):  \n  $$\n  D_{\\mathrm{KL}}\\left( P_{\\theta}(\\cdot \\mid p, s_{\\leq i}) \\| P_{\\theta}(\\cdot \\mid p, s_{<i}, s_i') \\right)\n  $$\n\n- Equation (2):  \n  $$\n  D_{\\mathrm{KL}}^{\\text{full}} \n  \\approx \n  \\sum_{j=1}^{\\min(T,L)} \n  D_{\\mathrm{KL}}\\left( \n  Q_{\\theta}(\\cdot \\mid p, s_{\\leq i}, t_{1:j-1}) \\| Q_{\\theta}(\\cdot \\mid p, s_{<i}, s_i', t_{1:j-1}) \n  \\right)\n  $$\n\nThese expressions measure the change in predicted continuation distributions before and after rewriting step $s_i$.\n\nWe now define the key variables and distributions involved in the KL computation:\n\n- $p$: the input prompt to the model.  \n- $s_i$: the $i$‑th reasoning step generated by the model.  \n- $s_{\\le i}$: the reasoning trace up to and including step $i$ (i.e., $s_1, s_2, \\dots, s_i$).  \n- $s'_i$: a rewritten version of step $s_i$ used in intra-step refinement.  \n- $t_j$: the $j$‑th token **after** $s_i$ (or $s'_i$), starting the continuation sequence.  \n- $t_{1:T}$: a full continuation sequence of $T$ tokens (including EOS).  \n\nWe define the following probability distributions:\n\n- $Q_\\theta(\\cdot \\mid \\cdot)$: the model’s **next-token distribution** under the given context (token-level), parameterized by $\\theta$.  \n- $P_\\theta(\\cdot \\mid \\cdot)$: the model’s **sequence-level distribution** over continuations $t_{1:T}$, given the input context.\n\nTo simplify notation, we define:\n\n- $\\mathsf{A}(t_{1:T}) := P_\\theta(t_{1:T} \\mid p, s_{\\le i})$\n- $\\mathsf{B}(t_{1:T}) := P_\\theta(t_{1:T} \\mid p, s_{< i}, s'_i)$\n\nThese represent the two sequence distributions compared in the KL divergence.\n\n---\n\n**Step 1: Sequence‑Level KL Definition**\n\nThe KL term in Equation (1) is:\n$$\nD\\_{\\mathrm{KL}}\\big( \\mathsf{A} \\| \\mathsf{B} \\big)=\\mathbb{E}\\_{t\\_{1:T} \\sim \\mathsf{A}}\n\\left[\n\\log \\frac{ \\mathsf{A}(t\\_{1:T}) }{ \\mathsf{B}(t\\_{1:T}) }\n\\right].\n$$\n\nExpanding both $\\mathsf{A}$ and $\\mathsf{B}$ autoregressively using $Q_\\theta$:\n$$\n\\begin{aligned}\n\\mathsf{A}(t\\_{1:T}) = \\prod\\_{j=1}^{T} Q\\_\\theta(t\\_j \\mid p, s\\_{\\le i}, t\\_{1:j-1}), \\\\\n\\mathsf{B}(t\\_{1:T}) = \\prod\\_{j=1}^{T} Q\\_\\theta(t\\_j \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1}).\n\\end{aligned}\n$$\n\nSubstituting into the KL definition:\n$$\n\\begin{aligned}\nD\\_{\\mathrm{KL}}(\\mathsf{A} \\| \\mathsf{B})\n&= \\mathbb{E}\\_{t\\_{1:T} \\sim \\mathsf{A}}\n\\left[\n\\sum\\_{j=1}^{T}\n\\log\n\\frac{\nQ\\_\\theta(t\\_j \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n}{\nQ\\_\\theta(t\\_j \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n}\n\\right] \\\\\n= \\sum\\_{j=1}^{T}\n\\mathbb{E}\\_{t\\_{1:j-1} \\sim \\mathsf{A}}\n\\left[\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right)\n\\right].\n\\end{aligned}\n$$\n\nThis shows that the **sequence-level KL** equals the expected sum of per-token conditional KL divergences.\n\n---\n\n**Step 2: Monte Carlo Approximation**\n\nComputing the full expectation over all possible token histories $t_{1:j-1}$ is intractable in practice. To address this, we use a Monte Carlo approximation: instead of averaging over all continuations, we compute the KL terms along an existing sampled trajectory $t_{1:T}$ drawn from $\\mathsf{A}$:\n$$\nD\\_{\\mathrm{KL}}^{\\text{full}}\n\\approx\n\\sum\\_{j=1}^{T}\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right).\n$$\n\n---\n\n**Step 3: Token-Window Truncation**\n\nTo reduce cost further, we truncate the sum at a fixed window size $L$ (e.g., 512 tokens):\n$$\nD\\_{\\mathrm{KL}}^{\\text{full}}\n\\approx\n\\sum\\_{j=1}^{\\min(T, L)}\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right).\n$$\n\nThis is exactly the formulation given in **Equation (2)** of the main paper.\n\nThe approximation introduces a residual term:\n$$\nR\\_L = \\sum\\_{j > L}\n\\mathbb{E}\\_{t\\_{1:j-1} \\sim \\mathsf{A}} \\left[\nD\\_{\\mathrm{KL}}(\\cdot \\| \\cdot)\n\\right] \\ge 0,\n$$\nwhich is small when the effect of rewriting $s_i$ fades over time.\n\n---\n\nIf the paper is accepted, we will include the full proof in the appendix of the camera-ready version."}, {"review_id": "oexNTPwc37", "reviewer": "Reviewer_Cefq", "summary": "Reasoning LLMs such as OpenAI-o1 and DeepSeek-R1 attain strong performance by producing CoT reasoning traces, but these verbose outputs incur high latency and memory costs. TokenSqueeze addresses this efficiency–accuracy trade-off via a self-supervised, three-stage Long2Short approach: (1) adaptive depth selection dynamically filters reasoning paths based on problem difficulty; (2) intra-step linguistic refinement rewrites each step under a KL-divergence constraint to maximize information density; and (3) a length-aware preference objective fine-tunes the model to balance correctness with shortness.", "questions": "As I mentioned in weakness:\n\n1. The generated reasoning lengths appear unstable. Were multiple random seeds used in the experiments to evaluate robustness?  \n2. It would strengthen the paper to compare results with state-of-the-art methods such as O1-Pruner, L1, and others.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n1. The paper is well-written and generally easy to follow.  \n2. The second idea—**intra-step refinement**—is particularly interesting.  \n3. The experimental results demonstrate strong performance, successfully reducing token length while maintaining accuracy.\n\n**Weaknesses:**\n\n1. In line 13, the authors mention \"we propose to selectively identify self-generated samples.\" While the general concept becomes clearer in subsequent sections, this point is initially vague and could benefit from clarification upfront.  \n2. In line 48, the paper states that when the reasoning length exceeds a certain token threshold, the correlation between token count and model performance significantly weakens. This observation would be more appropriately framed as part of the motivation in the Methods section rather than being introduced in the Experiments section.  \n3. The paper lacks an ablation study on the choice of \\( M \\), as referenced in line 139. This analysis is important for understanding the impact of the selection strategy.  \n4. The notation \\( s \\) is used inconsistently: in line 135 it refers to reasoning traces, while in line 151 it denotes a reasoning step. This dual usage creates confusion and should be clarified.  \n5. The symbol \\( \\alpha \\) appears twice with different meanings (e.g., line 132 and Equation 4). These should be clearly distinguished or redefined to avoid ambiguity.  \n6. In line 137, it is unclear how the \"longer incorrect response\" is obtained. More detail on this process would be helpful.  \n7. In line 157, the paper states that the level of information preservation is \"tunable.\" Is this tunability controlled manually, or is it learned during training? This should be specified.  \n8. In Section 3.2, the method for accurately segmenting a reasoning step is unclear, especially since reasoning steps may contain many tokens and ambiguous boundaries. How is the stopping criterion determined?  \n9. The generated reasoning lengths appear unstable. Were multiple random seeds used in the experiments to evaluate robustness?  \n10. It would strengthen the paper to compare results with state-of-the-art methods such as O1-Pruner, L1, and others.\n\n[1] O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.\n\n[2] L1: Controlling how long a reasoning model thinks with reinforcement learning.", "was_revised": true, "final_justification": "The authors addressed my concerns, and I would like to keep the positive rating.", "rebuttal": "We sincerely thank you for your careful reading and constructive feedback. We appreciate your recognition of the overall clarity of the paper, the novelty of the intra-step refinement component, and the effectiveness of our method in reducing token usage while preserving accuracy. Your comments have helped us identify several areas for clarification and improvement, and we address each of your points in detail below.\n\n**W1 W2 W3 W4:** \n\n- In line 13, the authors mention \"we propose to selectively identify self-generated samples.\" While the general concept becomes clearer in subsequent sections, this point is initially vague and could benefit from clarification upfront.\n\n- In line 48, the paper states that when the reasoning length exceeds a certain token threshold, the correlation between token count and model performance significantly weakens. This observation would be more appropriately framed as part of the motivation in the Methods section rather than being introduced in the Experiments section.\n\n- The notation $s$ is used inconsistently: in line 135 it refers to reasoning traces, while in line 151 it denotes a reasoning step. This dual usage creates confusion and should be clarified.\n\n- The symbol $\\alpha$appears twice with different meanings (e.g., line 132 and Equation 4). These should be clearly distinguished or redefined to avoid ambiguity.\n\n**Response:** Thank you for pointing out these important issues. If the paper is accepted, we will make the following revisions in the camera-ready version:\n\n1. In the abstract, we will revise the sentence \n   *“we propose to selectively identify self-generated samples that preserve the model’s competency while minimizing token usage”* \n   to \n   *“we propose to select self-generated samples whose reasoning depth is adaptively matched to the complexity of the problem”* \n   to improve clarity.\n\n2. In lines 132 and 135, we will revise the overloaded symbols used for quantile computation and reasoning trace collections to eliminate ambiguity and improve readability.\n\n3. Regarding the placement of the token-threshold observation (line 48), we acknowledge your suggestion and will revise it accordingly in the camera-ready version if the paper is accepted. Due to space limitations, we omit the details here.\n\n**W3:** The paper lacks an ablation study on the choice of $M$, as referenced in line 139. This analysis is important for understanding the impact of the selection strategy.\n\n**Response:** To further investigate the impact of the number of samples per instance $M$ during the data collection phase, we additionally trained DeepSeek-R1-Distill-Qwen-7B with $M = 32$. The results are presented below, where *Baseline* refers to the original unmodified DeepSeek-R1-Distill-Qwen-7B model:\n\n|          | AIME24 Acc | AIME24 AUC | AIME24 Len | MATH500 Acc | MATH500 AUC | MATH500 Len |\n| -------- | ---------- | ---------- | ---------- | ----------- | ----------- | ----------- |\n| Baseline | 55.5       | 41.6       | 7542       | 92.8        | 83.6        | 3638        |\n| $M = 32$ | 56.5       | 46.8       | 5596       | 92.4        | 87.1        | 1875        |\n| $M = 64$ | 57.5       | 48.5       | 5157       | 92.4        | 87.5        | 1773        |\n\nThese additional experiments allow us to assess how varying the sampling number $M$ influences the trade-off between reasoning performance and inference efficiency.\n\n**W6:** In line 137, it is unclear how the \"longer incorrect response\" is obtained. More detail on this process would be helpful.\n\n**Response:** The “long and incorrect” samples are defined relative to the chosen ones. Specifically, during data construction, for each chosen sample, we select rejected samples that satisfy both of the following conditions:\n\n1. The response is longer than the chosen sample in token length.  \n2. The response is classified as incorrect based on final answer comparison.\n\nThis design ensures that the preference objective simultaneously penalizes both verbosity and incorrectness, aligning the optimization with our intended trade-off between accuracy and efficiency.\n\n**W7:** In line 157, the paper states that the level of information preservation is \"tunable.\" Is this tunability controlled manually, or is it learned during training? This should be specified.\n\n**Response:** The tunability of information preservation is implemented at **training time during data preprocessing**.\n\nOur method allows adjusting the strictness of information preservation in the rewriting phase via a KL divergence threshold $\\varepsilon$. This hyperparameter governs how much semantic deviation is tolerated when rewriting each reasoning step. Specifically, given a set of candidate rewrites $\\{s_i^{(k)}\\}$, we select the shortest one that satisfies the following constraint:\n$$\n\\min_{s_i' \\in \\{s_i^{(k)}\\}} \\ell(s_i') \\quad \\text{subject to} \\quad D_{\\mathrm{KL}}\\left( P_{\\theta}(\\cdot \\mid p, s_{\\leq i}) \\,\\|\\, P_{\\theta}(\\cdot \\mid p, s_{<i}, s_i') \\right) < \\varepsilon\n$$\nHere, $\\ell(s_i')$ denotes the token length of the rewritten step, and $P_\\theta$ is the model's sequence-level distribution. This KL-based constraint ensures that the semantic integrity of the original reasoning trace is preserved during the offline data refinement stage.\n\nBelow, we provide a concrete example illustrating how different threshold values affect the resulting rewrites and their lengths:\n\n|                  | Original Text                                                | $\\varepsilon=0.005$                                          | $\\varepsilon=0.01$                                           | $\\varepsilon=0.02$                                           |\n| :--------------: | :----------------------------------------------------------- | :----------------------------------------------------------- | :----------------------------------------------------------- | :----------------------------------------------------------- |\n|     **Text**     | Now, since each test score is an integer between 0 and 100, inclusive, both $x$ and $y$ must be integers in that range. Also, since $y$ is higher than $x$, $x$ must be at least 0 and $y$ could be up to 100. | Now, since each test score is an integer between 0 and 100, inclusive, both $x$ and $y$ have to be integers in that range. Also, $y$ must be greater than $x$. | Also, since each test score is an integer between 0 and 100, inclusive, both $x$ and $y$ must be integers within that range. | Also, both $x$ and $y$ are integers between 0 and 100, inclusive. |\n| **Token Length** | 75                                                           | 52                                                           | 37                                                           | 25                                                           |\n\n**W8:** In Section 3.2, the method for accurately segmenting a reasoning step is unclear, especially since reasoning steps may contain many tokens and ambiguous boundaries. How is the stopping criterion determined?\n\n**Response:** We segment reasoning steps based on **double newline characters `\\n\\n`**, which is a common convention adopted by most mainstream open-source LLMs. This formatting is naturally used by model family such as DeepSeeks and Qwens when generating step-by-step reasoning outputs. As a result, our segmentation aligns well with model behavior and does not require additional annotation or post-processing.\n\n**W9:** The generated reasoning lengths appear unstable. Were multiple random seeds used in the experiments to evaluate robustness?\n\n**Response:** Thank you for raising this point. For smaller datasets such as AIME24 and AIME25, we adopted an evaluation strategy designed to ensure robustness against potential variability in response lengths. Specifically, we averaged the results over 16 independent inference runs to obtain stable performance estimates. All generations were conducted using the default random seed provided by the vLLM framework.\n\n**W10:** It would strengthen the paper to compare results with state-of-the-art methods such as O1-Pruner, L1, and others.\n\n**Response:** Since O1-Pruner and L1 are built upon different base models than ours, their results are not directly comparable to our setting. Therefore, in the paper, we compared our method with DAST and TrainEffi, which utilize the same base model.  \n\nConsidering that L1 focuses more on *controllable reasoning length*, while O1-Pruner emphasizes *reasoning efficiency*, we additionally reproduced O1-Pruner using its open-sourced codebase. Following the official configuration of O1-Pruner, we reproduced their method on a single node with 8 × A100 80GB GPUs, using DeepSeek-R1-Distill-Qwen-7B as the base model. We strictly followed the hyperparameter settings described in the original paper, without any modifications.  \n\nThe reproduction results are summarized below:\n\n|           | AIME24 Acc | AIME24 AUC | AIME24 Len | MATH500 Acc | MATH500 AUC | MATH500 Len |\n| --------- | ---------- | ---------- | ---------- | ----------- | ----------- | ----------- |\n| Baseline  | 55.5       | 41.6       | 7542       | 92.8        | 83.6        | 3638        |\n| O1-Pruner | 51.0       | 41.3       | 6281       | **93.0**    | 85.2        | 2764        |\n| Ours      | **57.5**   | **48.5**   | **5157**   | 92.4        | **87.5**    | **1773**    |\n\nAs shown in the results, our method achieves significantly higher accuracy on challenging tasks like AIME24, while also delivering superior AUC and substantially reduced reasoning lengths compared to O1-Pruner."}, {"review_id": "23mTutTGOe", "reviewer": "Reviewer_3A86", "summary": "TokenSqueeze proposes a training-time method to compress reasoning traces in LLMs via three innovations: (1) adaptive reasoning depth selection to retain essential steps based on problem difficulty, (2) KL-constrained intra-step linguistic refinement to boost conciseness without altering semantics, and (3) a length-aware preference objective (DPO-L) that jointly optimizes correctness and brevity. Evaluated on math (MATH500, AIME) and coding (LiveCodeBench) benchmarks, TokenSqueeze reduces token usage by 20–51% while preserving accuracy, outperforming methods like DAST and Kimi-k1.5.", "questions": "See the Weakness.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n\n- Intra-step refinement under KL constraints ensures semantic fidelity while shortening steps, a significant improvement over rule-based truncation.\n\n- 50% average token reduction on MATH500 with negligible accuracy drop (92.8% → 92.4%) for DeepSeek-R1-7B.\n\n- Validated across model sizes (1.5B/7B) and tasks (math/code), demonstrating broad applicability.\n\nWeakness\n\n- Intra-step refinement requires resampling K=64 candidates per step and KL calculations over token windows (Eq. 2), adding significant preprocessing costs. Training efficiency claims lack quantification (e.g., GPU hours vs. baseline).\n\n- Performance hinges on α (depth selection) and ε (KL threshold). No analysis explores how ε affects the brevity-fidelity trade-off.\n\n- The KL approximation (Eq. 2) lacks formal justification for preserving global reasoning integrity.", "was_revised": true, "final_justification": "The detailed response solves my concern.", "rebuttal": "We sincerely thank you for your careful review and positive assessment of our work. Your feedback is greatly valued, and we respond to your points in detail below.\n\n**Response to W1:** We conducted an experiment using 1,000 randomly sampled initial question prompts on a single node with 8×A100 GPUs. The runtime (in hours) for each major preprocessing stage is shown below:\n\n| Stage                         | Time (hours) |\n| - | - |\n| Data Sampling                 | 1.73         |\n| Intra-step Candidate Sampling | 3.91         |\n| KL Divergence Computation     | 3.23         |\n\nThis runtime is comparable to standard preprocessing pipelines in industrial practice, such as reward-based rejection sampling, and remains within acceptable bounds for large-scale training. Moreover, since our method constructs the dataset offline, it incurs this cost only once and can be reused across multiple training runs, unlike online reinforcement learning methods that require repeated sampling during training.\n\nTo further reduce computational cost, we have implemented several optimization strategies:\n\n- **KV-cache reuse:** \n  When processing step $s_{i+1}$, we reuse the key-value cache computed during the sampling of step $s_i$ in both resampling and KL divergence calculation, significantly reducing the cost of autoregressive decoding.\n\n- **Selective rewriting:** \n  We first filter generated samples to identify preference-worthy candidates (e.g., concise and correct answers) before rewriting. This ensures rewriting is applied only to a targeted subset.\n\n- **KL window approximation:** \n  The KL divergence is approximated within a fixed-length window of 512 tokens:\n\n  $$\n  D_{\\mathrm{KL}}^{\\text{full}} \n  \\approx \n  \\sum_{j=1}^{\\min(T, L)} D_{\\mathrm{KL}}\\left( Q_{\\theta}(\\cdot \\mid p, s_{\\leq i}, t_{1:j-1}) \\,\\|\\, Q_{\\theta}(\\cdot \\mid p, s_{<i}, s_i', t_{1:j-1}) \\right),\n  $$\n\n  This avoids full-sequence computation and offers significant resource savings.\n\n- **Choice of $K$:**  \n  We would like to clarify that the resampling parameter $K = 64$ was chosen as a strong setting to push performance to its limit for benchmark comparison. However, our empirical observations indicate that smaller values such as $K = 32$ still achieve highly competitive performance while significantly reducing computational cost. This suggests that such a large value is not strictly necessary in practice. We believe that a more comprehensive investigation of the impact of different $K$ values is an important direction for future work, and we plan to explore this systematically in subsequent versions of the paper or follow-up research. As shown in our response to Reviewer 1 (W1), the experiments on Phi-4-Reasoning-14B with $K = 16$ still demonstrate a significant improvement in the model’s efficiency.\n\n\n\n**Response to W2:** As shown in Figure 4 of the paper, we analyzed the performance impact of different $\\alpha$ values and found that extreme values (too large or too small) negatively affect both accuracy and token efficiency. We selected $\\alpha = 0.2$ as it achieves the best balance between brevity and correctness.  \n\nFor the KL threshold $\\varepsilon$, we selected its value based on empirical observations. To further investigate the effect of $\\varepsilon$, we additionally conducted a quantitative experiment with $\\varepsilon = 0.02$. The results are shown below:\n\n|                     | AIME24 Acc | AIME24 AUC | AIME24 Len | MATH500 Acc | MATH500 AUC | MATH500 Len |\n| -- | ---------- | ------ | ------ | ----------- | ----------- | ----------- |\n| Baseline            | 55.5       | 41.6       | 7542       | 92.8        | 83.6        | 3638        |\n| $\\varepsilon=0.02$  | 46.9       | 39.7       | 5019       | 88.0        | 83.5        | 1695        |\n| $\\varepsilon=0.005$ | 57.5       | 48.5       | 5157       | 92.4        | 87.5        | 1773        |\n\nThese results confirm that excessively high values of $\\varepsilon$ can negatively affect accuracy. We plan to explore the effect of varying $\\varepsilon$ more systematically in future work.\n\nIn addition to the quantitative results, we also provide a qualitative example that illustrates how different values of $\\varepsilon$ influence the form and length of the generated rewrites:\n\n|                  | Original Text     | $\\varepsilon=0.005$    | $\\varepsilon=0.01$    | $\\varepsilon=0.02$     |\n| ----- | ---------- | ----- | ---------- | ------ |\n| **Text**         | Now, since each test score is an integer between 0 and 100, inclusive, both $x$ and $y$ must be integers in that range. Also, since $y$ is higher than $x$, $x$ must be at least 0 and $y$ could be up to 100. | Now, since each test score is an integer between 0 and 100, inclusive, both $x$ and $y$ have to be integers in that range. Also, $y$ must be greater than $x$. | Also, since each test score is an integer between 0 and 100, inclusive, both $x$ and $y$ must be integers within that range. | Also, both $x$ and $y$ are integers between 0 and 100, inclusive. |\n| **Token Length** | 75   | 52    | 37     | 25    |\n\n**Response to W3:** Thank you for your question. Due to space limitations, we could not include the full derivation in the main paper. If the paper is accepted, we will provide the complete proof in the appendix of the camera-ready version.\n\nSince both data generation and step rewriting are performed on the same model, we define the original step $s_i$ and its refined version $s_i'$ as semantically equivalent if they satisfy the following condition:\n$$\nD_{\\mathrm{KL}}\\left( P_{\\theta}(\\cdot \\mid p, s_{\\leq i}) \\,\\|\\, P_{\\theta}(\\cdot \\mid p, s_{<i}, s_i') \\right) < \\varepsilon,\n$$\nas formulated in Equation (1). This constraint ensures that rewriting does not compromise the semantic integrity of the reasoning trace.\n\nBelow we provide the derivation from Equation (1) to Equation (2):\n\n**Proof (from Equation (1) to Equation (2))**\n\n---\n\n**Notation and Setup**\n\nWe begin by stating the two key KL divergence expressions from the paper:\n\n- Equation (1):  \n  $$\n  D_{\\mathrm{KL}}\\left( P_{\\theta}(\\cdot \\mid p, s_{\\leq i}) \\| P_{\\theta}(\\cdot \\mid p, s_{<i}, s_i') \\right)\n  $$\n\n- Equation (2):  \n  $$\n  D_{\\mathrm{KL}}^{\\text{full}} \n  \\approx \n  \\sum_{j=1}^{\\min(T,L)} \n  D_{\\mathrm{KL}}\\left( \n  Q_{\\theta}(\\cdot \\mid p, s_{\\leq i}, t_{1:j-1}) \\| Q_{\\theta}(\\cdot \\mid p, s_{<i}, s_i', t_{1:j-1}) \n  \\right)\n  $$\n\nThese expressions measure the change in predicted continuation distributions before and after rewriting step $s_i$.\n\nWe now define the key variables and distributions involved in the KL computation:\n\n- $p$: the input prompt to the model.  \n- $s_i$: the $i$‑th reasoning step generated by the model.  \n- $s_{\\le i}$: the reasoning trace up to and including step $i$ (i.e., $s_1, s_2, \\dots, s_i$).  \n- $s'_i$: a rewritten version of step $s_i$ used in intra-step refinement.  \n- $t_j$: the $j$‑th token **after** $s_i$ (or $s'_i$), starting the continuation sequence.  \n- $t_{1:T}$: a full continuation sequence of $T$ tokens (including EOS).  \n\nWe define the following probability distributions:\n\n- $Q_\\theta(\\cdot \\mid \\cdot)$: the model’s **next-token distribution** under the given context (token-level), parameterized by $\\theta$.  \n- $P_\\theta(\\cdot \\mid \\cdot)$: the model’s **sequence-level distribution** over continuations $t_{1:T}$, given the input context.\n\nTo simplify notation, we define:\n\n- $\\mathsf{A}(t_{1:T}) := P_\\theta(t_{1:T} \\mid p, s_{\\le i})$\n- $\\mathsf{B}(t_{1:T}) := P_\\theta(t_{1:T} \\mid p, s_{< i}, s'_i)$\n\nThese represent the two sequence distributions compared in the KL divergence.\n\n---\n\n**Step 1: Sequence‑Level KL Definition**\n\nThe KL term in Equation (1) is:\n$$\nD\\_{\\mathrm{KL}}\\big( \\mathsf{A} \\| \\mathsf{B} \\big)=\\mathbb{E}\\_{t\\_{1:T} \\sim \\mathsf{A}}\n\\left[\n\\log \\frac{ \\mathsf{A}(t\\_{1:T}) }{ \\mathsf{B}(t\\_{1:T}) }\n\\right].\n$$\n\nExpanding both $\\mathsf{A}$ and $\\mathsf{B}$ autoregressively using $Q_\\theta$:\n$$\n\\begin{aligned}\n\\mathsf{A}(t\\_{1:T}) = \\prod\\_{j=1}^{T} Q\\_\\theta(t\\_j \\mid p, s\\_{\\le i}, t\\_{1:j-1}), \\\\\n\\mathsf{B}(t\\_{1:T}) = \\prod\\_{j=1}^{T} Q\\_\\theta(t\\_j \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1}).\n\\end{aligned}\n$$\n\nSubstituting into the KL definition:\n$$\n\\begin{aligned}\nD\\_{\\mathrm{KL}}(\\mathsf{A} \\| \\mathsf{B})\n&= \\mathbb{E}\\_{t\\_{1:T} \\sim \\mathsf{A}}\n\\left[\n\\sum\\_{j=1}^{T}\n\\log\n\\frac{\nQ\\_\\theta(t\\_j \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n}{\nQ\\_\\theta(t\\_j \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n}\n\\right] \\\\\n= \\sum\\_{j=1}^{T}\n\\mathbb{E}\\_{t\\_{1:j-1} \\sim \\mathsf{A}}\n\\left[\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right)\n\\right].\n\\end{aligned}\n$$\n\nThis shows that the **sequence-level KL** equals the expected sum of per-token conditional KL divergences.\n\n---\n\n**Step 2: Monte Carlo Approximation**\n\nComputing the full expectation over all possible token histories $t_{1:j-1}$ is intractable in practice. To address this, we use a Monte Carlo approximation: instead of averaging over all continuations, we compute the KL terms along an existing sampled trajectory $t_{1:T}$ drawn from $\\mathsf{A}$:\n$$\nD\\_{\\mathrm{KL}}^{\\text{full}}\n\\approx\n\\sum\\_{j=1}^{T}\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right).\n$$\n\n---\n\n**Step 3: Token-Window Truncation**\n\nTo reduce cost further, we truncate the sum at a fixed window size $L$ (e.g., 512 tokens):\n$$\nD\\_{\\mathrm{KL}}^{\\text{full}}\n\\approx\n\\sum\\_{j=1}^{\\min(T, L)}\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right).\n$$\n\nThis is exactly the formulation given in **Equation (2)** of the main paper.\n\nThe approximation introduces a residual term:\n$$\nR\\_L = \\sum\\_{j > L}\n\\mathbb{E}\\_{t\\_{1:j-1} \\sim \\mathsf{A}} \\left[\nD\\_{\\mathrm{KL}}(\\cdot \\| \\cdot)\n\\right] \\ge 0,\n$$\nwhich is small when the effect of rewriting $s_i$ fades over time."}, {"review_id": "lPXbcBNXjR", "reviewer": "Reviewer_mo3k", "summary": "The paper introduces TokenSqueeze, a novel method for compressing reasoning traces in large language models (LLMs) to improve reasoning efficiency without sacrificing accuracy. TokenSqueeze addresses the issue of long chain-of-thought (CoT) traces in reasoning LLMs, which lead to increased inference latency and memory consumption. The method includes three main components: adaptive reasoning depth selection to maintain appropriate reasoning depth based on problem complexity, intra-step linguistic refinement to enhance the conciseness of reasoning steps while preserving their meaning, and a composite optimization objective that balances correctness and brevity. Through comprehensive experiments on mathematical reasoning and programming tasks, TokenSqueeze demonstrates significant reductions in token usage while maintaining or improving accuracy, achieving higher efficiency and better performance under token budget constraints compared to existing methods.", "questions": "See weakness.", "limitations": "No.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. The method for constructing contrastive preference data from self-generated reasoning traces is original and effective.\n2. The core idea of optimizing reasoning efficiency via length-aware preference modeling is reasonable and addresses a practical need.\n3. The paper is well-structured and easy to follow.\n\nWeaknesses：\n1. While the proposed DPO-L formulation integrates response length into the preference loss, the core idea of using length-based regularization in preference optimization has been explored in prior works.[1,2]\n2. The formulation in Equation (2) lacks clarity. Specifically, it is unclear whether the \\( j \\)-th token refers to the position following the \\( i \\)-th reasoning step, as implied in the main text. A more precise definition would improve interpretability.\n3. The paper would benefit from including experiments with alternative backbone models to verify the generalizability of the proposed method across different architectures.\n\n\n\n[1] O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning 2025.\n[2] Kimi k1.5: Scaling Reinforcement Learning with LLMs 2025.", "was_revised": true, "final_justification": "Thanks for the rebuttal. All of my concerns have been addressed. I have updated my score to Borderline accept.", "rebuttal": "We sincerely thank you for your careful review and positive feedback. We appreciate your recognition of the originality of our approach, the practicality of our objective, and the overall clarity of the paper. Your encouraging comments are very helpful to us, and we address the remaining points in detail below.\n\n**W1:** While the proposed DPO-L formulation integrates response length into the preference loss, the core idea of using length-based regularization in preference optimization has been explored in prior works.[1,2]\n\n**Response:** In contrast to approaches like *O1-Pruner* and *Kimi-k1.5*, which primarily rely on adding a length regularization term within a policy gradient framework, our core idea is not simply to impose a regularizer. Instead, we propose a systematic approach to preference optimization that improves inference efficiency **without sacrificing correctness**. Our method enables the model to implicitly prefer responses that are both accurate and concise, offering a fundamentally different paradigm from methods that focus solely on length penalties.\n\nMoreover, the way training data is constructed also differs fundamentally. We automatically derive $(\\mathrm{chosen\\ sample}, \\mathrm{rejected\\ sample})$ pairs from model outputs: for each prompt, the “chosen” response is correct and shorter, while longer, incorrect ones are labeled as “rejected.” In contrast, *O1-Pruner* and *Kimi-k1.5* construct their training data as $(\\mathrm{sample}, \\mathrm{reward})$ pairs, where each response is assigned a scalar reward.\n\n**W2:** The formulation in Equation (2) lacks clarity. Specifically, it is unclear whether the ( j )-th token refers to the position following the ( i )-th reasoning step, as implied in the main text. A more precise definition would improve interpretability.\n\n**Response:** We appreciate your comment and will clarify this in the camera-ready version if the paper is accepted. Specifically, in Equation $2$:\n$$\nD_{\\mathrm{KL}}^{\\text{full}} \n\\approx \n\\sum_{j=1}^{\\min(T,L)} \nD_{\\mathrm{KL}}\\left( \nQ_{\\theta}(\\cdot \\mid p, s_{\\leq i}, t_{1:j-1}) \\,\\|\\, Q_{\\theta}(\\cdot \\mid p, s_{<i}, s_i', t_{1:j-1}) \n\\right),\n$$\n\nWe clarify that $t_j$ refers to the $j$-th token in the consecutive reasoning steps following the $i$-th step $s_i$. Specifically, $t_1$ corresponds to the first token of $s_{i+1}$. If the tokens in $s_{i+1}$ are insufficient to fill the window, we continue accumulating tokens from subsequent steps until the window is filled or the response ends.\n\nTo provide more insight into Equation 2, we provide the derivation from Equation 1 to Equation 2 below.\n\n**Proof (from Equation (1) to Equation (2))**\n\n---\n\n**Notation and Setup**\n\nWe begin by stating the two key KL divergence expressions from the paper:\n\n- Equation (1):  \n  $$\n  D_{\\mathrm{KL}}\\left( P_{\\theta}(\\cdot \\mid p, s_{\\leq i}) \\| P_{\\theta}(\\cdot \\mid p, s_{<i}, s_i') \\right)\n  $$\n\n- Equation (2):  \n  $$\n  D_{\\mathrm{KL}}^{\\text{full}} \n  \\approx \n  \\sum_{j=1}^{\\min(T,L)} \n  D_{\\mathrm{KL}}\\left( \n  Q_{\\theta}(\\cdot \\mid p, s_{\\leq i}, t_{1:j-1}) \\| Q_{\\theta}(\\cdot \\mid p, s_{<i}, s_i', t_{1:j-1}) \n  \\right)\n  $$\n\nThese expressions measure the change in predicted continuation distributions before and after rewriting step $s_i$.\n\nWe now define the key variables and distributions involved in the KL computation:\n\n- $p$: the input prompt to the model.  \n- $s_i$: the $i$‑th reasoning step generated by the model.  \n- $s_{\\le i}$: the reasoning trace up to and including step $i$ (i.e., $s_1, s_2, \\dots, s_i$).  \n- $s'_i$: a rewritten version of step $s_i$ used in intra-step refinement.  \n- $t_j$: the $j$‑th token **after** $s_i$ (or $s'_i$), starting the continuation sequence.  \n- $t_{1:T}$: a full continuation sequence of $T$ tokens (including EOS).  \n\nWe define the following probability distributions:\n\n- $Q_\\theta(\\cdot \\mid \\cdot)$: the model’s **next-token distribution** under the given context (token-level), parameterized by $\\theta$.  \n- $P_\\theta(\\cdot \\mid \\cdot)$: the model’s **sequence-level distribution** over continuations $t_{1:T}$, given the input context.\n\nTo simplify notation, we define:\n\n- $\\mathsf{A}(t_{1:T}) := P_\\theta(t_{1:T} \\mid p, s_{\\le i})$\n- $\\mathsf{B}(t_{1:T}) := P_\\theta(t_{1:T} \\mid p, s_{< i}, s'_i)$\n\nThese represent the two sequence distributions compared in the KL divergence.\n\n---\n\n**Step 1: Sequence‑Level KL Definition**\n\nThe KL term in Equation (1) is:\n$$\nD\\_{\\mathrm{KL}}\\big( \\mathsf{A} \\| \\mathsf{B} \\big)=\\mathbb{E}\\_{t\\_{1:T} \\sim \\mathsf{A}}\n\\left[\n\\log \\frac{ \\mathsf{A}(t\\_{1:T}) }{ \\mathsf{B}(t\\_{1:T}) }\n\\right].\n$$\n\nExpanding both $\\mathsf{A}$ and $\\mathsf{B}$ autoregressively using $Q_\\theta$:\n$$\n\\begin{aligned}\n\\mathsf{A}(t\\_{1:T}) = \\prod\\_{j=1}^{T} Q\\_\\theta(t\\_j \\mid p, s\\_{\\le i}, t\\_{1:j-1}), \\\\\n\\mathsf{B}(t\\_{1:T}) = \\prod\\_{j=1}^{T} Q\\_\\theta(t\\_j \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1}).\n\\end{aligned}\n$$\n\nSubstituting into the KL definition:\n$$\n\\begin{aligned}\nD\\_{\\mathrm{KL}}(\\mathsf{A} \\| \\mathsf{B})\n&= \\mathbb{E}\\_{t\\_{1:T} \\sim \\mathsf{A}}\n\\left[\n\\sum\\_{j=1}^{T}\n\\log\n\\frac{\nQ\\_\\theta(t\\_j \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n}{\nQ\\_\\theta(t\\_j \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n}\n\\right] \\\\\n= \\sum\\_{j=1}^{T}\n\\mathbb{E}\\_{t\\_{1:j-1} \\sim \\mathsf{A}}\n\\left[\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right)\n\\right].\n\\end{aligned}\n$$\n\nThis shows that the **sequence-level KL** equals the expected sum of per-token conditional KL divergences.\n\n---\n\n**Step 2: Monte Carlo Approximation**\n\nComputing the full expectation over all possible token histories $t_{1:j-1}$ is intractable in practice. To address this, we use a Monte Carlo approximation: instead of averaging over all continuations, we compute the KL terms along an existing sampled trajectory $t_{1:T}$ drawn from $\\mathsf{A}$:\n$$\nD\\_{\\mathrm{KL}}^{\\text{full}}\n\\approx\n\\sum\\_{j=1}^{T}\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right).\n$$\n\n---\n\n**Step 3: Token-Window Truncation**\n\nTo reduce cost further, we truncate the sum at a fixed window size $L$ (e.g., 512 tokens):\n$$\nD\\_{\\mathrm{KL}}^{\\text{full}}\n\\approx\n\\sum\\_{j=1}^{\\min(T, L)}\nD\\_{\\mathrm{KL}}\\left(\nQ\\_\\theta(\\cdot \\mid p, s\\_{\\le i}, t\\_{1:j-1})\n\\Big\\|\nQ\\_\\theta(\\cdot \\mid p, s\\_{< i}, s'\\_i, t\\_{1:j-1})\n\\right).\n$$\n\nThis is exactly the formulation given in **Equation (2)** of the main paper.\n\nThe approximation introduces a residual term:\n$$\nR\\_L = \\sum\\_{j > L}\n\\mathbb{E}\\_{t\\_{1:j-1} \\sim \\mathsf{A}} \\left[\nD\\_{\\mathrm{KL}}(\\cdot \\| \\cdot)\n\\right] \\ge 0,\n$$\nwhich is small when the effect of rewriting $s_i$ fades over time.\n\n\n**W3:** The paper would benefit from including experiments with alternative backbone models to verify the generalizability of the proposed method across different architectures.\n\n**Response:** To further demonstrate the effectiveness of our model across different backbones and LLM scales, we conducted experiments using Phi4-Reasoning-14B. The results are presented below:\n\n| Model               | **Dataset** | **Accuracy (%)** | **AUC**  | **Avg. Length** |\n| ------------------- | ----------- | ---------------- | -------- | --------------- |\n| Phi4-Reasoning-14B  | AIME24      | 72.1             | 55.1     | 7768            |\n| TokenSqueeze (Phi4) | AIME24      | **73.8**         | **59.6** | **6326**        |\n| Phi4-Reasoning-14B  | MATH500     | 93.8             | 72.0     | 7644            |\n| TokenSqueeze (Phi4) | MATH500     | **96.0**         | **87.5** | **2921**        |\n\nIt is important to note that, due to time constraints, we reducing K in intra-step linguistic refinement from 64 to 16. We expect that further hyperparameter tuning would yield even better performance."}]

Author rebuttal:

Meta-review: This paper presents TokenSqueeze, a novel method to make reasoning LLMs more efficient by shortening their long chain-of-thought outputs without hurting accuracy. The approach works by first selecting the best reasoning paths and then refining the language in each step to be more concise, using a KL-divergence constraint to ensure the core logic remains intact. The model is then fine-tuned to prefer these shorter, correct answers. The core idea is quite intuitive and the paper is very well-written, with strong experimental results demonstrating significant token reduction. While the initial submission's evaluation was a bit limited to a specific model family and could have benefited from a deeper analysis of some hyperparameters, the authors have addressed these points in their rebuttal with new experiments and clarifications. Overall, this is a solid paper that tackles an important problem in a practical way, and I'm happy to recommend it for acceptance.
Decision: Accept (poster)