SynAdapt: Learning Adaptive Reasoning in Large Language Models via Synthetic Continuous Chain-of-Thought

## Abstract

Abstract While Chain-of-Thought (CoT) reasoning improves model performance, it incurs significant time costs due to the generation of discrete CoT tokens (DCoT).
Continuous CoT (CCoT) offers a more efficient alternative, but existing CCoT methods are hampered by indirect fine-tuning, limited alignment, or inconsistent targets.
To overcome these limitations, we propose SynAdapt , an innovative efficient reasoning framework.
Specifically, SynAdapt generates the synthetic CCoT to serve as a precise and effective alignment target for LLMs.
This synthetic CCoT explicitly guides the LLM to learn CCoT and derive accurate answers directly.
Furthermore, relying solely on CCoT is insufficient for solving hard questions.
To address this, SynAdapt integrates a difficulty classifier that leverages both question context and CCoT to identify hard questions.
CCoT can effectively help identify hard questions after some brief reasoning.
We then adaptively prompt the LLM to re-think these hard questions for improved performance.
Extensive experimental results across various benchmarks from different difficulty levels strongly demonstrate the effectiveness of our method, achieving the best accuracy-efficiency trade-off.

## 1 Introduction

Chain-of-Thought (CoT) reasoning (Kojima et al. 2022; Wei et al. 2022; Zhou et al. 2022) has shown remarkable potential in enhancing the problem-solving capabilities of Large Language Models (LLMs) for complex tasks (Guo et al. 2025; Yang et al. 2025; OpenAI 2025).
By decomposing problems into sequential steps, CoT allows LLMs to derive correct answers step-by-step.
However, a major drawback of CoT is its high computational cost due to the generation of numerous tokens, leading to substantial time consumption (Yu et al. 2024; Yeo et al. 2025).
While this cost is often acceptable in accuracy-sensitive scenarios, such as AI for Science (AI4S) (Lu et al. 2024) where accuracy is paramount, it becomes problematic in efficiency-sensitive scenarios.
For instance, in embodied intelligence, real-time human-computer interaction necessitates highly efficient reasoning to ensure a satisfactory user experience (Li et al. 2024a).
Consequently, a critical challenge emerges: how to reduce the length of generated CoT while preserving its effective reasoning capabilities

Figure: Figure 1: Comparisons between our SynAdapt and the other CCoT-based baselines. These baselines either train CCoT indirectly, provide only single-position alignment, or apply full alignment with incoherent targets.
Refer to caption: x1.png

Existing efficient reasoning approaches mainly involve fine-tuning or direct prompting LLMs to reduce the number of COT steps (Arora and Zanette 2025; Munkhbat et al. 2025; Xu et al. 2025a).
However, the remaining CoT steps still involve numerous discrete natural language tokens, which we refer to as DCoT.
As noted by Li et al. (2024b) and Lin et al. (2024), most of these verbalized tokens are mainly for communication and carry unnecessary linguistic details that do not contribute to the core reasoning process.
One promising approach is fine-tuning LLM to replace DCoT with a more compact and continuous CoT representation, known as CCoT. (Pfau, Merrill, and Bowman 2024; Goyal et al. 2023).
During reasoning, CCoT retains the hidden state of the LLM and skips generating one-hot token ID, allowing it to store more information than just a single token (Zhu et al. 2025).

Nonetheless, fine-tuning LLM to learn CCoT reasoning effectively remains challenging.
Coconut (Hao et al. 2024) gradually fine-tunes the LLM to replace DCoT with CCoT using a curriculum learning strategy (Deng, Choi, and Shieber 2024).
However, as shown in Figure [1](https://arxiv.org/html/2508.00574v1#S1.F1), it lacks explicit alignment between DCoT and CCoT, which limits its ability to effectively learn from the original DCoT.
CODI (Shen et al. 2025b) introduces explicit alignment between the last token hidden state of DCoT and the final hidden state of CCoT but ignores alignment for other intermediate tokens.
CompressCoT (Cheng and Van Durme 2024) attempts to identify a subset of important tokens from DCoT, whose length matches CCoT, and aligns the full CCoT with the hidden states of these tokens.
However, selecting only several isolated DCoT tokens leads to incoherence in the reasoning process.
This leads to significant performance degradation in CCoT learning.

To overcome these limitations, we propose a novel efficient reasoning framework called SynAdapt, which helps LLM learn Adaptive reasoning through Synthetic CCoT.
Our approach begins by generating a synthetic CCoT to serve as a comprehensive alignment target.
Specifically, we initialize a random CCoT, fix the LLM, and iteratively optimize the random CCoT into a synthetic CCoT to guide LLM towards correct answers.
The synthetic CCoT thereby serves as a better alignment target than only using several isolated and incoherent tokens from the original DCoT.
During fine-tuning, we apply the full alignment using the synthetic CCoT, as shown by Figure [1](https://arxiv.org/html/2508.00574v1#S1.F1).
This strategy helps LLM learn the full CCoT rather than only the last one.
Notably, we fine-tune the LLM to iteratively refine a meaningless draft to obtain the CCoT, rather than generating CCoT autoregressively.
This approach is more efficient (Jiang et al. 2025) and can boosts the reasoning ability of LLM by iterative refinement (Saunshi et al. 2025; Yu et al. 2025).

Moreover, according to the information theory (Nalewajski 2011), compressing DCoT into the dense CCoT inevitably leads to information loss and increases the complexity of solving hard questions (Koehn and Knowles 2017).
We provide an example in Figure [5](https://arxiv.org/html/2508.00574v1#S6.F5) of Appendix.
To address this, we train a difficulty classifier that assesses question difficulty based on both the question itself and the CCoT.
And then prompt the LLM to re-think hard questions using discrete CoT tokens for improved accuracy.
While CCoT may not be sufficient to solve these hard questions, it can help the classifier effectively identify them.
Some hard questions resemble simpler ones and can only be distinguished through the brief reasoning captured by CCoT.
We also present an illustrative example in Figure [6](https://arxiv.org/html/2508.00574v1#S6.F6) of Appendix.

We evaluate our method across various benchmarks with different difficulty levels, including GSM8K, MATH500, AMC23, AIME24, and AIME25.
By dynamically adjusting the ratio of re-think hard questions, our method demonstrates adaptability in both accuracy-sensitive and efficiency-sensitive scenarios.
Comprehensive experimental results demonstrate that our method outperforms other baselines in both scenarios, achieving an optimal accuracy-efficiency trade-off.
We also evaluate the identification performance of our difficulty classifier, showing its superior performance compared to other baselines.
The main contributions of this paper are as follows:

- •
We propose a novel efficient reasoning framework that generates synthetic CCoT, providing a better full alignment target to help LLMs learn CCoT more effectively.
- •
We introduce a difficulty classifier that more effectively distinguishes hard questions by considering both the question and the CCoT, enabling adaptive re-thinking for improved accuracy.
- •
The experimental results strongly demonstrate the effectiveness of our framework, achieving the best accuracy-efficiency trade-off.

## 2 Related Work

In this section, we introduce the mainstream related work on efficient reasoning in the LLMs, which can be mainly categorized into three types: SFT-based methods, RL-based methods, Prompt-based methods, and CCoT-based methods.

SFT-based methods either discard the CoT entirely or dynamically compress the CoT in the training data.
And then they apply supervised fine-tuning (SFT) on these compressed data to help LLM learn to reduce generation length.
While these methods are effective in shortening the generated output, they may ignore some crucial details of the original CoT during fine-tuning, leading to significant performance degradation (Yu et al. 2024; Ma et al. 2025b; Munkhbat et al. 2025; Xia et al. 2025; Kang et al. 2025).
RL-based methods primarily design length penalties to prevent the model from generating excessively long CoT.
While these methods can reduce reasoning length without sacrificing LLM performance, they require substantial resources for repeated data sampling to LLM training.
Additionally, the reduction in length is limited and may not be suitable for efficiency-sensitive scenarios, where minimizing generation length is crucial
(Arora and Zanette 2025; Luo et al. 2025; Yeo et al. 2025; Aggarwal and Welleck 2025; Shen et al. 2025a).
Prompt-based methods explicitly add length constraint instructions in the prompt for guiding LLM to reduce generation length.
Although these approaches are low-cost, their impact on length reduction is limited.
LLMs still tend to generate long, redundant reasoning CoTs, especially for those hard questions.
(Renze and Guven 2024; Xu et al. 2025a; Lee, Che, and Peng 2025; Han et al. 2024)

Instead of reasoning by numerous redundant tokens, CCoT-based methods aim to compress the reasoning steps by replacing the original discrete CoT (DCoT) with Continuous CoT (CCoT) in the latent space.
However, these methods often suffer from significant performance drops.
This is mainly because they either don’t explicitly align CCoT with DCoT or only use parts of the DCoT to conduct alignment.
These weak alignment signals can not effectively help LLM learn CCoT reasoning, leading to the performance degradation.
(Hao et al. 2024; Xu et al. 2025b; Shen et al. 2025b; Cheng and Van Durme 2024)

Due to the limited space, a detailed introduction of the above related works are shown in Appendix [6.1](https://arxiv.org/html/2508.00574v1#S6.SS1).

## 3 Methodology

Figure: Figure 2: Our SynAdapt framework consist of two stage. (1). In Synethetic CCoT Generation, we first generate the synthetic CCoT $Z_{\text{syn}}$ for each question. And then in Synethetic CCoT Enhance Fine-tuning, $Z_{\text{syn}}$ serves as the full alignment target. By using $Z_{\text{syn}}$, we fine-tune the LLM $\phi$ to effectively learn CCoT, enabling iterative refinement of a randomly initialized draft CCoT. Additionally, we train a difficulty classifier $\delta$ to assess question difficulty based on both the question and the generated CCoT. (2). During the inference stage, we use the fine-tuned LLM $\phi$ to iteratively refine and generate the final CCoT, while the difficulty classifier $\delta$ determines the question difficulty. For easy questions, the LLM directly generates the output, and for hard questions, it is prompted to re-think in order to generate the correct answer.
Refer to caption: x2.png

In this section, we present the details of our SynAdpat framework, which consists of two stages: the fine-tuning stage and the inference stage, as shown in Figure [2](https://arxiv.org/html/2508.00574v1#S3.F2).
During the fine-tuning stage, we first generate the synthetic CCoT by optimizing a randomly initialized one.
The optimization goal is to ensure that the LLM generates the correct answer when using the synthetic CCoT.
After generation, we fine-tune the LLM to learn CCoT by utilizing the synthetic CCoT as the alignment target.
Specifically, the LLM is trained to iteratively refine a draft CCoT until it aligns with the pre-generated synthetic CCoT
Additionally, we train a difficulty classifier that assesses a question’s difficulty based on both the question itself and its corresponding CCoT.

During the inference stage, the fine-tuned LLM generates the CCoT for the given question.
This generated CCoT, along with the original question, is then fed into the difficulty classifier to distinguish between easy and hard questions.
For easy questions, the LLM directly generates the answer based on the CCoT, ensuring high efficiency.
For hard questions, we discard the CCoT and prompt the LLM to re-think the question step by step, ensuring higher accuracy.

More details of the training stage and the inference stage are presented in Section [3.1](https://arxiv.org/html/2508.00574v1#S3.SS1) and Section [3.2](https://arxiv.org/html/2508.00574v1#S3.SS2), respectively.

### 3.1 Training Stage

#### Synthetic CCoT Generation.

To provide a more effective alignment target to learn CCoT representation during fine-tuning LLM, we firstly generate the synthetic CCoT before fine-tuning.

As shown in the upper-left part of Figure [2](https://arxiv.org/html/2508.00574v1#S3.F2), for each question $Q$, we randomly initialize a synthetic CCoT $Z_{\text{syn}}$ with a fixed length $m$.
We then concatenate $Q$ with $Z_{\text{syn}}$ and an end-of-think token to form $[Q,Z_{\text{syn}},\text{eot}]$.
Given that a well-constructed CCoT should guide the LLM to predict the correct answer based on the question and CCoT, we make $Z_{\text{syn}}$ trainable and optimize it by minimizing the following loss:

$$ $\mathcal{L}_{\text{ans}}=-\frac{1}{L_{a}}\sum_{i=1}^{L_{a}}\text{log}\mathcal{P}_{\theta}(A_{i}|Q,Z_{\text{syn}},\text{eot},A_{<i}),$ (1) $$

where $L_{a}$ is the length of the answer $A$, $A_{i}$ denotes the $i$-th token of $A$, and $\theta$ represents the parameters of the LLM.

Moreover, to prevent overfitting during CCoT optimization, we additionally align the hidden state of the eot token when using the synthetic CCoT with that obtained when using DCoT.
Assuming $\mathbf{h}^{l}_{\text{eot_syn}}$ is the hidden state of the eot token at the $l$-th layer of the LLM when provided with synthetic CCoT $Z_{\text{syn}}$ and $\mathbf{h}^{l}_{\text{eot_dcot}}$ is that when provided with DCoT, the alignment loss is defined as:

$$ $\mathcal{L}_{\text{dcot}}=\frac{1}{L}\sum_{l=1}^{L}\left\|\mathbf{h}^{l}_{\text{eot_syn}}-\mathbf{h}^{l}_{\text{eot_dcot}}\right\|_{1},$ (2) $$

where $L$ is the total number of layers in the LLM.
After optimizing using both $\mathcal{L}_{\text{ans}}$ and $\mathcal{L}_{\text{dcot}}$, we obtain the high-quality synthetic CCoT $Z_{\text{syn}}$, which serves a similar function to DCoT but is represented in a denser, continuous format.
These $Z_{\text{syn}}$ can serve as valuable alignment targets during fine-tuning LLM to learn CCoT.

#### Synthetic CCoT Enhanced Fine-tuning.

As demonstrated by Saunshi et al. (2025); Yu et al. (2025), iteratively looping an LLM can significantly enhance its reasoning capabilities and refine outputs.
Inspired by this, we fine-tune the LLM to iteratively refine the CCoT from a draft in a looping manner instead of generating it autoregressively.

As shown in Figure [2](https://arxiv.org/html/2508.00574v1#S3.F2), we concatenate the question $Q$ with a draft CCoT $Z_{\text{draft}}^{0}$.
The $Z_{\text{draft}}^{0}$ is initialized as the embedding of a repeated meaningless token sequence (i.e., <T>..<T>), with a fixed length of $m$.
We input the $Z_{\text{draft}}^{0}$ into LLM and use the corresponding output hidden state as the refined one.
The iterative refinement process can be formulated as:

$$ $Z_{\text{draft}}^{i}=f_{\phi}(Q,Z_{\text{draft}}^{i-1})[L_{q}:],$ (3) $$

where $Z_{\text{draft}}^{i}$ is the CCoT after refining $i$ iterations, $L_{q}$ is the length of the question $Q$, $\phi$ represents the fine-tuned LLM with a trainable LoRA module and $f_{\phi}(\cdot)$ returns the output hidden state from $\phi$.
After $k$ refining iterations, we obtain the final CCoT $Z_{\text{final}}=Z_{\text{draft}}^{k}$.
We explicitly align the full $Z_{\text{final}}$ with the synthetic CCoT $Z_{\text{syn}}$ and compute the $\mathcal{L_{\text{align}}}$ loss as:

$$ $\mathcal{L_{\text{align}}}=\left\|Z_{\text{final}}-Z_{\text{syn}}\right\|_{1}.$ (4) $$

Moreover, $Z_{\text{final}}$ should also guide the initial LLM to generate the correct answer.
Therefore, we compute an additional losses, similar to Equation [1](https://arxiv.org/html/2508.00574v1#S3.E1), as shown below:

$$ $\displaystyle\mathcal{L}^{\prime}_{\text{ans}}$ $\displaystyle=-\frac{1}{L_{a}}\sum_{i=1}^{L_{a}}\text{log}\mathcal{P}_{\theta}(A_{i}|Q,Z_{\text{final}},\text{eot},A_{<i}),$ (5) $\displaystyle\mathcal{L_{\text{refine}}}$ $\displaystyle=\mathcal{L_{\text{align}}}+\mathcal{L}^{\prime}_{\text{ans}},$ (6) $$

where $\theta$ represents the initial LLM without the LoRA module.
The $\mathcal{L_{\text{refine}}}$ loss fully utilizes the alignment information from $Z_{\text{align}}$.
After training using $\mathcal{L_{\text{refine}}}$, the fine-tuned LLM $\Phi$ effectively learns to iteratively refine the draft CCoT, ultimately generating the final CCoT to replace the original redundant DCoT.

#### Difficulty Classifier Training.

Additionally, we train a difficulty classifier $\delta$, composed of two MLP layers, to distinguish between hard and easy questions. It takes both the question itself and the CCoT as input.
Specifically, we construct question pairs $\left<Q_{\text{c}},Q_{\text{r}}\right>$ based on existing difficulty labels from the DeepMath dataset (He et al. 2025).
$Q_{\text{c}}$ is a hard question and $Q_{\text{r}}$ is an easy question.
Next, we input $Q_{\text{c}}$ and $Q_{\text{r}}$ to the fine-tuned LLM $\phi$ to obtain the corresponding CCoT $Z_{\text{final}}^{c}$ and $Z_{\text{final}}^{r}$.
Then we concatenate $Q_{\text{c}}$, $Z_{\text{final}}^{c}$ and one eot token and input to the initial LLM to obtain the output hidden state of eot as:

$$ $\mathbf{h}_{\text{eot_final}}^{c}=f_{\theta}(Q_{c},Z_{\text{final}}^{c},\text{eot})[-1],$ (7) $$

where $f_{\theta}$ represents the output hidden state from the initial LLM $\theta$ and $\mathbf{h}_{\text{eot_final}}^{c}$ denotes the output hidden state of the eot token.
Considering the attention mechanism of LLM, $\mathbf{h}_{\text{eot_final}}^{c}$ can fully capture the information in $Q_{c}$ and $Z_{\text{final}}^{c}$.
Similarly, we compute the $\mathbf{h}_{\text{eot_final}}^{r}$ for the easy question $Q_{r}$.
We train the difficulty classifier $\delta$ according to the following loss:

$$ $\mathcal{L}_{\text{diff}}=-\text{log}\sigma(f_{\delta}(\mathbf{h}_{\text{eot_final}}^{c})-f_{\delta}(\mathbf{h}_{\text{eot_final}}^{r})),$ (8) $$

where $f_{\delta}(\cdot)$ denotes the difficulty level predicted by $\delta$.
$\mathcal{L}_{\text{diff}}$ encourages the classifier to give higher score for hard question $Q_{c}$ and lower scores to easy ones $Q_{r}$.
By utilizing additional information from the CCoT, the classifier $\delta$ can more effectively distinguish between hard and easy questions.

### 3.2 Inference Stage

#### Adaptive Reasoning via CCoT.

During the inference stage, we concatenate the question with a draft CCoT and utilize the fine-tuned LLM $\phi$ to iteratively refine the draft CCoT to obtain the final CCoT.
And then we utilized the difficulty classifier to assign the difficulty score based on both the question and the CCoT.
Questions with a difficulty score below the threshold $\tau$ are considered easy, while those above are regarded as hard.

For easy questions, we just append a eot token after the CCoT and prompt the base LLM $\theta$ to directly output answer.
The generated CCoT effectively replaces the original discrete CoT reasoning process, which often contains numerous tokens and is time-consuming to generate, thereby achieving higher efficiency.

However, compressing DCoT into CCoT inevitably leads to information loss (Nalewajski 2011).
And as shown by Hao et al. (2024), relying solely on CCoT is insufficient for hard questions and may even lead to incorrect answer.
Therefore, we discard the generated CCoT and prompt the LLM to re-think the question via discrete CoT, using a more detailed reasoning process to generate the correct answer.
Additionally, inspired by Xu et al. (2025a), we explicitly prompt the LLM to condense each reasoning step, achieving a better trade-off between accuracy and efficiency.

Moreover, we can dynamically adjust the threshold $\tau$ to control the ratio of re-thinking.
This allows our method to simultaneously adapt to both accuracy-sensitive and efficiency-sensitive scenarios according the specific requirements of the real application.
All our used prompts are provided in Appendix [6.9](https://arxiv.org/html/2508.00574v1#S6.SS9).

## 4 Experiments

In this section, we conduct comprehensive experiments to demonstrate the effectiveness of our SynAdapt and address the following four key research questions:

- •
RQ1: Can SynAdapt offer a better accuracy-efficiency trade-off compared to other efficient reasoning baselines in both accuracy-sensitive and efficiency-sensitive scenarios? (see Section [4.1](https://arxiv.org/html/2508.00574v1#S4.SS1))
- •
RQ2: Does our difficulty classifier, which uses both the question and CCoT, can effectively distinguish between hard and easy questions? (see Section [4.2](https://arxiv.org/html/2508.00574v1#S4.SS2))
- •
RQ3: How about the training efficiency of SynAdapt? (see Section [4.3](https://arxiv.org/html/2508.00574v1#S4.SS3))
- •
RQ4: How well does the generalization capacity of SynAdapt? (see Section [4.4](https://arxiv.org/html/2508.00574v1#S4.SS4))

### 4.1 Evaluation of Accuracy-Efficiency Trade-off

**Table 1: Comparison between our SynAdapt and efficient reasoning baselines for both Accuracy-Sensitive Scenario and Efficiency-Sensitive Scenario. For the accuracy-sensitive scenario, we set the threshold $\tau=0.5$ for our method, meaning that questions with a difficulty score greater than 0.5 are routed to re-thinking, while others directly generate an answer based on the CCoT. For the efficiency-sensitive scenario, we set $\tau=1.0$, meaning all questions are answered directly using the CCoT to achieve high efficiency. Bold and underlined numbers represent the best and second-best average accuracy, generation length and Rel-G score for each scenario.**
| Methods | AIME25 | AIME24 | AMC23 | MATH500 | GSM8K | Average |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Acc | Len | Acc | Len | Acc | Len | Acc | Len | Acc | Len | Acc $\uparrow$ | Len $\downarrow$ | Rel-G $\uparrow$ |  |
| Raw Model | 36.7 | 13348.6 | 53.3 | 14071.4 | 92.5 | 6315.7 | 93.2 | 4087.4 | 90.7 | 1110.8 | 73.3 | 7786.84 | 1.00 |
| Accuracy-Sensitive Scenario |  |  |  |  |  |  |  |  |  |  |  |  |  |
| CoT-FT | 40.0 | 16427.3 | 40.0 | 15560.6 | 87.5 | 7049.3 | 88.6 | 3694.0 | 83.0 | 700.7 | 67.8 | 8686.4 | 0.83 |
| TokenSkip | 30.0 | 17811.3 | 36.7 | 14385.0 | 70.0 | 10030.8 | 78.4 | 16542.8 | 81.1 | 17165.5 | 59.2 | 15187.1 | 0.41 |
| NoThinking | 30.0 | 10623.6 | 40.0 | 11099.7 | 75.0 | 4143.6 | 82.4 | 1355.4 | 85.7 | 229.5 | 62.6 | 5490.4 | 1.21 |
| CoD | 40.0 | 10498.0 | 56.7 | 8488.5 | 80.0 | 2894.3 | 81.8 | 1591.1 | 84.2 | 286.2 | 68.5 | 4751.6 | 1.53 |
| TokenBudget | 36.7 | 15235.0 | 53.3 | 14897.7 | 82.5 | 5006.5 | 90.2 | 3186.8 | 86.9 | 573.0 | 69.9 | 7779.8 | 0.95 |
| SynAdapt ($\tau$=0.5) | 40.0 | 10198.3 | 56.7 | 8288.1 | 80.0 | 2881.6 | 82.4 | 1547.7 | 85.7 | 258.6 | 69.0 | 4694.8 | 1.58 |
| Efficiency-Sensitive Scenario |  |  |  |  |  |  |  |  |  |  |  |  |  |
| NoCoT-FT | 13.3 | 637.0 | 10.0 | 1680.1 | 50.0 | 513.1 | 74.8 | 478.9 | 87.1 | 209.5 | 47.0 | 703.7 | 7.13 |
| SelfTraining | 10.0 | 671.6 | 10.0 | 772.7 | 55.0 | 627.0 | 71.6 | 397.0 | 85.1 | 207.6 | 46.3 | 535.2 | 9.10 |
| Coconut | 6.7 | 647.2 | 13.3 | 1692.5 | 52.5 | 548.0 | 76.2 | 426.4 | 89.3 | 232.6 | 47.6 | 709.3 | 7.13 |
| CompressCoT | 10.0 | 623.1 | 6.7 | 1673.7 | 52.5 | 1356.1 | 75.0 | 445.8 | 88.2 | 207.7 | 46.5 | 861.3 | 5.73 |
| CODI | 13.3 | 2798.7 | 6.7 | 613.5 | 50.0 | 518.6 | 72.4 | 537.5 | 87.2 | 238.1 | 45.9 | 941.3 | 5.18 |
| SynAdapt ($\tau$=1.0) | 13.3 | 718.8 | 16.7 | 620.7 | 57.5 | 591.9 | 75.6 | 739.4 | 88.5 | 253.5 | 50.3 | 584.9 | 9.14 |
| - Synthetic CCoT | 10.0 | 1743.9 | 16.7 | 475.9 | 52.5 | 510.2 | 73.2 | 599.8 | 87.8 | 266.7 | 48.0 | 719.3 | 7.10 |
| - Iterative Refine | 6.7 | 767.6 | 10.0 | 700.2 | 50.0 | 1073.9 | 76.0 | 993.8 | 85.4 | 728.9 | 45.6 | 852.9 | 5.68 |

#### Experimental Settings

We use DeepMath (He et al. 2025) as the training set and evaluate our method and baselines on five widely adopted math-related benchmarks: AIME25, AIME24, AMC23, MATH500 (Lightman et al. 2023) and GSM8K (Cobbe et al. 2021). These datasets cover a diverse range of math questions across varying difficulty levels.
As for the evaluation metrics, we report accuracy (Acc) and generation length (Len) to assess both performance and efficiency. Additionally, we introduce the Relative Gain metric (Rel-G) defined as:

$$ $\text{Rel-G}=\frac{\text{Acc}/\text{Acc}_{\text{raw}}}{\text{Len}/\text{Len}_{\text{raw}}},$ (9) $$

where $\text{Acc}_{\text{raw}}$ and $\text{Len}_{\text{raw}}$ denote the accuracy and generation length of the raw model, respectively. A higher Rel-G indicates a better trade-off between accuracy and efficiency.

We adopt DeepSeek-R1-Distill-Qwen-7B as our raw model.
We set the length of the synthetic CCoT to $m=512$, and refining iterations for the draft CCoT to $k=4$.
The difficulty score ranges from 0 to 1.
In accuracy-sensitive scenarios, we set threshold $\tau=0.5$ to route difficult questions for re-thinking. In efficiency-sensitive scenarios, we set $\tau=1.0$ to prompt the LLM to directly generate answers based on CCoT for higher efficiency.

Further details on the datasets and implementation details are provided in Appendix [6.2](https://arxiv.org/html/2508.00574v1#S6.SS2) and Appendix [6.4](https://arxiv.org/html/2508.00574v1#S6.SS4) respectively.

#### Compared Methods

Here, we consider a broad range of existing efficient reasoning baselines, not limited to CCoT-based methods.
We categorize these baselines into two scenarios, accuracy-sensitive scenario and efficiency-sensitive scenario, based on their different focuses.

In the accuracy-sensitive scenario, CoT-FT directly uses the full discrete CoT from the training data to perform supervised fine-tuning (SFT) for improving performance.
TokenSkip (Xia et al. 2025) compresses the discrete CoT based on token importance and then applies SFT on the compressed CoT.
NoThinking (Ma et al. 2025a) skips the SFT process and directly prompts the model to skip reasoning and directly generate the answer.
CoD (Xu et al. 2025a) prompts the model to condense each reasoning step rather than skipping the reasoning process entirely.
TokenBudget (Han et al. 2024) let the LLM to predict a token budget for each question in advance and prompts the model do not exceed the token budget during further generation.

In the efficiency-sensitive scenario, NoCoT-FT (Yu et al. 2024) discards the discrete CoT and performs SFT using only the answer to improve efficiency.
SelfTraining (Munkhbat et al. 2025) applies best-of-$n$ sampling to extract the shortest correct CoT from the LLM and then fine-tunes the LLM on these CoT.
Coconut (Hao et al. 2024), CompressCoT (Cheng and Van Durme 2024), and CODI (Shen et al. 2025b) all belongs to CCoT-based methods, utilizing the CCoT to replace the DCoT for better efficiency.
Coconut adopts a curriculum learning strategy to gradually internalize DCoT into CCoT.
CompressCoT identifies key tokens in the DCoT and aligns the CCoT with their hidden states.
CODI employs self-distillation, aligning the last token hidden state of CCoT with that of DCoT during training.

More details of these compared method are provided in Appendix [6.3](https://arxiv.org/html/2508.00574v1#S6.SS3).

#### Main Results

For the accuracy-sensitive scenario, as shown in the upper part of Table [1](https://arxiv.org/html/2508.00574v1#S4.T1), our method with $\tau=0.5$ outperforms all other baselines by achieving the second-highest average accuracy while maintaining the shortest average generation length.
CoT-FT fine-tunes directly on the full DCoT, improving accuracy on hard questions but also increasing generation length.
TokenSkip selects parts of DCoT for fine-tuning, resulting in inconsistent CoT and performance degradation.
NoThinking can skip CoT for reducing length, but often causes accuracy drops.
CoD condenses each CoT step but cannot skip the unnecessary CoT in simple questions, resulting in a suboptimal accuracy-efficiency trade-off.
TokenBudget dynamically allocates more tokens to harder questions, preserving accuracy but not reducing generation length effectively.
In contrast, our method identifies hard questions and dynamically re-thinks them while directly generating answers for simple ones.
It maintains similar accuracy compared to the raw model while reducing generation length, achieving the highest Rel-G score of 1.55 in the accuracy-sensitive scenario.

For the efficiency-sensitive scenario, our method with $\tau=1.0$ significantly reduces the average generation length to just 584.9 tokens, while maintaining competitive accuracy compared to other baselines, as shown in the bottom part of Table [1](https://arxiv.org/html/2508.00574v1#S4.T1).
NoCoT-FT, which fine-tunes only on answers without CoT, leads to the accuracy drop.
SelfTraining allows the LLM to search for the shortest correct CoT via best-of-$n$ sampling.
But it struggles with harder questions and also results in a substantial drop in accuracy.

The three CCoT-based methods, Coconut, CompressCoT, and CODI, attempt to replace DCoT with CCoT.
However, these methods only use a portion of DCoT or the last token as the alignment target when fine-tuning the LLM to learn CCoT.
Due to the limited alignment signals, especially for hard questions, they achieve unsatisfactory accuracy.
In contrast, our method introduces a more effective alignment target, the synthetic CCoT.
By fully leveraging the alignment information from it, we enable more effective fine-tuning.
Consequently, our method achieves the highest accuracy and the second shortest generation length in average, yielding the best trade-off with a Rel-G score of 9.14.
We also present a representative case study in Figure [4](https://arxiv.org/html/2508.00574v1#S6.F4) of Appendix.

Moreover, we evaluate our method under various $\tau$ values.
As shown in Figure [3](https://arxiv.org/html/2508.00574v1#S4.F3)(a), our method consistently outperforms all other baselines, achieving the best accuracy-efficiency trade-off.
As shown in the bottom of Table [1](https://arxiv.org/html/2508.00574v1#S4.T1), we observe a significant performance decline when either Synthetic CCoT or Iterative Refinement is removed, which further highlights the importance of both components.

### 4.2 Evaluation of Difficulty Classifier Performance

Figure: Figure 3: (a) Accuracy-efficiency trade-off comparison between our method and other efficient reasoning baselines. (b). Difficulty ratio (The ratio of hard questions) of our method under different $\tau$ values across five benchmarks. (c/d). Accuracy under various difficulty ratios using different hard question identification methods on the MATH500 and MixD Datasets.
Refer to caption: x3.png

#### Experimental Settings

To evaluate the performance of our difficulty classifier, we use the MATH500 dataset, treating questions with a difficulty level of 5 as hard and the rest as easy.
Additionally, we construct the MixD dataset by combining AIME25/AIME24/AMC23 and part of GSM8K. Questions from AIME25/AIME24/AMC23 are considered hard, while those from GSM8K are regarded as easy.
We report macro precision (Pre), macro recall (Rec), and macro F1 (F1) of the hard question identification.
We also report the accuracy of our method using different identification approaches, maintaining the same ratio of hard questions.

#### Compared Methods

To demonstrate the effectiveness of our difficulty classifier, we consider several baselines for comparison: Seq_PPL (Mahaut et al. 2024) computes the PPL score for each question, treating those with high PPL as hard and others as easy.
PromptLLM (Han et al. 2024) directly prompts the LLM to assess question difficulty.
RouteLLM (Ong et al. 2024) trains an additional BERT model to judge question difficulty.
We directly use their released weights.
Probe_Q (Azaria and Mitchell 2023) trains a simple classifier, consisting of two MLP layers, to assess difficulty based solely on the question.

More details about the used datasets and the compared baselines are present in Appendix [6.5](https://arxiv.org/html/2508.00574v1#S6.SS5) and [6.6](https://arxiv.org/html/2508.00574v1#S6.SS6), respectively.

#### Main Results

As shown in Table [2](https://arxiv.org/html/2508.00574v1#S4.T2), our method, which identifies hard questions using both the question and CCoT, outperforms other baselines on both MATH500 and MixD datasets.
Seq_PPL relies solely on the PPL score, which does not strongly correlate with question difficulty.
PromptLLM prompts the LLM to assess difficulty, but this approach is unreliable due to the model’s limitations in identifying hard questions.
RouteLLM trains an additional BERT-based classifier, which incurs extra costs and struggles to effectively identify complex math questions requiring reasoning.
Probe_Q trains a classifier based only on the question, which can identify explicit hard questions but misses those that look simple but actually hard.
In contrast, our method can effectively identify those hard questions by using the reasoning information in corresponding CCoT.
As shown in Figure [3](https://arxiv.org/html/2508.00574v1#S4.F3) (b), it accurately identifies most difficult questions, such as those in AIME25/24, and AMC23.

Moreover, we also report the impact of different identification methods on overall performance in Figure [3](https://arxiv.org/html/2508.00574v1#S4.F3) (c/d).
We evaluate the problem-solving accuracy when using these methods under different difficulty ratios.
As shown in Figure [3](https://arxiv.org/html/2508.00574v1#S4.F3) (c/d), at the same difficulty ratio, our method can more accurately identify hard questions, route them for re-thinking, and achieve the best accuracy on both the MATH500 and MixD datasets.
However, we observe a decrease in accuracy when the difficulty ratio exceeds 0.6.
This is because easy questions are also routed for re-thinking, and excessive reasoning for simple questions will confuse the model, leading to incorrect answers.

**Table 2: Comparison of SynAdapt and those baselines for hard question identification on MATH500 and MixD Datasets. Bold and underlined numbers indicate the best and second-best results, respectively.**
| Methods | MATH500 | MixD |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Pre | Rec | F1 | Pre | Rec | F1 |  |
| Seq_PPL | 37.46 | 35.27 | 36.10 | 28.93 | 23.70 | 25.51 |
| PromptLLM | 45.83 | 47.01 | 45.86 | 49.95 | 49.94 | 48.47 |
| RouteLLM | 46.03 | 47.27 | 31.21 | 13.37 | 48.00 | 20.91 |
| Probe_Q | 73.24 | 58.75 | 58.90 | 70.95 | 74.66 | 63.81 |
| SynAdpat | 79.47 | 62.42 | 63.11 | 62.71 | 81.02 | 78.32 |

### 4.3 Analysis of Training Efficiency

To evaluate training efficiency, we report the training cost of our method and other CCoT-based methods.
As shown in Table [3](https://arxiv.org/html/2508.00574v1#S4.T3), our method offers comparable efficiency to the baselines.
While SynAdapt introduces additional synthetic CCoT generation, this process is highly efficient, accounting for only 9.89% of the total training cost.
Single CCoT generation only requires 10 seconds, which is very fast.

CompressCoT and CODI require autoregressive generation of CCoT during fine-tuning, leading to high training costs and low efficiency.
Coconut gradually internalizes DCoT, and since the initial CCoT length is small, the training cost is relatively low.
However, in the later stages, the cost still increases due to autoregressive generation.
In contrast, SynAdapt iteratively refines a draft CCoT rather than generating it autoregressively, effectively improving efficiency.
Therefore, our method achieves high training efficiency, demonstrating its practicality.

**Table 3: Training time costs for different CCoT-based methods. We use a batch size (bs) of 16 during synthetic CCoT generation.**
| Modules | Time (min) | Percentage |
| --- | --- | --- |
| Coconut | 740 | - |
| CompressCoT | 1192 | - |
| CODI | 1156 | - |
| SynAdapt | 1021 | 100% |
| LLM Training | 920 | 90.11% |
| Synthetic CCoT Generation (bs=16) | 101 | 9.89% |
| $\Rightarrow$ Single Synthetic CCoT Generation | 10s | 0.02% |

### 4.4 More backbones and Hyperparmeter analysis

To demonstrate the generalization ability of our method, we further evaluate it on more LLM backbones, such as DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-Distill-Qwen-1.5B.
As shown in Table [4](https://arxiv.org/html/2508.00574v1#S4.T4), when using $\tau=0.5$ to identify hard questions for rethinking, our method achieves comparable performance to the raw model while reducing generation length.
Even when using $\tau=1.0$, which means no rethinking of any questions, our SynAdapt still outperforms all other CCoT-based baselines.

We also conduct the hyperparameter analysis on CCoT length $m$ and refining iterations $k$.
As shown in Figures [7](https://arxiv.org/html/2508.00574v1#S6.F7) and [8](https://arxiv.org/html/2508.00574v1#S6.F8) in the Appendix, our method remains effective and robust across various hyperparameter settings.
Due to page limitations, more analyses and results are in Appendix [6.8](https://arxiv.org/html/2508.00574v1#S6.SS8).

**Table 4: More comparisons between our methods and those CCoT-based baselines on DeepSeek-R1-Distill-Llama-8B and DeepSeek-R1-Distill-Qwen-1.5B backbones. Acc and Len denote the average accuracy and generation length across all five benchmarks.**
| Methods | R1-Llama-8B | R1-Qwen-1.5B |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Acc $\uparrow$ | Len $\downarrow$ | Rel-G $\uparrow$ | Acc $\uparrow$ | Len $\downarrow$ | Rel-G $\uparrow$ |  |
| Raw Model | 67.2 | 7998.4 | 1.00 | 57.6 | 9166.2 | 1.00 |
| SynAdapt($\tau$=0.5) | 66.1 | 6406.2 | 1.23 | 57.3 | 8836.5 | 1.03 |
| Coconut | 45.5 | 572.6 | 9.46 | 39.6 | 1767.1 | 3.57 |
| CompressCoT | 44.6 | 1834.3 | 2.89 | 38.2 | 1166.0 | 5.21 |
| CODI | 38.3 | 488.2 | 9.34 | 40.1 | 1566.5 | 4.07 |
| SynAdapt($\tau$=1.0) | 48.0 | 582.7 | 9.80 | 42.1 | 690.8 | 9.70 |

## 5 Conclusion

We propose a novel and efficient reasoning framework, SynAdapt, designed to help LLMs learn continuous CoT (CCoT).
Before fine-tuning, we generate the synthetic CCoT, which serves as a more effective alignment target for learning CCoT.
Additionally, we train a difficulty classifier that identifies hard questions by considering both the question and its corresponding CCoT.
By dynamically prompting the LLM to re-think hard questions, our method can adapt to both accuracy-sensitive and efficiency-sensitive scenarios.
Extensive experimental results across various benchmarks consistently demonstrate the effectiveness of SynAdapt for efficient reasoning.

## References

- Aggarwal and Welleck (2025)
Aggarwal, P.; and Welleck, S. 2025.
L1: Controlling how long a reasoning model thinks with reinforcement learning.
*arXiv preprint arXiv:2503.04697*.
- Arora and Zanette (2025)
Arora, D.; and Zanette, A. 2025.
Training language models to reason efficiently.
*arXiv preprint arXiv:2502.04463*.
- Azaria and Mitchell (2023)
Azaria, A.; and Mitchell, T. 2023.
The internal state of an LLM knows when it’s lying.
*arXiv preprint arXiv:2304.13734*.
- Cheng and Van Durme (2024)
Cheng, J.; and Van Durme, B. 2024.
Compressed chain of thought: Efficient reasoning through dense representations.
*arXiv preprint arXiv:2412.13171*.
- Cobbe et al. (2021)
Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J. 2021.
Training Verifiers to Solve Math Word Problems.
*arXiv preprint arXiv:2110.14168*.
- Deng, Choi, and Shieber (2024)
Deng, Y.; Choi, Y.; and Shieber, S. 2024.
From explicit cot to implicit cot: Learning to internalize cot step by step.
*arXiv preprint arXiv:2405.14838*.
- Dong et al. (2024)
Dong, Y.; Jiang, X.; Liu, H.; Jin, Z.; Gu, B.; Yang, M.; and Li, G. 2024.
Generalization or memorization: Data contamination and trustworthy evaluation for large language models.
*arXiv preprint arXiv:2402.15938*.
- Goyal et al. (2023)
Goyal, S.; Ji, Z.; Rawat, A. S.; Menon, A. K.; Kumar, S.; and Nagarajan, V. 2023.
Think before you speak: Training language models with pause tokens.
*arXiv preprint arXiv:2310.02226*.
- Guo et al. (2025)
Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; et al. 2025.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*.
- Han et al. (2024)
Han, T.; Wang, Z.; Fang, C.; Zhao, S.; Ma, S.; and Chen, Z. 2024.
Token-budget-aware llm reasoning.
*arXiv preprint arXiv:2412.18547*.
- Hao et al. (2024)
Hao, S.; Sukhbaatar, S.; Su, D.; Li, X.; Hu, Z.; Weston, J.; and Tian, Y. 2024.
Training large language models to reason in a continuous latent space.
*arXiv preprint arXiv:2412.06769*.
- He et al. (2025)
He, Z.; Liang, T.; Xu, J.; Liu, Q.; Chen, X.; Wang, Y.; Song, L.; Yu, D.; Liang, Z.; Wang, W.; et al. 2025.
Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning.
*arXiv preprint arXiv:2504.11456*.
- Hu et al. (2022)
Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022.
Lora: Low-rank adaptation of large language models.
*ICLR*, 1(2): 3.
- Jiang et al. (2024)
Jiang, A. Q.; Sablayrolles, A.; Roux, A.; Mensch, A.; Savary, B.; Bamford, C.; Chaplot, D. S.; Casas, D. d. l.; Hanna, E. B.; Bressand, F.; et al. 2024.
Mixtral of experts.
*arXiv preprint arXiv:2401.04088*.
- Jiang et al. (2025)
Jiang, N.; Wu, Z.; Zhan, D.-C.; Lai, F.; and Lian, S. 2025.
DART: Distilling Autoregressive Reasoning to Silent Thought.
*arXiv preprint arXiv:2506.11752*.
- Kaddour et al. (2023)
Kaddour, J.; Harris, J.; Mozes, M.; Bradley, H.; Raileanu, R.; and McHardy, R. 2023.
Challenges and applications of large language models.
*arXiv preprint arXiv:2307.10169*.
- Kang et al. (2025)
Kang, Y.; Sun, X.; Chen, L.; and Zou, W. 2025.
C3ot: Generating shorter chain-of-thought without compromising effectiveness.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 39, 24312–24320.
- Koehn and Knowles (2017)
Koehn, P.; and Knowles, R. 2017.
Six challenges for neural machine translation.
*arXiv preprint arXiv:1706.03872*.
- Kojima et al. (2022)
Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y. 2022.
Large language models are zero-shot reasoners.
*Advances in neural information processing systems*, 35: 22199–22213.
- Lee, Che, and Peng (2025)
Lee, A.; Che, E.; and Peng, T. 2025.
How well do llms compress their own chain-of-thought? a token complexity approach.
*arXiv preprint arXiv:2503.01141*.
- Li et al. (2024a)
Li, M.; Zhao, S.; Wang, Q.; Wang, K.; Zhou, Y.; Srivastava, S.; Gokmen, C.; Lee, T.; Li, E. L.; Zhang, R.; et al. 2024a.
Embodied agent interface: Benchmarking llms for embodied decision making.
*Advances in Neural Information Processing Systems*, 37: 100428–100534.
- Li et al. (2024b)
Li, Z.; Liu, H.; Zhou, D.; and Ma, T. 2024b.
Chain of thought empowers transformers to solve inherently serial problems.
*arXiv preprint arXiv:2402.12875*, 1.
- Lightman et al. (2023)
Lightman, H.; Kosaraju, V.; Burda, Y.; Edwards, H.; Baker, B.; Lee, T.; Leike, J.; Schulman, J.; Sutskever, I.; and Cobbe, K. 2023.
Let’s Verify Step by Step.
*arXiv preprint arXiv:2305.20050*.
- Lin et al. (2024)
Lin, Z.; Liang, T.; Xu, J.; Lin, Q.; Wang, X.; Luo, R.; Shi, C.; Li, S.; Yang, Y.; and Tu, Z. 2024.
Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM’s Reasoning Capability.
*arXiv preprint arXiv:2411.19943*.
- Liu et al. (2024)
Liu, L.; Yang, X.; Lei, J.; Shen, Y.; Wang, J.; Wei, P.; Chu, Z.; Qin, Z.; and Ren, K. 2024.
A survey on medical large language models: Technology, application, trustworthiness, and future directions.
*arXiv preprint arXiv:2406.03712*.
- Lu et al. (2024)
Lu, C.; Lu, C.; Lange, R. T.; Foerster, J.; Clune, J.; and Ha, D. 2024.
The ai scientist: Towards fully automated open-ended scientific discovery.
*arXiv preprint arXiv:2408.06292*.
- Luo et al. (2025)
Luo, H.; Shen, L.; He, H.; Wang, Y.; Liu, S.; Li, W.; Tan, N.; Cao, X.; and Tao, D. 2025.
O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning.
*arXiv preprint arXiv:2501.12570*.
- Ma et al. (2025a)
Ma, W.; He, J.; Snell, C.; Griggs, T.; Min, S.; and Zaharia, M. 2025a.
Reasoning models can be effective without thinking.
*arXiv preprint arXiv:2504.09858*.
- Ma et al. (2025b)
Ma, X.; Wan, G.; Yu, R.; Fang, G.; and Wang, X. 2025b.
Cot-valve: Length-compressible chain-of-thought tuning.
*arXiv preprint arXiv:2502.09601*.
- Mahaut et al. (2024)
Mahaut, M.; Aina, L.; Czarnowska, P.; Hardalov, M.; Müller, T.; and Màrquez, L. 2024.
Factual confidence of llms: on reliability and robustness of current estimators.
*arXiv preprint arXiv:2406.13415*.
- mathai. (2024)
mathai. 2024.
AIME2025 dataset.
https://huggingface.co/datasets/math-ai/aime25.
- Maxwell-Jia. (2024)
Maxwell-Jia. 2024.
AIME2024 dataset.
https://huggingface.co/datasets/Maxwell-Jia/AIME˙2024.
- Munkhbat et al. (2025)
Munkhbat, T.; Ho, N.; Kim, S. H.; Yang, Y.; Kim, Y.; and Yun, S.-Y. 2025.
Self-training elicits concise reasoning in large language models.
*arXiv preprint arXiv:2502.20122*.
- Nalewajski (2011)
Nalewajski, R. F. 2011.
Elements of information theory.
In *Perspectives in Electronic Structure Theory*, 371–395. Springer.
- Ong et al. (2024)
Ong, I.; Almahairi, A.; Wu, V.; Chiang, W.-L.; Wu, T.; Gonzalez, J. E.; Kadous, M. W.; and Stoica, I. 2024.
Routellm: Learning to route llms with preference data.
*arXiv preprint arXiv:2406.18665*.
- OpenAI (2025)
OpenAI. 2025.
Learning to reason with LLMs.
- Pan et al. (2024)
Pan, Z.; Wu, Q.; Jiang, H.; Xia, M.; Luo, X.; Zhang, J.; Lin, Q.; Rühle, V.; Yang, Y.; Lin, C.-Y.; et al. 2024.
Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression.
*arXiv preprint arXiv:2403.12968*.
- Pfau, Merrill, and Bowman (2024)
Pfau, J.; Merrill, W.; and Bowman, S. R. 2024.
Let’s think dot by dot: Hidden computation in transformer language models.
*arXiv preprint arXiv:2404.15758*.
- Rasley et al. (2020)
Rasley, J.; Rajbhandari, S.; Ruwase, O.; and He, Y. 2020.
Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters.
In *Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining*, 3505–3506.
- Renze and Guven (2024)
Renze, M.; and Guven, E. 2024.
The benefits of a concise chain of thought on problem-solving in large language models.
In *2024 2nd International Conference on Foundation and Large Language Models (FLLM)*, 476–483. IEEE.
- Saunshi et al. (2025)
Saunshi, N.; Dikkala, N.; Li, Z.; Kumar, S.; and Reddi, S. J. 2025.
Reasoning with latent thoughts: On the power of looped transformers.
*arXiv preprint arXiv:2502.17416*.
- Shen et al. (2025a)
Shen, Y.; Zhang, J.; Huang, J.; Shi, S.; Zhang, W.; Yan, J.; Wang, N.; Wang, K.; Liu, Z.; and Lian, S. 2025a.
Dast: Difficulty-adaptive slow-thinking for large reasoning models.
*arXiv preprint arXiv:2503.04472*.
- Shen et al. (2025b)
Shen, Z.; Yan, H.; Zhang, L.; Hu, Z.; Du, Y.; and He, Y. 2025b.
Codi: Compressing chain-of-thought into continuous space via self-distillation.
*arXiv preprint arXiv:2502.21074*.
- Wei et al. (2022)
Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022.
Chain-of-thought prompting elicits reasoning in large language models.
*Advances in neural information processing systems*, 35: 24824–24837.
- Xia et al. (2025)
Xia, H.; Leong, C. T.; Wang, W.; Li, Y.; and Li, W. 2025.
Tokenskip: Controllable chain-of-thought compression in llms.
*arXiv preprint arXiv:2502.12067*.
- Xu et al. (2025a)
Xu, S.; Xie, W.; Zhao, L.; and He, P. 2025a.
Chain of draft: Thinking faster by writing less.
*arXiv preprint arXiv:2502.18600*.
- Xu et al. (2025b)
Xu, Y.; Guo, X.; Zeng, Z.; and Miao, C. 2025b.
Softcot: Soft chain-of-thought for efficient reasoning with llms.
*arXiv preprint arXiv:2502.12134*.
- Yang et al. (2025)
Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; et al. 2025.
Qwen3 technical report.
*arXiv preprint arXiv:2505.09388*.
- Yeo et al. (2025)
Yeo, E.; Tong, Y.; Niu, M.; Neubig, G.; and Yue, X. 2025.
Demystifying long chain-of-thought reasoning in llms.
*arXiv preprint arXiv:2502.03373*.
- Yu et al. (2024)
Yu, P.; Xu, J.; Weston, J.; and Kulikov, I. 2024.
Distilling system 2 into system 1.
*arXiv preprint arXiv:2407.06023*.
- Yu et al. (2025)
Yu, Q.; He, Z.; Li, S.; Zhou, X.; Zhang, J.; Xu, J.; and He, D. 2025.
Enhancing Auto-regressive Chain-of-Thought through Loop-Aligned Reasoning.
*arXiv preprint arXiv:2502.08482*.
- Zhou et al. (2022)
Zhou, D.; Schärli, N.; Hou, L.; Wei, J.; Scales, N.; Wang, X.; Schuurmans, D.; Cui, C.; Bousquet, O.; Le, Q.; et al. 2022.
Least-to-most prompting enables complex reasoning in large language models.
*arXiv preprint arXiv:2205.10625*.
- Zhu et al. (2025)
Zhu, H.; Hao, S.; Hu, Z.; Jiao, J.; Russell, S.; and Tian, Y. 2025.
Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought.
*arXiv preprint arXiv:2505.12514*.
- zwhe99. (2024)
zwhe99. 2024.
AMC23 dataset.
https://huggingface.co/datasets/zwhe99/amc23.

## 6 Appendix

Appendix Overview: The appendix is organized into two main parts:
Appendices 6.1–6.6 provide detailed related works and more experimental setup of our SynAdapt.
Appendices 6.7–6.9 present additional experimental results, further demonstrating the effectiveness of our SynAdapt.

### 6.1 Details of Related Work

In this section, we provide a detailed overview of related works on LLM efficient reasoning, which can be broadly categorized into four main types: SFT-based methods, RL-based methods,Prompt-based methods, and CCoT-based methods.

For SFT-based methods, Yu et al. (2024) proposes to collect CoT and answer data from reasoning LLMs and directly discard the CoT part. And then they fine-tune the LLM using only the answers to help the model reduce reasoning length.
Ma et al. (2025b) fine-tunes the LLM simultaneously on data with CoT and data without CoT, using specific instructions to distinguish between the two.
During inference, they use the instructions to prevent the model from outputting CoT.
Munkhbat et al. (2025) applies best-of-n sampling to LLM, selecting the shortest CoT, and fine-tune the model on these short CoTs to reduce reasoning length.
Xia et al. (2025) assesses the semantic importance of tokens in the initial CoT, retaining only the most important tokens for fine-tuning the LLM.
Kang et al. (2025) dynamically samples simplified CoTs from the model after each fine-tuning epoch for the next round of fine-tuning.

All of the above methods either discard the CoT or use a simplified version for fine-tuning the LLM to reduce reasoning length. While these approaches effectively shorten the reasoning length, they overlook important details in the original CoT, leading to significant performance degradation during further fine-tuning.

For RL-based methods, Arora and Zanette (2025) introduces a length-based reward, where shorter correct answers receive higher rewards, and uses policy gradient (PG) methods to fine-tune the LLM to reduce reasoning length.
Luo et al. (2025) enhances this reward by comparing the generated answer length to a reference answer and applies PPO optimization for LLM fine-tuning.
Yeo et al. (2025) further introduces a cosine-based reward and applies a penalty for exceeding the length limit.
Aggarwal and Welleck (2025) uses length-constrained prompts to sampling data during RL fine-tuning.
Shen et al. (2025a) employs SimPO to fine-tune the LLM using a length-preference dataset.

Although these RL-based methods can reduce reasoning length to some extent while maintaining LLM performance, RL fine-tuning requires significant resources.
For example, they need to repeatedly sample new data for updating the action of LLM.
Moreover, the reduction in length is limited and cannot be applied to those efficiency-sensitive scenarios.
For instance, in real-life medical QA scenarios, efficiency is critical. Diagnosis advice must be concise, enabling doctors and patients to quickly access key details and conclusions, especially in emergencies. Previous studies (Kaddour et al. 2023; Liu et al. 2024) have highlighted that overly long responses can lead to errors, such as confusing similar drug names or omitting critical contraindications.

For Prompt-based methods, Renze and Guven (2024) propose to prompt the LLM to perform CoT reasoning while explicitly instructing it to be concise.
Xu et al. (2025a) focuses on adding instructions in the prompt to condense each reasoning step and limit verbosity.
Lee, Che, and Peng (2025) explores various prompt types to reduce reasoning length, such as prompting to output only numbers or only use bullet points.
Han et al. (2024) estimates a token budget for each question, allocating more tokens for harder questions, and instructing the LLM to stay within this budget during reasoning for efficiency.

Most prompt-based methods reduce reasoning length by adding additional length constraint instructions in the prompt.
While this approach is low-cost, its impact on reducing length is limited.
LLMs still tend to generate redundant reasoning CoTs, especially when faced with hard questions.

For CCoT-based methods, Hao et al. (2024) was the first to propose to fine-tune the LLM to reason continuously and utilize the last hidden state as the continuous CoT (CCoT) to replace traditional discrete CoT (DCoT), which often contain redundant tokens.
They introduce curriculum learning to gradually replace DCoT with CCoT during fine-tuning, without explicit alignment with the original DCoT.
Xu et al. (2025b) is similar to Coconut, but it incorporates an additional assistant LLM with a projection module to generate the CCoT.
Although it provides slight improvements, it also incurs additional resource costs.
Shen et al. (2025b) employs self-distillation to learn CCoT by simultaneously fine-tuning on both DCoT and CCoT and explicitly aligns the last token hidden state between the two.
Cheng and Van Durme (2024) measures token importance in advance and aligns the CCoT only with the hidden states of those important tokens in the DCoT.

Current CCoT-based methods can successfully compress reasoning steps into a latent space, replacing the original DCoT with a more efficient CCoT and significantly reducing generation length.
However, they often suffer from unsatisfactory performance degradation.
This is mainly because they either do not apply explicit alignment between DCoT and CCoT or only use partial DCoT (e.g., the last token or a subset of important tokens) to supervise CCoT learning.
These weak supervisory signals fail to help LLM to learn a well CCoT representation, leading to significant performance drops.
Therefore, designing stronger supervisory signals for CCoT learning is crucial for real-world applications.

### 6.2 Dataset Details for Trade-off Evaluation

For the training set, we use the DeepMath-103K dataset (He et al. 2025), which contains numerous math problems with three distinct reasoning paths from DeepSeek-R1 (Guo et al. 2025), covering various math topics and difficulty levels.
For each question, we randomly select one reasoning path as the discrete CoT and exclude samples with reasoning paths exceeding 12,000 tokens.
Moreover, as pointed out by Dong et al. (2024), the public datasets, containing numerous samples, suffer from a ’data contamination’ issue, where some samples may be similar to evaluation benchmark.
Directly training on this data may cause the model to memorize these samples, leading to unnaturally high performance.
Additionally, including too many training samples introduces excessive training costs, which contradicts our goal of high efficiency.
Therefore, we only sample a portion of the original DeepMath-103K dataset for training. Specifically, we randomly sample 10% of the training samples for each difficulty level to create the final DeepMath dataset, ensuring the distribution of question difficulty remains consistent.
The total size of the DeepMath dataset is 9,660.

For the test set, we consider several widely adopted math-related benchmarks: AIME25 (mathai. 2024), AIME24 (Maxwell-Jia. 2024), AMC23 (zwhe99. 2024), MATH500 (Lightman et al. 2023), and GSM8K (Cobbe et al. 2021). The difficulty of these benchmarks gradually decreases, covering a wide range from complex math competitions to simple grade school math. The details of both the train and test dataset sizes are shown in Table [5](https://arxiv.org/html/2508.00574v1#S6.T5).

**Table 5: The size of our used train dataset and five math-related evaluation benchmarks, covering various difficulty levels.**
| Train Dataset | Test Dataset |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| DeepMath | AIME25 | AIME24 | AMC23 | MATH500 | GSM8K |
| 9660 | 30 | 30 | 40 | 500 | 1319 |

### 6.3 Baselines Details for Trade-off Evaluation

Here, we provide more details about all the compared efficient reasoning baselines. We consider not only CCoT-based baselines but also other SFT-based and prompt-based methods.
We exclude RL-based methods, as these require substantial resources to apply RL learning to LLMs, making them inefficient and impractical for real-world applications.

We further mainly categorize these baselines into two scenarios based on their focus.
Baselines for the accuracy-sensitive scenario primarily aim to maintain performance while shortening the generation length.
Here are the details of these baselines:

CoT-FT belongs to SFT-based methods. We directly uses the CoT and answers from the training set, to supervise fine-tune (SFT) the LLM. This method aims to maintain accuracy while slightly reducing the generation length.

TokenSkip (Xia et al. 2025) belongs to SFT-based methods.
As proposed by TokenSkip, different tokens in the CoT have varying semantic importance, and tokens with low semantic value can be skipped during SFT of the LLM.
Specifically, we use LLMLingua-2 (Pan et al. 2024) to assess the importance of each token and obtain a compressed CoT.
We set the compression ratio to 0.7 because too low ratio will make the CoT inconsistent for fine-tuning while too low ratio only provides a slight reduction in generation length.
We utilize the compressed CoT along with corresponding answer to fine-tune LLM to reduce generation length while maintaining performance.

NoThinking Ma et al. (2025a) is a prompt-based method.
NoThinking proposes to directly prompt the LLM to avoid generating a CoT, which effectively reduces the generation length with fine-tuning process.
Specifically, we append the instruction “Okay, I think I have finished thinking.</think>” to the initial prompt, instructing the LLM to skip reasoning and directly output the answer without CoT.

CoD (Xu et al. 2025a) is another prompt-based method.
Different from NoThinking directly prompts LLM to skip reasoning and do not output CoT, Chain-of-Draft (CoD) preserves the reasoning process but condenses each reasoning step by inserting the “only keep a minimum draft for each thinking step, with 5 words at most.” instruction.

TokenBudget Han et al. (2024) is also a prompt-based method.
Following TokenBudget, we prompt the LLM in advance to estimate the difficulty of each question and determine the essential token budget.
During inference, we incorporate the token budget into the initial prompt by adding the instruction, “Let’s think step by step and use fewer than [[Token Budget]] tokens”, guiding the LLM to reduce unnecessary generation.

In contrast, baselines for the efficiency-sensitive scenario prioritize improving efficiency, even at the cost of performance.
Here are the details of these baselines:

NoCoT-FT Yu et al. (2024) is an SFT-based method.
However, unlike previous SFT-based methods, NoCoT-FT distills the ability from the reasoning model to the model that does not output any CoT, by fine-tuning solely on the answer part from the reasoning model.
Specifically, we discard the CoT part in our training set and fine-tune the LLM only with the answer.

SelfTraining Munkhbat et al. (2025) is another SFT-based method.
As proposed by SelfTraining, we apply best-of-n sampling to the LLM to generate multiple answers for each question, then select the shortest correct answer to fine-tune the LLM and reduce generation length.
During sampling, we also provide demonstrations as few-shots to instruct the LLM to generate the answer directly without CoT.
The sampled answers are then used to fine-tune the LLM to skip the CoT.

Coconut (Hao et al. 2024) is one of CCoT-based methods.
According to Coconut, we apply curriculum learning to help the LLM gradually learn Continous CoT (CCoT).
Specifically, we fine-tune the LLM for 3 epochs, gradually reducing the initial DCoT tokens to none as the epochs progress, and replacing them with CCoT.
Finally, we can internalize the DCoT into the CCoT.

CompressCoT (Cheng and Van Durme 2024) belongs to CCoT-based methods
Following CompressCoT, we first identify important tokens in the discrete CoT using LLMLingua-2 (Pan et al. 2024) and compute the mid-layer hidden states of these tokens as the target.
We then fine-tune the LLM with the LoRA module to generate the CCoT similar to target.
Simultaneously, we fine-tune another LoRA module to predict the correct answer based on the CCoT.
During inference, we first use the prior LoRA module to generate the CCoT and then use the other LoRA module to generate the answer based on it.

CODI (Shen et al. 2025b) is another CCoT-based methods.
As proposed by CODI, we fine-tune the LLM with two tasks: the teacher task, which generates the discrete CoT tokens and the final correct answer, and the student task, which generates the CCoT and the correct answer.
We then explicitly align the last token hidden states from the DCoT and CCoT to achieve self-distillation from DCoT to CCoT.

### 6.4 Implementation Details of our SynAdapt

We adopt the DeepSeek-R1-Distill-Qwen-7B (Guo et al. 2025) as the LLM backbone and we also evaluate the our method on other backbones in Section [4.4](https://arxiv.org/html/2508.00574v1#S4.SS4).
For the Synthetic CCoT generation, we fix the LLM backbone and make the randomly initialized synthetic CCoT to be trainable.
The length of synthetic CCoT is set as $m=512$ and the we optimize it using the learning rate at 1e-3 for 32 steps.
During optimization, we use a batch size of 16 to ensure high efficiency.

For Synthetic CCoT Enhanced Fine-tuning, we use LoRA (Hu et al. 2022) to fine-tune the LLM for learning CCoT.
The lora rank is set to be 8 and the alpha value at 32.
We use the Deepspeed (Rasley et al. 2020) framework to fine-tune the LLM.
We fine-tine LLM for 3 epochs with a batch size of 1 and a gradient accumulation step of 16.
We employ the AdamW optimizer with a learning rate set to 4e-5.
The refinement steps of the draft is $k=4$ and the length of CCoT is also $m=512$.
We also analyze these hyperparameters in Section [4.4](https://arxiv.org/html/2508.00574v1#S4.SS4).

For Adaptive Reasoning via CCoT, we firstly generate the CCoT with the length of 512 and use the difficulty classifier to judge the difficulty score $\tau$ based on question and CCoT.
The score ranges from 0 to 1, with scores below the threshold $\tau$ considered as simple, and those above as hard.
For the efficiency-sensitive scenario, we set $\tau=1.0$, treating all questions as simple.
For the accuracy-sensitive scenario, we set $\tau=0.5$ to classify some questions as hard.
We also try more $\tau$ values, as shown in Figure [3](https://arxiv.org/html/2508.00574v1#S4.F3)(a).
During answer generation, we use greedy decoding and set the maximum generation length to 32,768 tokens.
The generation prompt and the prompt for re-thinking hard questions are provided in Appendix [6.9](https://arxiv.org/html/2508.00574v1#S6.SS9).
All our training and evaluation experiments are conducted on the H20 GPU.

### 6.5 Dataset Details for Difficulty Classifier Evaluation

Here, we will introduce the details of the two datasets used to evaluate the hard question identification performance.
For the MATH500 dataset (Lightman et al. 2023), we use the original difficulty labels, which range from 1 to 5, with higher values indicating more difficult questions.
Questions with a difficulty level of 5 are considered hard, while the others are easy.
The detailed statistics are shown in Table [6](https://arxiv.org/html/2508.00574v1#S6.T6).

For the MixD dataset, we combine questions from AIME25, AIME24, and AMC23 to form the hard question set.
Questions from the GSM8K dataset are considered easy.
We random select 20% questions from GSM8K randomly selected to form the easy question set to avoid severe data imbalance problem.
We then mix both the hard and easy questions to create our MixD dataset.
The detailed statistics are shown in Table [6](https://arxiv.org/html/2508.00574v1#S6.T6).

**Table 6: The statistics for our used test dataset used to evaluate hard question identification performance.**
| Dataset | Total Cnt | Number of Hard | Number of Easy |
| --- | --- | --- | --- |
| MATH500 | 500 | 134 | 366 |
| MixD | 363 | 100 | 263 |

### 6.6 Baselines Details for Difficulty Classifier Evaluation

In this section, we provide a more detailed introduction to the baselines for hard question identification as follows:

Seq_PPL (Mahaut et al. 2024) uses sequence probability (PPL) to reflect the confidence of the LLM.
We compute the PPL of the LLM on each question, which is equivalent to the sequence probability of the question.
We treat those questions with high PPL are considered hard, while those with low PPL are categorized as simple.

PromptLLM (Han et al. 2024) prompts the LLM to assess the difficulty of a question and predict the essential token budget required for solving it.
We also prompt the LLM to predict the token budget and restrict the range to 128-32,768 tokens.
Questions that require a high token budget are considered hard, while those with a low token budget are classified as simple.

RouteLLM (Ong et al. 2024) trains a hard question classifier using a BERT backbone.
The classifier assigns high scores to hard questions and routes them to stronger LLMs, such as GPT-4 (OpenAI 2025), while easier questions are processed by weaker LLMs, like Mixtral-8x7B (Jiang et al. 2024).
Therefore, we directly use their released model weights (^1^11https://huggingface.co/routellm/bert_gpt4_augmented) and classify those questions with high scores as hard.

Probe_Q (Azaria and Mitchell 2023) trains a classifier based on the LLM’s hidden state to assess truthfulness.
Similarly, we provide the LLM with the question and train a classifier to evaluate difficulty based on the last token’s hidden state from LLM.
This approach is similar to ours, but it does not leverage information from the CCoT for assessing question difficulty.

### 6.7 Case studies

Figure: Figure 4: An representative example of the generated output from our SynAdapt and other CCoT-based baselines is provided. We highlight the crucial wrong steps that lead to incorrect answers in red, and the correct reasoning steps in green. Redundant parts in the answer are marked with a gray background. We also provide short analyses explaining the disadvantages or advantages of the generated responses.
Refer to caption: x4.png

#### Response Example from Various Baselines

We provide a representative example to demonstrate the effectiveness of our SynAdapt by comparing its generated response with those from other CCoT-based baselines, including Coconut, CompressCoT, and CODI.

As shown in Figure [4](https://arxiv.org/html/2508.00574v1#S6.F4), the response from Coconut contains numerous redundant parts, which primarily serve communication or linguistic purposes, rather than contributing to the reasoning process needed to derive the correct answer.
Moreover, the answer generated is incorrect, highlighting that indirect training without explicit alignment with DCoT fails to effectively learn CCoT.
CompressCoT successfully generates a concise response without redundancy but still outputs the wrong answer.
This is because it aligns only with a subset of isolated, incoherent DCoT tokens, which fail to capture the full reasoning process, resulting in performance degradation.
For CODI, the generated response provides the correct answer but retains redundant parts. This occurs because it applies alignment only at the final position, limiting its ability to learn CCoT and produce concise output.

In contrast, our method generates both a concise and correct answer. This is due to our use of synthetic CCoT as the fine alignment target and applying full alignment during CCoT fine-tuning. These results strongly demonstrate the effectiveness of our method for efficient reasoning.

#### CCoT for Hard Question Example

We provide an illustrative example demonstrating that solely relying on CCoT is insufficient to solve hard questions.
As shown in Figure [6](https://arxiv.org/html/2508.00574v1#S6.F6), when the LLM relies only on CCoT, it generates a concise but incorrect answer.
It may be because CCoT restricts the LLM’s ability to verify reasoning steps, confining it to the incorrect answer.
However, when prompted to re-think the question, the LLM can rectify the previous mistake and derive the right answer.
This effectively demonstrates that compressing DCoT into CCoT inevitably results in information loss, limiting the model’s reflective ability and leading to incorrect answers.

Figure: Figure 5: An illustrative example of solving hard question relying solely on CCoT or rethinking via DCoT. We highlight the crucial wrong steps that lead to incorrect answers in red, and the correct reasoning steps in green.
Refer to caption: x5.png

#### Indistinguishable Hard Question Example

We provide an illustrative example showing how some hard questions are similar to simple ones, making them difficult to distinguish.
As seen in Figure [6](https://arxiv.org/html/2508.00574v1#S6.F6), both the easy and hard questions are very similar, both focusing on the quaternions topic and are short in length.
If we only assess difficulty based on the question itself, both would be categorized as easy, leading to performance degradation.

However, when considering the CoT process, there exist significant differences.
For the easy question, the CoT is short and easily leads to the correct answer. In contrast, the hard question involves more reasoning steps and a longer CoT.
By incorporating both the CoT and the question, we can accurately identify these indistinguishable hard questions.
This highlights the value of reasoning information in identifying hard questions.
This is also why our difficulty classifier is build up on both the question and CCoT, which can effectively utilize the information in CCoT.

Figure: Figure 6: An illustrative example of an easy question and the similar hard question, with their corresponding CoT processes. We also present the identification results using only the question or both the question and CoT. The key differences from the CoT of the hard question, compared to the easy question, are highlighted in red color.
Refer to caption: x6.png

### 6.8 Hyperparameter Analysis

#### The Length of CCoT m m italic_m .

We analyze the hyperparameter $m$ in our method, which controls the length of CCoT.
As shown in Figure [7](https://arxiv.org/html/2508.00574v1#S6.F7)(a), as the value of $m$ increases, both accuracy and generation length both rise.
To further measure the balance between high accuracy and low generation length, we compute the Rel-G score to capture the actual performance gain, as shown in Equation [9](https://arxiv.org/html/2508.00574v1#S4.E9).
As illustrated in Figure [7](https://arxiv.org/html/2508.00574v1#S6.F7)(b), $m=512$ achieves the best Rel-G score, indicating the optimal accuracy-efficiency trade-off.

Figure: Figure 7: The performance of our methods when using different CCoT Length $m$. We report the average accuracy, average generation length and the Rel-G score across five benchmarks.
Refer to caption: x7.png

#### The Refining Iterations of CCoT k k italic_k .

We also analyze the hyperparameter $k$ in our method, which controls the refining iterations of CCoT.
As shown in Figure [8](https://arxiv.org/html/2508.00574v1#S6.F8)(a), in the initial stage, as $k$ increases, accuracy improves while generation length decreases.
However, when $k$ exceeds 4, accuracy starts to drop and generation length increases.
To assess the accuracy-efficiency trade-off, we compute the Rel-G score.
As shown in Figure [8](https://arxiv.org/html/2508.00574v1#S6.F8)(b), our method achieves the best Rel-G score at $k=4$, indicating the optimal trade-off.

Figure: Figure 8: The performance of our methods when using different refining iterations $k$ for CCoT generation. We report the average accuracy, average generation length and the Rel-G score across five benchmarks.
Refer to caption: x8.png

### 6.9 Used Prompt Templates

In this section, we present the prompts used in our method.
For easy questions, we directly prompt the LLM to generate an answer based on the CCoT, as shown in Figure [9](https://arxiv.org/html/2508.00574v1#S6.F9).
For hard questions, we prompt the LLM to re-think and generate discrete CoT, condensing each reasoning step, as illustrated in Figure [10](https://arxiv.org/html/2508.00574v1#S6.F10).

Figure: Figure 9: The prompt used for directly generating answers based on the CCoT.
Refer to caption: x9.png

Figure: Figure 10: The prompt used for re-thinking hard questions via discrete CoT process while condensing each CoT step.
Refer to caption: x10.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: See Weaknesses.
soundness: 3
strengths: 1. Novel, tightly-coupled design: The three-stage pipeline (find optimal CCoT for LLM A → train LLM B to mimic it → deploy B+A) is novel and interesting.

2. The authors conduct extensive experiments to prove the superiority of SynAdapt against baselines.
summary: The paper introduces SynAdapt, a framework that synthesises CCoT for a target reasoning LLM, where a adapter LLM is trained iteratively to predict a good CCoT needed for reasoning problem solving. The authors show high sequence length efficiency while preserving most accuracy.
weaknesses: > Compute cost is only partially accounted for

Training LLM B requires n iterations and each forward pass concatenates the full current CCoT (length m) with the question.  Complexity O(n*m) is paid during synthesis, yet Table 1 quotes only the final inference length.  If n≈4 and m≈512, the total FLOPs *before seeing a single test example can be already large.  A FLOPs count that includes the iterative stage is needed to argue for true efficiency.

> Performance concerns

Although high token efficiency, the accuracy drop in Table 1 is equally large. Since the “length” column only counts the final tokens for reasoning parts, any claimed trade-off between accuracy and sequence length is meaningless.

---

confidence: 4
contribution: 1
initial_rating: 2
presentation: 2
questions: How do you decide which baselines are for accuracy-sensitive
scenario or efficiency-sensitive scenario? The current division does not make sense to me. For example, tokenbudget is designed to be efficient.  Isn’t accuracy-sensitive scenario for reasoning models that take super long reasoning time to get superior performance? 

What is the length here? Is it the number of tokens?

What is difference between CoT-SFT and raw model? How do you do SFT here? 

What are some particular reasons for refinement? Is it better than directly optimizing  toward the target CCoT?
soundness: 2
strengths: The authors implement wide experiments to evaluate the performance of their method.The proposed method is empirically shown to be effective in reasoning tasks in the author’s setting. The authors compare their method with various baselines on different datasets.
summary: The paper proposes an efficient reasoning framework that replaces long discrete Chain-of-Thought (DCoT) with a compact continuous CoT (CCoT) plus adaptive routing. First, it optimizes a synthetic CCoT per question while lightly aligning hidden states with the DCoT endpoint. Then it fine-tunes the LLM with LoRA to iteratively refine a draft CCoT to match the synthetic target, teaching the model to think in latent space without autoregressively emitting CoT tokens. A difficulty classifier, conditioned on the question and the refined CCoT, routes “hard” items to re-think with discrete CoT and lets “easy” ones answer directly from CCoT. Across math, scientific QA, and coding, SynAdapt reports shorter generations with competitive accuracy.
weaknesses: This paper introduces a quite complex framework based on different components proposed by past works, while it lacks really interesting or important insights/ findings.

The effectiveness of the framework may be questionable. Based on Table 1, when fully using CCoT, the performance is basically the same as directly prompting the model to give the output. The better performance in the accuracy-sensitive scenario is because the full CoT of the model is used. So this naturally questions why we ever need the proposed framework? A difficulty estimator is good enough if it can decide when to directly output the answer and when to use full CoT.

The experiments are only done on a single model. It remains unclear whether the framework is ad-hoc to that model.

There are many parameters to tune to make the framework work, for example, the number of iterations in refinement.  The refinement process also takes time, which is not taken into consideration for efficiency.

The paper implies good CCoT should be distilled step by step from DCoT (this assumption is not very intuitive and lacks evidence). However, the framework itself is not distilling each step from DCoT but last token (which is similar to CoDI). This is a bit against the advantages they claimed in their framework.  The motivation for Synthetic CCoT is not strong or maybe this is because of writing.

---

confidence: 2
contribution: 2
initial_rating: 4
presentation: 3
questions: See weaknesses
soundness: 2
strengths: 1. The paper brings a new perspective to CCoT learning in LLMs, addressing weaknesses of prior partial or indirect alignment approaches. Specifically, SynAdapt’s use of synthetic, explicitly optimized CCoTs as full alignment targets for fine-tuning is clearly articulated and represents a concrete methodological advance.
2. The inclusion of an adaptive, CCoT-informed difficulty classifier is well-motivated and shows robust empirical performance over alternatives that rely on perplexity, prompting, or question-only signals.
summary: This paper introduces SynAdapt, a framework for efficient and adaptive reasoning in LLMs by generating and aligning synthetic continuous Chain-of-Thought (CCoT) representations. The authors propose synthesizing a continuous CoT to serve as an explicit and optimized alignment target, rather than relying on discrete or partial alignments as in prior work. SynAdapt also integrates a difficulty classifier that uses both the input question and its CCoT to identify challenging problems, adaptively prompting the LLM to re-think harder cases using discrete, step-wise reasoning.
weaknesses: 1. Missing Discussion of Key Related Recent Works. [1] also proposed a module to classify the questions based on the questions’ complexity. 
[1] X. Chen, S. Zhou, K. Liang, and X. Liu, “Distilling reasoning ability from large language models with adaptive thinking,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–14, 2025.
2. Clarity and Interpretation of Mathematical Descriptions: 
- the motivation for aligning only the eot token’s hidden states is given for overfitting prevention, but why this is preferable to multi-token or more structured alignments is left vague.
- The reason why iteratively refine is needed is not clearly stated in the text.
- The hidden dimensions such as Zsyn are not marked, which greatly affects the reader's understanding.
3. The authors did not conduct experiments on the sensitivity of the length of the synthetic CCoT, denoted as m. In my opinion, the value of m should have a significant impact on the performance of CCoT. Moreover, since m is set to a fixed value, I wonder whether this is appropriate for chain-of-thought reasoning, whose information content can vary greatly across different problems.
4. Generalization beyond mathematical reasoning is claimed but not demonstrated. The method is never tested on tasks outside math QA
5. Using token count or sequence length as an efficiency metric is unfair, since the proposed method includes several preprocessing steps—such as iterative refine and difficulty estimation—before generating the chain of thought. Therefore, inference time would be a fairer basis for comparison.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. In my eyes, one of the major weakness in this paper is the deficiency in model accuracy. The strengths of CCoT lies in generating less tokens, which improves the efficiency. However, it harms the effectiveness, as shown in Table 1.
2. The classifier is used to predict easy and hard queries. However, how to set the threshold is difficult. In other words, how hard is hard and how easy is easy?
3. The generated synthetic CCoT is used to fine-tune LLM for better CCoT understanding, which is reasonable. But the generation of synthetic CCoT is separately optimized from fine-tuning. Hence, I am concerned about the problem that the generated $Z_{syn}$ could be in different subspace from $Z_{final}$, which leads to sub-optimal results.
soundness: 3
strengths: 1. Adaptive reasoning is of practical use due to the need to save resources.
2. The paper is very well written and easy to follow.
3. Experiments have clearly demonstrated the capability of the proposed model in balancing effectiveness and efficiency for inference.
summary: The paper proposes an innovative efficient reasoning framework for adaptive reasoning in LLMs via synthetic continuous CoT. The synthetic CCoT explicitly guides the LLM to learn CCoT and derive accurate answers directly. Further, the paper proposes a classifier to distinguish easy and hard queries. Finally, extensive experiments are conducted to verify the usefulness of the proposed method.
weaknesses: 1. The proposed method is relatively weak in effectiveness. 
2. CCoT method lacks explainability, which deviates from explainable inference in daily use.
3. The classifier introduces an additional hyperparameter to control easy and hard queries, which is difficult to set.