Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization
Authors: Haotian Luo
Haiying He
Yibo Wang
Jinluan Yang
Rui Liu
Naiqiang Tan
Xiaochun Cao
Dacheng Tao
Li Shen
Venue: NeurIPS 2025 poster ()
Keywords: Reasoning
LLM
Long-CoT
Efficiency
R1
Primary area: deep_learning

Abstract: Recently, long-thought reasoning models achieve strong performance on complex reasoning tasks, but often incur substantial inference overhead, making efficiency a critical concern. Our empirical analysis reveals that the benefit of using Long-CoT varies across problems: while some problems require elaborate reasoning, others show no improvement—or even degraded accuracy. This motivates adaptive reasoning strategies that tailor reasoning depth to the input. However, prior work primarily reduces redundancy within long reasoning paths, limiting exploration of more efficient strategies beyond the Long-CoT paradigm. To address this, we propose a novel two-stage framework for adaptive and efficient reasoning. First, we construct a hybrid reasoning model by merging long and short CoT models to enable diverse reasoning styles. Second, we apply bi-level preference training to guide the model to select suitable reasoning styles (group-level), and prefer concise and correct reasoning within each style group (instance-level). Experiments demonstrate that our method significantly reduces inference costs compared to other baseline approaches, while maintaining performance. Notably, on five mathematical datasets, the average length of reasoning is reduced by more than 50\%, highlighting the potential of adaptive strategies to optimize reasoning efficiency in large language models.

Ada-R1: Hybrid CoT via Bi-Level Adaptive Reasoning Optimization
Haotian Luo 122 Haiying He Yibo Wang', Jinluan Yang', Rui Liu
Naiqiang Tan Xiaochun Cao', Dacheng Tao? Li Shen 12f
Shenzhen Campus of Sun Yat-sen University;
2 Center for AI Theoretical Foundation and Systems, Shenzhen Loop Area Institute; China Agricultural University; Tsinghua University; Zhejiang University; 6 Didichuxing Co. Ltd; ? Nanyang Technological University
Abstract
Recently, long-thought reasoning models achieve strong performance complex reasoning tasks, but often incur substantial inference overhead, making efficiency a critical concern. Our empirical analysis reveals that the benefit of using Long-CoT varies across problems: while some problems require elaborate reasoning, others show no omprovement--r even degraded accuracy. This motivates adaptive reason- ing strategies that tailor reasoning depth to the input. However, prior work primarily reduces redundancy within long reasoning paths, limiting exploration of more ef- ficient strategies beyond the Long-Col CoT paradigm. To address this, we propose a novel two-stage framework for adaptive and efficient reasoning. First, construct a hybrid reasoning model by merging long and short CoT models to enable diverse reasoning styles. Second, we apply bi-level preference training guide the model to select suitable reasoning styles (group-level), and prefer concise and correct reasoning within each style group (instance-level). Experiments demonstrate that our method significantly reduces inference costs compared to other baseline ap- proaches, while maintaining performance. Notably, five mathematical datasets, the average length of reasoning is reduced by more than 50%, highlighting the potential of adaptive strategies optimize reasoning efficiency in large language models. Our code is coming soon at http://hthubb.com//ttttewww
Introduction
Recent large language models (LLMSs) such as OpenAl's [1 and Deepseek's R1 adopt extended and structured reasoning processes (Long-CoT)) for LLMs to enhance problem-solving,, achieving strong performance through human-like deliberation. Similarly, 3 and [ enable MLLMs to possess such capabilities as well. However, the improved reasoning comes with high inference costs, including increased latency and resource consumption (5 which limits deployment in real-time or resource-constrained scenarios. Existing efficiency-orienter methods operate within the Long-CoTT distribution, aiming to reduce redundancy through pruning or compression g [ While effective some extent, these approaches do not question whether long reasoning is necessary, overlooking potential gains from fundamentally shorter reasoning strategies. CoT- Valve [10] enables both long and short outputs but lacks adaptive selection based on input complexity, leading to suboptimal results.
Our investigation (presented in Section 3) about the benefit of Long-CoT reasoning reveals a crucial insight: the utility of long, elaborate reasoning chains is highly problem-dependent. While complex
'Equal contribution
"Corresponding Author: Li Shen (shenli6 @
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

problems genuinely benefit from detailed, step-by-step derivations, many other problems can be solved accurately and more efficiently with shorter, more direct reasoning paths. In fact, for simpler problems, forcing Long-CoTT process might not only be wasteful but can sometimes even introduce errors or degrade performance. This observation strongly motivates the need for adaptive reasoning strategies systems that can tailor the depth and style of their reasoning process to the specific demands of the input problem.
Inspired by these limitations, we propose a two-stage framework for efficient and adaptive reasoning by enabling models to choose between distinct reasoning strategies. The first stage constructs a hybrid model capable of generating both Long-CoT and Short-CoT outputs. The second introduces Bi- Level Adaptive Reasoning Optimization a training method comprising: (i) Group-Level Preference, guiding the model to select an appropriate reasoning style based on input complexity, and (ii) Instance- Level Preference, encouraging concise yet accurate reasoning within the chosen style. This dual-level adaptation allows dynamic allocation of computational resources, yielding substantial efficiency gains without sacrificing performance. On MATH([I], our method reduces reasoning length by 58% with accuracy loss, and on GSM8K[12), by 74% with improved accuracy. These results highlight the effectiveness of adaptive reasoning in balancing quality and efficiency in large-scale models,
Our contributions can be summarized as follows:
We conduct empirical analysis investigating the benefits of long Chain-of-Thought (CoT) reasoning relative to shorter CoT approaches, identifying the conditions under which extended reasoning paths offer tangible advantages.
We propose using Adaptive Hybrid Reasoning Model to enhance inference efficiency, ac- companied by novel training pipeline (Ada-R1). Comprehensive experiments demonstrate that our proposed method achieves excellent performance, significantly improving efficiency while maintaining high accuracy.
We perform further analyses on the resulting Adaptive Hybrid Reasoning Model to gain deeper insights into its characteristics and operational behavior. And we will release the model weights of the Adaptive Hybrid Reasoning Model to the public to encourage further research and application by the community.
2 Related Work
Model Merging Model merging 13 is an emerging technique that fuses parameters from multiple trained models into without access to original training data. Recent methods include parameter
interpolation [14] and alignment-based strategies [15]. with applications in LLMs, multimodal
models, and other machine learning subfields. Beyond simple linear averaging, advanced methods such as DARE [6]. TeES-Merging [17]. and AdaMerging [8] have been proposed. DARE reduces redundancy by dropping and rescaling delta parameters. TIES- Mer-ing mitigates interference by trimming and aligning parameter signs. AdaMerging improves performance via entropy-basee layer or task weighting on unlabeled data. In contrast to traditional model merging that consolidates capabilities from multiple models, our work enables single model to adaptively choose between Long-CoT and Short-Cof reasoning for each instance, aiming to optimize computational efficiency rather than multi-task performance.
Efficient Reasoning A variety of methods have been proposed for improved reasoning efficiency. Several techniques apply post-training strategies to shorten reasoning paths. (6] constructs preference datasets using DPO and SimPO, guiding models toward concise reasoning through preference- based fine-tuning. O1-Pruner[3 [ samples CoT's to build baselines for length and accuracy, then applies offline optimization to reduce reasoning length without harming performance. Similarly, [19] leverages simple fine-tuning on self- self-generated concise CoTs obtained via best-of-N sampling and few-shot prompting. Some approaches focus on token-level compression. TokenSkipl (00, for instance, removes tokens selectively based on their estimated importance within the COT. CoT- Valve 10], in contrast, manipulates the parameter space to produce CoTs with varying degrees of compression. Besides, various methods adopt different reasoning paradigms for efficiency. For instance COCONUT21) and CCOT(5) enable reasoning within the latent space, reducing the need for explicit token-level generation. Speculative Thinking(22] enhances small model inference by allowing large models to guide them during reasoning. Similarly, LightThinker 23] achieves efficiency by dynamically compressing intermediate thoughts throughout the reasoning process. Also,

---

design novel reasoning paradigms for efficiency, [30 also explores model merging technical for reasoming efficiency. Different from most works, our work solves reasoning efficiency in a novel adaptive reasoning perspective.
some works (244)
25 26
27 28
(29])
3 Motivation
3.1 Problem Setup
Chain-of-Thouge (CoT) prompting has emerged a powerful technique for enhancing the reasoning capabilities of large language models. Within the CoT paradigm, a distinction can be made between Long-CoT, which involves generating detailed and extensive thinking steps, and Short-CoT, which directly generate solving steps.
3.2 When Do We Need Long-CoT?
Simply applying Long-CoT to all problems introduces unnecessary overhead, especially for easier tasks where detailed reasoning brings little or no benefit. To understand when Long-CoT is truly needed, we empirically analyze its effectiveness across different problem types. We compare Long- CoT and Short-CoT on mixed dataset (MixMathematics) composed of samples from 31ME MATH, and GSM8K (details Section 5. We use DeepSeek-Ke Binieekk-R---- RI-Distilll-OPAA -welll--eee-7B for Long-Coo Con,, and fine-tune it with 2,000 Short-CooT samples from Srren2.5-MMth-.. Bihhh--B--nnttttt [32) to create consistent Short-CoT model. We avoid using Qwen2.5 directly due to its differing training format, which may affect later merging and sampling. From 2,500 problems, we generate 12 responses per model per question and remove cases where both models fail completely. We then calculate accuracy gains [Long-CoT accuracy minus Short-CoT accuracy).
As shown in Figure (left), nearly half the samples show no improvement from Long-Col and some even suffer performance drops. Further analysis (Figure [ right) groups samples by the average length of their Long-Cof outputs-longee CoTs tend to correspond harder problems. We find that Long-CoT significantly improves accuracy complex questions but provides little or no benefit for simpler ones.
gainn
geing
Gain
Rotio
0.4
Proportiol tion
48%
45%
-04
006
1k-2k
2k-4k
4k
6lk-B
8k-16k
Proportion Accuracy Diff by Length Birs
The proportion of gain in the data (left) and the relationship between CoT length and accuracy improvement (right), Long-CoT CoT reasoning improves accuracy difficult problems but has little effect harms performance on easy ones.
Figure 1:
3.3 A New Perspective on CoT Efficiency
Prior methods (Table such as Overthinking (6) kimi-1.5 [33] and O1-Pruner, typically operate within a limited optimization scope but generally maintain performance stability incur only a slight drop, with O1-Pruner notably achieving no performance decrease. In contrast, methods designed for a broad optimization scope, including Model Merge and CoT-Valve. did not consider how to tackle easy and different problems, rendering according to the inherent difficulty of the task. Thus they frequently result in significant performance degradation. a nutshell, methods with a restricted optimization can generally preserve performance but lose the chance utilize shorter COT. However, approaches capable of utilize broader CoT
model incapable of determining its reasoning depth

---

Method
CoT Optimization Scope
Performance (Accuracy)
Overthinking 6
Limited
Slightly Dropped Slightly Dropped
kimi-1.5
Limited
O1-Pruner
Limited
Not Dropped /
Naive Merge CoT-Valve
Broad v
(mostly) Dropped x
Broad v
Dropped x
Ada-RI(Ours)
Broad V
Slightly Dropped v
Table 1: Comparison of Different Methods. "Limited" indicates optimization within the Long-Co'l distribution, restricting efficiency. "Broader" covers both Long- and Short-CoT, enabling shorter, more efficient responses. "Slightly dropped" means accuracy decreased by less than 3%, while "dropped" refers to decrease greater than 3%.
distribution have struggled to maintain accuracy due to their inability to adapt adequate reasoning depth problem complexity.
The finding mentioned in last section motivates us to address the efficiency challenge of Long-Cor models from novel perspective: enabling the reasoning model to adaptively select an appropriate reasoning mode (long or short CoT) for different problems, and then generate a correct and concise CoT in the determined mode. Our proposed method (Ada-R1) differentiates itself by successfully achieving a broad optimization scope while incurring only a marginal performance decrement. This demonstrates more favorable trade-off between efficiency and accuracy compared existing broad-scope optimization techniques.
Bi-Level Adaptive Reasoning Optimization
4.1 Problem Setup
We consider a LLM parameterized by 0 and denoted as Te. In the context math problem solving, the LLM accepts sequence 1 ..... x.), commonly termed as the problem, and then generate a corresponding solution y = [y' ...,, v. Hence, solution y is construed as sample drawn from the conditional probability distribution no (-1z). The conditional probability distribution ( can be decomposed as follows:
m
II To
We consider two LLMs: one trained to generate long, reflective Chain-of- Thought (CoT) reasoning (Long- CoT model, denoted as 0y.) and the other trained for short and concise reasoning paths (Short- CoT model, denoted as 0s). These two models are typically fine-tuned with different CoT and demonstrate distinct reasoning patterns.
4.2 Method Overview
Our method consists of two stages, shown in Figure 2 First, merge Long-CoT model and Short-CoT model to obtain unified reasoning model capable of generating both types of reasoning paths. This allows exploration over broader CoT distribution. Bi-Level Preference Training: for group-level preference, the model learns to choose between long and short reasoning group based on the input; for instance-level preference, it learns compress the reasoning path to improve efficiency within the chosen group determined by group-level preference.
the second stage, we apply
4.3 Stage I: Long-and-Shee Reasoning Merge
To enable flexible reasoning behaviors within single model, we first perform model merging with long and short models. We adopt a simple yet effective strategy linearly merging their parameters. Given two models with parameters 0r and As. compute the merged model as:
0g a0, + - (1-a)6g,
(2)

---

F
Somple
l'roup
Adz
m M
Bi- avel Optinization
+
Kay
Sample
May
Stage
Stage II
Figure 2: Pipeline of Ada-R1. At Stage I, we fused the models to obtain Rōw- In Stage II, we sample from both long and short models and then elicit the group-level and instance-leve preference. After this, we optimize T0 at both group and instance level to obtain a hybrid adaptive reasoning model.
where a E [0, 1] is a merging coefficient that balances the contribution from each model. The resulting hybrid reasoning model, Tey inherits the capacity to generate both long and short CoT depending on the input.
This merged model expands the diversity of the CoT distribution it can produce, laying the foundation for adaptive reasoning. By combining the strengths of both reasoning styles, it enables the model to potentially match different problem types with suitable reasoning strategies, which is key to improving efficiency in the next stage.
4.4 Stage II: Bi-Level Preference Training
In this stage, we introduce a Bi-Level Preference Training strategy to fine-tune the model toward efficient reasoning. The core idea is to train the model to: (1) select the appropriate reasoning style (long or short) for each problem (group-level preference) and (2) further compress the reasoning within the determined chosen group instance-leved preference).
Group Labels. We define group label to denote the reasoning style of a response group. Let gr denote the long reasoning group and gs denote the short reasoning group. For given input problem I, a generated resposne (solution) y belongs to one of the two groups. We use {yi to denote the set of K Long-Col CoT responses generated by the Long- CoT model 0,, and {8, g==s for the corresponding short responses from the Short-CoT model 0g.
Group-Level Preference. For each math problem in the dataset D, we sample K solutions from both the long and short reasoning models. Let {v! ) and {uş 1 Jj=1 be the respective sample sets. We define the approximated accuracy expectation for each group as:
K
(I)((())
1[Correct(y/))
[()(
1
Σ
[[Correct(y]) ).
(3)
K
where [||] is the indicator function. Then we introduce a preference margin threshold > 0. The group-level preference for is then determined as:
JL > gs
-(C)) æ if B[C((x) (()
x
gs > 9L
Given the group-level preference for an input x, we form training pairs from the Cartesian product of the two groups. For example. if > gs x. we construct the preference pairs as:
P Perop((1) (I) (ax v. 5 i [1, K],j [1.
From this set of pairs, we randomly sample a subset contain Mj pairs to construct DPO training tuples (x, yw 21), where you is the preferred (chosen) response and y is the less preferred (rejected).
5

---

For all 2 E D, we perform group-level preference assignment by comparing the sampled long and short responses as described above. These tuples are then aggregated into a new dataset D group {(r, Nes D1 ), which serves supervision for optimizing the DPO objective at the group level.
Instance-Level Preference. Once the preferred group g" E {g1. 9s) is determined for given 2t we further construct instance-level preferences within that group to encourage more concise reasoning. We compare response pairs (yo: yb) such that both belong to the same group (c.g.. .. Vas yb E {u! }). and prefer the shortest correct response. For dispreferred samples, we select M2 longest responses. Formally, for each x E "D with preferred group 9², we first identify the subset of correct responses {Vi g=g' Among these, we select the shortest correct response as the preferred instance: {ys correct S
Vve arg
min WEfyE.oom
To construct instance-level preference pairs, we then select the M2 longest responses from the entire group )= Denote these {ui ) This yields dataset of instance-level training tuples:
D Pisstacce
{1r, Wwr 31) Vhe arg min yE vElv. )amss lyl. y arg max max
(Ma) ve lu-
|
These instance-level preferences encourage the model not only reason correctly, but also do so concisely within the preferred reasoning style.
We sample such intra- -rtru pairs and them as additional training data for DPO encourage the model to favor more concise reasoning within each group.
Objective. Given collelcted preference datasets D groupp and D Dinstance sampled from p" which contains N preference pairs (ar, You Vw: 3/ With a parameter controlling the deviation from the reference model Pref, DPO optimize the model by:
Nb (Ner x)) - 3 log
THw (3/ (x) To (3t x)) ]
max (2 (e 311 (rl))rDDD Denar UD entence Rdp
log a (Blg
Tor, (Be x.)
5 Experiments
5.1 Setup
Long-CoT Models. The long thought models chosen for experiment DeepSeek-R 1-Distill- Qwen-7B and -i--asei-----isiil---e---- 1.5B, which have demonstrated excellent performance on most math problem-solving tasks. For both models, we utilize full-parameter fine-tuning.
Short CoT Models. Since model merging requires Shot-CoT models, we face two issues with existing Shot-CoT models: (1) they often employ templates that differ from those used in Long- CoT models; (2) they tend to exhibit substantial parameter deviations from the base model, which fine-tune the Long-CoT models using a small number of short CoT examples to obtain the corresponding Shot-CoT models. This approach ensures consistency in template usage and maintains closer parameter proximity between the two models.
introduces instability during the merging process [8] (34]. To address these challenges,
Dataset. Following s (35 and Light-RI (36), we construct mixed training dataset ensure coverage across mathematical problems of varying difficulty levels. Specifically, we combine GSM8K, MATH, and AIME datasets in a ratio of 1:3:1, resulting in a total of 2,500 diverse math problems.
Evaluation. We use the GSM8K test set, the MATH test set, and AIME25 as in-distribution evaluation data, while Olympiad 37 and Minerva 38 are employed as out-of-distributinn test sets. For evaluation metries, we consider both accuracy and sequence length. For small-scale datasets such AIME25, we report results averaged over four independent runs to reduce randomness and improve reliability. Additionally, we report the average accuracy degrade rate and the average length reduction rate across all test sets.

---

Table 2: Accuracy (shown above) and length (shown below) of models and methods on different benchmarks. Avg represents the change in length and accuracy compared to the Long model (+ for increase, for decrease).
Bench
AIME25
MATH500
GSM8K
Olympiad
Minerva
Avg.(%)
Model
7B Models
38.3
90.2
88.9
54.4
35.7
Long(RI-distil)) 1-ditilll)
(11005)
(3534)
(1014)
(7492)
(4533)
10.0
78.6
89.5
39.4
28.6
19.97%
Short
(957)
(591)
(272)
(910)
(579)
(-84.57%)
21.7
79.4
88.4
41.2
25.7
18.63%
Merge
(9079)
(916)
(236)
(3743)
(1734)
(-56.02%)
35.8
89.4
86.0
55.2
35.6
-3.56%
DPO
(9976)
(2334)
(360)
(5309)
(3281)
(-33.26%)
40.0
92.4
89.4
55.3
35.3
+2.48%
O1-Pruner
(9353)
(2212)
(377)
(5295)
(3259)
(-34.53%)
22.5
78.6
87.9
39.6
29.4
18. 18.11%
CoT-Valve
(5024)
(747)
(235)
(2313)
(629)
(-73.06%)
35.8
90.2
90.3
52.4
34.1
-1.65%
Ada-RI(Ours)
(8426)
(1468)
(260)
(4889)
(1647)
(-50.93%)
1.5B Models
23.3
81.0
80.9
41.6
26.1
Long(RI-disin))
(12307)
(4416)
(1481)
(7687)
(5789)
9.0
69.4
78.2
30.7
22.4
26.34%
Short
(1098)
(740)
(269)
(1373)
(725)
(-85.15%)
20.8
71.8
74.2
28.6
20.0
10.12%
Merge
(9226)
(1740)
(251)
(3767)
(1399)
(-59.10%%
20.8
81.4
74.8
42.8
24.3
-5.93%
DPO
(10224)
(3055)
(374)
(6319)
(3905)
(-34.57%)
23.3
82.6
84.6
44.7
28.3
+2.18%
O1-Pruner
(9496)
(2782)
(726)
(5658)
(3964)
(-33.75%)
14.2
69.6
76.3
28.7
19.5
19.61%%
CoT-Valve
(7744)
(1299)
(205)
(3169)
(867)
(-67.52%)
23.0
80.8
79.2
42.1
23.5
-1.21%
Ada-RI(Ours)
(9516)
(2455)
(341)
(5802)
(3021)
(-43.28%)
5.2
Competitive Methods
DPO. DPO are widely used baselines in reasoning optimization area. Follwoing the setting of 33 we choose shortest sample as chosen samples and longest sample rejected sample.
Valve. CoT-Valve enables dynamic control of Chain-of-Thoughh length using a single model by identifying and leveraging a controllable direction in the model's parameter space generate compressed COT.
O1-Pruner. O1-Pruner is method designed to reduce reasoning overhead while maintaining model accuracy. begins by establishing a bascline through pre-sampling, and then applies reinforcement learnin-bbased finetuning.
5.3
Main Results
can seen from the Table that: the Short and Merge models achieve the most significant length reduction compared to the Long Model. However, this efficiency gain is accompanied by notable degradation in accuracy, exceeding 10 percentage points. Among the models that not suffer significant accuracy degradation, our method achieves the best length reduction performance,

---

Table Ablation study of each component on several benchmarks, showing that the Merge + bi-level achieves the best trade-off, with 52.08% average length reduction and minimal 0.51% accuracy degradation compared to others.
Bench
AIME25
MATH500
GSM8K
Avg.(%
Model
38.3
90.2
88.9
Long(RI-oo) -ddsttll))
(11005)
(3534)
(1014)
21.7
79.4
88.4
-2.83%
Merge
(9079)
(916)
(236)
(-56.10%)
35.8
84.6
88.7
-3.82%
Merge + SFT
(11222)
(2314)
(375)
(-31.86%)
24.2
81.6
88.0
10.86%
Merge + instance level
(8514)
(886)
(212)
(03))00%
30.8
87.8
91.6
-3.31%
Merge group level
(9049)
(1565)
(359)
(-46.03%)
35.8
90.2
90.3
-0.51%
Merge bi level
(8426)
(1468)
(260)
(-52.08%)
reaching 50.93% for the 7B model and 43.28% for the 1.5B model. Compared to DPO, our approach demonstrates both more substantial length reduction and significantly less accuracy degradation. While O1-Pruner maintains high accuracy, its length reduction effect is considerably weaker than that of our method.
5.4 Ablation Study
To assess each component's impact in our framework, we conduct an ablation study on AIME25, MATH500, and GSM8K. As shown in Table 3 the Merge model reduces average output length by 56. 10%, but with a notable 12.83% drop accuracy.
Supervised Fine-Tunie Tuning (SFT) on the merged model (using the chosen sample in our group level preference dataset), helps recover significant portion of the lost accuracy, bringing the average degradation down to 3.82 3.82%. However, its average length reduction is less pronounced (31.66% compared the Merge model without further training.
Although optimization at the instance level leads to a substantial reduction in sequence length (38%), the model fails to distinguish which problems require long reasoning and which can be solved with shorter responses, resulting in more than a 10% drop in accuracy.
Introducing the group-level preference training after merging (Merge group level) yields better results than SFT and only instance level. achieves higher average length reduction (46.03%) and slightly better accuracy recovery, with only 3.31% average degradation relative the baseline. This indicates that training the model to select the appropriate reasoning style effective in balancing efficiency and accuracy.
The full method (Merge bi level), combining group and instance level preference training, offers the best trade-off: 52.08% length reduction with only 0.51% accuracy loss. This result highlights the complementary benefits of the bi-level training approach: the group level guides the model towards suitable reasoning styles, and the instance level further refines the chosen style by favoring concise and correct responses, leading to a highly efficient and accurate hybrid reasoning model.
Further Evaluation
6.1 Thinking Ratio Study
To investigate the thinking characteristics of different models, we propose the "Thinking Ratio" metric. This metric is designed to detect whether response constitutes a deep thinking (Long-CoT)) sample. Long-CoT responses typically include unique keywords (e.g.. 'wait', 'recheck)). By detecting the

---

presence of these keywords in a response, we can determine if it is deep thinking sample. This detection method more generalizable than relying solely on response length. We use subset of Math Testset. Using the method described above, we analyzed the proportion of deep thinking samples for each model. Furthermore, for each category (thinkingnnon-thinking samples), we also calculated their accuracy.
The results are shown in Figure The baseline Long-CoT model predominantly employs deep thinking (0.98), yielding high accuracy. In contrast, the Naive Merge model drastically shifts towards non son-thinking responses (0.94) but suffers significant accuracy degradation on both thinking (0.68) and non- -on-thinking (0.81) paths. DPO shows a moderate shift to non- non-thinking (0.34) while preserving accuracy. Our Ada-R1 model achieves a more significant shift towards non- oon-thinking (0.72) than DPO. yet crucially maintains high accuracy for these dominant non- non-thinking responses (0.96), unlike the Naive Merge. This demonstrates Ada-R1 1 's effective adaptation, utilizing efficient shorter paths without substantial accuracy loss.
Naerts CCT
TD Poiiing
Popprtee
066
Add1
15/3
Nergag
Figure 3: The proportion and accuracy of thinking and non- non-thinking in different methods, Ada-RI R1 can achieve a good balance and accuracy between thinking and non-thing non-thinking.
6.2 Adaptive Reasoning Study
This section evaluates the adaptive reasoning ability of Ada-R1 (7B) on the MATH dataset, which is divided into five difficulty levels (Level 1-5). We analyze both the model's thinking ratio (Long-Co'I usage) and its average accuracy across these levels. As shown in the left part of Figure the thinking ratio increases significantly with task difficulty. Level I problems have the lowest Long-CoT usage, while Level 5 shows the highest, indicating that Ada-RI adaptively chooses to think more on harder problems. In terms of accuracy (Figure right), Ada-R1 achieves strong performance across difficulty levels. Its accuracy is comparable to that of a full Long-CoT model (Deepseek-RF --eek--------------- -B---iill and consistently higher than the Short-CoT model, especially on Levels 3 to 5. These results support our hypothesis from Section 3: Ada-RI can selectively apply Long-CoT -CoT when needed, achieving a better balance between accuracy and efficiency.
6.3 Merge Model Analysis
To verify the effectiveness of the model merging strategy, we compute the average loss of the merged model on both Long-CoT and Short-CoT samples, and compare these values with those of the original Long-CoT and Short-CoT models. As shown in Table the merged model exhibits similar losses to the corresponding original models on both types of samples. This indicates that the merged model preserves the reasoning capabilities of both individual models, thereby cnabling it to fully leverage population-leve level preference signals during the second-stage fine-tuning. These results support the effectiveness of our dual-level preference training setup, where the merged model benefits from a broader reasoning space while being guided by meaningful preference supervision.
6.4 Out of Domain Test
To further assess the effectiveness of our method, we additionally evaluate it on mon-mathematical reasoning benchmarks, including (39]. [40)[40], and MMLU[4] (we report results both

---

Thinking CDT
Non- Non-Thesing
ASRL
Lang
Shart
Propartions
0.6
0.75
0.2
Level
Level
Level
Level
Leael
Lavel
Level
Lnaal
Lanel
Diffenent
Difficulty Levels
Different Difficulty
Figure 4: The ratio of thinking and non-thinking CoTs Ada-RI-7B on different MATH levels (left) and the accuracy on different MATH levels of different models (right). As the difficulty increases, Ada-R1 able think more on harder problems and maintain higher accuracy.
Table 4: Average losses of the merged model compared with the original Long-CoT and Short-CoT models, showing that the merged model retains both reasoning abilities.
Model
Long-Cof Togg-CoTuee Loss
Short-CoT Loss
Long (RI-distill)
0.2106
Short
0.1570
Merge
0.2141
0.2028
Table 5; Evaluation results of Ada-RI on non-mathematical reasoning tasks.
Bench
LogiQA
GPQA
MMLU(stem)
MMLUUhumanities)
Av).
Model
45.1
30.3
43.3
16.3
Long(RI-distin)
(3269)
(7302)
(2336)
(1476)
44.2
31.3
48.3
21.0
+7.26%
DPO
(2650)
(6592)
(1577)
(1270)
(-15.95%)
48.6
32.8
54.6
30.9
+23.63%
Ada-RI(ours)
(2794)
(5101)
(1909)
(930)
(-25.37%)
STEM and Humanities subsets), as shown in Table 5 The results indicate that Ada-RI maintains strong efficiency-accureee trade-off's beyond purely mathematical problems, demonstrating its generalizability to broader reasoning domains.
7 Conclusion
In this paper, demonstrate through empirical analysis that the benefits of Long-CoTT reasoning vary significantly depending on the problem. Motivated by this, we propose novel two-stage training framework for adaptive reasoning. Experiments show that model trained with our method can reason adaptively to different problems. And our method significantly reduces inference costs while preserving performance, highlighting the promise of adaptive strategies for optimizing reasoning efficiency in large language models.
10

---

8 Acknowledgement
This work is supported by National Key R&D Projects (NO. 2024YFC33007000, NSFC Grant (No. 62576364), National Natural Science Foundation of China (No.62025605 6241 1540034), Shenzhen Basic Research Project (Natural Science Foundation) Basie Research Key Project (NO, JCYJ20041200124440044 CCF-DiDi GAIA Collaborative Research Funds (NO. CCF-DiDi GAIA 202419 and CCF-DiDi GAIA 202519).
References
[1] OpenAl. Learning to reason with Ilms.
https://openaai.com/
learning- to reason- wwwssnewwithhleeeeeeeeeeeeeee-ee with-1lms/ 2024. [Accessed 19-09-2024).
[2] DeepSeek-A Al team. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025.
[3] Huanjin Yao, Qixiang Yin, Jingyi Zhang. Min Yang, Yibo Wang. Wenhao Wu, Fei Su, Li Shen, Minghui Qiu, Dacheng Tao, and Jiaxing Huang. R RI-sharevl: Incentivizing reasoning capability of multimodal large language models via share-grpo, 2025.
[4] Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang. Yuxin Song, Haocheng Feng, Li Shen, and Dacheng Tao. Mulberry: Empowering milm with ol-like reasoning and reflection via collective monte carlo tree search, 2024.
[5] Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations, 2024.
[6] Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? on the overthinking of ol-like llms, 2025.
[7] Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang. Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models. 2025.
[8] Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1 O1-pruner: pruning. 2025.
fine-tuning for ol-like reasoning
[9] Daman Arora and Andrea Zanette. Training language models to reason efficiently, 2025.
[10] Xinyin Ma. Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-compressible chain-of-thought tuning, 2025.
[11] Dan Hendrycks, Collin Burns, Sauray Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurlPS, 2021.
[12] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021.
[13] Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang. Xiaochun Cao, Jie Zhang, and Dacheng Tao. Model merging in Ilms, milms, and beyond: Methods, theories, applications and opportu- nities, 2024.
[14] Yue Zhou, Yi Chang, and Yuan Wu. Mixup model merge: Enhancing model merging perfor- mance through randomized linear interpolation, 2025.
[15] Rishabh Bhardwaj. Do Duc Anh, and Soujanya Poria. Language models are homer simpson! safety re-alignment fine-tuned language models through task arithmetic, 2024.
[16] Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as free lunch, 2024.
11

---

[17] Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. Ties mieging Resolving interference when merging models, 2023.
[18]
Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. Adamerging: Adaptive model merging for multi-task learning, 2024.
[19] Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjin Yang. Yujin Kim, and Se- Young Yun. Self-training elicits concise reasoning in large language models, 2025.
[20] Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in Ilms, 2025.
[21] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space, 2024.
[22] Wang Yang, Xiang Yue, Vipin Chaudhary, and Xiaotian Han. Speculative thinking: Enhancing small-model reasoning with large model guidance at inference time, 2025.
[23] Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujic Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. Lightthinker: Thinking step-by-step compression, 2025.
[24] Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Zheng Lin, Li Cao, and Weiping Wang. Dynamic early exit in reasoning models, 2025.
[25] Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models, 2025.
[26] Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking, 2025.
[27] Ziqing Qiao, Yongheng Deng, Jiali Zeng, Dong Wang, Lai Wei, Fandong Meng, Jie Zhou, Ju Ren, and Yaoxue Zhang. Concise: Confidence-gund compression in step-by-step efficient reasoning. 2025.
[28] Ren Zhuang, Ben Wang, and Shuifa Sun. Accelerating chain-of-thought reasoning: When goal-gradient importance meets dynamic skipping. 2025.
[29] Junjie Yang, Ke Lin, and Xing Yu. Think when you need: Self-adaptive chain-of- th-thuuhttt learning, 2025.
[30] Han Wu, Yuxuan Yao, Shuqi Liu, Zehua Liu, Xiaojin Fu, Xiongwei Han, Xing Li, Hui-Ling Zhen, Tao Zhong, and Mingxuan Yuan. Unlocking efficient long-to-short Ilm reasoning with model merging, 2025.
[31] MAA. American invitational mathematics examination aime. In American Invitational Mathematics Examination AIME 2024, February 2024.
[32] Qwen, :, An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.
[33] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, Chuning Tang, Congcong Wang, Dehao Zhang, Enming Yuan, Enzhe Lu, Fengxiang Tang. Flood Sung, Guangda Wei, Guokun Lai, Haiqing Guo, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haotian Yao, Haotian Zhao, Haoyu Lu, Haoze Li, Haozhen Yu, Hongcheng Gao. Huabin Zheng, Huan Yuan, Jia Chen, Jianhang Guo, Jianlin Su, Jianzhou Wang, Jie Zhao, Jin Zhang, Jingyuan Liu, Junjie Yan, Junyan Wu, Lidong Shi, Ling Ye, Longhui Yu, Mengnan Dong, Neo Zhang, Ningchen Ma, Qiwei Pan, Qucheng Gong, Shaowei Liu, Shengling Ma, Shupeng Wei, Sihan Cao, Siying Huang, Tao Jiang, Weihao Gao, Weimin Xiong, Weiran He, Weixiao Huang, Wenhao Wu, Wenyang He,
12

---

Xianghui Wei, Xianqing Jia, Xingzhe Wu, Xinran Xu, Xinxing Zu, Xinyu Zhou, Xuehai Pan, Y. Charles, Yang Li, Yangyang Hu, Yangyang Liu, Yanru Chen, Yejie Wang, Yibo Liu, Yidao Qin, Yifeng Liu, Ying Yang. Yiping Bao, Yulun Du, Yuxin Wu, Yuzhi Wang, Zaida Zhou, Zhaoji Wang, Zhaowei Li, Zhen Zhu, Zheng Zhang, Zhexu Wang, Zhilin Yang. Zhiqi Huang, Zihao Huang, Ziyao Xu, and Zonghan Yang. Kimi k1.5: Scaling reinforcement learning with 1mss, 2025.
[34] Yuyan Zhou, Liang Song, Bingning Wang, and Weipeng Chen. Metagpt: Merging large language models using model exclusive task arithmetic, 2024.
[35] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025.
[36] Liang Wen, Yunke Cai, Fenrui Xiao, Xin He, Qi An, Zhenyu Duan, Yimin Du, Junchen Liu, Lifu Tang, Xiaowei Lv, Haosheng Zou, Yongchao Deng, Shousheng Jia, and Xiangzheng Zhang. Curriculum sft, dpo and for long cot from scratch and beyond, 2025.
Light-rl:
[37] Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. Olympiadbench: challenging benchmark for promoting agi with olympiad-leve bilingual multimodal scientific problems, 2024.
[38] Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models, 2022.
[39] Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. 2020.
[40] David Rein, Betty Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-proof q&a benchmark, 2023.
[41] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt, Measuring massive multitask language understanding. 2021.
13

---

A Selection of a in Stage (Model Merge)
To select an appropriate value of a during the merge phase, adopted the following approach. We randomly sampled 100 problems from the AIME exams (AIME 2025 is excluded). Using the 7B model mentioned in our experiment, we evaluated performance under different values of a (0.9, 0.8, 0.7), computing both accuracy and thinking ratio for each setting. We selected a = 0.8 as a balanced choice for Stage training it has relatively high accuracy and moderate thinking ratio.
Table 6: Performance of merged 7B models on 100 AIME problems.
Accuracy
Thinking Ratio
0.9
54.0
93%
0.8
40.0
48%
0.7
27.0
9%
B
Training Details
For both models, we selected 2,500 problems from the mixed Mathematics training data. For each problem, we sample 12 times. From each set of solutions, we randomly selected 2 solutions for training. After computing the rewards, normalized the reward values. Both models trained with 8 A800-80G GPUS. The other hyperparameters used in the training process are presented in the table below.
Table 7: Hyperparameters for the Deepseek- Distill--Diiiil111555 and Deepseek-DDssill--7 Distill-7B.
Hyperparameter Deepseek-Distill Disttll--D- Seepseek-Disttil--
cutoff len
4096
4096
batch_size learning rate
32
32
5.0c-7
5.0e-7
num trrii__epocil
2.0
2.0
Ir wwwsscchherryyy. M M2
constant
constant
4
4
2
beta
0.05
0.1
C Further Evaluation of Different Methods
We further evaluate the performance and efficiency of different methods (Ada-R1, DPO, O1-Pruner) across varying levels of problem difficulty, illustrated in Figure Figure 6 F the accuracy ratio of each method relative to a baseline model across different difficulty levels within the MATH dataset. The results indicate that while performance trends may vary, proposed Ada-Rl R1 method demonstrates strong robustness. Specifically, as the inherent difficulty of the mathematical problems increases, Ada-RI is able to consistently maintain a high accuracy ratio.
Figure 6 show the ratio of average tokens consumed by each method to solve problems across the same difficulty spectrum. As expected, solving more difficult problems generally requires more reasoning steps and thus more tokens. However, Figure 6 reveals that Ada-RI exhibits favorable token efficiency. Critically, when faced with increasing problem difficulty, Ada-R1 manages to solve these complex problems while utilizing relatively fewer tokens compared to other evaluated methods, showcasing its ability to achieve efficient reasoning
for demanding tasks.
Collectively, these figures highlight Ada-RI's ability to strike beneficial balance between accuracy and efficiency. It not only maintains high performance on challenging problems (Figure 5) but also does so computationally efficient manner, particularly evident in lower token usage for difficult instances (Figure 6) addressing limitations observed in prior methods.
14

---

AGdR1
DPO
OL Fruner
Long
Shart
0.5
Level1
Level
Lecel
Level4
Level
Different Difficulty Levels
Figure 5: The ratio of accuracy at different MATH levels different models. As the difficulty increases, Ada-RI is able to maintain high accuracy.
AdoRI
DPO
O1- O1-Pruner
Long
Short
5400
4750
4100
3450
2800
1200
850
200
Different Difficulty Levels
Figure 6: The ratio of average tokens on different models. As the difficulty increases, Ada-R1 is able to use relatively fewer tokens to solve difficult problems.
D Why Does Ada-R1 Work?
D.1 Early Mode Selection Assumption
While Ada-RI significantly reduces inference cost by adaptively selecting a reasoning strategy during the inference stage, its design relies important assumption: the model determines the reasoning mode (Long-Co'l or Short-CoT) immediately after receiving the problem input, without relying on any intermediate computation external signals. In other words, the model is expected to assess the complexity of the problem and select an appropriate reasoning path before beginning the actual problem-solving process.
D.2 Visualization Setup
To investigate this question and better understand how Ada-RI works, we design an experiment. We randomly select 500 problems from the training data and evaluate them using the 7B models (R1, and Ada-RI). each problem, we extract the hidden states of the final token in the input sequence and use the last layer's hidden states as the internal representation of the problem. Based previously computed group-level preferences (i.e., whether the problem should solved using Long-CoT) or Short-CoT), assign color label to each sample-red red for problems requiring Long-CoT and blue
15

---

R1.78
Ads R1-78
Lang hnleroo Iaannt
Lang lpaswdi
Figure Visualization of R1 model
Figure Visualization of Ada-RI model
for those suitable for Sho.t-CoT. We then apply t-SNE to project the high-dimensional hidden states into a two-dimensional space for visualization.
D.3 Ada-RI Learns an Implicit Problem Classifier
From the visualization, we observe that after sifferencebbssee training, Ada-RI is able to partially separate problems that require Long-CoT from those that do not, based solely on their internal representations. This suggests that the model learns encode problem complexity in representation space, enabling early and efficient reasoning mode selection. Such a capability underpins effectiveness of Ada-R1: by making informed decision on the reasoning strategy at the problem stage, the model avoids unnecessary computation for simpler problems while retaining full reasoning capacity for more complex ones.
E
Case Study
We select two cases and 10) of R1 and Ada-R1 to better understand our proposed method. In case Ada-R1 can infer answers simple problems with small amount of CoT; in case I0, Ada-RI can infer answers to difficult problems with relatively few long CoT.
F
Limitations
While our proposed adaptive hybrid reasoning framework shows strong performance on benchmark datasets, its applicability in real-world scenarios presents new challenges. In particular, real- world tasks often involve diverse input distributions, domain-speciffe knowledge, and evolving requirements that differ from curated academic datasets such as MATH GSM8K. The current design assumes clear patterns in problem complexity that can guide reasoning strategy selection; however, practical applications, this complexity may be harder to estimate reliably without additional context or metadata. We view this as promising direction for future research to bridge the gap between controlled evaluation settings and real-world reasoning applications.
16

---

Convert the point (0,3) in rectangular coordinates to polar coordinates. Enter your answer in the form (r, 9). where r > 0 and S < 2x. Let's think step by step and output the final answer within boxed(). Ground Truth: (3, )
Question:
Ada-RI: To convert the rectangular coordinate (0, 3) to polar coordinates, we use the formulas: / y and 0 = arctan (#). For the point (0,3): v02 + 3 v9 3 Since the point lies on the positive y-axis, the angle 7 Thus, the polar
coordinates are (3, 2). Final Answer: (3. 2
Okay, so need to convert the rectangular coordinate (0, 3) to polar coordinates. Hmm, polar coordinates are represented as (r, 8), where is the radius (distance from the origin) and 0 is the angle made with the positive x-axis. Let me recall the formulas to convert from rectangular to polar coordinates.
R1:
First, remember that the radius r can be found using the Pythagorean theorem. Since in rectangular coordinates, the point is (z, y), the distance from the origin is r = /2² + y². So in this case, is 0 and is 3. Plugging that into the formula, should be v + 3 which is + = /9 = 3. Okay, that seems straightforward.
Now, the angle 0. I know that 0 calculated using the arctangent function because tan 0 y But wait, in this case, x is O. Hmm, arctangent of y But dividing by zero is undefined. need think about what angle corresponds to when In rectangular coordinates, when is O, the point lies on the y-axis. Since y is positive here (y = 3), the point is on the positive y-axis. So, in terms of polar coordinates, the angle 0 should be the angle that points directly upwards along the y-axis. I think that's 2 radians, which is 90 degrees. Because starting from the positive x-axis, if we rotate radians coumterclockwise,
end up pointing straight up along the positive y-axis.
Let me double-check that. If 2. then converting back to rectangular coordinates, æ should be cos 0 cos 5) and should be sin 0 3 ( cos( 5 is 0 and sin( is 1, so that gives x = and = 3. which exactly the point we started with. So that makes sense. Wait, but could 0 also be 3r No, because 3 is pointing downward along the negative y-axis, which would give us = -3, not 3. So has to be in the range where the point is on the positive y-axis, which 5.
Just to make sure, let's 's think about the unit circle. At = 0, we're on the positive x-axis; at 0 = -. positive y-axis; 0 n, negative x-axis; 0 3r negative y-axis. So yeah, 0 m is correct for the positive y-axis.
So, putting it all together, is 3 and 0 is T Therefore, the polar coordinates should be (3, 3). Let me just write that out to make sure didn't I make any mistakes. Wait, another thought: sometimes people use different ranges for 0, like from -T to n. But in this case, the problem specifies that should be between 0 and 2n,, so is within that range. So, I don't need to adjust or anything.
Just to recap: the point (0, 3) in rectangular, so 0, y 3. Calculating r: v02 -d
Calculating 0: since x = 0 and y is positive, 0 is 3. So, (3, F) is the polar coordinate.
I think that's solid. Let me visualize too. If plot (0.3) on the rectangular coordinate system, it's on the y-axis, three units up. In polar coordinates, that's radius of 3 and an angle of т from the positive x-axis. Yep, that lines up perfectly.
y
Just to add another check, let me compute the sine and cosine with 0 F cos is 0, so x = + 0 O, which matches. sin( 3 1, so y = 3 = 3, which also matches. Perfect, no discrepancies there.
So, I think mm confident that the polar coordinates are (3, T). don't see any mistakes in my reasoning, and all the checks seem to confirm that this is the correct answer.
π
Final Answer: The polar coordinates are (3,
2
Figure 9: Ada-R1 can infer answers to simple problems with a small amount of CoT, whereas the R1 model requires long COT.
17

---

Question:
over from 1 to n-1. Let me write that down: Ej=1 Lk=1 (f+k)a -n Ej=1 n² Yes, that seems right. Because when k n, J can go from to n-1, and for each j. k is determined as n n-j.. So. the inner sum just Simplify that expression: Ei=i कo and q Σx=1 मo. Find a way to write Σj=1 Ex=1 G+E in terms of and q. Let's think step by step and output the fi- nal answer within boxed(). Ground Truth: p
Define p
Ada-R1: Okay, so have this problem where need to express a double sum in terms of two known series, p and q. Let me first write down what's given. We have: p = Lk=1
and q = 1 gs. And the expression we need evaluate Žj=1 Lk=1 G+R]3. Hmm, so this is a double sum over j and starting from to infinity, and each term is 1 divided by (j + k) cubed. I need to find way write this in terms of p and q. Let me think about how to approach this. Maybe I can change variables to simplify the expression. If let = k, then for each fixed n, can range from 1 to n-1, and k would be -j. So, for each n, the number of terms where j + k = is n So, maybe can rewrite the double sum single sum over and for each
adding n (n-1) times. So, the double sum becomes: _n= - //- So now, the sum
En=2 n=1 ( ) Yes, because n-1 En=z n But notice that these sums are just
n be split into separate sums: Σn=2
p and q minus their first term. Let's write that: The first sum is: En= n² = p - 1a = pp-1. 1.
1
Similarly, the second sum is: En= 2 # = - 1 = - 1. Therefore, the original
[OMITTED]
= = - 11-p))==---==ppp p- Therefore, Thus, the original double sum becomes: (p - 1) - (q -
the final answer is: 17-9 gl.
R1: Okay, so I have this problem where I need to express a double sum in terms of two known series, p and q. Let me first write down what's given. We have: p = - A and
q q= -= R3 And the expression we need to evaluate is Σ)= Ei=1 (z++)3 Hmm, so this is a double infinite sum where both j and k start from I and go to infinity, and each term is I over (j k) cubed. need find a way to write this in terms of p and q. Let me think about how to approach this. Maybe I can find substitution or change variables to simplify the expression. Since the denominator is (j + 3=)3, maybe if let sum in terms of n and another variable. Let's try that. Let me set n + k. Then, for each fixed n, j can range from 1 to So, for each n, j goes from 1 to 1, and k n- So, substituting, the double sum becomes: nn
j + k. then can rewrite the
because k would be j.. which also needs to be at least 1.
E Ej=1 n.. Wait, that seems promising. Because now for each n, we have term ş multiplied by the number of j's, which is (n- So, the inner sum is just a-1 Therefore, the entire expression becomes: Σn=2 n-1 1
n ).). Which is: Σ 1 - >m 2 n But know that: E 1 and Σ -==1 nz = So, subtracting 0 and Ln substituting back into the expression: (p -----(---------- --=--l-++1-pp
n² Simplify that: En n2
nス the first term from both series: Em nz
q - Therefore, q. Wait, so the entire double sum simplifies to p q Let me verify my steps to make sure I didn't make mistake. I started by changing variables, setting n n=j+k: Then for each n. goes from to n 1, so k n- also goes from 1 to 1. That seems correct. 2. Then, rewrote the double sum as a single sum over n, with each term being n-1 That makes sense because for each n, there are (n 1) terms contributing a valid step because a-1 = a infinite series starting 2 = 2, which correct because the original sum starts from = 5. Finally, expressed the sums in terms of and by subtracting the n=1 term from each series. That's correct because the original p and q start from n=l. So, putting all together... [OMITTED]
1 each. 3. Then, I split the fraction into पे _ That's म.
ş T.. 4. Then, I separated the sums into two
Therefore, the final answer is:
Figure 10: Ada-R1 can infer answers to difficult problems with relatively few long CoT, whereas the RI model requires longer COT.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We provide our contributions and scope both in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in the Appendix this paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification:
Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
Yes, we provide all necessary information to reproduce the experimental
results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification:
We provide code and data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/ddeeeeee Cooeeeoodeesssseeeeee //nips.com / ssiniiiiccypyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The training and test details are described in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper does not report error bars, confidence intervals, standard devia- tions for the experimental results. But our models are evaluated with multiple benchmarks.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
21

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide this information in the paper in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
Our research follows the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes. All assets are properly credited and used under their respective licenses.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do provide assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: This research does not involve LLMs any important, original, non- standard components.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
25


Reviews:
[{"review_id": "l04irfgn2V", "reviewer": "Reviewer_Mw4b", "summary": "The paper proposes Ada-R1: a two-stage LM training framework for adaptive reasoning that can potentially limit the length of the reasoning chain depending on the problem. For this, the authors construct a hybrid reasoning model by merging long and short CoT models, then use preference training to guide the model to select suitable reasoning styles. They show that the method reduces the average length of reasoning by ~50%, with minimal degradation to final performance.", "questions": "1. Did the authors perform any analysis of samples from AdaR1 vs DPO vs O1-pruner etc to learn what is the difference in reasoning chain? what exactly AdaR1 is doing differently?\n2. why upsample MATH dataset for training?\n3. Section 6.1, line 254: \"This detection method is more generalizable than relying solely on response length.\" , why?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The proposed two stage training approach, Ada-R1 is novel.\n- Ada-R1reduces the average reasoning length by ~50% (on five mathematical datasets,), with minimal degradation to final performance\n\nWeaknesses:\n- The two stage approach is more complicated than a baseline e.g. O1-pruner while incurring a limited degradation in performance.\n- There is no clear motivation on why Ada-R1 is a better approach than \"self training for concise reasoning [17]\" or O1-pruner.\n- The authors mention in Section 3.3 that other approaches like Overthinking or O1-Pruner \"typically operate within a limited optimization scope\", whereas their approach is \"broad\" without explaining what it means or why.", "was_revised": true, "final_justification": "The authors have addressed some concerns, helping in an improved score.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive feedback. We have carefully considered all comments and provide detailed responses below to address the concerns raised.\n\n---\n\n**weakness 1** \n\nThank you for your concern about the method process. Although our method adopts two stages, neither is complex. **Stage I (model merging)** is simple weight-space arithmetic (parameter addition/subtraction) and finishes within tens of seconds. **Stage II** performs a standard DPO training loop after constructing the bi-level datasets and mixing them, which is also not complex. While our approach incurs a small drop in accuracy, it **substantially reduces the number of generated tokens**, achieving a favorable accuracy–cost trade‑off. Moreover, it offers a novel and practical perspective for enabling the model to **dynamically allocate reasoning mode** to each problem for efficient reasoning.\n\n---\n\n**weakness 2** \n\nOur motivation is explicit in **Sec. 3.3** and **Table 1**. Prior “self‑training for concise reasoning” methods (e.g., DPO‑style shortest‑vs‑longest) and O1‑Pruner **optimize within the Long‑CoT distribution**, pruning or compressing long traces. This preserves accuracy but **forfeits exploration of genuinely Short‑CoT solutions**. In contrast, Ada‑R1 first merges Long and Short models to expand the output distribution to both styles, then uses bi‑level preference training to select the style per input and compress within the chosen style, enabling broad‑scope optimization without the large accuracy drops.\n\n---\n\n**weakness 3**\n\nIn our paper, scope explicitly refers to the **CoT distribution being optimized**:\n**Limited scope**: optimization is confined to the **Long‑CoT** distribution. (only thinking paths)\n**Broad scope**: optimization covers **both Long‑ and Short‑CoT** (i.e., thinking and non‑thinking paths).\n\nPrior approaches like Overthinking or O1-Pruner operate on a **limited scope**, meaning they focus on optimizing **only one type** of reasoning path—typically **Long-CoT**—assuming that longer reasoning is always preferable.\n\nIn contrast, our method operates over a **broad scope**, optimizing over **both Long-CoT and Short-CoT** distributions. This design reflects the intuition that the optimal reasoning path may vary by input: some queries benefit from detailed reasoning (Long-CoT), while others are better answered succinctly (Short-CoT).\n\n---\n\n****question 1****\n\nActually, our evaluation demonstrates that **Ada-R1 is capable of generating substantially shorter CoTs** (non-thinking CoTs) compared to O1-Pruner and DPO, especially when solving simpler problems. This class of concise CoTs lies **outside the original distribution** that DPO and O1-Pruner are optimized over, which inherently biases toward longer, detailed (thinking-style) CoTs due to their reliance on Long-CoT initialization or compression.\n\nIn Figure 3 (Thinking Ratio Study), Ada-R1 shows a **higher proportion of non-thinking CoTs** compared to both DPO and the Naive Merge. More importantly, Ada-R1 **maintains high accuracy** even on these short responses.\n\nFigure 4 (Adaptive Reasoning Study) further validates this behavior: **Ada-R1 adaptively increases the use of Long-CoTs** (thinking responses) as the problem difficulty grows, while still using **short and efficient non-thinking responses for easier problems**. This dynamic range of CoT generation is absent in DPO and O1-Pruner, which primarily shorten existing long CoTs (i.e., compressed thinking CoTs) rather than introducing fundamentally shorter reasoning paths (non-thinking).\n\n---\n\n**question 2**\n\nIn fact, MATH is the most widely used dataset that includes different levels of difficulty. We upsample the MATH dataset to ensure the model sees a wide range of problem difficulties during training. This diversity is essential for learning adaptive reasoning—i.e., when to use long vs. short reasoning. Without enough complex problems, the model cannot learn to make this distinction effectively.\n\n---\n\n**question 3**\n\nThank you for pointing this out. This method is more generalizable than length-based heuristics for the following reasons:\nThinking tokens explicitly reflect the model’s reasoning intent or self-reflection (e.g., hesitation, checking). This makes them a more *semantically meaningful* indicator of deep reasoning, as opposed to length, which can be inflated by irrelevant or verbose content. Beside, token-based cues are more transferable across different model architectures or training setups. Length distributions may vary significantly, but the presence of reasoning-related tokens like “wait” tends to be consistent among models employing Chain-of-Thought prompting.\nMoreover, recent studies such as [1][2][3] also determines whether the model is thinking by detecting words such as “wait”. [4] shows that certain tokens like “wait” and “let’s think step by step” are associated with **peaks in mutual information**, reflecting critical moments such as redirection or self-reflection during reasoning.\n\n---\n\n**Final Remark**\nWe thank the reviewer for their valuable comments. We hope that our clarifications and revisions have adequately addressed the concerns, and we respectfully ask the reviewer to consider a more favorable score in light of these improvements. If there are any remaining concerns, we would be glad to further clarify them.\n\n---\n\n**References**\n\n[1] Dynamic Early Exit in Reasoning Models\n\n[2] R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search\n\n[3] Wait, We Don’t Need to “Wait”! Removing Thinking Tokens Improves Reasoning Efficiency\n\n[4] Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning"}, {"review_id": "UdXDCQmRqs", "reviewer": "Reviewer_sm7M", "summary": "This paper proposes Ada-R1, a two-stage framework for adaptive and efficient reasoning in LRMs.\nThe first stage constructs a hybrid reasoning model by merging Long-CoT and Short-CoT models, enabling diverse reasoning styles. \nThe second stage introduces bi-level preference training based on DPO, guiding the model to select appropriate reasoning styles at a group level and encouraging concise, correct reasoning at an instance level.\nExperiments on mathematical datasets demonstrate that Ada-R1 reduces inference costs and reasoning length while maintaining performance.", "questions": "* Why is model merging chosen for Stage I over directly training a single model on a mixed dataset of Long-CoT and Short-CoT examples? Please provide a comparative experiment or a more detailed theoretical argument to justify the choice of model merging, demonstrating its advantages over mixed training strategies for creating a hybrid reasoning model.\n* Could the authors elaborate on the precise definition of \"CoT Optimization Scope\" (Limited vs. Broad) and provide more intuitive examples of how these scopes directly influence the model's output and the final task performance?\n* How is the preference margin threshold for the group-level preference determined?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* The core idea of enabling LLMs to adaptively choose reasoning depth (Long-CoT vs. Short-CoT) based on problem complexity addresses a critical efficiency concern in current LRMs.\n* The proposed bi-level optimization (group-level and instance-level preference) is a well-structured approach to guide the model towards both appropriate reasoning styles and conciseness within those styles.\n* The results show large reductions in reasoning length without substantial accuracy loss.\n\nWeaknesses:\n* The methodology for creating the Short-CoT model, specifically fine-tuning a Long-CoT model (DeepSeek-R1-Distill-Qwen-7B) with 2,000 Short-CoT samples from Qwen2.5-Math-7B-Instruct, raises concerns. This approach is uncommon, and the quality and representativeness of these 2,000 samples could significantly influence the fine-tuned Short-CoT model's performance (in Table 2 main results) and the entire analysis in Section 3.2. A more standard or thoroughly justified method for obtaining a consistent Short-CoT baseline would strengthen the experiments.\n* In Section 3.3, the concept of \"CoT Optimization Scope\" as either \"Limited\" or \"Broad\" and its direct implications on model output or final results are not clearly defined or intuitively explained. A more precise definition and illustrative examples of how different scopes manifest in model behavior and outcomes would greatly enhance the understanding of the motivation.\n* The paper lacks statistical significance tests for its experimental results, particularly concerning datasets like AIME25, which has a small test set size (30 samples). The sensitivity of results to hyperparameter settings necessitates multiple evaluation runs with reported averages and error bars to confirm the robustness and generalizability of the findings.\n* Based on the ablation study in Table 3, the \"Merge + group level\" configuration already achieves a significant length reduction with a relatively small accuracy degradation. The incremental benefit of the instance-level preference training appears less pronounced compared to the group-level. The authors should provide a more compelling argument or further analysis to justify the necessity and significant impact of the instance-level preference beyond the group-level.", "was_revised": true, "final_justification": "I acknowledge the methodological novelty of the Ada-R1 and have updated my score.", "rebuttal": "We thank the reviewer for the time and effort spent reviewing our work. We greatly value the feedback and have done our best to address each point thoroughly in the responses below.\n\n---\n\n**weakness 1**\n\nWe thank the reviewer for pointing out the importance of the Short-CoT baseline's construction. We acknowledge that the approach of fine-tuning a Long-CoT model (DeepSeek-R1-Distill-Qwen-7B) using 2,000 Short-CoT samples from Qwen2.5-Math-7B-Instruct may appear uncommon. However, this design choice was necessitated by the characteristics of existing Long-CoT models such as the DeepSeek distillation series.\n\nSpecifically, these Long-CoT models adopt a distinct set of special tokens—different from their base or instruct counterparts—as evidenced by their configuration files on Hugging Face. These special tokens are often critical, serving functions such as marking speaker roles and delineating dialogue boundaries. In the context of model merging, it is generally required that participating models share identical special token vocabularies to ensure compatibility. Therefore, a Short-Cot model obtained by SFT from DeepSeek-R1-Distill-Qwen-7B allowed us to maintain consistency in tokenization and avoid incompatibilities caused by divergent vocabularies.\n\nWe agree that the choice and representativeness of the 2,000 Short-CoT samples could influence the performance of the Short-CoT model. However, we emphasize that the final performance of Ada-R1 (as shown in Table 2 and Section 3.2) remains stable and competitive, suggesting that the effect of this approximation on the overall conclusions is minimal.\n\nAdmittedly, a more standard alternative would involve distilling a Long-CoT model from a Qwen-base model using consistent token vocabularies, and using Qwen-instruct directly as the Short-CoT model. Nevertheless, due to the computational and time-intensive nature of large-scale distillation, this approach remains infeasible for academic research settings. We hope the reviewer can understand this resource limitation and recognize our efforts to ensure empirical rigor within these constraints.\n\n---\n\n**weakness 2**\n\nThank you for this valuable suggestion.Specifically, we define:\n\n*Limited Scope*: The model is optimized over a narrow region of the CoT space—typically only one reasoning style, mostly Long-CoT (i.e., “thinking” paths). This restricts the model to produce lengthy explanations regardless of whether such reasoning is necessary. As a result, the model struggles to discover shorter, more efficient solutions even when they exist.\n\n*Broad Scope*: The model is optimized over a wider CoT space that includes both Long-CoT and Short-CoT styles (i.e., both “thinking” and “non-thinking” paths). This enables the model to adaptively choose between concise and detailed reasoning based on the input, leading to higher reasoning efficiency without sacrificing correctness.\n\nIn practice, a model trained under a limited scope (e.g., using only Long-CoT data) tends to over-explain even simple problems, resulting in unnecessarily long outputs. In contrast, our method allows the model to flexibly produce shorter answers for easy tasks while still preserving detailed reasoning when required. This adaptivity is a key contributor to the efficiency gains observed in our experiments.\n\nWe appreciate the reviewer’s feedback and we will add a more detailed explaination in our final version of paper.\n\n---\n\n**weakness 3**\n\nFor small-scale datasets such as AIME25 (30 samples), we indeed recognize the risk of variance. The results reported in the paper are averaged over 4 independent runs to mitigate randomness. We will explicitly clarify this detail in the revised version to better reflect the robustness and reliability of our findings.\n\n---\n\n**weakness 4**\n\nWe agree that group-level preference training already achieves a strong efficiency–accuracy trade-off, reducing average CoT length by 46.03% with only a 3.31% drop in accuracy. However, adding instance-level preference training further improves this trade-off: our full bi-level method achieves a 50.93% length reduction with only a 1.65% accuracy drop—recovering nearly 2% accuracy while gaining additional compression, as shown in Table 3.\n\nMore importantly, instance-level preference serves a complementary optimization role. While group-level training focuses on selecting between long and short reasoning styles based on input difficulty, instance-level training focuses on refining within the chosen group by promoting more concise correct reasoning. These two levels target orthogonal axes —reasoning mode and reasoning verbosity—and optimizing both yields more compact and adaptive reasoning behaviors than either alone.\n\n---\n\n**question 1**\n\nWe choose model merging over mixed training for Stage I due to both **efficiency and stability**. First, merging is **orders of magnitude faster** (e.g., only tens of seconds ), making it far more suitable for hyperparameter search and rapid deployment. Second, mixed training requires careful data balance tuning, which is brittle and often leads to suboptimal trade-offs between long and short reasoning styles.\n\nAdditionally, we trained an additional model(DeepSeek-R1-Distill-Qwen-7B) as base model on mixed CoT data(2k long cot and 2k short cot) but observed worse performance compared to our merged model, despite significantly longer training time in Stage-I.\n\n|  | AIME25 | MATH | GSM8k | Olympiad | Minerva |\n| --- | --- | --- | --- | --- | --- |\n| Ada-R1(mix) | 27.5 (5102) | 83.8 (1021) | 89.6 (279) | 47.1 (2755) | 31.6 (941) |\n| Ada-R1(merge) | 35.8(8426) | 90.2 (1468) | 90.3 (260) | 52.4 (4889) | 34.1 (1647) |\n\nFurthermore, prior work like [1] demonstrates that mixed training can degrade existing capabilities due to task interference, whereas merging preserves task-specific strengths without retraining while [2] presents merging as a cost-effective strategy to unify diverse LLM capabilities post-training, avoiding the instability and inefficiency of retraining. What's more, model merge is very fast and only takes tens of seconds.\n\n[1] Mix Data or Merge Models? Optimizing for Diverse Multi-Task Learning\n\n[2] Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities\n\n---\n\n**question 2**\n\nsee our response to weakness 2\n\n---\n\n**question 3**\n\n\nSpecifically, we set the threshold as  1/(2K), K is the number of candidate responses per question. We will clarify this in the final revision.\n\n---\n\n**Final Remark**\n\nWe once again thank the reviewer for their valuable comments. We hope that our clarifications and revisions have adequately addressed the concerns, and we respectfully ask the reviewer to consider a more favorable score in light of these improvements. If there are any remaining concerns, we would be glad to further clarify them."}, {"review_id": "q9ZLNMCHFY", "reviewer": "Reviewer_dmsV", "summary": "This paper proposes Ada-R1, a hybrid reasoning model that combines long and short CoT models to enable diverse reasoning strategies. Ada-R1 is trained using a bi-level preference training approach: at the group level, the model learns to select appropriate reasoning styles, while at the instance level, it is trained to prefer concise and correct reasoning. Experimental results demonstrate both the effectiveness and efficiency of Ada-R1, achieving strong performance with reduced token usage.", "questions": "1. Could the group-level preference stage be removed, keeping only instance-level preference training, since the latter already accounts for reasoning efficiency during data construction?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The research problem is highly relevant, especially in the context of optimizing test-time computation.\n2. The results are impressive, showing substantial token cost reductions while maintaining performance.\n3. The paper is clearly written and easy to follow.\n\n**Weaknesses**\n1. More out-of-distribution (OOD) and simple reasoning datasets should be included to convincingly demonstrate the model's adaptability. Currently, only two OOD testbeds are used, and the overthinking problem is not exclusive to math. Including a wider range of tasks would strengthen the evaluation.\n2. The definition of group-level preference raises concerns. Since preferences are derived from samples generated by different models—each with distinct problem-solving capabilities—it's questionable whether these preferences can effectively guide the training of a combined model in Stage 1.\n3. The overall training pipeline is quite complex. It involves: 1) training separate models to develop distinct reasoning strategies, 2) merging these models, 3) performing group-level preference training, and 4) conducting instance-level preference training. This multi-step process may limit scalability and practical deployment.", "was_revised": true, "final_justification": "The rebuttal addresses my questions well. I will keep the scores.", "rebuttal": "We thank the reviewer for the time and effort spent reviewing our work. We greatly value the feedback and have done our best to address each point thoroughly in the responses below.\n\n---\n\n**weakness 1**\n\nThank you for this valuable suggestion. To demonstrate our model’s out-of-distribution adaptability, we applied the version trained on mathematical reasoning to two classic OOD benchmarks: MMLU and GPQA. Below are the results:\n\n| Model (7B) | MMLU(stem) | MMLU(humanities) | gpqa |  |\n| --- | --- | --- | --- | --- |\n| Base | 43.3 (2336) | 16.3 (1476) | 30.3 (7302) |  |\n| DPO | 48.3(1577) | 21.0 (1270) | 31.3 (6592) |  |\n| Ada-R1 | 54.6 (1909) | 30.9 (930) | 32.8 (5101) |  |\n\nThese results show that our model maintains strong performance, and shorter generation lengths—beyond the math domain. We will include the GPQA evaluation in the final manuscript to further substantiate our claims.\n\n---\n\n**weakness 2**\n\nWe appreciate the reviewer’s insightful concern regarding the validity of group-level preferences. To address this, we conducted a focused loss-based analysis to examine whether the distributional shift between the merged model and the pre-merged models is substantial.\n\nSpecifically, we compute the average loss of the merged model on Long-CoT and Short-CoT samples respectively, and compare these values with the original Long-CoT and Short-CoT models.\n\n| Model | Long-CoT Loss | Short-CoT Loss |\n| --- | --- | --- |\n| short | * | 0.1570 |\n| long | 0.2106 | * |\n| merge | 0.2141 | 0.2028 |\n\nAs shown, the loss of the merged model on both Long-CoT and Short-CoT samples remains close to that of the respective original models. This indicates that the merged model retains the reasoning capabilities of both models and is thus well-positioned to leverage group-level preference signals during Stage 2 fine-tuning. These results support the effectiveness of our Bi-Level Preference Training setup, where the merged model benefits from a broader reasoning space while being guided by meaningful preference supervision.\n\nMoreover, the effectiveness of this setup is also reflected in both our main results and ablation studies, where applying group-level preference training to the merged model leads to substantial efficiency gains with limited accuracy degradation.\n\n---\n\n**weakness 3**\n\nThanks for you feeback. We would like to clarify following points: \n\n(1) Training separate models is not strictly required. In practice, one can directly utilize existing long-CoT and short-CoT models. In our experiments, we re-trained a short-CoT model only because the released long-CoT model from DeepSeek uses a completely different special token vocabulary (e.g., beginning-of-text tokens, user tokens, etc.) from its base model. To ensure fair comparison and academic rigor, we avoided introducing this confounding factor by training a compatible short-CoT model. \n\n(2) The model merging step is lightweight and efficient. We use the open-source framework Mergekit, and the merge operation completes in under one minute. \n\n(3 & 4) Stage 2 training is conducted as a unified process. While we refer to both group-level and instance-level preferences, the training is carried out jointly using a combined dataset that includes samples from both sources. Our loss function is designed to accommodate both types of supervision simultaneously, and the data sampling is completed in a single pass.\n\nOverall, the pipeline is not very complex in practice and can be implemented efficiently using off-the-shelf tools and pre-existing models.\n\n---\n\n**question 1**\n\nWe appreciate the reviewer’s suggestion. While instance-level preference training does incorporate reasoning efficiency during data construction, our experiments indicate that group-level preference provides complementary benefits that are not fully captured by instance-level training alone.\n\n| Model | AIME25 | MATH | GSM8k |\n| --- | --- | --- | --- |\n| Long | 38.3 (11005) | 90.2 (3534) | 88.9 (1014) |\n| Merge | 21.7 (9079) | 79.4 (916) | 88.4 (236) |\n| Merge + SFT | 35.8 (11222) | 84.6 (2314) | 88.7 (375) |\n| Merge + group level | 30.8 (9049) | 87.8 (1565) | 91.6 (359) |\n| Merge + instance level | 24.2 (8514) | 81.6 (886) | 87.95 (212) |\n| Merge + bi level | 35.8 (8426) | 90.2 (1468) | 90.3 (260) |\n\nAs shown in the table, **Merge + group level** achieves significantly better performance than **Merge + instance level** on **MATH** (87.8 vs. 81.6) and **GSM8k** (91.6 vs. 87.95), while maintaining a higher score on **AIME25** (30.8 vs. 24.2). This suggests that group-level preferences help promote more robust generalization and better tradeoffs across tasks. Furthermore, **bi-level training**, which combines both group- and instance-level preferences, consistently performs the best or on par with the best across all benchmarks, further supporting the complementary nature of the two.\n\nIn particular, our merged model inherits two distinct reasoning styles from its component models. Instance-level preference training focuses on fine-grained selection between individual outputs, but it does not explicitly guide the model in choosing between these broader reasoning modes. Therefore, removing the group-level stage would lead to notable performance drops. We believe this justifies the inclusion of group-level preference training in our framework."}, {"review_id": "j39OMhv8ia", "reviewer": "Reviewer_NgJ5", "summary": "The authors propose Ada-R1, a two-stage framework for adaptive and efficient reasoning. First, a hybrid model is created by merging Long-CoT and Short-CoT models to enable diverse reasoning styles. Then, bi-level preference training is applied: group-level preference selects suitable reasoning styles based on input complexity, while instance-level preference encourages concise and correct reasoning within each style.", "questions": "1. The current experiments focus exclusively on mathematical reasoning tasks (e.g., GSM8K, MATH). How do the authors expect Ada-R1 to perform on other domains (e.g., natural language inference, coding)? Are there plans to validate the framework on non-mathematical tasks? Particularly, i'd like to see the performance on logical reasoning benchmarks such as LogiQA.\n2. The paper reports performance differences (e.g., length reduction, accuracy changes) but does not include error bars, confidence intervals, or statistical tests (e.g., t-tests). Are the observed improvements statistically significant, or could they be due to random variation?\n3.  The paper focuses on reasoning length but does not address the latency or computational cost of dynamically selecting between Long-CoT and Short-CoT during inference. How does the adaptive selection process impact real-time performance?\n4. Recommended citation: Break the chain: Large language models can be shortcut reasoners", "limitations": "Clearly state in the main text the limitations of the current approach, such as generalization to non-mathematical domains, the choice of linear merging, and the reliance on synthetic Short-CoT models.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strenghts:\nThe two-stage design—merging Long-CoT and Short-CoT models followed by bi-level preference training (group-level style selection and instance-level compression)—is both theoretically sound and empirically validated. \nThe authors ensure reproducibility by detailing training procedures, hyperparameters, and dataset compositions. They also plan to release model weights, fostering further research. \nAdaptive reasoning analysis showing that Ada-R1 appropriately increases Long-CoT usage for harder problems (Level 5 in MATH) while favoring Short-CoT for simpler tasks, aligning with theoretical expectations. The contribution is clear.\n\nWeaknesses:\nWhile the experiments focus on mathematical reasoning (GSM8K, MATH, etc.), the paper does not address how Ada-R1 performs on other task types (e.g., logical reasoning, coding, or multi-step decision-making). \nThe paper’s reliance on linear merging might overlook more sophisticated interactions between the two reasoning styles.\nThe paper does not report error bars, confidence intervals, or statistical tests (e.g., t-tests) to validate the significance of performance differences between Ada-R1 and baselines.\nThe Short-CoT models are fine-tuned from Long-CoT models using a small dataset (2,000 samples), which might not fully capture the diversity of short reasoning patterns. This setup could introduce bias, as the Short-CoT models are not independently trained but derived from their Long-CoT counterparts. Comparing against standalone Short-CoT models (trained from scratch) would strengthen the validity of the hybrid approach.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for these insightful observations. We address each concern below:\n\n**weakness 1: While the experiments focus on mathematical reasoning (GSM8K, MATH, etc.), the paper does not address how Ada-R1 performs on other task types**\n\nOur primary experiments focus on mathematical reasoning (e.g., GSM8K, MATH, AIME), and we also evaluate OOD dataset **Olympiad** and **Minerva**（In paper Table2），also on the **LogicQA, GPQA, MMLU**. Results show that Ada-R1 continues to exhibit strong efficiency-accuracy trade-offs beyond purely mathematical tasks, supporting its generalizability to other reasoning domains. We plan to further explore other tasks in future work. (prompt is Let’s think step by step and output the final answer within \\boxed{}. (e.g., \\boxed{A}) with zero shot)\n\n| Model | LogiQA | **GPQA** | MMLU(stem) | MMLU(humanities) |  |\n| --- | --- | --- | --- | --- | --- |\n| Long | 45.1 (3269) | 30.3 (7302) | 43.3 (2336) | 16.3 (1476) |  |\n| DPO | 44.2 (2650) | 31.3 (6592) | 48.3(1577) | 21.0 (1270) |  |\n| AdaR1(ours) | 48.6 (2794) | 32.8 (5101) | 54.6 (1909) | 30.9 (930) |  |\n\n---\n\n**weakness 2: The paper’s reliance on linear merging might overlook more sophisticated interactions between the two reasoning styles**\n\nWe acknowledge that our current merging strategy adopts a simple linear interpolation of Long- and Short-CoT models, which may not capture all possible interactions. However, this simplicity is intentional for two reasons: (1) The goal of Stage I is **not to coordinate** the two reasoning modes but merely to **enable the hybrid model to represent both styles** Coordination and preference learning are then handled effectively in **Stage II (bi-level preference training)** (2) The linear merging provides a **transparent and interpretable baseline**, making it easier to isolate and analyze the effectiveness of group- and instance-level preferences, as demonstrated in our ablation studies.\n\n---\n\n**weakness 3: The paper does not report error bars, confidence intervals, or statistical tests (e.g., t-tests) to validate the significance of performance differences between Ada-R1 and baselines.**\n\nIn line with common practices in this field (e.g., [1][2][3][4][5][6][7][8]), we report **point estimates** on standard benchmarks following the evaluation protocols established in prior work. For smaller test sets such as AIME25, we **run 4 random sampling rounds and report the average** to mitigate variance, we will explicitly clarify this detail in the revised version. \n\n---\n\n**weakness 4: The Short-CoT models are fine-tuned from Long-CoT models using a small dataset (2,000 samples), which might not fully capture the diversity of short reasoning patterns.**\n\nWe trained a short model because Deepseek uses a different special token vocabulary for distillation, making it impossible to directly merge with these short cot models. Furthermore, we tested the existing short cot model (Qwen2.5-7B-Instruct), and found that it performed worse than the model we trained.   Based on these considerations, we chose to fine-tune the new short cot model.\n\n|  | AIME25 | MATH500 | GSM8K | Olympiad | Minerva |\n| --- | --- | --- | --- | --- | --- |\n| Qwen2.5-7B-Instruct (off-the-shelf) | 7.5 (1121) | 76.0 (574) | 90.1(280) | 37.2(977) | 24.3(613) |\n| Merge | 21.4 (9079) |  79.4 (916) |  88.4 (236) |  41.2 (3743) | 25.7 (1734) |\n| Short (in paper) | 10.0 (957) | 78.6 (591) | 89.5 (272) | 39.4 (910) | 28.6 (579) |\n\n---\n\n**References**\n\n[1] CoThink: Token-Efficient Reasoning via Instruct Models Guiding Reasoning Models\n\n[2] L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning\n\n[3] Scalable Chain of Thoughts via Elastic Reasoning\n\n[4] Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards\n\n[5] Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models\n\n[6] ThinkSwitcher: When to Think Hard, When to Think Fast\n\n[7] Let LLMs Break Free from Overthinking via Self-Braking Tuning\n\n[8] REST: Stress Testing Large Reasoning Models by Asking Multiple Problems at Once\n\n---\n\n**question 1**\n\nSee our response to weakness 1\n\n---\n\n**question 2**\n\nSee our response to weakness 3\n\n---\n\n**question 3**\n\nThe adaptive selection in Ada-R1 does not rely on an external module or classifier to decide between Long-CoT and Short-CoT. Instead, the selection is implicitly learned and performed within the model’s standard autoregressive decoding. Thus, there is no additional latency or computational overhead during inference. Moreover, by significantly reducing the number of generated tokens (e.g., over 50% reduction on average in Table 2), Ada-R1 naturally improves real-time inference performance.\n\n---\n\n**question 4**\n\nWe fully agree with your suggestion and will add this citation to our paper in future revisions\n\n---\n\n**limitations**\n\nThank you for this valuable feedback. We agree that the manuscript should explicitly clarify the limitations of our approach, such as the challenges it may encounter when extended beyond mathematical domains. We will add a detailed discussion in the revised version to further enhance the paper’s completeness and transparency."}]

Author rebuttal:

Meta-review: Summary of the paper: Ada-R1 is a two-stage training recipe that equips LRMs with the ability to decide, on a per-problem basis, how long CoT is necessary. Stage 1 fuses a long-CoT model (high-accuracy, verbose) and a short-CoT model (lower-cost, concise) into a single hybrid backbone that can emit both reasoning styles. Stage 2 then performs bi-level DPO: (i) a group-level loss teaches the model to pick the style whose historical accuracy matches the perceived difficulty of the prompt, and (ii) an instance-level loss rewards concise yet correct reasoning within the chosen style. Across 5 mathematical benchmarks (MATH, GSM8K, OCWCourses, OlympiadBench, AMC), Ada-R1 cuts the average number of reasoning tokens by roughly 50 % (up to 60 % on easier splits) while holding task accuracy nearly constant (drops ≤ 1 %). A post-hoc analysis shows that Ada-R1 automatically allocates longer reasoning chains to the hardest MATH Level-5 problems and shorter ones to the easiest, matching the intuitive “effort-vs-difficulty” curve.

Strengthes of the paper:

1. Novel and timely idea: One of the early efforts that trains a single model to dynamically trade off accuracy and inference cost by selecting short vs. long reasoning on the fly after many LRMs are released early this year.

2. Technically solid design: The hybrid merging step avoids catastrophic forgetting, and the bi-level DPO objective cleanly disentangles “which style” from “how concise.”

3. Strong empirical evidence: Extensive experiments on math datasets and beyond, ablations on each training stage, and detailed token/accuracy curves.

4. Reproducibility: Authors submit the paper with code, including training recipes, hyper-parameters, data splits.

Weaknesses of the paper: After reading the rebuttal, I think the authors' response addresses most of the reviewers' concerns and questions. No major concerns are identified for this paper. Do incoporate the content during rebuttal into the camera-ready version of the paper.

Reasons for the decision: This paper attacks a pressing problem—test-time compute efficiency in LRMs—and solves it with a conceptually simple yet technically rigorous method. The reduction in token cost is not incremental (≈50 %), and the fact that this is achieved without hurting accuracy makes the result non-trivial. Reviewers unanimously praise the novelty, clarity, and empirical strength, and no major concerns (missing baselines, unfair comparisons, or reproducibility issues) were raised.
Decision: Accept (poster)