QFFT, Question-Free Fine-Tuning for Adaptive Reasoning
Authors: Wanlong Liu
Junxiao Xu
Fei Yu
Yukang Lin
Ke Ji
Wenyu Chen
Lifeng Shang
Yasheng Wang
Yan Xu
Benyou Wang
Venue: NeurIPS 2025 spotlight ()
Keywords: Large language models
Efficient Reasoning
Large reasoning model
Math reasoning
Primary area: deep_learning
TL;DR: We propose  a new distillation approach that removes the input question for adaptive and efficient reasoning.

Abstract: Recent advancements in Long Chain-of-Thought (CoT) reasoning models have improved performance on complex tasks, but they suffer from overthinking, which generates redundant reasoning steps, especially for simple questions. This paper revisits the reasoning patterns of Long and Short CoT models, observing that the Short CoT patterns offer concise reasoning efficiently, while the Long CoT patterns excel in challenging scenarios where the Short CoT patterns struggle. To enable models to leverage both patterns, we propose Question-Free Fine-Tuning (QFFT), a fine-tuning approach that removes the input question during training and learns exclusively from Long CoT responses. This approach enables the model to adaptively employ both reasoning patterns: it prioritizes the Short CoT patterns and activates the Long CoT patterns only when necessary. Experiments on various mathematical datasets demonstrate that QFFT reduces average response length by more than 50\%, while achieving performance comparable to Supervised Fine-Tuning (SFT). Additionally, QFFT exhibits superior performance compared to SFT in noisy, out-of-domain, and low-resource scenarios.

QFFT, Question-Free Fine-Tuning for Adaptive Reasoning
Wanlong Liu Junxiao Xu" Fei Yu², Yukang Lin', Ke Ji Wenyu Chen'
Yan Xu Yasheng Wang2, Lifeng Shang?. Benyou Wang* 2f
University of Electronic Science and Technology of China, Chengdu, China 2 The Chinese University of Hong Kong, Shenzhen 3 Huawei Noah's Ark Lab
Abstract
Recent advancements in Long Chain-of- -hhuuuhh (CoT) reasoning models have improved performance on complex tasks, but they suffer from overthinking. which generates redundant reasoning steps, especially for simple questions. This paper revisits the reasoning patterns of Long and Short CoT models, observing that the Short CoT patterns offer concise reasoning efficiently, while the Long CoT patterns excel in challenging scenarios where the Short CoT patterns struggle. To enable models to leverage both patterns, propose Question-Fiee Freee Fine-'Tuning (QFFT), a fine-tuning approach that removes the input question during training and learns exclusively from Long CoT responses. This approach enables the model adaptively employ both reasoning patterns: it prioritizes the Short CoT patterns and activates the Long CoT patterns only when necessary. Experiments on various mathematical datasets demonstrate that QFFT reduces average response length by more than 50%. while achieving performance comparable to Supervised Fine- Tuning (SFT). Additionally, QFFT exhibits superior performance compared to SFT in noisy, out-of-, and low-resource scenarios. QFFT is publicly available Tuning
Introduction
Recent advancements in Long CoT reasoning models, such OpenAl ol [ and DeepSeck-RR R1 have significantly improved performance on complex tasks, such as mathematics and coding 3
(
77 These improvements are largely attributed the test-time scaling paradigm, where models generate Long CoT, consuming more tokens, to effectively simulate human-like deep thinking behavior like self-reftection, error correction, and exploration of multiple solution strategies. Building on this success, a growing body of research has focused on distillation methods that transfer the Long CoT reasoning abilities of OpenAl ol and DeepSeek-RI into smaller LLMs, achieving notable
performance gains [ 9 10 2
[2] have identified a critical limitation in current Long CoT models (c.g.. DeepSeck-RII RRI): they often exhibit overthinking, generating unnecessarily complex or redundant reasoning steps even for simple problems. As result, models distilled from these Long CoT models tend to inherit this drawback, leading to inefficiencies during inference.
However, recent studies
To mitigate this issue, recent Long-to- to-oort methods 13
explore compressing the length of Long CoT responses. However, these methods incur substantial additional training costs while achieving only limited reductions in token usage.
14
15
16 17
Wanlong Liu and Junxiao Xu contributed to this work equally. Yan Xu and Benyou Wang are the corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

For Difficult Questions
For Simple Questions (Both Models can Solve)
75.1%
74.8%
AIME 24: 18.1
AIME 24: 72.6
Avg, Token: 415
Avg. Token: 1646
Concise
Overthinking
Qwen-32B-nnstruct (Short CoT)
RI-Distill-22B (Long CoT)
Owen-32BBinntruet (Short CoT)
RI-Distill-22B (Long CoT)
Figure 1: Comparison of short and Long CoT reasoning patterns. The left subplot shows that the Long CoT patterns perform better on difficult questions, while the right subplot demonstrates that, for simple questions solvable by both, the Short CoT patterns provide significantly more concise answers.
In this paper, we compare the reasoning patterns Short CoT models and Long CoT models: (1) For efficiency: Short CoT models provide concise and efficient reasoning, whereas Long CoT models often generate lengthy outputs, which typically involve unnecessarily complex or redundant steps for simple problems. (2) For effectiveness: Short CoT models struggle with more difficult problems due to their simplistic reasoning patterns, while Long CoT models, with their reflective reasoning, demonstrate clear advantages in such scenarios. This contrast motivates natural question: Can design models that adaptively select between Short CoT and Long CoT reasoning patterns? Such models would ideally combine the strengths of both reasoning patterns employing Short CoT reasoning for simple problems to enhance efficiency, and leveraging Long CoT reasoning for difficult problems to pursuit effectiveness.
We revisit current Long CoT Supervised Fine-Tuning (SFT): models are trained on large-scale (question, Long CoT response) pairs, suffering from overthinking issues. Inspired by [18], we hypothesize that this issue arises because SFT enforces the mapping from questions Long CoT responses, overriding the model's original Short CoT patterns and causing it to apply Long CoT reasoning indiscriminatel--eeee
when concise reasoning would suffice. To design models that could leverage both Short CoT and Long CoT patterns for better balance between efficiency and effectiveness, we set two goals: (1) preserve the model's default Short CoT reasoning patterns, and (2) enable it to acquire Long CoT patterns that trigger reflective behaviors when facing uncertainties or errors. To achieve these objectives, we propose Question- Free Fine- Tuning (QFFT), a fine-tuning method that discards input questions and only fine-tunes Long CoT responses. This approach avoids overriding the Short CoT patterns while still enabling the model to learn reflective reasoning patterns of Long COT.
Our extensive experiments demonstrate that the QFFT method adaptively integrates both reasoning patterns: it employs Short CoT reasoning for simple problems to enhance efficiency, and leverages Long CoT reasoning for more challenging problems. Further analysis reveals that the QFFT model prioritizes Short CoT patterns by default, transitioning refleetive Long CoT reasoning when encountering errors or uncertainties. Notably, QFFT achieves performance comparable to SFT on six math datasets, while significantly reducing the average number of generated tokens by up to 50%, thereby effectively mitigating the issue of overthinking. Moreover, QFFT exhibits superior performance compared to SFT in noisy, out-of-domain, and low-resource scenarios.
2 Towards Adaptive Reasoning
2.1 Definition of Adaptive Reasoning
Chain-of- Thhuught (CoT), since its introduction, been widely adopted in various models (e.g., Breen..5-3B--Instteee significantly enhancing its reasoning capabilities. With recent advancements, modern large reasoning models (e.g., DeepSeek-R1) increasingly employ Long CoT reasoning patterns, characterized by reflective verification behaviors [19 6). Reasoning patterns exhibiting

---

Table 1: Comparison of Reasoning Modes: Short CoT, Long CoT, and Adaptive Reasoning
Short CoT
Long CoT
Adaptive Reasoning
Reasoning Style
Direct, concise.
Reflective, self-corretion, con-
Adaptively uses Short/Long
taining reflective keywords (c.g.. "wait"),
CoT based on question difficulty
Application Scenarios
Effective for simple questions.
Strong difficult questions.
Optimal for both simple and dif- ficult questions.
Cost
(fewer tokens).
High (redundant tokens).
Moderate (adaptive needs).
Risk
Oversinplification.
Overthinking.
Balanced (mitigates both risks).
these reflective verification behaviors can be categorized Long CoT [20]. whereas traditional CoT reasoning patterns lacking such behaviors are classified as Short CoT.
Despite the effectiveness of Long CoT in difficult questions, Short CoT maintains efficiency advan- tages for simple questions. This creates a fundamental trade-off. As shown in Table the reflective capabilities of Long CoT' improve accuracy but induce "overthinking" resulting in approximately 74.8% redundant tokens on simple questions; conversely, the conciseness of Short CoTT, while effi- cient, risks "oversimplificatoon" leading to 75.1% 1% average accuracy decline in difficult questions (shown in Figure
Key Observation I: Long CoT patterns significantly outperform Short CoT difficult questions, whereas Short CoT patterns achieve comparable performance with substantially fewer tokens on simple ones.
Key Observation motivates us to explore methods to develop adaptive models to flexibly use Long CoT and Short CoT reasoning patterns based on question difficulty. In this paper, we formally define this capability as Adaptive Reasoning:
Definition 1 Adaptive Reasoning refers to model's ability to adaptively prioritize Short CoT reasoning patterns for simple questions and prioritize Long CoT reasoning patterns for difficult questions when Short CoT is ineffective.
Here, simple questions are those solvable by the model using Short CoT patterns, while difficult questions are those for which Short CoT fails.
2.2 Quantitative Metric Adaptive Reasoning Ability
The adaptive reasoning capability of a model can be evaluated by measuring the alignment between its reasoning patterns and question difficulty. Here, question difficulty is based whether the Short CoT of the evaluated model can correctly answer the question.
Assumption If Long CoT model ML is derived (e.g. distilled) from a Short CoT model Ms, we approximate the Short CoT capability of model ML by that of model Mg.
Consequently, introduce the Short CoT model Ms reference model to estimate whether the evaluated model can correctly answer the question using Short CoT reasoning.
We introduce the Reasoning Adaptability Cohen's Kappa (RAK), inspired by Cohen's Kappa 21 a statistical measure evaluating the agreement between two raters beyond chance. In our context, the two "raters" correspond to the question difficulty provided by the reference model (simple or difficult) and the reasoning patterns (Short or Long CoT) used by the evaluated model.
Definition 2
Reasoning Adaptability Cohen's Kappa (RAK). If the Long CoT model Mr. is derived (e.g., distilled) from the original Short CoT model Ms (reference model), RAK measures the performance of model Mr in adaptively selecting the appropriate reasoning pattern, accounting for

---

100
S1.1-SFT-78
45.7
Short Col
40
R1-Disbil Qwen-iBB
$1.1
Ratio of Short CoT (%)
RAK
35
R1-Distill Dwon-328
RAK
30
51.1-QFFT-78
11-1-QFFT-328
60-
25
40-
20
10
110
4.2
Models
0%
0.1%
1%
10%
200%
Ratio Question (%)
(a)
(b)
Figure 2: Subfigure (a) presents the Reasoning Adaptability Cohen's s Kappa (RAK) scores different Long CoT models. Subfigure (b) illustrates that during Long CoT SFT. the Q (question) - R (Long Co'T) mapping leads to Long CoT patterns overriding the model's 's original Short CoT patterns.
chance agreement.
Po -
RAK
1 Pc.
where the observed agreement
TP+ and the expected agreement Pe.
TP+TX
(T computed based on the marginal probabilities of the predicted and (TP+ (TP4PP DX)+
actual classes
A higher RAK indicates greater consistency between the evaluated model's reasoning pattern selection and task difficulty, demonstrating stronger adaptive reasoning capability. Conversely, a lower RAK reflects weaker consistency, indicating poorer adaptive reasoning capability.
2.3 Pilot Study on Adaptive Reasoning
We evaluate the reasoning adaptability of existing Long CoT models on MATH500 [2] dataset. As shown in Figure 2a we observe that current Long CoT models (obtained by SFT) have low RAK scores, which attributes to their consistent over-reliance on Long CoT reasoning patterns.
Hypothesis. In Long CoT SFT, models trained on large-scale (Q: question, R: Long CoT response) pairs, teaching them to solve questions with Long CoT reasoning patterns. This training paradigm poses a potential risk [18]: models may default to Long CoT reasoning for all inputs, a phenomenon we term Override. We hypothesize that this phenomenon occurs because the new Q R mapping-from questions to Long CoT Sepoones--vverniies the original mapping Short CoT responses, causing the model to over-rely on Long CoT patterns.
To verify this hypothesis, conduct the following experiment: select proportion a (0 < a 1) of training samples and retain their original question-respons. pairs. For the remaining (1 - a) proportion of samples, remove the questions, keeping only the Long CoT responses. We then evaluate how the ratio of the model's Short CoT changes as we gradually increase the value of a.
during Long CoT SFT, randomly
Results. As shown in Figure 2b. when questions are removed from all training samples, the model exhibits over 50% Short CoT patterns and does not over-rely on Long CoT reasoning. However, once samples with questions are included, even with a minimal logarithmic-scale increase (from 0.1 % 1%), the proportion of Short CoT patterns drops dramatically (from 40.95% to 13.24%). This
"Specifically, TP is the number of solvable questions of Ms where model Mr. selects Short CoT patterns, FWl is the number of solvable questions of Ms where Mr. selects Long CoT patterns, FP is the number of unsolvable questions of Ms where model Mr, selects Short CoT patterns, and TN the number of unsolvable questions of Ms where model Mr selects Long CoT patterns. Here, "solvable" indicates that reference model Ms answers the question correctly.
We will discuss Section why significant number of Long CoT patterns still appear.

---

indicates that even an extremely small proportion of Q - R mappings is sufficient to override the model's original Short CoT patterns, leading to over-reliance on Long CoT reasoning.
Key Observation 2: Current Long CoT models over-rely on Long CoT patterns. This is because the questions (Q) Long CoT responses (R) mapping in SFT stage overrides their original Short CoT patterns.
Methodology: Question-Free Line-Tuning
3.1
Motivation
To achieve adaptive reasoning, we aim for the model to (1) default Short CoT patterns, while (2) adaptively triggering the reflective Long CoT behaviors when enrors or uncertainty occur.
First, preserve the model's 's default Short CoT patterns and prevent it from being overridden, need to avoid training the model to learn a fixed Q - R mapping.
Second, the model should effectively learn Long CoT reasoning patterns, which could trigger reflective behaviors when the model uncertain about its solution or encounters mistakes. Prior studies 66 23) suggest that the core of Long CoT patterns lies in the structure of responses, rather than the questions. This implies that models distilled solely from Long CoT responses, even without access to the corresponding questions, can still acquire Long CoT reasoning capability.
The Method
To preserve the original Short CoT patterns while enabling adaptive switching to reflective Long CoT patterns for challenging questions during distillation, we propose Question-Free Fine-Tunine (QFFT). Different from SFT, remove the input question Q entirely, training the model exclusively on the reasoning response R. Specifically, the model is optimized using a standard causal language modeling objective over the reasoning sequence:
COFFT
Σ
log (o. Rct.
R
where R is the set of token indices in the response sequence R and Po is the model's output probability. Φ indicates that QFFT removes the question component compared SFT.
By omitting the question Q during training and training solely on the reasoning response R, QFFT avoids learning fixed mapping from questions Long CoT responses while still acquiring Long CoT reasoning capability.
3.3
Why QFFT Leads to Adaptive Reasoning?
We explain why QFFT achieves adaptive reasoning from both training and inference stages.
Training: Preserving Short CoT and Learning Reflection. During the QFFT training, the model avoids learning direct mapping from questions (Q) Long CoT responses (R), thereby preserving the model's default Short CoT patterns to answer questions. Additionally, the model is trained exclusively on Long CoT responses. Thus, the model theoretically acquires the capability for Long CoT reasoning and learns to exhibit reflective behaviors when encountering uncertainty errors within the context of Long CoT reasoning. Formally, let Un denote the event of encountering uncertainty or errors during Long CoT reasoning. Then, the model learns the conditional probability:
PB UL).
where B, denotes the occurrence of reflective behaviors, and Ur represents uncertainty or errors arising specifically within the context of Long CoT reasoning.

---

Inference: Default to Short CoT with Adaptive Reflection. At inference time, the QFFT model defaults the Short CoT patterns. However, since the model has only explicitly learned the conditional probability Po Br Uz) during training, it is not immediately obvious why it can still trigger reflective behaviors in the context of Short CoT reasoning. We explain this phenomenon from a transfer learning 24 25 perspective:
Assumption If a model has learned the conditional probability Po(B, UL). this reflective capability can be transferred to Short CoT, enabling it to implicitly learn Po Us), where Us denotes uncertainty or errors that arise specifically within Short CoT reasoning.
We empirically verify this assumption with experiments in Appendix F.5 Consequently, when the model detects errors or uncertainty in Short CoT reasoning context, it naturally triggers reflective behaviors to reconsider and correct its reasoning process.
In summary, QFFT enables the model reason in adaptive manner: it defaults to efficient Short CoT patterns and adaptively uses reflective Long CoT patterns when necessary, thus achieving both effectiveness and efficiency.
4 Experiments
In this section, we conduct experiments on both in-domain and out-of-domain datasets to validate the effectiveness of QFFT. Additional results (e.g. on different backbones) can be found in the appendix.
Table 2: Main results on 3 mathematical benchmarks. The reported accuracy and Avg. Len (Tokens)
averaged over 16 random sampling runs. RAK is defined reasoning adaptability metric.
Data
Method
GSMSK
MATH
AIME25
Average
Ace
Tokens
RAK
Tokens
RAK
Tokens
RAK
Tokens
RAK
7B Models (Based on Owen2.S-7PP 5-7-5-5-t-s-tcc
SFT
90.6
1.7K
1.8
80.8
5.3K
3.5
18.2
17.7K
0
63.2
8.2K
1.8
S1.1
QFFT
91.0
0.4K
28.4
2.8K
47.7
12.8K
28.0
62.8
5.3K
34.7
+0.4
-6..5%
+26.6
.0.6
47.2%
+44,2
-1.0
.2.7%
+28.0
-0.4
-30.5%
+32.9
SFT
88.2
1.8K
0.2
80.4
5.8K
6.1
16.8
17.1K
0.2
61.8
8.2K
2.2
LIMO
QFFT
88.0
0.7K
26.7
80.6
4.1K
40.1
17.2
15.6K
34.2
61.9
6.8K
33.7
-61.1%
+26.5
-293%%
+31.
SFT
91.0
1.4K
2.3
81.6
5.7K
4.6
19.8
13.8K
0.1
64.1
7.0K
2.3
BS-17K
QFFT
90.4
0.4K
29.7
81.4
2.2K
44.6
18.3
9.7K
29.7
63.4
4.1K
34.7
A
-0.6
-71.4%
+27.4
-0.2
-61.%
+40.0
-1.5
-29.7%
+29.6
-0.8
-54,2%
432.3
32B Models (Based
SFT
92.8
2.1K
0.2
93.1
4.1K
4.2
48.6
16.2K
0
7.5K
1.5
SI.1
QFFT
93.6
0.6K
31.2
92.2
2.4K
45.7
46.8
12,9K
29.3
77.5
5.3K
35.4
A
+0.8
-71.4%
+31.0
-0.9
-11.5%
+41.5
-1.8
-20.4%
29.3
-0.6
-44.4%
+33.9
SFT
91.2
1.9K
1.0
93.0
3.9K
8.8
45.8
13.2K
0
76.6
6.3K
3.3
LIMO
QFFT
92.6
0.8K
27.4
92.6
2.9K
38.9
45.0
12.5K
33.2
76.7
5.4K
33.2
+1.4
-57.9%
+26.4
-044
+30.1
-5.3%
+33.2
+0.1
-9.66
+29.9
4.1 Experimental Setup
Dataset. We select several high-quality distillation datasets, including S1.1 (1k) 19. LIMO (871) [10) and Bespoke- Stratos (17k) [8], where all responses are distilled from the DeepSeek- RI model. Detailed data description is shown in Appendix E.I
Training details. We use S.ven2.5-Inntruuct and Owen2.55 26 as the base models. All experiments are conducted with LLaMA Factory [27]. with a maximum sequence length of 16,384 tokens. Detailed hyperparameters can be found the Appendix E.2
Evaluation. We evaluate model performance on six in-domain math datasets, including two sim- ple datasets: GSM8K (28 and MATH500 22) two medium-difficulty datasets: the American Mathematics Competitions (AMC23) and Minerva 29 which includes undergraduate-level STEM

---

Table 3: Comparison with other Long-to- to-rro baselines. AES is used as metric that balances performance and response length.
Method
GSMMK
MATH
AIME25
Average. Tokens
Acc
Tokens ; AES
Acc
Tokens
AES
Acc
Tokens
AES
Acc
AES
Loug Short Methods (7B)
LIMO 7B (basc)
88.2
L8K
80.4
5,9K
16.8
17.8K
61.8
8.5K
SFT Shortest
88,9
1.2K
3,4
78.3
-0.7
17.3K
0.9
61.7
1.2
DPO Shortest
89.8
1.6K
1.5
79.5
5.4K
-0.1
17.3
17.1K
0.7
62.2
8.0K
0.7
SimPO Shortest
87.2
1.2K
2.2
75.8
3.2K
-1.0
14.0
8.8K
-12.0
59.0
4.4K
-3.6
O1-pruner
90.8
0.8K
5.8
78.2
1.8
14.2
12.2K
-12.7
61.0
5.4K
-1.7
Distilled Methads (7B)
DAD-7B
90.0
0.9K
5.2
80.2
4.8K
1.6
17.3
17.7K
0.3
62.5
7.8K
2.4
QFFT (Ours)
5.9
Long-to-Shoo Shhor Methods (32B)
LIMO 32B (buse)
91.2
1.9K
93.0
3.9K
45.8
13.2K
76.7
6.3K
SFT Shortest
93.2
9.2K
-38.2
91.4
3.1K
0.4
46.3
14.5K
-0.9
76.9
8.9K
-12.9
SimPO shortest
94.5
0.6K
7.2
89.8
2,4K
0.4
40.0
13.0K
-12.5
74.8
6.7K
-1.6
O1-pruner
94.8
0.6K
7.2
90.5
2.0K
2.1
29.2
7.3K
-136.9
71.5
3.3K
-42.5
Distilled Methods (328)
QFFT (Ours)
92.6
0.8K
5.9
92.6
2.9K
2.1
45.0
12.5K
-1.2
76.7
5.4K
2.3
problems, and two high-difficulty datasets: AIME 2024 and AIME 2025. Additionally, we also evaluate on out-of-domain non-math datasets (in Section (B22) GPQA (D) and MMLU--Po B1 We report average accuracy by generating 16 responses per question using a sampling temperature of 0.6 and maximum decoding length of 32,768 tokens. For calculating the Reasoning Adaptability score, we average the RAK scores of 16 samples to mitigate random fluctuations. Specifically, for the 7B SFT and QFFT models, we use Qwen2.5-77 7a-..---7-Instrree as the reference model, while for the 32B models, we use Owen2.5-322B-o ---32B-Insttuct. Further details regarding prompts and inference settings are provided in Appendix E.3
4.2 Baselines
Puffiuulty-ddative Distillation. To enable adaptive reasoning, a straightforward approach is Difficulty-A Adaptive Distillation (DAD), which combines short and Long CoT data for SFT based on question difficulty. Specifically, we use GSM8K as simple questions and LIMO as challenging questions, distilling Short-CoT responses for the former and Long-Cot responses for the latter. The combined data is then used SFT. Further details are provided in Appendix E.4
Long-to-Short Baselines. We compare with four competitive Long-to- thort baselines: (1) SFT- Shortest [I] SFT on the shortest correct responses directly. (2) DPO-Shortest & SimPO- Shortest [ widely used baselines in reasoning optimization. Following the setup of the original work, the shortest correct response is selected as the preferred sample, and the longest correct re- sponse as the rejected one. (3) O1-Pruner [14] reinforcement learning- basedd method that reduces reasoning length while preserving accuracy. It first builds a baseline via pre-sampling, then guides the model generate more concise reasoning under performance-preservve constraints. We reproduce their results based on the LIMO model, following the experimental setup and hyperparameters. as described in their paper.
Efficiency Metric. Following 14
adopt the Accuracy- -ffeccc--ffiiieeyy Score (AES) quantify the trade-off between model accuracy and token reduction. Let (Apase- Liaas) and (Amodels Imodel) denote the accuracy and token count for the baseline and evaluated models, respectively. In the Long-to-Short scenario, the baseline is the original Long CoT model (e.g.. LIMO-7B); for QFF,, the model fine-tuned via SFT the same dataset.
We define relative changes as: AL = computed as:
Lamar Tamental
AA
Amertel Avaen Apase
The AES is then
Lsace
a CALL + 4AL+9AANI
AA
AES-
LAL 7
AA 1460 0,

---

with default parameters 0.1, = 0.1, and 1.0. The AES is calculated by weighting and summing the model's s solution token length and accuracy. In this metric, we prioritize maintaining accuracy (i.e., avoiding performance degradation) over reducing token usage.
4.3
Main Results
4.3.1 Comparison with SFT
As shown in Table 2 we observe that across three mathematical evaluation datasets, QFFT methods significantly reduce the average token length while achieving performance comparable to that of SFT. In addition, QFFT substantially increases the RAK score, indicating improved adaptive reasoning capabilities. This suggests that QFFT effectively leverages Short CoT patterns when handling simple and solvable problems, reducing token consumption without sacrificing accuracy. more detailed analysis of performance and reasoning adaptability is provided in the Section 5
We further observe that the degree of token reduction varies with dataset difficulty. Specifically, QFFT achieves more substantial token savings on simpler datasets such GSM8K and MATH, where the model adaptively retains a higher proportion of the Short CoT patterns. In contrast, on the more challenging AIME25 dataset, the model must rely more heavily the Long CoT patterns, resulting in less pronounced reduction in token consumption.
Finally, QFFT demonstrates strong adaptability across all three training datasets, including S1.1, LIMO, and BS-17K, when applied to both 7B and 32B model scales. This highlights the robustness and scalability of the QFFT approach.
Key Observation QFFT achieves performance comparable to SFT, significantly improving Reasoning Adaptability (RAK), thus substantially reducing the required token budgets.
4.3.2 Comparison with Long-to-Short Baselines
As shown 3 compared to Long-to-Short methods like SFT-Shortest and DPO, QFFT achieves greater token reduction with less performance degradation. On the other hand, while O1-Pruner and Simpo-FCS result in higher token reduction, they come at the cost of significant drop in performance. In contrast, our QFFT method achieves higher AES values, providing a better trade-off between performance and token efficiency.
5 An In-depth Analysis
In this section, we conduct an in in-depth analysis to address two key questions: Q1: How QFFT Enables Adaptive Reasoning? (Section 5.1) and Q2: Why QFFT Matches SFT Performance? (Section 5.2
5.1 How QFFT Enables Adaptive Reasoning? (Q1)
In Section 3.3] we have explained why QFFT en- ables adaptive reasoning. In this part, aim to explore, in the inference stage, how QFFT mod- els utilize Long CoT patterns. To this end, first present a detailed case study illustrating the model's transition from the default Short CoT to the Long CoT reasoning pattern. Subsequently. we analyze the primary scenarios and conditions under which the model adaptively adopts Long CoT reasoning, providing deeper insights into QFFT's adaptive reasoning capabilities.
Verification
1500
Backtracking
Subgoal Setting
233
Counts
1000
Backward Chaining
1.7
1.96
500
7449
392
2
Math Difficulty Level
Figure Long CoT Behavior Classification Trend on MATH500, The proportion of Long CoT in- creases by difficulty level.
Case Study. We check some cases of the rea- soning chains of QFFT. As illustrated in Fig-

---

Question
Q: Every morning Aya gocs for 9-kilometer--ooe long walk and stops at a coffee shop afterwards... Find the number of minutes the walk takes her, including the minutes spent in the coffee shop.
QFFT Long Co'I Response
To solve problem, we need determine
Aya spends walking and in the coffee shop for different speods.
#Ml Step I: Convert all time units to hours. Since speed kilometers per hour...
Short CoT
##* Step 2: First scenario: Walking s km/h Total time for the walk including coffee shop: hours ... km/h Here, 8, b /22 8) (-11 + sqrt(12] + 2880)) / 16 = (-11 sqr((991))) 16.
##t Step 3: Second scenario: Walking
-90:
sqr((111222000000111111111111111 1122 (-90) *
Check if sqrt (2991) integer. 2991 331,
sqr((2991) 3 sq11(331), which not integer. This suggests calculation Wait, let's recheck the quadratic oquation sctup. Wait, after moving terms ... Wait, perhaps solving incorrectly, Let's compute the discriminant.... Therefore, no solution? Wait, check equations. Wait, original equations Walking at km/h is /.0 hours 180 minules. Total time inchuding is 180 minutes + 24 minutes = 204 minutes. Thus, the answer is boxed
Long CoT
Figure 3: A case study. This case illustrates that QFFT initially adopts the Short CoT patterns. Upon encountering error, the model triggers the reflective Long CoT refine its reasoning.
we observe that in responses of the QFFT model, the reflective Long CoT patterns initially follow a concise Short CoT style expected. Then, once the model encounters uncertainty or detects potential error, it triggers Long CoT reasoning patterns, marked by frequent use of reflective keywords such as "wait".
ure 3
Conditions Triggering Long CoT Reasoning. We further investigate under what conditions the QFFT model would switch Long CoT behavior. According (32), there four main Long CoT behaviors: (1) Verification, verification or the systematic checking of intermediate results. (2) Backtracking, which involves explicit revision of approaches when errors are detected. (3) Sub-goal Setting, where a complex problem is broken down into manageable steps. (4) Backward Chaining, where in goal-directed reasoning task, the solution is derived by working backward from the desired outcome. Notably, adopting verification and backtracking behaviors suggests the model faces uncertainties or errors. The four behaviors identified in our case study are illustrated in Appendix F.4
Then, check responses generated by the QFFT model and observe that most Long CoT responses are triggered by uncertainties or errors. Specifically, we use GPT-4o to classify the triggered Long CoT behaviors in MATH into the four categories. We regard the first "wait" the boundary between the two patterns, providing GPT-4o with two sentences before and after the first "wait" for classification. As shown Figure verification is the most common one across all difficulty levels, accounting for approximately 53% on average in Long CoT behaviors. This indicates the QFFT model tends to trigger Long CoT patterns when is uncertain about the previous steps. The second most common trigger is backtracking, which accounts for 26% on average, that is, rechecking and updating previously incorrect steps. Interestingly, as the difficulty of the questions increases, the proportion of backtracking gradually increases, suggesting that the model reflects and updates its step more frequently in such questions.
Conclusion. Our experiments above illustrate Long CoT reasoning in QFFT is mostly triggered by errors or uncertainties from the initial Short CoT patterns. The QFFT model defaults Short CoT reasoning and dynamically activates Long CoT patterns when these uncertainties and errors arise, demonstrating the reasoning adaptability.
5.2
Why QFFT Matches SFT Performance? (Q2)
Our experimental results indicate that the QFFT model achieves performance comparable to SFT distillation primarily due two key factors:
The model selectively employs Short CoT reasoning for simple problems, maintaining accuracy while significantly reducing token consumption.

---

(FFT-322((Uus)
1003
QFFT-328(OUrs)
4093
SFT-328
Long CuT CMT)
00
OFFT7BIOur
Accurary((:
5000
Qwen 32B Ins.
SFT-78
60
4000
10
R1-0kiil-78
40-
QFFT-326(DurS)
20
20
Qwen-32B-ins.
1000
Math Difiiuitt Lecel
Short
Long
Short
Lona
(39H)
(116)
1386)
[116)
(a)
(b)
(c)
Figure 5: Subfigure (a) presents the proportion of Long CoT patterns used by different models across difficulty levels; Subfigures (b) and (c) evaluate the capabilities of Short CoT and Long CoT for QFFT, respectively, demonstrating that QFFT acquires Long CoT abilities comparable to SFT.
2.
The model acquires the long-Cof reasoning capability comparable to that SFT, enabling it to effectively handle challenging problems.
QFFT Models Switch to Long CoT as Difficulty Rises. To investigate how the QFFT model adapts its reasoning patterss according question difficulty, we analyze the proportion of Long CoT responses generated by the QFFT model on the MATH500 dataset. As illustrated in Figure 5a the frequency of Long CoT responses clearly increases with problem difficulty-from just 7.41% the easiest questions to 51.12% on the most challenging ones. This trend demonstrates that QFFT models effectively adopt an adaptive reasoning strategy, relying predominantly on Short CoT for simpler problems and increasingly invoking Long CoT reasoning complexity grows.
QFFT Acquires Long CoT Reasoning Capability Compared to SFT. To further evaluate the reasoning capabilities of the QFFT model under both short and Long CoT patterns, we conduct detailed analysis using responses from the S1.1-QFFT-32B model on the MATH500 dataset. Specifically, we define two subsets: MATHsystem I, consisting of questions answered using Short CoT reasoning, and MyTHsy consisting of questions that trigger Long CoT reasoning. We then compare the performance of $1.1-QQ.RRF -FFFFF--22 against Qwen2.5-320 --322--Isttuctt and S1.1-SFT-32B on these subsets.
As shown in Figures 5b and 5c. the S1.1-QFFT-32B demonstrates slightly improved Short CoT performance compared the original Short CoT model, while its Long CoT capability remains comparable to the SFT model. Consequently, QFFT achieves overall performance on par with SFT distillation, while offering enhanced computational efficiency through adaptive reasoning.
Conclusion
In this paper, we revisit the complementary reasoning patterns of Short CoT and Long CoT models, and introduce Question-Free Fine-Tuning to effectively balance these two patterns. By removing explicit queries during training, QFFT preserves concise reasoning patterns from Short CoT while selectively leveraging the reflective reasoning capabilities of Long COT. Experimental results on mathematical datasets demonstrate that approach achieves performance comparable to SFT, while reducing the number of generated tokens by approximately 50%. Furthermore, QFFT exhibits superior generalization capabilities in out-of-domain tasks, noisy data, and low-resourg scenarios.
7
Acknowledgement
This work was supported by Major Frontier Exploration Program (Grant No. C10120250085) from the Shenzhen Medical Academy Research and Translation (SMART), the Shenzhen Science and Technology Program (JCYJ202208 18 18000001002) NSFC grant 72495131, Shenzhen Doctoral Startup Funding (RCB$2022 CCCSS20220000093330066 Foundation of China (NSFC) (12326608), Shenzhen Science and Technology Program (Shenzhen Key Laboratory Grant No. SSOSSS222200200010222222 and Shenzhen Stability Science Program 2023.
Tianyuan Fund for Mathematics of National Natural Science
10

---

References
[1] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed HI-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameyy, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
[2] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in IIms via reinforcement learning. arXiv preprint arXiv:2501.12945, 2025.
[3] Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. Reasoning with large language models, survey. arXiv preprint 2rXiv:2407.11511, 2024.
[4] Dayu Yang. Tianyang Liu, Daoan Zhang, Antoine Simoulin, Xiaoyi Liu, Yuwei Cao, Zhaopu Teng, Xin Qian, Grey Yang, Jiebo Luo, al. Code think, think to code: A survey on code-enbanced reasoning and reasoning-drin code intelligence in llms. arXiv preprint arXiv:2502. 19411, 2025.
[5] Ahmed Ki-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, et al. Competitive programming with large reasoning models. arXiv preprint arXiv:2502. 06807, 2025.
[6] Dacheng Li. Shiyi Cao. Tyler Griggs, Shu Liu, Xiangxi Mo. Eric Tang. Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G Patil, Matei Zabaria, et al. Lims can easily learn to reason from demonstrations structure, not content, what matters! arXiv preprint arXiv:2502.07374 2025.
[7] Menghao Huo, Kuan Lu, Qiang Zhu, and Zhenrui Chen. Enhancing customer contact effi- ciency with graph neural networks in credit card fraud detection workflow. arXiv preprint MxXiv:2504.02275: 2025.
[8] NovaSky Team. Sky-tl: Fully open-source reasoning model with ol-preview performance in 450 budget. https://novaskyy-a wtt.hubboooossss/kkyttttt 2025. Accessed: 2025-01-09.
[9] Niklas Muennighoff, Zitong Yang. Weijia Shi. Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.19395 2025.
[10] Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint UrXiv:2502.03387 2025.
[11] Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, al. Do not think that much for 2+ 3=? the overthinking of ol-like llms. arXiv preprint arXiv:24/2.21188 187, 2024.
[12] Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song. Dian Yu, Juntao Li, Zhuosheng Zhang, et al. Thoughts are all over the place: On the underthinking of ol-like llms. arXiv preprint arXiv:2501.18555 2025.
[13] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.125000 2025.
[14] Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naigiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmoninn pruning. arXiv preprint arXiv:2501. 12570, 2025.
fine-tuning for ol-like reasoning
[15] Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697, 2025.
[16] Xinyin Ma. Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang Cot-valve: sength-compressible chain-of-thought tuning. arXiv preprint arXiv:2502. 09601, 2025.
[17] Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable th-huuht compression in 1lms. arXiv preprint 2xXx:22502.12077 2025.
11

---

[18] Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan. Understanding catastrophic forgetting in language models via implicit inference. In The Twelfth International Conference on Learning Representations, 2024.
[19] Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. Towards large reasoning models: A survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.090 100686, 2025.
[20] Huifeng Yin, Yu Zhao, Minghao Wu, Xuanfan Ni, Bo Zeng, Hao Wang, Tianqi Shi, Liangying Shao, Chenyang Lyu, Longyue Wang, et al. Towards widening the distillation bottleneck for reasoning models. arXiv preprint arXiv:2503.0146/ 2025.
[21] Tarald O Kvålseth. Note on cohen's 's kappa. Psychological reports, 65(1):223-226, 1989.
[22] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050 2023.
[23] Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in Ilms. arXiv preprint arXiv:2502.03333, 2025.
[24] Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. survey of transfer learning. Journal of Big data, 3:1-40, 2016.
(25] Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongho Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong. and Qing He. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):33-76, 2020.
[26] An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024.
[27] Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 400-410, 2024.
[28] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint StV:21100114166 2021.
[29] Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil. Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843-388 2022.
[30] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024.
[31] Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
[32] Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01 2025.
[33] Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. Data engineering for scaling language models to 128k context, 2024.
12

---

[34] Alex Zhuang, Ge Zhang, Tianyu Zheng, Xinrun Du, Junjie Wang, Weiming Ren, Stephen W. Huang, Jie Fu, Xiang Yue, and Wenhu Chen. Structlm: Towards building generalist models for structured knowledge grounding, 2024.
[35] Liyan Tang. Philippe Laban, and Greg Durrett. Minicheck: Efficient fact-checking of ilms on grounding documents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 8818-8847 2024.
[36] Arrigo Schieppati, Jan-Inge Henter, Erica Daina, and Anita Aperia. Why rare diseases are an important medical and social issue. The Lancet, 37 371(9629)2039--244 2008.
[37] Peter E Gretener. Significance of the rare event in geology. AAPG Bulletin, 51(11):2177-2206 1967.
[38] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025.
[39] Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system 1 to system 2: A survey of reasoning large language models. arXiv preprint arXiv:250..7419, 2025.
[40] Alireza S Ziabari, Nona Ghazizadeh, Zhivar Sourati, Farzan Karimi-Malekabal Payam Piray, and Morteza Dehghani. Reasoning on a spectrum: Aligning Ilms to system and system thinking. arXiv preprint arXiv:2502. 2025.
[41] Violet Xiang, Charlic Snell, Kanishk Gandhi, Alon Albalak, Anikait Singh, Chase Blagden, Duy Phung, Rafael Rafailov, Nathan Lile, Dakota Mahan, et al. Towards system 2 reasoning in lims: Learning how to think with meta chain-of- -ffthugh... arXiv preprint arXiv:2501.04682 2025.
[42] Hugging Face. Open rl: A fully open reproduction deepseck-rl. r-,, January 2025.
[43] Ivan Moshkov, Darragh Hanley, Ivan Sorokin, Shubham Toshniwal, Christof Henkel, Benedikt Schifferer, Wei Du, and Igor Gitman. Aimo-2 winning solution: Building state tatt--of-the-art math- ematical reasoning models with openmathreasoning dataset. arXiv preprint arXiv:2504. 16891, 2025.
[44] Han Wu, Yuxuan Yao, Shuqi Liu, Zehua Liu, Xiaojin Fu, Xiongwei Han, Xing Li, Hui-Ling Zhen, Tao Zhong, and Mingxuan Yuan. Unlocking efficient long-to-shor so-sort Ilm reasoning with model merging, arXiv preprint arXiv:2503.20641 2025.
[45] Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502./8600 2025.
[46] Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. Concise thoughts: Impact of output length 11m reasoning and cost. arXiv preprint arXiv:2407. 19825, 2024.
[47] Tingxu Han, Zhenting Wang. Chunrong Fang, Shiyu Zhao, Shiqing Ma. and Zhenyu Chen. Token-buuget-awaree 11m reasoning. arXiv preprint arXiv:2412.18544 2024.
[48] Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, Yan Wang, Wei Shen. Qing Gu, Anh Tuan Luu, See-Kiong Ng, Zhiwei Jiang, and Bryan Hooi, Longrecipe: Recipe for efficient long context generalization large language models, 2024.
[49] Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Processbench: Identifying process errors in mathematical reasoning. In The 63dd Annual Meeting of the Association for Computational Linguistics, 2025,
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction clearly describe our contribution, the algorithm, and the experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We mentioned our limitation in Appendix. D
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We mentioned our Theory Assumptions in Section 3.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibiliyy
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: the Appendix. we provide a detailed description of our model architecture and present all the training details, including dataset and hyperparameter settings.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
We provide our code in the supplemental material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/dde/eeee Cooeeeoodeesssseeeeee 7/nips.com / casssnnnnccccyyc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
We provide the full details Appendix. A.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We mentioned using random seeds repeat least 3 times to caleulate the average in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We mentioned in the Appendix.A.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
This paper strictly adheres the NeurlPS Code of Ethics
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work is foundational research and not tied to particular applications
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Yes, the paper properly credits the creators or original owners of assets (e.g..oode, data, models) and explicitly mentions and respects the relevant licenses and terms of use.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcone com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
18

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets introduced in this article
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datasettcoodemmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subject- sarticle.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
19

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: LLM is used for writing polish; Our proposed method is based the scope of LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non ntn-standard components.
Please refer to LLM policy lhttps://neurip:: http///rrppp.ccoooofeeeeeeeeeeee /ctc//ooneeeenccco222//////// for what should or should not be described.
20

---

A Two Equivalence of the QFFT Method
To understand (1) how QFFT preserves the model's original Short CoT capability and prevents them from being overridden and, (2) how QFFT enhances the model's Long CoT capabilities, we examine Iwo theoretical equivalences of our method.
Supervised Fine-tuning with Null Questions. QFFT can be conceptualized as a specialized form of SFT with null questions. In conventional SFT, models learn to map from questions Q Long CoT responses R, which overrides the original Short COT. In contrast, with QFFT, since the questions are empty, the model does not learn any concrete - R mappings. Consequently, during inference, any non-empty input will not trigger Long CoT patterns, thereby preserving the model's original Short CoT patterns. In practice, this "null-question SFT" approach both elegant and efficient, requiring no complex architectural modifications or specialized training strategies; it can be implemented simply by removing the question from the standard SFT frameworks.
Continued Pre-training. From another perspective, QFFT functions as a specialized form of continued pre-training. Among training stages of LLMss, continued pre-training has emerged as essential approach for expanding foundational capabilities and developing domain expertise [33] 34]. Unlike SFT, continued pre-training does not necessarily incorporate complete question-answer pairs and typically operates without question masking, incorporating loss calculations across all tokens. Consequently, QFFT can be conceptualized as specialized form of continued pre-training that systematically enhances the model's 's general Long CoT proficiency, including reflective reasoning capabilities, by training exclusively on reasoning responses, without reliance on specific question- answer pair formats.
B
More Applications of QFFT
Despite being trained without explicit question inputs. QFFT still consistently surpasses SFT in several targeted scenarios. In this section, we highlight three representative applications where QFFT demonstrates superior performance over SFT.
B.1 Noisy Scenarios
In real-world applications, training data often contains various quality issues. In the context of Long CoT distillation, these issues may include incomplete reasoning processes, incorrect conclusions, or entirely irrelevant responses. Traditional SFT methods are highly sensitive to such noisy data, potentially leading significant performance degradation.
To evaluate QFFT's robustness under various noise conditions, we design four progressively chal- lenging noise levels as shown in Figure 6al
Level I: Normal Data As baseline, this level uses original high-quality data containing complete and correct question-answer pairs.
Level II: Incorrect Conclusions
This level preserves the original reasoning process but introduces numerical errors in the final steps, leading to incorrect conclusions.
Level III Incomplete Reasoning This level randomly truncates approximately half of the answer content while maintaining the semantic completeness sentences.
Level IV: Irrelevant Answers This level creates completely mismatched question-answer pairs, where each question paired with an answer from another question.
For each noise level, we construct datasets in both standard SFT and QFFT formats. We use the LIMO dataset our distillation corpus, with Qwen2.5-7B --B-Inntrutct as the Short CoT model for distillation training. Performance is evaluated on MATH dataset by averaging results from 16 sampling runs per question.
Experimental results in (b) declines dramatically, from 76.5% at Level I to 0.4% at Level IV, indicating high sensitivity to noisy data. In the most severe case, with completely irrelevant responses (Level IV), the model completely loses its reasoning capabilities, In contrast, QFFT exhibits remarkable robustness, maintaining 78.6%
that as noise levels increase, standard SFT performance
21

---

I:
Q
Step1
Step2
Step
78.6
78.6
74.2
Step2
I
Step2
Step
30
SFT
Stap1
Stap2
Step
OFFT
0.4
Leve
Level 11
Lovel
Level
N
Noise Leve!
(a)
(b)
Figure 6: Subfigure (a): Schematic illustration of different noise levels; Subfigure (b): Performance comparison between SFT and QFFT methods across different noise levels. As noise levels increase from Level (normal data) to Level IV (irrelevant answers), SFT performance deteriorates dramati- cally, dropping from 76.5% to 0.4%. In contrast, QFFT maintains robust performance across all noise levels, achieving 78.6% even under extreme noise conditions such as Level IV.
performance even under extreme noise conditions at Level IV, matching its Level I performance. These results provide compelling evidence QFFT's advantage in handling noisy training data.
Key Observation 4: QFFT significantly outperforms SFT in handling noisy training data, maintaining robust performance even under extreme noise conditions where SFT methods completely fail.
B.2 Out-of-Domain Scenarios
To assess the generalization ability on out-of-domain evaluation sets, we further conducted evaluations the GPQA and MMLU-Pro datasets (detailed performance specific subsets is provided in Appendix F33 As shown in Table 4 (1) QFFT demonstrates superior performance compared to SFT both GPQA 30 and MMLU-Pro datasets, highlighting its strong generalization out-of-domain data.
Additionally, to investigate whether the removal of queries by QFFT increases model hallucinations. we further evaluate the models on LLLM-AggrFacc(35)) a benchmark specifically designed for hallucination detection. As shown in Table 4 models trained with SFT exhibit noticeable decline in performance compared to the baseline Dwen2.55-Insrun models, especially for the 7B scale models. We observe that such decline is attributed to the models' poor instruction following ability. which leads to failures in extracting the final answers. In contrast, QFFT achieves slightly better results than the baseline on
indicating that it does not exacerbate the risk of hallucinations.
Key Observation 5: QFFT demonstrates superior out-of-domain generalization compared
SFT, and reduce hallucination risks.
B.3 Low-Resource Scenarios
certain low-resource scenarios (e.g., rare medical conditions (36 and unique geological phenom- ena 37), high-quality data are often scarce, limiting the availability of extensive Long CoT training data. Under these conditions, Long CoT SFT typically struggles enable models to effectively learn the Long CoT reasoning capability.
22

---

Table 4: Comparison OOD dataset. L.M-- AggreFact is used to evaluate hallucination. means that SFT 7B models have poor instruction following ability and fail to extract most answers.
0.80
0.78
Data
MMLU-PPo
GPQA
LIM-AggreFact
OFFT
0.72
Qwen225-77-inssrutt
SFT
Qwen--.5-Instruuch
S1.1-SFT
43.1
40.0
41.8
36.4
25.5*
55.2
+ LJMOOSFT
29.3
43.2
8.9
0.70
20
25 Epach
30
50 40
45
$1.1-QFFT
53.2
44,4
58.9
LIMO-QFFT
48.4
44.2
58.7
Performance of SFT and QFFT on MATH 500 in the low-resource scenario. The pass @ score is averaged over samples.
Figure 7:
Qwen-2.5 9-no-22.--nstruut-3288
62.2
49.5
75.3
S1.1-SFT
64.9
60.6
60.5
LIMO-SFT
52.4
65.3
62.2
S1.1-QFFT
73.6
65.7
76.8
LIMO QFFT
73.4
67.9
75.8
To investigate the performance of QFFT in such scenarios, we simulate a low-resource scenario: We randomly sample 10 data points from the $1.1 dataset. To ensure training stability and sufficient data volume, each sampled data point is distilled 10 responses using Deepseek-RI, yielding total of 100 training instances. Subsequently, we train both SFT and QFFT using these data with identical parameters for 50 epochs.
As depicted in Figure 7 QFFT consistently outperforms SFT in the low-resource scenario. Models trained with SFT predominantly rely on Long CoT patterns, which are not sufficiently internalized during training. resulting in limited overall performance. In contrast, QFFT not only retains the original Short CoT patterns but also adaptively uses Long CoT reasoning as needed. Consequently, QFFT achieves enhanced performance by integrating reflective Long CoT behaviors on top of the foundational Short CoT abilities, demonstrating superior performance low-resource scenarios.
Key Observation 6: QFFT demonstrates superior performance in low-resource scenarios compared to SFT.
C Related Work
Large Reasoning Models Recent advancements in large reasoning models, such as OpenAl-ol [. DeepSeek-Re [2], and QWQ [3 have significantly improved performance on complex tasks, such
as mathematics and coding 33 3 39 These improvements largely stem from the test-time scaling
paradigm, which enables models to adopt Long CoT reasoning nattern-simulating human-like deep thinking through self-reflection, error correction, and multi-step solution exploration [39 40
Building on this success, recent studies have explored distilling the capabilities of powerful Long CoT models into smaller, cost-effective ones. Some approaches leverage large-scale, diverse, and challenging datasets to equip smaller models with test-time scaling abilities. For example, works such as DeepSeek (2). OpenR1 422. Sky-T1 8) and epessnngi
4% curate broad, high-quality datasets and distill responses from DeepSeek-Ri. These responses are used to fine-tune smaller dense models (e.g., Qwen2.5-32B)
reasoning patterns generated by the DeepSeek-RI - model, enabling them outperform GPT-4 on mathematical reasoning benchmarks. On the other hand, methods such as S1 [ and LIMO [10 fine-tune smaller models using only around 1000 high-quality and diverse examples paired with reasoning trajectories from DeepSeek-RI. Despite relying on fraction of the data and computational resources, these approaches empower Short CoT models with strong reasoning capabilities, achieving competitive or even superior performance on challenging benchmarks.
Long-to-Shee to-rt Approaches While Long CoT has significantly improved the ability of LLMs to handle complex reasoning tasks, often comes with substantial computational overhead and tendency toward overthinking [I] [9]. To solve this problem, work []] explores methods such as
23

---

DPO and SimPO, which sample multiple responses and train models to prefer the shortest and correct responses. Ol-Pruner [14 proposes Length Harmonizing Fine-Tuning framework that formulates an optimization objective to prune unnecessary steps while maintaining accuracy. Cal-Valve [16] dynamically controls the reasoning chain length, allowing the model to use shorter chains on simple tasks and retain longer ones for complex reasoning. On the other hand,
[4 merges short and Long CoT models to achieve efficient reasoning. Other approaches [45] 46 47 48) further enhance the
token efficiency of models by adopting approaches such as adaptively allocating reasoning budgets based on task difficulty. However, most existing methods focus primarily on improving the efficiency of Long CoT reasoning. This paper proposes novel perspective: enabling models to adaptively learn to use Long CoT and Short CoT reasoning adaptively, achieving greater efficiency and flexibility in inference.
D Deeper Insights into the QFFT Method
The QFFT method provides unique advantage in its ability to inject new patterns into model without overriding the model's default patterns. Specifically, consider a scenario where model initially possesses default pattern (Pattern A, e.g. Short CoTT). When additional distinct patterns (Patterns and C, e.g., Long CoTT) are introduced through training data, QFFT facilitates the seamless integration of these new patterns while preserving the original Pattern A. Consequently, the model can adaptively activate Patterns B and C based on their respective triggering conditions, achieving an adaptive utilization of multiple patterns.
Importantly, when Patterns A, B, and C Cach exhibit unique capabilities, meaning that no single pattern can fully replicate or substitute the functionality of another, the QFFT-trained model demonstrates clear superiority over traditional SFT. This because SFT typically overwrites the default patterns (losing the capabilities of the default pattern A). In contrast, QFFT maintains the coexistence and adaptive triggering of multiple patterns, enabling the model to leverage the capabilities of diverse patterns.
Looking ahead, we plan to explore the injection of additional specialized patterns, such as tool- oriented patterns (e.g.. API-calling patterns, code patterns) or even custom-designed patterns tailored explicitly to specific tasks. This direction promises further enhancement of model flexibility and adaptability, opening new avenues for advanced pattern integration and utilization.
E
Experimental Details
E.1
Training Data
In our experiments, both QFFT and SFT are trained using three primary datasets: S1.1, LIMO, and Below, we provide overview of each dataset.
9 The S1.1 1 dataset consists of 1,000 carefully curated questions paired with detailed reasoning traces. These questions were selected based on three criteria: difficulty, diversity, and quality. The dataset covers range of subjects, primarily focusing mathematics, but also includes other scientific disciplines such as biology, physics, and economics.
S1.1
The LIMO dataset contains 817 high-quality training samples, each comprising mathematical problem and its corresponding detailed reasoning chain. The reasoning chains were distilled from the DeepSeek- R1 model to ensure accuracy and clarity.
LIMO 10]:
The Bespoke Bespoke-Stratos- 17k dataset includes 17,000 reasoning exam- ples, each containing a question, a reasoning trace, and an answer. This dataset was generated by replicating and improving the Berkeley Sky-T1 data pipeline and utilizing SFT distillation data from DeepSeek-R1.
Bespoke-Stratos--Iz
For all three datasets, the responses used in our experiments were based on the original responses from the datasets, which were distilled from the DeepSeek-RP model. This ensures that the reasoning chains used in both training and evaluation are of high quality.
24

---

E.2
Training Details
For the SFT baselines, follow the official hyperparameters from S1, LIMO. and Sky-T1 during training. For QFFT, we use the same set of hyperparameters as those used for SFT. We use Llama Factory [27] for training. During the QFFT training, we modify the template in the Llama Factory by removing the question and all system prefixes, focusing solely on training g the responses. Other detailed hyperparameters used the training process are presented in Table
Table Hyperparameters for training different sizes of models in our experiments. Both SFT and QFFT share the same set of hyperparameters.
Hyperparameter 7B Models
32B Models
Phi4-Mini-Instruu
Cutoff len
16384
16384
16384
Batch_size
8
32
Learning_ rate
le-5
le-5
le-5
Epochs
Lr scheduler type
Cosine lc-4
Cosine
Cosine
Weight decay
le-4
le-4
Warmup_raid
0.1
0.1
0.1
Evaluation Prompt
System:
Please reason step by step, and put your final answer within ()
User: [Question] Assistant:
Figure 8: The prompt of evaluation in experiments.
E.3 Evaluation Details
We conducted evaluation experiments on all models using 0.6 and token budget 32,768. For the mathematical datasets, we sample 16 times to compute the average accuracy. For the Out-of- Domain tasks (including GPQA, MMLU-Pro, and FLMMAggreFact) we run 4 times and report the average accuracy. We use the VLLM reasoning architecture, and the inference setup is aligned with LIMO. The specific prompts used are shown in Figure g
E,4 Long-to-Short Baseline Details
To enable adaptive selection between long-CoT and short-CoT patterns, straightforward approach is Difficulty-Aaapir Distillation (DAD) for short and long reasoning chains. Specifically, we identify the challenging subset Dhard (850 examples) by selecting questions from S1.1 that Dwwen.55IInsrruet77 fails to solve. For the simple subset Peasy. we combine questions correctly answered by Qwen2.5- Innstuttttt from S1.1 and randomly sampled questions from the GSM8K training set, totaling 850 examples. For Daard, we use the solutions provided by $1.1, which are distilled from DeepSeek-RR R1. For easy, we distill responses from the model.
Diffiuuty-Adaaptive Distillation
Long-to-Short training Long-to-Shor methods are typically employed to reduce the redundant reasoning lengths of long-CoT models, such as distillation models. In our experiments, selected the LIMO-7B and 32B distillation models. We compared our approach against four competitive Long-to-Short baseline methods:
(1) SFT-Shortest IT Following the procedure in [II, constructed dataset by selecting the shortest correct responses and then performed SFT.
For the LIMO dataset, we used LIMO distillation models to sample ten times.
(2) DPO-Sportteeee and SimPO-Shorten 11
Adhering to the setup described in we sampled ten times using the distilled LIMO model on the LIMO dataset. The shortest correct response was chosen the preferred sample, while the longest correct response served as the rejected sample.
25

---

We replicated its results based on the LIMO distillation model. The experiments were conducted on the LIMO dataset, and all hyperparameters strictly followed the specifications outlined in the original paper.
(3) O1-Pruner [14]
F
Additionally Experiments
F1
Experimental Results on More Datasets
In this section, we report the experimental results on AMC, AIME, and Minerva. Here we draw the same conclusion as in the main text. As shown in Table 6 (1) observe that across three mathematical evaluation datasets, QFFT methods significantly reduce the average token length while achieving performance comparable to that of SFT. This can be attributed to the substantial improvement in reasoning adaptability brought about by QFFT, which leverages System patterns for simple and solvable questions, thus reducing token consumption. (2) We observe that QFFT achieves a more significant token reduction on the simpler datasets (e.g.. AMC) compared to AIME24. This is due to the relative simplicity of these datasets, which allows the model retain higher proportion of System 1 patterns, thus significantly reducing the average token length. In contrast, on the more challenging AIME24 dataset, QFFT retains fewer System patterns, leading less pronounced token reduction.
Table 6: Main results on 4 mathematical benchmarks. The reported accuracy and Avg. Len (Tokens) averaged over 16 random sampling runs.
AMC
AIME24
Minerva
Average
Data
Method
Acc
Tokens
RAK
Acc
Tokens
RAK
Acc
Tokens
RAK
Acc
Tokens
RAK
7B Models (Based on Qwen2.5-7BIInstmee
SFT
56.1
12.9K
3.2
19.0
13.7K
0.5
36.2
8.5K
0.7
37.1
11.7K
1.5
S1.1
QFFT
55.3
7.4K
43.7
20.6
10.3K
29.6
38.2
2.9K
27.8
38.0
6.9K
33.7
A
-0.8
-42.4
+40.5
+1.6
-24.6
+29.1
+2.0
-65.9
+27.1
+0.9
-44.3
+32.2
55.8
12.6K
19.1
13.9K
0.0
38.2
7.3K
37.7
11.3K
LIMO
QFFT
9.8K
39.8
19.6
11.2K
28.9
37.7
4.6K
24.2
38.2
8.5K
+1.4
-21.9
+39.1
+0.5
-19.6
+28.9
-0.5
-37.5
+23.1
+0.5
-26.3
+30.4
SFT
61.6
11.2K
3.6
19.4
13.4K
0.0
40.8
4.6K
0.9
40.6
9.7K
1.5
BS-17K
QFFT
61.6
7.2K
40.1
20.6
9.0K
27.7
39.3
3.4K
26.8
40.5
6.5K
31.5
+36.5
+27.7
-31.5
+30.0
32B Models (Based on (wwe22.5.32B-Inttrre
SET
87.2
8.9K
0.8
56.9
12.2K
0.0
48.3
4.9K
1.1
64.1
8.7K
0.6
SI.1
QFFT
86.9
5.7K
38.7
56.7
9.4K
24.9
47.1
3.1K
31.1
63.6
6.1K
31.6
-35,9
+37.9
-0.2
+24.9
-1.2
-37.2
+30.0
-32.0
+30.9
46.7
8.1K
LIMO
QFFT A
89.5
7.2K
42.1
54.6
9.5K
22.8
46.1
3.5K
28.9
63.4
6.8K
31.3
-2.2
-21.5
+39.7
-2.1
-10.7
+22.2
-0.6
-20.4
+27.5
-1.6
-17.5
+29.8
F.2
Analysis with Different Model Backbones
To investigate whether the QFFT method limited to specific backbone architectures (c.g., Qwen), evaluate QFFT using alternative backbone, Phi4-mini- Instrutt the LIMO and S1.1 datasets. As shown in Table 7 compared to SFT, QFFT significantly enhances the reasoning adaptability under the Phi architecture, while achieving comparable overall performance. These results consistent with those on Qwen architecture, demonstrating that QFFT is not constrained to a specific backbone and generalizes well to other architectures.
F.3 Detailed Analysis on Out-of-Domain Benchmarks
In this section, we further analyze the generalization capabilities of QFFT on various subtasks from the MMLU-Pro benchmark across diverse domains. shown Table g after SFT, performance significantly decreases in knowledge-intensiv domains such as Law, History, and Economics com- pared to the baseline (Qwen2.5-1 7---2.557--nn Instrucuc. In contrast, improvements are notable in Physies and Chemistry.
26

---

Table Comparison of SFT and QFFT based on the Phi-4-Mini Insti--nstructtt base model.
GSM8K
MATH
AIME25
Average
Data
Method
Acc
Tokens
RAK
Acc
Tokens
RAK
Acc
Tokens
RAK
Acc
Tokens
RAK
Based
Phi-4-mini-Ima
(3.8B)
SFT
88.6
2.4K
69.8
5.3K
2.7
9.7
15.8K
56.0
7.8K
S1.1
QFFT
88.9
0.9K
31.4
69.7
2.8K
42,9
10.2
12.8K
26.5
56.3
5.5K
33.6
A
+0.3
-62.5%
+30.3
-0.1
-47.2%
+40.2
+0.5
-19.0%
+26.5
+0.2
-42.9%
+32.3
SFT
88.1
1.7K
67.2
5.2K
7.1
8.3
16.1K
1.8
54.5
7.7K
LIMO
QFFT
88.0
0.6K
29.3
66.9
3.7K
38.1
9.1
12.5K
31.6
54.7
5.6K
33.0
A
-0.1
-64.7%
+28.5
-0.3
-88.8%
+31.0
+0.8
-22,4%
+29.8
+0.1
-38.6%
+29.8
We analyze and have the following explanation: Physics and Chemistry share substantial knowledge overlap with Mathematics, and solving problems in these fields often inherently requires mathematical reasoning skills. Therefore, training the model on mathematical problems naturally enhances its generalization Physics and Chemistry tasks. This observation aligns with findings reported in the study [1O].
In contrast, domains such Law, History, and Economies primarily knowledge-intensiv relying more heavily on factual recall and contextual understanding rather than structured reasoning. Since SFT is driven by queries during training, it tends align the model closely with specific question formats. This overfitting may limit the model's flexibility in generalizing to tasks outside the training distribution, particularly in domains that require broad knowledge rather than reasoning. The reliance on task-specific queries may overly anchor the model to narrow cues, leading to catastrophic forgetting of general knowledge and ultimately degrading performance in unrelated domains.
QFFT mitigates these question-inde inducedd biases by learning reasoning patterns independent of specific input queries. By doing QFFT effectively reduces task-specific overfitting, preserving general knowledge and improving the model's ability to generalize across diverse domains.
Table 8: Accuracy (%) on Various Categories of the MMLP-Pro Dataset. The background red indicates performance improvement, and the green indicates performance decline compared to Swen2.5-7B-Innsrrus Results are averaged over four runs.
Category
LIMO-7B-FFFT LIMO-7B-SFT
Qwen2.5-7B-Instruce
Business
42.3
54.8
24.3
Law
29.5
5.5
53,0
Psychology
54.0
9.3
53.5
Chemistry
29.8
42.8
11.8
History
50.3
12.0
55.0
Other
50.8
21.3
49.0
Hcalth
52.5
16.5
51.5
Economics
58.0
27.8
57.5
Physics
48.8
52.0
20.3
Computer Science
41.8
33.0
39.5
Philosophy
43.5
26.5
49,0
Engineering
20.5
31.0
15.8
All
48.4
29.3
40.0
F.4
Case Study
In this section, we present specific cases to illustrate the four distinct behaviors exhibited by QFFT with Long CoT, shown in Figure 9
The Verification behavior involves reflective processes such as double-checking and validating intermediate results. For example, the model uses "let me double-check", which explicitly verifies intermediate computations to ensure correctness.
The Backtracking behavior involves explicitly revising previous steps when enrors or inconsistencies are detected. For instance, upon encountering an error such as "but already used" the model traces back to earlier steps identify and rectify the mistake.
27

---

The Sub-goal Setting behavior decomposes complex problems into multiple manageable sub-steps or sub- sub-goals, facilitating step-by-step reasoning and moblem-solvingg
The Backward Chaining behavior refers to goal-directed reasoning, where the solution is derived by reasoning backward from the desired outcome. For example, the model may reason: "Wait, actually, if we can express S in terms of T', then..." to guide its reasoning process toward the goal.
Verification
Backtracking
Let's all possible ways factor 84 into distinct positive integers and compute their sums First, let's consider the prime factors: 2, 3. 7 Their product is 3 x 7 42, which is less than 84, so we noed another factor Wait, but 2 x = 42, and 42 2 = 84, but 84, but is already set So, we need to combine some factors
So, the 4th value is 29, and the 5th value is 29 Therefore, the median is (29 29)/2 = 29 Altematively, since both the 4th and 5th values are 29, their average is 29 Wait, but let me double-check the sorted list to make didn't miss anything Original values
already used Altematively, 42
Subgoal Setting
Backward Chaining
alternate interior angles are equal, and consecutive interior angles supplementary In the diagram, there is transversal from point Q point and another transversal from point R point I' Wait, maybe lines PQ and TR are transversals Let me try to figure out the relationships between the angles First, angle at x degrees, angle at T 2x degrees
So, 2T + G S Our goal is to find the number of treeks that equal one squig, which is S/T But since the problem asks for how many treeks equal the weight of one squig, we need find S/T Wait, actually. express in terms of T, then S/T would the number of trocks equivalent to one squig But perhaps the problem is asking for the number such that
can
Figure 9: Behavior Classification Examples.
F.5
Generalization of Reflective Behaviors from Short CoT to Long CoT
To validate our second assumption -tht reflec- tive capabilities learned by Long CoT models can transfer to Short CoT contexts- designed an experiment examine whether models trigger re- flective behaviors when errors occur during Short CoT reasoning.
80
71.2%
70
Percentage (%))
60
57.4%
50
Experimental Design We utilized the Process- Bench [49] dataset, which contains detailed step- by-step solutions mathematical problems pro- duced by Short CoT models, with each step anno- tated as either correct or incorrect. This dataset enabled precise control over error conditions in our experiments. We constructed two types of rea- soning sequences: completely correct reasoning step sequences and sequences containing errors. We then prompted Long CoT models (Deepseek- E1-QQen-Distill-7
30
20
No Reflection
10:
Reflection
Reasoning
Reasoning
Reasoning Sequence Type
Figure 10: Reflection Behavior Comparison.
to continue writing from these sequences and analyzed the model's 's behav- ioral differences, particularly focusing on the emergence of reflective behaviors.
Results Our experimental results demonstrate that when Long CoT models are provided with
correct reasoning steps, they tend to directly continue the reasoning process with minimal reflection as shown in Figure TO Specifically, models exhibited reflective behaviors in approximately 42.6% of cases following correct reasoning sequences. In contrast, when presented with sequences containing errors, the proportion of reflective behaviors increased dramatically 71.2%. These reflective
28

---

behaviors manifested as identifying calculation errors, revisiting reasoning paths, providing corrected solution approaches, and validating intermediate results.
These findings strongly support our hypothesis: the reflective capabilities of Long CoT models, characterized by Po B, UL), successfully transfer to Short CoT contexts, represented as Po (B, Us).
G Limitations
The QFFT approach is simple yet efficient. Compared other efficiency-orienter methods, avoids complicated rejection sampling procedures and thus significantly saves computational resources. However, we observe that QFFT does not effectively optimize the efficiency of Long CoT reasoning. As result, the overthinking issues still occur on challenging questions, such as AIME24 and AIME25, To address this limitation, we further explore combining QFFT with various Long-to-Shon to-Short methods, aiming to effectively improve the efficiency of Long CoT reasoning within the QFFT framework.
Table 9: Comparison with other Long-to-Shor baselines. performance and response length.
AES used as a metric that balances
Method
GSMBK
MATH
AIME25
AVG.
Acc f Tokens
AES
Tokens
AES
Acc
Tokens
AES
Tokens
AES
Long-to-Shen Methods (7B)
LIMO (base)
88.2
1.8K
80.4
5.9K
16.8
17.8K
8.5K
SFT Shortest
88.9
1.2K
3,4
78.3
4.8K
-0.7
17.9
17.3K
0.9
61.7
7.8K
1.2
DPO Shortest
89.8
1.6K
1.5
79.5
5.4K
-0.1
17.3
17.1K
0.7
62.2
8.0K
0.7
SimPO Shortest
1.2K
3.2K
-3.6
01 O1-pruner
90.8
0.8K
5.8
78.2
3.2K
1.8
14.2
12.2K
-12.7
61.0
5.4K
-1.7
Distilled Methods (7B)
DAD-7B
90.0
0.9K
5.2
80.2
4.8K
1.6
17.3
17.7K
0.3
62.5
7.8K
2.4
QFFT (Ours)
88.0
1.7K
5.9
80.6
4.1K
2.9
17.2
15.6K
1.4
61.9
6.9K
3.4
DPO Shortest
0.7K
4.0
80.5
4.0K
16.7
15.2K
0.5
61.2
6.6K
2.6
SimPO Shortest
89.0
0.5K
7.3
80.2
3.4K
4.1
17.7
13.6K
2.9
62.3
5.8K
4.8
QFFT + Long-to-Short We investigate whether the QFFT-trained model can further benefit from the Long-to-Short strategy to enhance the efficiency of Long CoT reasoning. Specifically, we explore applying
and SimPO-sshortss methods to the QFFT model. To achieve this, first sample 10 responses from the MAMO-7B-QFFT model on the LIMO dataset and select the shortest correct solution among them for subsequent DPO and SimPO training. As shown in Table 9 observe that both DPO and SimPO methods can further improve the overall token efficiency of the QFFT model. Notably, these methods significantly enhance the AES metric with minimal performance degradation.
We further compare the average length of Long CoT and Short CoT after applying DPO and SimPO. As shown Figure [T we find that the average token count of Long CoT decreases significantly. This indicates that the Long-to-Short method can be effectively applied to QFFT to further enhance the efficiency of Long COT. This demonstrates the scalability and adaptability of the QFFT method, highlighting its potential to effectively integrate with Long-to- Sore-t--Shoon Short strategies for further efficiency gains.
29

---

AIME25 Dataset
MATH500 Dataset
QFFT
16.2K
QFFT
8.7K
Average Token Length (K)
15.0
QFFT SimPO
QFFT DPO
14.7K
Average Token Length (0)
8.0
QFFT + SimPO
QFFT DPO
7.8K
7.4K
12.5
5.0
2.0
2.5
0.8K
0.7K
0.7K
0.6K
0.6K
Short CoT
Long CoT
Short CoT
Long CoT
(a)
(b)
Figure 11: Average token length comparison of LIMO-7B-QFFT models on mathematical problem- solving datasets. Subfigure (a) presents the AIME25 dataset and subfigure (b) presents the MATH500 dataset, showing the average lengths Short CoT and Long CoT responses across three model variants: the base LIMO-7B-QFFT model, and the same model enhanced with either DPO or SimPO length reduction (Long-to-Short).
30


Reviews:
[{"review_id": "fR8HBHOjll", "reviewer": "Reviewer_wE8y", "summary": "This work introduces a method to train the reasoning models (by removing the Question while training) such that the method now generates less steps (less number of tokens)  while not degrading the accuracy much as compared to baseline. They do it in Supervised Fine Tuning Regime.", "questions": "(1) Qwen is trained on short COT and now you are trying to make it adapt to long COT to make it think more and improve performance on difficult questions. To do this you distill long answers from Deepseek R1 and use it in SFT training and QFFT training (removing Q from the training). Intuitively by SFT training one would assume that the model becomes better at long COT. Because of being trained on long answers SFT will automatically learn to generate larger tokens.  If we remove the question and do QFFT training, its seems to me that the length of each datapoint has become smaller and hence the model will learn to generate lesser token as compared to SFT. It will be interesting to me if we can compare:\n1. Qwen base model's token count for gsm8k with that of QFFT model.\n2. Qwen base model's token count for AIME25 with that of QFFT model.\n\n\n(II) Gains over strong long-to-short baselines (e.g., O1-Pruner, SimPO) are not too much. Is it possible to provide a more detailed justification?", "limitations": "N/A", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The method is original.\n2. The method significantly reduces the token count while being trained on small number of samples only. The method don't really benefit much by training on larger number of samples (BS-17k vs S1.1). \n3. Extensive experiments are provided on in-domain and OOD as well SOTA Qwen model. \n\nWeaknesses:\n1. While the method is original, I am struggling to get an intuitive understanding of why the method works ? And why removing Q from the training does not affect performance ?\n2. Table 1 should include a row of actual base model's performance to understand how the training data affects SFT and QFFT. When directly checking the results on Qwen model, I see 3%-4% drop. Doesn't that signify that the training data is not useful for the model training and decrease performance.\n3. As the data difficultly increases, accuracy drop of QFFT as compared to SFT also becomes larger. \n4. This work has tested its method on one series of model which is Qwen. To be sure of the method's generalizability it would be more convincing if the method was tested on other series of model too which follows short COT. \n\nTypos:\n1. Line 195: setup of, ... (missing citation)\n2. Line 286:  Is it achieves ?", "was_revised": true, "final_justification": "My questions were resolved fully by the rebuttal especially the intuitive explanation and additional generalizability results. Hence I maintained my score of leaning towards accept.", "rebuttal": "Thank you for your constructive comments. We have carefully considered your suggestions and would like to provide the following clarifications.\n\n> **Strength 1**: The method significantly... The method don't really benefit much by training on larger number of samples (BS-17k vs S1.1). \n\n**Re**:    Thank you for your recognition of our work and your valuable questions. Regarding the question: **our method does not really benefit much by training on larger number of samples (BS-17k vs. S1.1)**, we provide the following explanations:\n  \n  (1) In Table 1 of our paper, we compared the performance of QFFT and SFT on both the BS-17k and S1.1 datasets. QFFT achieves results comparable to SFT on both datasets. As for the lack of greater improvement on S1.1 compared to BS-17k, **we believe this may be attributed to the inherent quality of the training data, rather than a limitation of our QFFT method.**\n  \n  (2) As noted in Paper [1], just 1K high-quality samples from S1.1 can achieve effects comparable to the full 59K dataset. Therefore, although S1.1 has fewer samples, its superior quality is sufficient to endow the model with strong Long CoT capabilities, yielding performance close to that of the BS-17k dataset.\n\n  [1] s1: Simple test-time scaling.  \n\n> **Weakness 1**: While the method is original, I am struggling to get an intuitive understanding of why the method works? And why removing Q from the training does not affect performance?  \n\n**Re**:    The core goal of QFFT is to simultaneously achieve **preserving Short CoT capability** and **triggering Long CoT for challenging problems**, and this mechanism can be intuitively explained through conditional probability:\n\n(1) The original model already possesses the capability of generating concise reasoning (Short CoT) for a question $Q$, corresponding to the conditional probability of Short CoT inference $P(\\text{Short CoT} \\mid Q)$. Unlike SFT, which forces the model to learn the direct mapping from questions to Long CoT as $P(\\text{Long CoT} \\mid Q)$, QFFT avoids overwriting the original $P(\\text{Short CoT} \\mid Q)$ by removing the question $Q$ from training data. This ensures the model defaults to Short CoT patterns during inference.\n\n(2) Meanwhile, Long CoT response data inherently contains patterns like reflection (denoted as $B_r$), which emerge when encountering uncertainties or errors. Specifically, during training, QFFT enables the model to learn Long CoT reasoning capabilities and the conditional probability of reflective behaviors. Formally, let $U_L$ denote uncertainties or errors arising in Long CoT reasoning; the model learns: $P_\\theta(B_r \\mid U_L)$, where $B_r$ represents the occurrence of reflective behaviors, and $P_\\theta$ denotes the model-parameterized conditional probability distribution.\n\nDuring inference, the model defaults to Short CoT. When Short CoT fails to solve challenging problems (accompanied by uncertainties or errors), denoted as $U_S$, the model learned $P_\\theta (B_r \\mid U_L)$ in the QFFT training. This enables the activation of $B_r$ when $U_S$ occurs, thereby triggering Long CoT for correction.\n  \n  > **Weakness 2**: Table 1 should include a row of actual base model's performance. When directly checking the results on Qwen model, I see 3%-4% drop. Doesn't that signify that the training data is not useful for the model training and decrease performance.\n\n**Re**: Thank you for your valuable question. We have added the performance of the base models for comparison, as shown in the table below:\n\n  | Model | GSM8K | MATH500 | AIME25 |\n  |-|-|-|-|\n  | Qwen2.5-7B-Instruct   | 90.7  | 74.8 | 9.8  |\n  | S1.1-SFT-7B | 90.6 | 80.8    | 18.2 |\n  | S1.1-QFFT-7B | 91.0  | 80.2 | 17.2 |\n  | Qwen2.5-32B-Instruct  | 93.6  | 82.8  | 17.5   |\n  | S1.1-SFT-32B | 92.8  | 93.1 | 48.6 |\n  | S1.1-QFFT-32B| 93.6  | 92.2 | 46.8   |\n  \n  (1) Our reported results are slightly lower (by about 2%) compared to those in the Qwen official blog [2]. This is because we report pass@16 values (averaging over 16 random samplings), whereas the official Qwen results use greedy sampling once. As a result, our metrics are more robust.\n  \n  (2) On the MATH500 and AIME25 datasets, both SFT and QFFT show clear improvements over the base models. For simpler datasets like GSM8K, the advantages of Long CoT over Short CoT are not as pronounced as they are on more challenging datasets (e.g., AIME25).\n\n  [2] Qwen2.5-LLM: Extending the boundary of LLMs\n\n  > **Weakness 3**: As the data difficultly increases, accuracy drop of QFFT as compared to SFT also becomes larger.\n\n**Re**: Thank you for your insightful question. However, we believe that the observed larger accuracy drop as data difficulty increases is due to random fluctuations.\n\n(1) The AIME25 dataset consists of only 30 questions, which introduces significant randomness and  high variability (although run at 16 times).\n\n(2) In Appendix C.1 Table 5 (Supplementary Material), we report the performance of QFFT on AIME24 and AMC, as shown below:\n\n| Data   | Method | AMC | AIME24 |\n|-|-|-|-|\n| S1.1   | SFT | 56.1 | 19.0 |\n| S1.1   | QFFT | 55.3    | 20.6  |\n| S1.1   | Δ | -0.8 | +1.6  |\n| LIMO   | SFT | 55.8| 19.1  |\n| LIMO   | QFFT | 57.2    | 19.6 |\n| LIMO   | Δ | +1.4 | +0.5 |\n| BS-17K | SFT | 61.6 | 19.4|\n| BS-17K | QFFT | 61.6  | 20.6  |\n| BS-17K | Δ | 0.0  | +1.2 |\n\nOn the challenging datasets AIME24 and AMC, QFFT performs better than SFT on average. Therefore, we believe that QFFT can match the performance of SFT on both difficult and simple problems.\n\n  > **Weakness 4**: This work has tested its method on Qwen. To be sure of the method's generalizability it would be more convincing if the method was tested on other series of model.\n\n**Re**:  Thank you for your insightful suggestion. In Appendix C.2 (Table 6), we include experiments on the Phi-4-mini-Instruct model, which validate the effectiveness of QFFT on the Phi-4 architecture. Additionally, we have added experiments on the LLaMA3 architecture, as shown below:\n\n| Data | MATH500 (Acc/Tokens/TAK) | GSM8K  (Acc/Tokens/TAK)| AIME25  (Acc/Tokens/TAK)|\n|-|-|-|-|\n| Llama3.1-8B-Instruct| 44.2/1.1K/0 | 67.4/0.8K/0 | 0.6/2.4K/0  |\n| + 1k S1.1 QFFT |  56.1/4.8K/28.6 | 76.5/1.3K/26.1 | 4.4/16.5K/9.9 |\n| + 1k S1.1 SFT|  56.3/9.7K/4.4| 75.2/5.1K/2.1| 3.7/24.8K/0|\n| + Bespoke-Stratos-17k QFFT | 68.1/3.8K/36.8  | 86.8/1.2K/27.2 | 12.3/13.7K/22.4  |\n|+ Bespoke-Stratos-17k SFT  |  68.8/7.1K/0.4 | 86.3/4.0K/3.8 | 12.5/19.6K/0  |\n\nWe observe that for LLaMA3.1-8B, both SFT and QFFT perform poorly on AIME25 with only 1k (S1.1) data. We believe this is because 1k samples are insufficient for the LLaMA model to effectively learn Long CoT capabilities. In contrast, with BS-17k data, there is a substantial improvement over the 1k results. We attribute this to characteristics inherent to LLaMA3 itself. Inspired by works [3] and [4], we note that LLaMA3 lacks sufficient injection of Long CoT patterns such as reflection and backtracking, during its pre-training phase (compared to Qwen), thus requiring more data to learn Long CoT patterns effectively.\n\n[3] OctoThinker: Mid-training Incentivizes  Reinforcement Learning Scaling\n\n [4] Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs\n\n  > **Question 1**:  Qwen is trained on short COT and now you are trying to make it adapt to... 1. Qwen base model's token count for gsm8k with that of QFFT model. 2. Qwen base model's token count for AIME25 with that of QFFT model.\n\n**Re**:   Thank you for your thoughtful insights. The table below compares the accuracy and token counts for the Qwen base model, QFFT on GSM8K and AIME25:\n  \n  | Model | GSM8K (Acc/Tokens) | AIME25 (Acc/Tokens)  |\n  |-|-|-|\n  | Qwen2.5-7B-Instruct   | 90.7 / 436  | 9.8 / 1865|\n  | + S1.1 QFFT | 91.0 / 443 | 17.2 / 12789 |\n  | + S1.1 SFT   | 90.6 / 1766  | 18.2 / 17740 |\n  \n  But we think your understanding of QFFT needs to be clarified. \n\n(1) Long CoT responses are typically much longer than the question lengths (e.g., distilled DeepSeek-R1 responses for MATH and AIME can reach 5k or more than 10k tokens). Compared to the response, the question length is almost negligible.\n\n(2) The reason QFFT works is shown in our response to your **Weakness 1**.\n\n  > **Question 2**: Gains over strong long-to-short baselines (e.g., O1-Pruner, SimPO) are not too much. Is it possible to provide a more detailed justification?\n\n**Re**: Thank you for your valuable question. \n\n(1) While QFFT achieves efficiency comparable to strong long-to-short baselines, it has unique advantages. Its training is exceptionally simple: based on SFT, it only removes input queries, needing no extra data or complex reinforcement learning (unlike O1-Pruner), thus reducing training overhead. Moreover, QFFT outperforms in areas beyond long-to-short methods' scope, As shown in Section 4.4 (Table 3) of our paper, it exhibits better hallucination mitigation on LLM-AggreFact and superior out-of-domain generalization (e.g., on GPQA and MMLU-Pro), granting it distinct value.\n\n(2) QFFT and long-to-short methods are orthogonal. This means that the Long CoT outputs from QFFT models can be further compressed using long-to-short techniques, thereby elevating the upper limit of QFFT's efficiency. We provided experiments demonstrating this in the table below.\n\n| Method | GSM8K Acc ↑ | GSM8K Tokens ↓ | GSM8K AES ↑ | MATH Acc ↑ | MATH Tokens ↓ | MATH AES ↑ | AIME25 Acc ↑ | AIME25 Tokens ↓ | AIME25 AES ↑ |\n|-|-|-|-|-|-|-|-|-|-|\n| LIMO 7B (base)   | 88.2 | 1.8K   | -    | 80.4 | 5.9K  | -    | 16.8 | 17.8K | -   | \n| O1-pruner  | 90.8  | 0.8K | 5.8   | 78.2  | 3.2K | 1.8| 14.2 | 12.2K   | -12.7 |\n| QFFT (Ours)  | 88.0 | 0.7K  | 5.9  | 80.6 | 4.1K | 2.9 | 17.2 | 15.6K | 1.4 | \n| QFFT + SimPO | 89.0  | 0.5K   | **7.3** | 80.2| 3.4K  | **4.1** | 17.7| 13.6K | **2.9** |"}, {"review_id": "z3mEBPF74X", "reviewer": "Reviewer_rhbK", "summary": "The paper addresses the efficiency problem in current long chain-of-thought (CoT) reasoning models, which, while strong on complex tasks, often suffer from overthinking—producing unnecessarily verbose reasoning even for simple problems. The authors propose Question-Free Fine-Tuning (QFFT), a distillation approach where the model is fine-tuned only on reasoning chains (CoT responses), with the input questions removed during training. This approach aims to preserve concise short-CoT reasoning for easy problems while enabling adaptive invocation of long-CoT (reflective) reasoning only when necessary. Experimental results on a range of mathematical benchmarks (e.g., GSM8K, MATH, AIME) and out-of-domain tasks show that QFFT reduces average response length by over 50% while maintaining comparable performance to standard SFT, and demonstrates better generalization and adaptivity. A new metric, Thinking Adaptive Cohen’s Kappa (TAK), is introduced to quantify the adaptivity in reasoning pattern selection.", "questions": "1. Have the authors considered or evaluated hybrid fine-tuning schemes, such as partial masking of questions, curriculum masking, or re-introducing questions at inference time? Would this further improve adaptivity or mitigate risks of instruction-following degradation?\n2. Is there any observed degradation in instruction-following or user prompt adherence, especially for non-mathematical tasks, due to removal of questions during training? If so, can the authors comment or provide measurements?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper identifies the important yet overlooked problem of overthinking in long-CoT distilled models.\n2. QFFT, which removes the input question in fine-tuning, is a straightforward method.\n3. Across multiple mathematical reasoning datasets, QFFT matches or slightly exceeds SFT on accuracy, while dramatically reducing token usage.\n4. The introduction of the TAK metric and detailed analyses (e.g., how QFFT models switch to long-CoT patterns only when needed) provides clear evidence for the method’s claimed adaptivity.\n\nWeaknesses:\n1. The main contribution is empirical and conceptual, not algorithmic.\n2. While the paper analyzes adaptivity, it could benefit from a more in-depth ablation, e.g., testing hybrid or partial question masking, or examining pathological cases where QFFT may fail.\n3. The discussion of potential negative impacts (e.g., removing questions during training could risk instruction-following degradation in some applications) is not fully explored in the main paper, though some limitations are covered in the appendix.", "was_revised": true, "final_justification": "The results in the original paper and rebuttal are very impressive, although I am still a bit suspicious on how this can work this well, I want to maintain my positive score to reflect solely based on the objective results.", "rebuttal": "Thank you for your constructive comments. We have carefully considered your suggestions and would like to provide the following clarifications.\n\n> **Weakness 1**: The main contribution is empirical and conceptual, not algorithmic.\n\n**Re**:   Thank you for your feedback. We acknowledge that the QFFT method may appear relatively simple. However, our approach is highly meaningful and provides valuable insights.\n\n(1) First, QFFT is a simple yet highly effective method. It only requires removing the input question from the training data, with the rest of the process identical to standard SFT. Additionally, QFFT matches SFT's performance while significantly improving token efficiency (reducing average response length by over 50%) and achieving better generalization to out-of-domain tasks.\n  \n(2) QFFT offers a novel methodological insight: it enables the model to learn new patterns without overwriting existing ones, while ensuring the model adaptively employs the new patterns under its default behavior. This provides a foundational framework for flexibly incorporating additional patterns.\n\n(3) We provide a probabilistic explanation of why QFFT achieves adaptive reasoning:\n\nThe core goal of QFFT is to simultaneously achieve **preserving Short CoT capability** and **triggering Long CoT for challenging problems**, and this mechanism can be intuitively explained through conditional probability:\n\n- The original model already possesses the capability of generating concise reasoning (Short CoT) for a question $Q$, corresponding to the conditional probability of Short CoT inference $P(\\text{Short CoT} \\mid Q)$. Unlike SFT, which forces the model to learn the direct mapping from questions to Long CoT as $P(\\text{Long CoT} \\mid Q)$, QFFT avoids overwriting the original $P(\\text{Short CoT} \\mid Q)$ by removing the question $Q$ from training data. This ensures the model defaults to Short CoT patterns during inference.\n\n- Meanwhile, Long CoT response data inherently contains patterns like reflection (denoted as $B_r$), which emerge when encountering uncertainties or errors. Specifically, during training, QFFT enables the model to learn Long CoT reasoning capabilities and the conditional probability of reflective behaviors. Formally, let $U_L$ denote uncertainties or errors arising in Long CoT reasoning; the model learns: $P_\\theta(B_r \\mid U_L)$, where $B_r$ represents the occurrence of reflective behaviors, and $P_\\theta$ denotes the model-parameterized conditional probability distribution.\n\n- During inference, the model defaults to Short CoT. When Short CoT fails to solve challenging problems (accompanied by uncertainties or errors), denoted as $U_S$, the model learned $P_\\theta (B_r \\mid U_L)$ in the QFFT training. This enables the activation of $B_r$ when $U_S$ occurs, thereby triggering Long CoT for correction.\n\n\n> **Weakness 2**:  While the paper analyzes adaptivity, it could benefit from a more in-depth ablation, e.g., testing hybrid or partial question masking, or examining pathological cases where QFFT may fail.\n\n**Re**: We provide detailed clarification to your questions in our response to **Weakness 3** and **Question 1**.\n\n\n> **Weakness 3**: The discussion of potential negative impacts (e.g., removing questions during training could risk instruction-following degradation in some applications) is not fully explored in the main paper, though some limitations are covered in the appendix.\n\n**Re**:  Thanks for your insightful suggestions.   We conducted experiments on pathological scenarios where QFFT may fail, focusing on two key scenarios: Noisy Scenarios and Low-Resource Scenarios. **While our experiments did not reveal performance degradation of QFFT relative to SFT in these scenarios, we acknowledge that investigating more diverse scenarios would be valuable and plan to explore them as future work.**\n\n**(1) We first performed QFFT training in Noisy Scenarios.** \n\nTo assess QFFT's robustness to noisy data—common in real-world Long CoT distillation (e.g., incomplete reasoning, incorrect conclusions, irrelevant responses)—we tested it across four noise levels using the LIMO dataset and Qwen2.5-7B-Instruct, evaluating on MATH with 16 runs per question. We aligned the hyperparameters across all settings and, due to resource constraints, trained each model for 3 epochs.\n\nLevel I (normal data) served as the baseline. Levels II–IV introduced errors with gradually increasing damage to the data: Level II involved incorrect conclusions, Level III included truncated reasoning, and Level IV contained mismatched question-answer pairs, respectively. As shown in the table below: \n\n| Data   | Method | MATH Pass@1(%) |\n|---|--|-|\n| Level1 | SFT | 76.54|\n| Level1 | QFFT | 78.61|\n| Level2 | SFT | 68.03|\n| Level2 | QFFT | 74.69  |\n| Level3 | SFT  | 37.39|  \n| Level3 | QFFT | 76.89 |\n| Level4 | SFT  | 0.41  |\n| Level4 | QFFT | 78.61  |\n\nSFT performance plummeted from 76.5% (Level I) to 0.4% (Level IV), losing reasoning ability with extreme noise. QFFT, however, remained robust, retaining 78.6% performance at Level IV—matching its Level I results—demonstrating clear advantages in handling noisy training data.\n\n**(2) We also conducted experiments in Low-Resource Scenarios.**\n\nTo examine QFFT's performance in low-resource settings, we sampled 10 data points from the S1.1 dataset, with each generating 10 responses via Deepseek-R1 to form 100 training instances. Both SFT and QFFT were trained on these data for 50 epochs with identical parameters. As shown in the table below: \n\n| Training Steps  | SFT Accuracy (%) | QFFT Accuracy (%) |\n|--|--|-|\n| 35| 71.95| 76.45|\n| 70 | 75.85 | 78.75 |\n| 105 | 76.85| 79.15 |\n| 140| 74.20| 78.20|\n| 175| 74.25 | 79.00|\n| 210| 75.05 | 77.75 |\n| 245 | 74.35| 76.50 |\n| 280| 74.40 | 77.95 |\n| 300（50 epoch）| 76.00 | 76.75|\n\nQFFT consistently outperforms SFT across all checkpoints in this low-resource setting. \n\n> **Question 2**: Is there any observed degradation in instruction-following or user prompt adherence, especially for non-mathematical tasks, due to removal of questions during training? If so, can the authors comment or provide measurements?\n\n**Re**:  Thank you for your valuable question.  We have conducted experiments on two general instruction-following datasets (IFEval [1] and AlignBench-v1 [2]), as shown below:\n\n| Model | Configuration | IFEval All | IFEval Short CoT (Score / Percentage) | IFEval Long CoT (Score / Percentage) | AlignBench-v1 All | AlignBench-v1 Short CoT (Score / Percentage) | AlignBench-v1 Long CoT (Score / Percentage) |\n|-|---|---|--|-|-|-|-|\n| Qwen2.5-7B-Instruct | -| 72.9| 80.9 / 0| - | 7.13 | 7.13 / 100%  | 0|\n| Qwen2.5-7B-Instruct| + S1.1 1K SFT | 62.8| 68.4 / 70.4% | 49.4 / 29.6% | 5.33 | 5.93 / 60.2%  | 4.43 / 40.8% |\n| Qwen2.5-7B-Instruct | + S1.1 1K QFFT  | 69.3  | 71.0 / 96.2%  | 51.2 / 1.5% | 6.06  | 6.12 / 96.9%   | 4.28 / 3.1% |\n| Qwen2.5-7B-Instruct | + BS 17K SFT | 63.2 | 69.7 / 62%| 52.6 / 38.0% | 5.01 | 5.64 / 50.7%  | 4.36 / 49.3%  |\n| Qwen2.5-7B-Instruct | + BS 17K QFFT| 71.7| 71.0 / 98% | 50.3 / 2.0% | 6.03 | 6.05 / 98.4% | 4.63 / 1.6% |\n| Qwen2.5-32B-Instruct | -  | 80.9 | 80.9 / 0 | -  | 7.13 | 7.13 / 100% | 0  |\n| Qwen2.5-32B-Instruct | + S1.1 1K SFT| 67.4 | 74.0 / 71.7%  | 50.8 / 28.3% | 5.60  | 6.08 / 69.7% | 4.49 / 30.3% |\n| Qwen2.5-32B-Instruct| + S1.1 1K QFFT  | 75.1 | 79.5 / 93.3%  | 52.8 / 1.7% | 6.71 | 6.81 / 95.0% | 4.87 / 5% |\n\nBased on these experiments, we draw the following conclusions:\n\n- Long CoT SFT can degrade the model's instruction-following capabilities on general tasks. Similar findings have been noted in work [3].\n\n- Models fine-tuned with QFFT exhibit stronger instruction-following abilities compared to SFT. This is because SFT leads to relying more on Long CoT patterns, which significantly impairs instruction-following. In contrast, QFFT preserves Short CoT patterns in most cases, thereby maintaining better instruction-following performance on general tasks.\n\n[1] Instruction-Following Evaluation for Large Language Models\n\n[2] AlignBench: Benchmarking Chinese Alignment of Large Language Models\n\n[3] Scaling Reasoning, Losing Control: Evaluating  Instruction Following in Large Reasoning Models\n\n> **Question 1**: Have the authors considered or evaluated hybrid fine-tuning schemes, such as partial masking of questions, curriculum masking, or re-introducing questions at inference time? Would this further improve adaptivity or mitigate risks of instruction-following degradation?\n\n**Re**:   Thank you for your insightful suggestions. This is a point well worth exploring. \n\nWe conducted hybrid question masking experiments: **progressively incorporating samples with questions into the QFFT training process, which, from the perspective of SFT, is equivalent to progressively masking questions.**  We have supplemented our work with the following experiment: For the S1.1 1k dataset, we progressively masked the questions in α% of the samples and then performed SFT. Subsequently, we evaluated the trained model on the MATH500 dataset. We tracked changes in adaptivity (TAK) and the proportion of Short CoT usage as the ratio of samples with questions increased, as shown in the table below:\n\n| Ratio(α) of Data with Question (%) | Ratio of Short CoT (%) | TAK  |\n|-|-|-|\n| 0%| 56.2| 38.4|\n| 0.1% | 40.6| 30.5|\n| 1% | 13.1| 8.8|\n| 10% | 10.2 | 8.4|\n| 100%| 8.0 | 6.2 |\n\nThis experiment leads to the following conclusions: As the proportion of masked questions increases(fewer samples with questions), the model's adaptivity (TAK) steadily improves.  This partially validates the hypothesis in Section 2.3 of our paper: Even an extremely small proportion of samples with question (eg. 1%) mixed into the data can override the model's original mapping to Short CoT responses, erasing the model’s prior ability to respond concisely. This demonstrates the rationality of the motivation behind the QFFT method."}, {"review_id": "0Q5o45blSS", "reviewer": "Reviewer_dV3j", "summary": "The paper proposes a novel approach to solve the problem of \"overthinking\" in reasoning models. They train the model only on the answer reasoning chains removing the original question. This results in using shorter reasoning chains for simpler problems and longer for more complex ones.", "questions": "Please see the weaknesses", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper proposes a novel approach that targets the problem of overthinking by eliminating the input question while finetuning.\n- Despite significant reduction in number of output tokens, QFFT does not degrade the performance much. \n- QFFT is able to switch between long and short CoT depending upon the complexity of problem. \n- QFFT achieves superior performance on out-of-domain tasks, demonstrating that question free finetuning results in more transferable.\n\n\nWeaknesses:\n- The paper could benifit from a deeper manual analysis of failure modes of QFFT compared to other methods. \n- It is not clear whether QFFT would generalize to toher types of reasoning like commmon-sense, scientific etc.\n- Does QFFT lead to any reduction in other model capabilities?", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive comments. We have carefully considered your suggestions and would like to provide the following clarifications.\n\n>  **Weakness 1:** The paper could benifit from a deeper manual analysis of failure modes of QFFT compared to other methods.\n\n**Re**: Thanks for your insightful suggestions. To identify the failure modes of QFFT, we carried out experiments on pathological scenarios where QFFT might underperform.\n\n**(1)  We first performed QFFT training in Noisy Scenarios.**\n\nTo assess QFFT's robustness to noisy data—common in real-world Long CoT distillation (e.g., incomplete reasoning, incorrect conclusions, irrelevant responses)—we tested it across four noise levels using the LIMO dataset and Qwen2.5-7B-Instruct, evaluating on MATH with 16 runs per question. We aligned the hyperparameters across all settings and, due to resource constraints, trained each model for 3 epochs.\n\nLevel I (normal data) served as the baseline. Levels II–IV introduced errors with gradually increasing damage to the data: Level II involved incorrect conclusions, Level III included truncated reasoning, and Level IV contained mismatched question-answer pairs, respectively. As shown in the table below: \n\n| Data   | Method | MATH Pass@1(%) |\n|--------|--------|----------------|\n| Level1 | SFT    | 76.54          |\n| Level1 | QFFT   | 78.61          |\n| Level2 | SFT    | 68.03          |\n| Level2 | QFFT   | 74.69          |\n| Level3 | SFT    | 37.39          |  \n| Level3 | QFFT   | 76.89          |\n| Level4 | SFT    | 0.41           |\n| Level4 | QFFT   | 78.61          |\n\n SFT performance plummeted from 76.5% (Level I) to 0.4% (Level IV), losing reasoning ability with extreme noise. QFFT, however, remained robust, retaining 78.6% performance at Level IV—matching its Level I results—demonstrating clear advantages in handling noisy training data.\n\n**(2)  We also conducted experiments in Low-Resource Scenarios.**\n\nTo examine QFFT's performance in low-resource settings, we sampled 10 data points from the S1.1 dataset, with each generating 10 responses via Deepseek-R1 to form 100 training instances. Both SFT and QFFT were trained on these data for 50 epochs with identical parameters. As shown in the table below: \n\n| Training Steps       | SFT Accuracy (%) | QFFT Accuracy (%) |\n|----------------------|------------------|-------------------|\n| 35                   | 71.95            | 76.45             |\n| 70                   | 75.85            | 78.75             |\n| 105                  | 76.85            | 79.15             |\n| 140                  | 74.20            | 78.20             |\n| 175                  | 74.25            | 79.00             |\n| 210                  | 75.05            | 77.75             |\n| 245                  | 74.35            | 76.50             |\n| 280                  | 74.40            | 77.95             |\n| 300（50 epoch）      | 76.00            | 76.75             |\n\nQFFT consistently outperforms SFT across all checkpoints in this low-resource setting. SFT models over-relied on insufficiently internalized Long CoT patterns, limiting performance. In contrast, QFFT retained original Short CoT patterns and adaptively used Long CoT when needed, integrating reflective behaviors with foundational abilities to achieve better results.\n\n**While the aforementioned experiments did not reveal obvious shortcomings of QFFT, theoretically, our method may have a limitation: it cannot be trained on base models (e.g., Qwen2-7B-Base). This is because QFFT requires the model to inherently possess basic Short CoT capabilities, which is why our existing experiments need to use instruction-tuned models (e.g., Qwen2.5-7B-Instruct). We look forward to subsequent researchers with interest exploring this direction more deeply.**\n\n\n>  **Weakness 2:** It is not clear whether QFFT would generalize to other types of reasoning like commmon-sense, scientific etc.\n\n**Re**:   Thank you for your valuable question. We rebuttal from the following two aspects:\n\n(1) Training with mathematical data (LIMO) and evaluating on OOD tasks: In Section 4.4 (Table 4) of our paper, we evaluated performance on out-of-domain tasks, including the general task MMLU-Pro and the scientific task GPQA. The specific results are as follows:\n  \n  | Model  | MMLU-Pro | GPQA  |\n  |---|----------|-------|\n  | Qwen-2.5-Instruct-7B        | 40.0     | 36.4  |\n  | &nbsp;&nbsp;+ LIMO-SFT      | 29.3     | 43.2  |\n  | &nbsp;&nbsp;+ LIMO-QFFT     | 48.4     | 44.2  |\n  | Qwen-2.5-Instruct-32B       | 62.2     | 49.5  |\n  | &nbsp;&nbsp;+ LIMO-SFT      | 52.4     | 65.3  |\n  | &nbsp;&nbsp;+ LIMO-QFFT     | 73.4     | 67.9  |\n  \nThe conclusion is that models trained with QFFT outperform SFT on out-of-domain tasks, including general and scientific reasoning. This demonstrates the superior out-of-domain generalization ability of the QFFT approach.\n  \n(2) We also conducted experiments **training** on the Bespoke-Stratos-17k dataset, which includes a mix of mathematical, scientific, and code-related data. Our Table 1 (in our manuscript) includes the results for models trained on Bespoke-Stratos-17k data, where QFFT matches SFT in performance while significantly reducing token count and improving efficiency. This demonstrates that QFFT can also be effectively trained on other types of reasoning data.\n\n>  **Weakness 3:** Does QFFT lead to any reduction in other model capabilities?\n\n**Re**:   This is a question well worth exploring.\n\n  (1) First, like other SFT-based Long reasoning models, QFFT may slightly harm the model's instruction-following capabilities (compared to the base model). We systematically evaluated QFFT and SFT against the base models on instruction-following benchmarks, as shown in the table below:\n\n| Model | Configuration | IFEval All | IFEval Short CoT (Score / Percentage) | IFEval Long CoT (Score / Percentage) | AlignBench-v1 All | AlignBench-v1 Short CoT (Score / Percentage) | AlignBench-v1 Long CoT (Score / Percentage) |\n|-|---|---|--|-|-|-|-|\n| Qwen2.5-7B-Instruct | -| 72.9| 80.9 / 0| - | 7.13 | 7.13 / 100%  | 0|\n| Qwen2.5-7B-Instruct| + S1.1 1K SFT | 62.8| 68.4 / 70.4% | 49.4 / 29.6% | 5.33 | 5.93 / 60.2%  | 4.43 / 40.8% |\n| Qwen2.5-7B-Instruct | + S1.1 1K QFFT  | 69.3  | 71.0 / 96.2%  | 51.2 / 1.5% | 6.06  | 6.12 / 96.9%   | 4.28 / 3.1% |\n| Qwen2.5-7B-Instruct | + BS 17K SFT | 63.2 | 69.7 / 62%| 52.6 / 38.0% | 5.01 | 5.64 / 50.7%  | 4.36 / 49.3%  |\n| Qwen2.5-7B-Instruct | + BS 17K QFFT| 71.7| 71.0 / 98% | 50.3 / 2.0% | 6.03 | 6.05 / 98.4% | 4.63 / 1.6% |\n| Qwen2.5-32B-Instruct | -  | 80.9 | 80.9 / 0 | -  | 7.13 | 7.13 / 100% | 0  |\n| Qwen2.5-32B-Instruct | + S1.1 1K SFT| 67.4 | 74.0 / 71.7%  | 50.8 / 28.3% | 5.60  | 6.08 / 69.7% | 4.49 / 30.3% |\n| Qwen2.5-32B-Instruct| + S1.1 1K QFFT  | 75.1 | 79.5 / 93.3%  | 52.8 / 1.7% | 6.71 | 6.81 / 95.0% | 4.87 / 5% |\n\nBut our experimental results indicate that QFFT models exhibit stronger instruction-following capabilities than SFT. On one hand, this highlights the inherent limitations of Long CoT in instruction-following, a point also mentioned in [1]. On the other hand, compared to SFT, QFFT preserves Short CoT patterns, resulting in better instruction-following performance overall.\n\n**We have not observed degradation in other model capabilities from QFFT training. We will continue to monitor and investigate this aspect in future work.**\n\n[1] Scaling Reasoning, Losing Control: Evaluating  Instruction Following in Large Reasoning Models"}, {"review_id": "aTty09G1YZ", "reviewer": "Reviewer_CTuE", "summary": "This paper introduces Question-Free Fine-Tuning (QFFT), a new method to improve the efficiency and adaptability of large language models. Instead of training with question–answer pairs, QFFT removes the input questions and fine-tunes only on reasoning chains. This allows the model to keep its original concise reasoning for simple tasks while selectively using longer, reflective reasoning for complex ones. Experiments show that QFFT reduces response length by over 50% without losing accuracy and generalizes better to new tasks. It effectively balances performance and token efficiency by enabling adaptive reasoning.", "questions": "1. As mentioned in the Weakness, could the authors consider including additional and stronger baselines?\n2. How does the proposed method perform on other models, such as LLaMA? It would be helpful to include additional experiments to demonstrate the generalizability of the method across different model architectures.\n3. Since the proposed fine-tuning method removes human user queries, it may potentially weaken the model’s instruction-following ability. Could the authors include experiments on instruction-following benchmarks to rule out this possibility?", "limitations": "See Weaknesses and Questions", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. The paper conducts comprehensive experiments on both in-domain (e.g., GSM8K, AIME) and out-of-domain (e.g., GPQA, MMLU-Pro) benchmarks, showing that QFFT maintains or slightly improves accuracy while significantly reducing token usage.\n2. The paper clearly identifies the problem of overthinking in long CoT models and the inefficiency of existing distillation approaches. Besides, the paper is well-organized, with distinct sections for background, method, experiments, and analysis, each flowing logically from one to the next.\n3. The core innovation—fine-tuning without the input question—is simple yet effective. It breaks from standard supervised fine-tuning and opens a new direction in adaptive reasoning.\n\n**Weaknesses:**\n1. To the best of my knowledge, there has been extensive research on effective reasoning. However, this work includes only a limited number of relatively weak baselines. Stronger and more diverse baselines should be introduced to enable a more rigorous and convincing comparison with the proposed method.\n2. According to the results in Table 2, the proposed method shows limited improvement on relatively simple tasks. On the 32B model, no performance gains are observed on either the GSM8K or MATH datasets.", "was_revised": true, "final_justification": "The paper presents a well-structured study that introduces a simple yet effective method QFFT.  Through comprehensive in-domain and out-of-domain experiments, it demonstrates that QFFT maintains or slightly improves accuracy while significantly reducing token usage. Through the rebuttal, the authors effectively addressed my concerns regarding additional models and instruction-following performance, which led me to raise my score from 4 to 5.", "rebuttal": "Thank you for your constructive comments. We have carefully considered your suggestions and would like to provide the following clarifications.\n\n> **Weakness 1:** To the best of my knowledge, there has been extensive research on effective reasoning. However, this work includes only a limited number of relatively weak baselines. Stronger and more diverse baselines should be introduced to enable a more rigorous and convincing comparison with the proposed method. As mentioned in the Weakness, could the authors consider including additional and stronger baselines?\n\n**Re:** Thank you for your valuable suggestions. We have incorporated additional baselines focused on efficient reasoning. These include training-free methods including Chain-of-Draft [1], Self-Doubt[2] model merge[3], as well as training-based methods CoT-Valve [4] and O1-Pruner [5].\n\nFor Chain-of-Draft and Self-Doubt, we followed the original prompts and reasoning strategies, applying them to our LIMO SFT-trained model to evaluate performance. For the Model Merging and CoT-Valve baselines, we selected the highest-performing variants reported in their respective original papers. Specifically, for Model Merging, we employed Ties-Merging to combine our LIMO SFT-trained model with the original instruct model, following the methodology outlined in [3]. For CoT-Valve, we adopted the CoT-Valve+P training approach and utilized their publicly available MixChain-Z dataset for training, as described in [4]. During evaluation, we sampled each question 16 times and computed the average performance across all methods to ensure fair and consistent comparisons. The results, summarized in the table below, demonstrate that QFFT exhibits strong competitiveness across various metrics.\n\nAdditionally, QFFT and long-to-short methods are orthogonal. This means that the Long CoT outputs from QFFT models can be further compressed using long-to-short techniques, thereby elevating the upper limit of QFFT's efficiency. We provide experiments demonstrating this in the table below.\n\n| Method | GSM8K Acc ↑ | GSM8K Tokens ↓ | GSM8K AES ↑ | MATH Acc ↑ | MATH Tokens ↓ | MATH AES ↑ | AIME25 Acc ↑ | AIME25 Tokens ↓ | AIME25 AES ↑ | Avg. Acc ↑ | Avg. Tokens ↓ | Avg. AES ↑ |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| LIMO 7B (base) | 88.2 | 1.8K | - | 80.4 | 5.9K | - | 16.8 | 17.8K | - | 61.8 | 8.5K | - |\n| Training Free baselines |  |  |  |  |  |  |  |  |  |  |  |  |\n| Chain-of-Draft | 84.2 | 1.5K | -2.9 | 77.1 | 5.6K | -3.6 | 15.7 | 17.6K | -6.4 | 59.0 | 8.23K | -4.3 |\n| Self-Doubt | 87.7 | 1.1K | 3.4 | 79.4 | 4.0K | 1.7 | 15.9 | 12.3K | 2.2 | 61.0 | 5.8K | 2.4 |\n| Model-Merge(Ties-Merging) | 85.0 | 1.1K | 0.3 | 77.0 | 4.1K | -1.2 | 12.7 | 15.9K | -23.3 | 58.23 | 7.03K | -8.1 |\n| Taining-based Methods |  |  |  |  |  |  |  |  |  |  |  |  |\n| CoT-Valve(CoT-Valve+P-MixChain-Z) | 82.7 | 0.5K | 1.0 | 66.2 | 3.1K | -12.9 | 14.4 | 12.8K | -11.5 | 54.43 | 5.47K | -7.8 |\n| O1-pruner | 90.8 | 0.8K | 5.8 | 78.2 | 3.2K | 1.8 | 14.2 | 12.2K | -12.7 | 61.07 | 5.4K | -1.7 |\n| QFFT (Ours) | 88.0 | 0.7K | 5.9 | 80.6 | 4.1K | 2.9 | 17.2 | 15.6K | 1.4 | 61.93 | 6.8K | 3.4 |\n| QFFT + SimPO | 89.0 | 0.5K | 7.3 | 80.2 | 3.4K | 4.1 | 17.7 | 13.6K | 2.9 | 62.3 | 5.83K | 4.8 |\n\n[1] Chain of Draft: Thinking Faster by Writing Less\n\n[2] Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt\n\n[3] Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging\n\n[4] CoT-Valve: Length-Compressible Chain-of-Thought Tuning\n\n[5] O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning\n\n> **Weakness 2:** According to the results in Table 2, the proposed method shows limited improvement on relatively simple tasks. On the 32B model, no performance gains are observed on either the GSM8K or MATH datasets.\n\n**Re:** Thank you for your question. Indeed, as shown in Table 2 in our paper, QFFT provides limited efficiency improvements (AES) on relatively simple evaluation datasets (such as GSM8K), especially when compared to strong long-to-short baselines like O1-Pruner. This is because QFFT is fundamentally an adaptive reasoning method: its efficiency gains come from replacing Long CoT with Short CoT for simple questions, rather than compressing the Long CoT responses. In contrast, long-to-short methods compress the redundant parts of Long CoT responses.\n\nHowever, long-to-short methods have a notable drawback: they can significantly harm performance on more difficult problems. As discussed in work [6][7], adequate reasoning length is critical for \"slow thinking\" models to effectively solve complex tasks, and compressing the Long CoT chain often leads to severe performance degradation on such tasks. In this regard, QFFT avoids this pitfall and is able to match SFT performance on challenging problems.\n\nAdditionally, **QFFT and long-to-short methods are orthogonal**. This means that the Long CoT outputs from QFFT models can be further compressed using long-to-short techniques, thereby elevating the upper limit of QFFT's efficiency. We provide supporting experiments in the table below:\n\n| Method | GSM8K Acc ↑ | GSM8K Tokens ↓ | GSM8K AES ↑ | MATH Acc ↑ | MATH Tokens ↓ | MATH AES ↑ | AIME25 Acc ↑ | AIME25 Tokens ↓ | AIME25 AES ↑ |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| LIMO 7B (base) | 88.2 | 1.8K | - | 80.4 | 5.9K | - | 16.8 | 17.8K | - |\n| O1-pruner | 90.8 | 0.8K | 5.8 | 78.2 | 3.2K | 1.8 | 14.2 | 12.2K | -12.7 |\n| QFFT (Ours) | 88.0 | 0.7K | 5.9 | 80.6 | 4.1K | 2.9 | 17.2 | 15.6K | 1.4 |\n| QFFT + SimPO | 89.0 | 0.5K | 7.3 | 80.2 | 3.4K | 4.1 | 17.7 | 13.6K | 2.9 |\n\n[6] DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models\n\n[7] s1: Simple test-time scaling.\n\n\n> **Question 2:** How does the proposed method perform on other models, such as LLaMA? It would be helpful to include additional experiments to demonstrate the generalizability of the method across different model architectures.\n\n**Re:** Thank you for your insightful suggestion. In Appendix C.2 (Table 6), we include experiments on the Phi-4-mini-Instruct model, which validate the effectiveness of QFFT on the Phi-4 architecture.\n\nAdditionally, we have added experiments on the LLaMA3 architecture, as shown below:\n\n| Data | MATH500 (Acc/Tokens/TAK) | GSM8K (Acc/Tokens/TAK) | AIME25 (Acc/Tokens/TAK) |\n| --- | --- | --- | --- |\n| Llama3.1-8B-Instruct | 44.2/1.1K/0 | 67.4/0.8K/0 | 0.6/2.4K/0 |\n|+ 1k S1.1 QFFT | 56.1/4.8K/28.6 | 76.5/1.3K/26.1 | 4.4/16.5K/9.9 |\n|+ 1k S1.1 SFT | 56.3/9.7K/4.4 | 75.2/5.1K/2.1 | 3.7/24.8K/0 |\n|+ Bespoke-Stratos-17k QFFT | 68.1/3.8K/36.8 | 86.8/1.2K/27.2 | 12.3/13.7K/22.4 |\n|+ Bespoke-Stratos-17k SFT | 68.8/7.1K/0.4 | 86.3/4.0K/3.8 | 12.5/19.6K/0 |\n\nWe observe that for LLaMA3.1-8B, both SFT and QFFT perform poorly on AIME25 with only 1k (S1.1) data. We believe this is because 1k samples are insufficient for the LLaMA model to effectively learn Long CoT capabilities. In contrast, with BS-17k data, there is a substantial improvement over the 1k results. We attribute this to characteristics inherent to LLaMA3 itself. Inspired by works [8] and [9], we note that LLaMA3 lacks sufficient injection of Long CoT patterns such as reflection and backtracking, during its pre-training phase (compared to Qwen), thus requiring more data to learn Long CoT patterns effectively.\n\n[8] OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling\n\n[9] Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs\n\n\n> **Question 3:** Since the proposed fine-tuning method removes human user queries, it may potentially weaken the model’s instruction-following ability. Could the authors include experiments on instruction-following benchmarks to rule out this possibility?\n\n**Re:** Thank you for your valuable question. We have conducted experiments on two general instruction-following datasets (IFEval [10] and AlignBench-v1 [11]), as shown below:\n\n| Model | Configuration | IFEval All | IFEval Short CoT (Score / Percentage) | IFEval Long CoT (Score / Percentage) | AlignBench-v1 All | AlignBench-v1 Short CoT (Score / Percentage) | AlignBench-v1 Long CoT (Score / Percentage) |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Qwen2.5-7B-Instruct | - | 72.9 | 80.9 / 0% | - | 7.13 | 7.13 / 100%| 0 |\n| Qwen2.5-7B-Instruct | + S1.1 1K SFT | 62.8 | 68.4 / 70.4% | 49.4 / 29.6% | 5.33 | 5.93 / 60.2% | 4.43 / 40.8% |\n| Qwen2.5-7B-Instruct | + S1.1 1K QFFT | 69.3 | 71.0 / 96.2% | 51.2 / 1.5% | 6.06 | 6.12 / 96.9% | 4.28 / 3.1% |\n| Qwen2.5-7B-Instruct | + BS 17K SFT | 63.2 | 69.7 / 62% | 52.6 / 38.0% | 5.01 | 5.64 / 50.7% | 4.36 / 49.3% |\n| Qwen2.5-7B-Instruct | + BS 17K QFFT | 71.7 | 71.0 / 98% | 50.3 / 2.0% | 6.03 | 6.05 / 98.4% | 4.63 / 1.6% |\n| Qwen2.5-32B-Instruct | - | 80.9 | 80.9 / 0 | - | 7.13 | 7.13 / 100% | 0 |\n| Qwen2.5-32B-Instruct | + S1.1 1K SFT | 67.4 | 74.0 / 71.7% | 50.8 / 28.3% | 5.60 | 6.08 / 69.7% | 4.49 / 30.3% |\n| Qwen2.5-32B-Instruct | + S1.1 1K QFFT | 75.1 | 79.5 / 93.3% | 52.8 / 1.7% | 6.71 | 6.81 / 95.0% | 4.87 / 5% |\n\nBased on these experiments, we draw the following conclusions:\n\n(1) Long CoT SFT can degrade the model's instruction-following capabilities on general tasks. Similar findings have been noted in work [12].\n\n(2) Models fine-tuned with QFFT exhibit stronger instruction-following abilities compared to those with SFT. This is because SFT leads to over-reliance on Long CoT patterns, which significantly impairs instruction-following. In contrast, QFFT preserves Short CoT patterns in most cases, thereby maintaining better instruction-following performance on general tasks.\n\n[10] [Instruction-Following Evaluation for Large Language Models](https://arxiv.org/abs/2311.07911)\n\n[11] AlignBench: Benchmarking Chinese Alignment of Large Language Models\n\n[12] Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models"}]

Author rebuttal:

Meta-review: This paper introduces Question-Free Fine-Tuning (QFFT), a simple yet novel approach to mitigating the inefficiency of long chain-of-thought (CoT) reasoning in large language models. Instead of training on question–answer pairs, QFFT fine-tunes solely on reasoning chains, encouraging models to adaptively balance concise reasoning on simple problems and reflective reasoning on harder ones. Experiments on multiple benchmarks suggest that QFFT reduces token usage by over 50% while maintaining comparable or slightly improved accuracy, with some evidence of stronger generalization to out-of-domain tasks.

All reviewers agree that the method is conceptually simple, original, and effective, addressing the important and underexplored issue of overthinking in long-CoT reasoning models. Comprehensive experiments demonstrate that QFFT significantly improves token efficiency while preserving or modestly improving task accuracy. The introduction of the Thinking Adaptive Cohen’s Kappa (TAK) metric provides a useful lens for measuring adaptive reasoning. The paper is clearly written, well structured, and supported by both in-domain and out-of-domain evaluations. Reviewers highlight that the rebuttal satisfactorily addressed concerns regarding generalizability to additional model families and potential degradation of instruction-following.

On the other hand, several reviewers point out that the set of baselines is limited. Comparisons against stronger and more diverse long-to-short reasoning approaches (e.g., SimPO, pruning-based methods) would strengthen the empirical case. Some concerns remain about generalizability, as the evaluation focuses primarily on Qwen, with limited results on other architectures, though the rebuttal provided additional supporting evidence. A deeper analysis of failure modes, hybrid schemes (partial question masking), or potential impacts on instruction following could make the work more robust.

Overall, this paper makes a meaningful and original contribution to adaptive reasoning in large language models. While not without limitations, the simplicity and effectiveness of QFFT, together with the strong empirical evidence, make it a valuable addition to the literature. I recommend acceptance.
Decision: Accept (spotlight)