DLoFT: Gradient-Decoupled Fine-Tuning for Generalizable Long Chain-of-Thought Reasoning
Authors: Sitong Wu
Haoru Tan
Jingyao Li
Shaofeng Zhang
XIAOJUAN QI
Bei Yu
Jiaya Jia
Venue: NeurIPS 2025 poster ()
Keywords: Reasoning
LLM
Primary area: deep_learning

Abstract: Long chain-of-thought (LongCoT) has emerged as a powerful reasoning paradigm for enabling large language models (LLMs) to solve complex tasks through a systematic and thorough thinking phase.
Although supervised fine-tuning (SFT) on high-quality LongCoT traces has proven effective to activate LongCoT abilities, we find that models trained in this way tend to overfit problem-specific knowledge and heuristics, leading to degraded out-of-distribution performance.
To address this issue, we propose a Decoupled LongCoT Fine-Tuning (DLoFT) algorithm, which enables the model to learn generalizable LongCoT reasoning abilities while preventing overfitting to the reasoning content with problem-specific information.
The key idea is to decouple the gradient into two orthogonal components: 1) a paradigm-relevant gradient corresponding to the general LongCoT paradigm and 2) a content-relevant gradient reflecting the problem-specific information, where only the former gradient is used to update model parameters.
Specifically, by leveraging the unique two-phase composition (thinking and solution) of the LongCoT response, our gradient decoupling mechanism isolates the content-relevant gradient via a projection operation and separates the paradigm-relevant gradient through orthogonalization.
Our DLoFT ensures the model concentrate on internalizing the LongCoT paradigm rather than memorizing problem-specific knowledge and heuristics.
Extensive experiments demonstrate that our DLoFT significantly improves the generalization behavior of LongCoT abilities compared to SFT while maintaining strong in-distribution performance.

DLoFT: Gradient-Decoupled Fine-Tuning for
Generalizable Long Chain-of-Thought Reasoning
Sitong Wu Haoru Tan
Jingyao Li!
Shaofeng Zhang?
Xiaojuan Qi
Bei Yu It Jiaya Jia
The Chinese University of Hong Kong
The University of Hong Kong
Shanghai Jiao Tong University *The Hong Kong University of Science and Technology
Abstract
Long chain-of-thought (LongCoT) has emerged as a powerful reasoning paradigm for enabling large language models (LLMs) to solve complex tasks through a systematic and thorough thinking phase. Although supervised fine-tuning (SFT) on high-quality LongCoT traces has proven effective to activate LongCo'T abilities, we find that models trained this way tend overfit goblem-sspecifg knowledge and heuristics, leading to degraded out-of-distributon performance. To address this issue, we propose a Decoupled LongCoT Fine-Tuning (DLoFT) algorithm, which enables the model to learn generalizable LongCoT reasoning abilities while preventing overfitting to the reasoning content with problem-specift information. The key idea is to decouple the gradient into two orthogonal components: 1) a paradigm-relevan gradient corresponding to the general LongCoT paradigm and 2) content-rclevant gradient reflecting the croblem-specific information, where only the former gradient is used to update model parameters. Specifically, by leveraging the unique two-phase composition (thinking and solution) of the Long- CoT response, our gradient decoupling mechanism isolates the content-relevan gradient via a projection operation and separates the paradigm-reles relevvnn gradi- ent through orthogonalization. Our DLoFT ensures the model concentrate on internalizing the LongCoT paradigm rather than memorizing problem-speccfie knowledge and heuristics. Extensive experiments demonstrate that our DLoFT significantly improves the generalization behavior of LongCoT abilities compared to SFT while maintaining strong in-distribution performance. The code is available https::/ github. cww/dvlalllleecccooooo research/DLoFd
Introduction
Recent large language models (LLMs), for example, OpenAI ol-o3 [1-3], have achieved remarkable breakthroughs in difficult reasoning tasks [4-6]. A key technique driving these advances is the long chain-of-thought (LongCoT)) reasoning paradigm, which encourages models conduct systematic and thorough thinking process involving reflection on intermediate steps, error correction, feasibility assessment, and exploration of alternative ideas.
Following these breakthroughs, the community has attempted to replicate such LongCoT reasoning abilities, which remains significant challenge. Recent efforts typically rely imitating high-quality LongCoT traces via supervised fine-tuning (SFT) [7-9] or conducting costly reinforcement learning (RL) on tasks with deterministic final answer [10]. Notably, existing works are predominantly focused
"qqual contribution (stone-wa wu Corresponding author
hdu. hrtan hkk.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

SFT (in-domain)
SFT [out-domain) DLFFT (out-domain)
- SFT lin-domain)
SFT (out-domain
DLoFT (in-domain)
DLoFT (in-domain!
DOFFT (out-domain)
70
100
80
LongCoT Response Ratio (96)
80
Accuracy
10
60
20
10
100
200
300
400
500
800
700
800
100
200
300
400
500
600
700
800
Training Step
Training Step
Figure 1: Illustration of the overfitting problem. We train the Qwen2.5-788 1eem2..5-7BIIstrrrre Instrrcctt [11] [] on OpenR 1- Math-5K LongCoT dataset [12] using SFT and our DLoFT respectively, and test on AIME24 math competition [4] as in-domain evaluation and MedQA [13] as out-domain evaluation. (a) shows the change test accuracy during training. and (b) shows the percentage of the model's response the test data containing LongCo'I behavior training progresses. The yellow dotted line indicates the turning point where out-domain performance begins to degrade continuously. Please refer to Sec. for more analysis of this overfitting problem.
on narrow fields, such as competition-level mathematics [4], code generation [6], and PhD-level science question answering [5]. Considering the data sparsity and proprietary for some fields, is
non-trivial and impractical to construct a large-scale dataset with comprehensive knowledge coverage, and it is extremely expensive to annotate the high-quality LongCo'T response. Consequently, practical question arises: Can the LongCoT reasoning ability acquired from specific fields (e.g., mathematics) effectively generalize unseen fields (e.g., engincering, economics, literature, etc)?
Observations. To investigate this, we conducted a series of empirical studies as Figure Our experiments reveal surprisingly poor generalization performance novel fields. By analyzing the training dynamics, we found that the -o--off-iiittttionn performance degradation arises from the overfitting to problem-specifie knowledge and heuristics present in the training data. In addition, have tried common overfitting mitigation strategies, such as early stopping. Although simply applying early stopping produces slight improvement in unseen fields, it simultaneously leads to insufficient learning in seen fields and does not always exhibit the LongCoT paradigm. We attribute this to the inherent coupling of general LongCoT reasoning abilities with the problem-speeific knowledge and heuristics in supervised fine-tuning, These findings underscore the necessity for a training algorithm capable of decoupling problem-speciff knowledge from underlying LongCoT reasoning abilities.
Our Approach. In this paper, we propose a novel training algorithm, named Decoupled LongCoTT' Fine-Tuning (DLoFT), which enables the model master the generalizable LongCoT reasoning paradigm while avoiding overfitting to reasoning content. The central idea to filter out the gradient component induced from problen-specific information via gradient decoupling mechanism. Specifi- cally, in each training iteration, we first calculate the full gradient giill of the entire LongCoT response, which blends the influence of two learning signals: other is the knowledge and skills required to solve the problem. Then, we decouple the full gradient gradient gpar exclusively capturing
the LongCoT reasoning paradigm, and the
grull into two orthogonal components: (1) the paradigm-relewee the general LongCoT reasoning paradigm, (2) the content-relevant gradient gcon corresponding to the problem-specific knowledge and heuristics. A key insight of our decoupling mechanism that the inherent two-phase structure of LongCoT responses (a thinking phase followed by solution phase) naturally facilitates the disentanglement of these two gradient components. The thinking phase captures both LongCoT reasoning paradigm and problem- sppecifc information, whereas the solution phase only encodes the latter (see Sec. 4. and 4.2 for more explanation). Therefore, by comparing the gradients induced by the full LongCoT response and the solution phase alone, we can isolate the gradient gcon using projection operation and separate gpar via an orthogonalization operation (claborated in Sec, 4.3). Finally, the model is updated using only the paradigm-relevant relevant gradient com-

---

ponent. This ensures the model focuses on internalizing the LongCoT paradigm without memorizing problem-speeific knowledge and heuristics. The overall algorithm is outlined Algorithm 1.
We compare our DL.FT with SFT on the challenging benchmarks 17 diverse domains for both generalist and domain-specific L.MMs. Extensive experiments demonstrate that: (1) our DLoFT effectively mitigates the overfitting problem of SFT and learns stronger generalizable LongCoT reasoning ability. For example, it significantly outperforms SFT by an average of +16.4 points on out-domains and has a notable advantage of +11.5 points on in-domains. (2) As a cold-start stage for RL, our DLoFT can free the cold-start from the dependence on in-domain LongCoTT data, and serves as stronger foundation for subsequent RL, amplifying the outcome of RL.
2
Related Work
2.1 LLM Reasoning from CoT to LongCoT
Reasoning is a core capability of large language models (LLMs), reflecting their ability to address complex real-world problems. Since 2022, Chain-of- Chain-of-Thought (CoT) [14] has enhanced the ability of LLM on reasoning tasks (like mathematics [15] and basic programming [16]) by encouraging them to generate intermediate steps before deriving a final answer. While LLM with CoT reasoning paradigm still struggles with highly complex reasoning tasks like competition-leve math [4], coding competitions [6], and PhD-level science QA [5]. For example, GPT-4o [17] only achieves 13.4%
accuracy on the well-known math competition AIME24 [4]. The release of OpenAl's ol model [1] broke this status and made great progress in these tasks. The underlying reason for ol's success is the shift in the reasoning paradigm from a short-formed CoT to a more comprehensive Long Chain-of- of-Thhught (LongCoT). which scales up the inference computation of LLMs by engaging in deeper thinking phase before delivering the final answer. Unlike a deterministic solution, this thinking process allows the model to freely and thoroughly explore a given problem, including several typical characteristics: (I) exploring different ideas, (2) backtracking to prior steps if the current step hopeless, (3) reflecting the correctness of previous steps, and (4) correcting errors.
2.2 Techniques for Enabling LongCoT Reasoning
Following the breakthrough of ol [1], the community has attempted to cultivate the LongCoT reasoning abilities for LLMs. These efforts can be categorized into four groups:
Inducing Reflection via Inference- Intervention. The first line of work encourages the model to engage in self-reflection through an inference-time intervention mechanism. For example, [7] demonstrated that simply inserting the token "Wait" into the model's current reasoning trace can effectively prompt the model to reflect on its current step. [18] further revealed that this self-reflection behavior already acquired during the pre-training phase, which offers a plausible explanation for the effectiveness of inference-time intervention techniques. However, inference-ting time intervention less practical because it relies on manual intervention to trigger reflective behavior rather than spontaneously performing self-reflection and correction when necessary (like ol [1]).
Imitate High-Quality LongCoT Responses via Supervised Fine-Tuning (SFT). Inspired by knowledge distillation [19, 20), the most straightforward way to enable LongCoT abilities is fine- tuning the LLM to imitate the high-quality LongCoTT reasoning traces, which can be curated in two ways: one is to collect the responses from existing models with strong LongCoTT abilities (e.g., DeepSeek-RI [10] and QwQ [21, 22], etc.), the other is to manually synthesize the tree search sampling trajectories of CoT model into the LongCoT form [23, 24]. Existing studies narrowly focus on few topics for both training and evaluation, including competition-level mathematics [4], code generation [6]. and PhD-level science question answering [5]. In contrast, our experiments reveal that SFT on LongCoT data with narrow topies leads to poor generalization on diverse novel topics, which is caused by the fact that the model is prone to overfitting the groblem-specine knowledge and skills involved in the training data. To address this issue, we propose Decoupled Fine- Fine-Tuning (DLoFT) algorithm for learning generalizable LongCoTT reasoning, which decouples the gradient exclusively corresponding to general LongCoT reasoning paradigm and the gradient caused by problem-specific information.
3

---

LongCoT Emerges through Zero Reinforcement Learning (Zero-RL), DeepSeck-RI-Zero [10] first found that LongCoT reasoning abilities can naturally emerge through reinforcement learning (RL) with rule-based rewards for base L.M,, referred to as Zero-RI. training. In particular, they experimented on 671B model [25] and demonstrated that the model naturally learned to solve reasoning tasks with more thinking time (increased average response length) and to reflect using an anthropomorphic tone (called "aha moment"). Subsequent works [26-28] have attempted to reproduce its performance and emergence phenomenon by demystifying its proprietary training data and settings.
Reinforcement Learning (RL) with LongCoT Cold-start. Despite the success of DeepSeek- -1- [e]o[10], recent studies [29] have found that Rero-RL does not always stably emerge LongCoT reasoning paradigm, particularly in terms of extended reasoning length and complexity. A SFT cold-start phase on LongCoT-style data essential first instill the LongCoTT reasoning paradigm in the model before applying RL. for further enhancement. This cold-start phase is proven to make further RL improvement easier [10, 29]. Under this "RL with Cold-start" training scheme, our DLoFT algorithm can effectively replace the SFT in the cold-start phase. Experiments show that RL with our DLoFT cold-start achieves better generalization and superior RL gains compared to SFT counterpart.
3 Overfitting Problem Investigation
observe that supervised fine-tumine (SFT) LongCoT data involving few specific domains leads to notable performance drops on 11 out-domain benchmarks, with degradations ranging from -1.9 to -1.0 points. This clearly highlights the poor generalization behavior of standard LongCoT Supervised Fine- TuninTn (SFT).
From Figure 1,
To further investigate the underlying reason for this out-of-distribution generalization problem, we monitor two key metries throughout the training process on both in-domain and out-domain benchmark: (1) the accuracy; (2) the percentage of model's response to the test data that exhibits LongCoT behavior. In particular, we use GPT-4o-mini API to detect whether the model response contains LongCoT behaviors such reflection, error correction, and exploration of new idcas.
As shown in Figure 1, although the accuracy on the in-domain benchmark steadily improves during training, the accuracy on the out-domain benchmark initially increases slightly (from 55.3 to 57.0) and then begins to consistently decline after 250 step, eventually dropping to 46.9. This trend suggests that the out-of-distribution performance degradation is due to the model overfitting to problem-specife knowledge and heuristics present in the training data.
In addition, common overfitting mitigation strategy, such as early stopping (stopping training before out-domain performance degradation), has been shown be ineffective. Although simply applying early stopping can prevent performance degradation on out-domain benchmark. it leads to both the limited performance gains and insufficient learning on the LongCoT reasoning paradigm. As shown in Figure (()), if training is stopped early at step 250 (indicated by the yellow dashed line), we forfeit approximately 7 points on the accuracy of in-domain benchmark. Simultaneously. as Figure ((b), only 3.3% of the in-domain test examples exhibit LongCoTT behavior at the early stopping moment, indicating that the model has not yet sufficiently internalized the LongCoT reasoning paradigm.
These observations underscore a key challenge: naive supervised fine-tuning (SFT) is prone to overfit to moollem-specine knowledge and heuristics present in the training data, rather than internalizing a generalizable LongCoT reasoning paradigm. In contrast, our DLoFT not only ensures steady increase in in-domain and out-domain performance (as Figure (a)), but also demonstrates faster learning of the LongCoT paradigm (see Figure 1(b)).
4 Method
In this section, we introduce our Decoupled LongCoT Fine-Tuning (DLoFT) algorithm, which enhances the generalizability of learned LongCoT reasoning abilities while preventing overfitting to specific problems in training data. First, we review the characteristics of LongCoT data in Sec. 1, providing the necessary background for understanding the motivation and algorithm. Then, in Sec. 4.2, we discuss the challenges in LongCoT fine-tuning, and highlight our motivation for leveraging the unique structure of LongCoT data. Finally, we elaborate our algorithm in Sec.

---

4.1 Preliminary: LongCoTT Reasoning Format
The LongCoT-style response is structured into two phases: (1) Phase-1: a systematic thinking phase that includes analyzing problems, summarizing findings, exploring new ideas, reflecting on conrectness, and correcting errors. (2) Phase-2: a solution phase that presents deterministic step-by- step solution derived from the successful attempts during the thinking phase. This two-phase structure in LongCoT enables more thorough, exploratory and reflective reasoning process, in contrast the conventional CoT which typically follows a single-phase structure focused solely on step-by- by-sepp reasoning, Figure 5 in Appendix provides example of LLM's response under LongCoT Reasoning.
4.2 Motivation
LongCoT fine-tuning aims to enable the model to adopt LongCoT reasoning paradigm on problem solving: first engaging deliberate thinking to make a deep and structured exploration of the problem, and then synthesizing a deterministic solution based on the successful reasoning traces.
The mainstream training objective is to directly imitate high-quality LongCoT responses thousands of representative reasoning problems [7, 8]. This optimization objective inherently couples two types of learning signals: (1) reasoning paradigm, i.e., how structure thoughts, analyze problem, reflect on previous steps, correct errors, assess feasibility, backtrack to prior step, and brainstorm alternative ideas; (2) reasoning content, i.e., the relevant knowledge and skills required for solving the problem. Typically, models require more training iterations to internalize strong and stable LongCoT abilities due to the complexity of the LongCoT reasoning process. However, this increases the risk of overfitting and skills covered in the training data. As a result, the model learns reason effectively within seen topics, but struggles to generalize to unseen ones.
not to the reasoning paradigm itself, but to the -abecff-ppccfic knowledge
To address this issue, we propose encouraging the model to focus on learning the LongCoTI reasoning paradigm, rather than memorizing problem- specific knowledge or heuristics. Concretely, we decouple the gradients into two orthogonal components: one corresponding problem-specific content and the other capturing exclusive LongCoTT reasoning paradigm. Only the latter gradient component is adopted update model parameters.
key insight of our work is that the unique two-phase structure of LongCoT responses naturally facilitates the decoupling of puradigm-relevant and content-relevant gradients. As described in 4.c, the thinking phase encodes both the reasoning paradigm and problem-specific content, while the solution phase reflects only the latter. By comparing the gradients induced by the full LongCoT response and the solution part alone, we can isolate the gradient component that exclusively corresponds to the LongCoT reasoning paradigm. The algorithm details are elaborated in the following sub-section.
4.3 Decoupled LongCoT Fine-Tuning
This section provides a detailed introduction to our training methodology designed to enhance the long reasoning abilities of large language models, which we name Gradient- Gradient-Decouplee Decoupled Our core idea is a gradient decoupling mechanism that explicitly separates the supervisory signals related to long reasoning from those that are purely problem-specific. This approach fundamentally differs from conventional Supervised Fine-Tuning (SFT), where the model's learning objective is typically replicate memorize the provided annotated long reasoning process and the final answer. The overall procedure is outlined in Algorithm 1. The core of the entire optimization algorithm consists of the following three sequential steps, used for gradient decoupling and parameter updating.
Step 1: Compute the Full Response Gradient
For each training example in the problem, Ti signifies the exploratory thinking process, and S, indicates the deterministic solution, we input the question P; into the model O. We then compute the negative log-likelihood (NLL) loss over the complete Long Chain-of- Thought (LongCoT) response, represented as T, @ S,, where () denotes the concatenation of the two text components. Formally, the gradient gtall of the averaged
represented as a tuple (P,, St)., St). where P. denotes the
5

---

Algorithm 1: Decoupled LongCoT Fine-Tuning (DLoFT)
Input: Training datasct D, initial model parameters 0, learning rate 1), maximum iterations T, batch size B
for = to do
Sample mini-batch {(P,, (.)) from D, where
Pa the i-th problem
T: the exploratory thinking process in LongCoT response for P:
S: the deterministic solution in LongCoT response for P.
Step Compute the Full Response (Ti S.) Gradient
If (:: concatenation If NLL loss
Crn >>
Lk-1
17;651
log pe ((2 e S.)* (T. @ S.)=*,a P.):
Slull = Cialls
I/ compute gradient
Step 2: Gradient Decoupling
Compute the reference gradient (w.zf. problem-speeific information):
Let -i=1 E log po (St [,5*, PP)::
// NLL loss with S; target compute gradient
gier
VaLier
Decouple the content-relevant gradient (wazt. problem-specific information) from
Son
[Rail.Red)
gir:
// projection
Decouple the paradigm- relevant gradient (w.rt. LongCoT reasoning paradigm) from giill orthogonalization
Spz Biall
gcomi
Step 3: Update Model Parameters
- 0 1) gpuri
If use decoupled paradigm-rolevant gradient to update model
return 0:
loss function Lrall for this mini-batch is defined as:
IT:88:
Biill VoLran where Liall
1
Σ
log po (T: a S:) (T. ( ( (I)
B
k=1
where the gradient contains two intertwined types of supervisory information. First, there is problem- specific content, which includes the facts, formulas, or procedures needed for particular problem. Focusing too much on this during training can cause the model to memorize details, limiting its ability develop generalizable reasoning skills. Second, there is information aimed activating the model's LongCoT abilities. This involves guiding the model generate and follow logical steps to reach a solution, rather than just memorizing input-output pairs. This approach is crucial for enabling the model generalize beyond specific training examples and apply learned reasoning patterns to new and unseen problems.
Step 2: Gradient Decoupling via Projection and Orthogonalization
In this step, decouple the full response gradient grall defined by Eq.(1) into the paradigm-relevant gradient and the content-relde relevant gradient. The former contains supervisory information for long reasoning abilities that is beneficial for generalization, while the latter contains problem-specific supervisory information.
Compute the reference gradient. To isolate the content-specific gradient from giull we first compute reference gradient that captures only the problem-speciff content. For each training example (P,, T,, S:),, we evaluate the negative log-likelihood (NLL) loss solely over the solution component S,, defined as follows:
S1
1
log pe ( (5 ...)
gref VeLref where Lref
B
This gradient, Seet, signifies how the model would adjust its parameters if it were trained to directly produce the final deterministic solution S., without undergoing an extensive exploratory reasoning phase before generating the answer. Consequently, gie serves as reference gradient for problem- specific reasoning content, as the NI.. loss target S, encompasses only the knowledge and heuristics pertinent to the problem, devoid of any LongCoT reasoning framework.

---

Decouple the content-relevant gradient via projection. Using the reference gradient giet. which focuses on problem-specte reasoning, we can extract the content-relevant component from the full gradient giull through straightforward projection operation. The projection is defined as follows:
(guill+ gre()
Econ
2 gref
Bref
where (:,-) denotes the inner product and ||. represents the & norm. Conceptually, gee defines the subspace that captures the influence of problem-specific content model updates. By projecting giull onto gref, we isolate the component of giull that pertains exclusively to problem-specific reasoning.
This results in goon, which represents the content-relevant gradient component.
Decouple the paradigm-
gradient via orthogonalization. After isolating the content-reles rellvvntt gradient gcon. we extract the paradigm-relevant component by removing the influence of problem- specific reasoning from the full gradient giull This is accomplished through an orthogonalization operation defined as:
Etull
Epar
Spar
gfull
Bcon
(4)
Bref
By subtracting geon from giall we eliminate the component associated with problem-specífic reasoning. resulting in purified gradient Epar that is orthogonal to 8con- This ensures that Epar captures aspects of grull that are independent of mroblem-speciffe knowledge. Consequently, Epar represents the paradigm-releww LongCoT reasoning paradigm. It indicates how the model should adjust its parameters to enhance its LongCoT reasoning abilities, irrespective the specific problems present in the training data. Figure illustrates the relationships between these gradients.
Scon
Figure 2: Relationships be- tween the involved gradients in DLoFT.
gradient, focused solely on the
Step 3: Update the Model with Seaadigm-Relevan Gradient
To avoid the model overfitting to the problem-specine knowledge involved in training data, we drop the content-relevant gradient Scon and update the model using only the paradigm-relevant gradient gpar:
0 0
1j Epary
where v is the learning rate. This gradient decoupling mechanism ensures that the model concentrately internalizes generalizable LongCoT reasoning abilities rather than memorizing problem-specine information in training data.
Experiments
In this section, we first introduce the experimental setups in S.1. Then, in Sec. 5.2, demonstrate the superiority of our DLoFT algorithm over supervised fine-tuning (SFT), both as a standalone supervised learning stage and cold-start stage prior reinforcement learning (RL). Ablation studies on our key designs and further discussions are detailed in Appendix A.
5.1 Experimental Setup
Models. To evaluate the effectiveness and generality of our DLoFT algorithm, we conduct ex- periments on both generalist and domain- -opeiic LLMs with varying sizes. For generalist LLMs, adopt the Qwen2.5- across a variety tasks. For domain-specific LLMs, we consider three major domains: math, code, and medicine. Concretely, Qwen2.5-Coder Innstrruct [31] for code generation, and Meditron- 7B [32] for medical reasoning.
series [11] at 7B and 32B scales due their strong performance
Qwen2.5- Mathh7BBIInstruc [30] for mathematical reasoning,
Evaluation. To investigate the comprehensive generalization, we conduct evaluation a variety of benchmarks across 20 domains. For major domains with many well-known benchmarks, we test on the most commonly-used and challenging ones, such as the competition-level AIME24 [4], MATH-500 [33] and OlympiadBench [34] for mathematics, LiveCodeBench (v2) [6] for code generation, and MedQA [13] for medical reasoning. For other less-studied domains with few
7

---

well-established benchmarks, we directly use the corresponding subset from SuperGPQA [35], benchmark focused on graduate-level reasoning problems, as our test set. These domains cover STEM disciplines, social sciences, and humanities, including physics, chemistry, biology, computer, mechanical, electronic, communication, astronomy, geography, civil, agriculture, economics, history, law, literature, philosophy, and sociology.
Training Datasets. Our experiments utilize two types of LongCoT datasets that differ in coverage: Dataset. We adopt the well-known s1K [7] dataset due to its good trade-off between effectiveness and efficiency. It contains one thousand high-quality data covering mathe- matics, programming, science, and puzzles, and is carefully curated based on quality, difficulty, and diversity simultaneously. Each data has a LongCoTT response generated by Google Gemini Flash Thinking model [36].
Single-Domaie Dataset. We use open-source datasets that focus on three major domains: math, code, and medicine. Specifically, we take the -Mathh2222 dataset [12] for mathematics, the programming-red
subset from OpenThoughts-1 114K dataset [37] for code, and the Medical- ol dataset [38] for medicine. Prior studies [39. 7] have revealed that learning LongCoT reasoning capabilities does not require hundreds of thousands of data, because it focuses on changing the reasoning paradigm rather than memorizing a large amount of knowledge. Therefore, we randomly sample 5K data from each of the above three datasets for our training, denoted as OpenR1-Mathen
TpennThoughts-to hhoughts-Code-5K, and Medical-ol -5K, respectively.
Training Settings. For fair comparison, we use the same training settings for SFT and our DLoFT. The models are trained for 10 epochs using the AdamW optimizer with weight decay of zero. The learning rate is first increased from 0 to le-5 with a warm-up ratio of 0.03, and then decreased following a cosine decay schedule. We set the batch size as 16 when training on s s1K [7] dataset, and 64 for other training datasets. During the RL training stage, we use the recent popular GRPO algorithm [40] with a KL penalty coefficient of 0.001. The batch size is set to 128, and the learning rate is set 1e-6 keeping constant throughout the training. We train the model for epochs, because it is found that the reward curve converges after about 3 epochs.
5.2 Main Results
5.2.1
DLoFT Generalizes Better than SFT in all Evaluation Settings
Results on Generalist LLMs. Figure: 3 compares our DLoFT with SFT on two generalist LLMs (Qwen2.5-7B/32B- -nsttttttt We evaluate across 17 diverse domains and report the relative accuracy compared to the baseline model to intuitively illustrate the gain and degradation in performance after training. It can be observed that SFT suffers from clear overfitting training topics. leading to notable performance drops on out-of-distribution domains. For instance, SFT causes a degradation of -16.0 points on "communication" for 7B model. In stark contrast, our DLoFT consistently improves performance across both in-domain and oft-o-- benchmarks, demonstrating a clear advantage in generalization ability. For example, compared with SFT, our DLoFT exhibits an advantage of + +12.4 and 16.4 points on 7B and 32B models. This shows that our method not only mitigates overfitting but also promotes the transferability of LongCoT reasoning ability across various domains. Notably, DLoFT still outperforms SFT even on in-domain benchmarks, yielding additional +8.9 and +1 +11.5 accuracy gains for 7B and 32B models, respectively. This reveals that the gain brought by SFT is still limited due to overfitting to narrow problem-specific content in the training set with narrow topics, which hinders its capacity to generalize even within the same domain. Averaged over all 17 domains, our DLoFT achieves a substantial improvement of 49.8 for Soven..5-BB-Insiee a degradation of -18.. In summary, these results indicate that our DLoFT enables generalist LLMs to efficiently acquire generalizable LongCoT capabilities using only a small amount of representative data, without the necessity to construct a large-scale dataset that exhaustively covers a wide range of topics. This not only significantly reduces the cost of data collection, curation, and annotation, but also greatly saves the required training time and computational resources.
while SFT leads to
Results on Domain-specific LLMs DLoFT's strong generalization can also be observed in the case of domain-specific LLMs. Figure 4 compares our DLoFT and SFT on three excellent domain-specific LLMs trained with either in-domain or out-domain LongCoT data. Results show that our DL consistently outperforms SFT across all evaluation settings, regardless of whether the training data is in-domain or out-domain. In particular, when models are trained out-domain LongCoT data,
8

---

SFT
DLaFT (ours)
In-domain benchmarks
Out-domain benchmarks
all benchmerks
Accuracy Gain (Relative to Baseline Model)
20
Baseline Medel: Qwen2--78-nnttruu
10
13.
140
12
120
140
-10
-20
Baseline Model (ven2.5-22-Inutreee
30
31.2
21.0
20
10
10
-10
meth
physies
chemistry
biology
code
medicire
computer
electronic
communication
mechanical
astronamyy
Reograal
Chvil
agriculture
economics
history
low!
leeertuue
philosophy
sociology
everage
Figure 3: Comparison between SFT and our DLoFT on two generalist LLMs trained on s dataset [7] and evaluated on 17 domains. The last three columns with green edges show the average gain on these domains for all benchmarks, in-domain benchmarks, and out-domain ones, respectively. To intuitively observe which domains experienced performance degradation after training, we only report the relative performance (pass@) accuracy) compared to the baseline model. It can found aut-of-distribution domains, while our DLoFT boosts the performance of baseline model on both in in-domain and out-domain benchmarks.
that SFT leads to significant performance degradation
SFT hurts the performance by -3.3, -1.6, and -7.5 points for these three baseline models in math, code, and medicine domains, respectively. In contrast, our DLoFT achieves substantial improvements accordingly: +10.0 on AIME24 (math), +1.8 LiveCodeBench (code), and +9.1 on MedQA (medical). Interestingly, our DLoFT shows comparable performance when training on in-domain or out-domain data, which indicates that our DLoFT algorithm effectively isolates the influence related to LongCoT reasoning paradigm from the training data, while being robust to domain-specifie information that may otherwise act as confounding noise. Moreover, even when in-domain LongCoT data is available, our DLoFT still provides moderate improvements over SFT. We attribute this to SFT's tendency to focus on domain- and problem-specific knowledge or skills present in the training data, thereby limiting its capacity to learn general reasoning ability. In contrast, by decoupling domain-specific signals from reasoning ability signals, our DLoFT concentrates on learning general reasoning ability, leading to more robust gains on in-domain benchmarks as well. In summary, these results demonstrate that our DLoFT offers an efficient and practical way for enabling LongCoT reasoning in domain-specifie LLMs. With DLoFT, domain LLM can activate its LongCoT capability by leveraging publicly available LongCoT data from other domains, freed from constructing its own in-domain LongCoT datasets costly. This not only reduces the expense of data annotation and training, but also provides a fundamentally viable solution for domains with privacy constraints (data cannot be annotated through public APIs) or domains whose data is intrinsically difficult to collect.
5.2.2 DLoFT is a More Effective LongCoT Cold-Start for Reinforcement Learning (RL)
We highlight two key benefits when using our DLoFT as cold-start training algorithm before RL:
Relax the Dependence on In-domain Data in Cold-Start. It can observed from Figure 4 that SFT cold-start relies heavily on in-domain LongCoT data. The performance degrades consistently when performing SFT cold-start on out-domain LongCoT data. RL may not always able to completely recover the performance loss caused by the SFT cold-start, so that the final performance after RL. may worse than the baseline. For example, as Figure 4c, SFT+RL still falls behind the baseline model by 4.9 points. In contrast, DLoFT frees the cold-start from the indispensability for in-domain

---

SFF
Ouaff Darrinn
Boseiin 136.7
Ozuelina 03.50
DueF
Bood 147.3
SPT ML
SFT+R
3TT-AL
BurTinL
Sudine Mal Que225 Mark 78 iurract
Accuracy on LiveCadellenee (%u
Burdhe Model Quea25 Cader Instratt
65
Budis Nakt Mediree 73
Accuracy on AIME24/9)
-2
38.3
Accuracy on MedOA (%6
60
62.5
-0
33.3
33.3
36.4
58.9
30.0
33.7
13
20-
16.7
26.7
35.3
35.5
45
57.0
13.3
Thoughts Code-5
Opanñ1 Math SK
OpenR1-Matth SK
OpenThoughtte Satgt--CCdd-eeee (ih-domain)
OperRt: Math 5K
Medical 01. Iin-domaini
loub: domain)
(nn domain)
louk donaidm
(out-domain)
LongCoT Trein Deteset
LongCoT Train Dotaset
LongCoT Train Detaset
(a)
(b)
(c)
Figure 4: Compare SFT with our DLoFT on three domain-specift specific LLMs. Models are trained on out-domain and in-domain LongCoTT data, respectively. In contrast to the degraded performance after SFT on out-domain LongCoTT data, our DLoFT shows much better generalization and further promotes the subsequent reinforcement learning (RL).
LongCoT data. Results show that our DLoFT cold-start, using publicly available LongCoT data from other domains, can achieve performance comparable to that obtained with an equal amount of in-domain data. Such strong generalization and robustness bring great convenience for the data collection and curation in the cold-start stage.
Amplify the Outcome of RL. Even with access in-domain LongCoT cold-start data, DLoFT offers stronger foundation for subsequent RL, boosting RL to achieve higher gains. For example, as Figure 4c, RL after SFT cold-start brings +3.7 gain, while RL after DLoFT cold-start achieves larger +5.0 improvement, One reasonable explanation is that SFT cold-start, even when performed on in-domain data, may still introduce slight overfitting to domain-specifie information, which provides suboptimal foundation for the subsequent RL stage and increases the burden for further improvement.
6 Conclusion
We introduced Decoupled LongCoTT Fine- Tine-Tuning (DLoFT), an algorithm designed to endow LLMs with generalizable LongCoT reasoning abilities while mitigating overfitting to problem-speeific information. By decoupling gradients into paradigm-relevant and content-relevant components using the intrinsic two-phase structure of LongCoT responses, DLoFT updates the model solely with the reasoning-relevant signal. Our approach preserves in-distribution performance and yields substantial gains in nut-of-distribution generalization, offering robust and efficient path toward scalable and efficient LongCoT activation across diverse tasks and domains. We compare our DLoFT with SFT on challenging benchmarks 17 diverse domains for both generalist and domain-specific LLMs. Extensive experiments demonstrate that: (1) our DLoFT effectively avoids the overfitting problem in SFT and learns stronger generalizable LongCoT reasoning ability.
(2) a cold-start stage prior to RI, our DLoFT can free the cold-start from the dependence on in-domain LongCo'T data, and serves as a stronger foundation for subsequent RL, amplifying the gain of RL.
Acknowledgements
This work supported in part by the Research Grants Council under the Areas of Excellence scheme grant AoF/E--01/22-R, Hong Kong Research Grant Council Early Career Scheme (Grant No. 27209621). General Research Fund Scheme (Grant No. 17202422, 17212923, 17215025), Theme-based Research (Grant No. T45-701/22-R) and Shenzhen Science and Technology Innovation Commission (SGDX202205001 1405040), Part of the described research work is conducted in the JC STEM Lab of Robotics for Soft Materials funded by The Hong Kong Jockey Club Charities Trust.
10

---

References
[1] OpenAL. Learning reason with Ilms, 2024.
[2] OpenAL. Introducing openai o3 and o4-mini, 2025.
[3] OpenAL. Openai 03-mini, 2025.
[4] AIME, American invitational mathematics examination, 2024.
[5] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang. Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-prodd q&a benchmark. In First Conference Language Modeling, 2024.
[6] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. Livecodehench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07997 2024.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arviv preprint arXiv:2501 2025.
[8] Yingqian Min, Zhipeng Chen, Jinhao Jiang, Jie Chen, Jia Deng, Yiwen Hu, Yiru Tang, Jiapeng Wang. Xiaoxue Cheng. Huatong Song, et al. Imitate, explore, and self-improve A reproduction report on slow wow-thinking reasoning systems, arXiv preprint uxxv:2412.094133 2024.
[9] Bespoke Labs. Bespoke-stratos: The unreasonable effectiveness of reasoning distillation, 2025.
[10] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang. Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in ilms via reinforcement lcarning. arXiv preprint wrxiv2551111944 2025.
[11] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, et Qwen2. technical report. arXiv preprint arXiv:2412.15115, 2024.
[12] Open-R1 Team. Openr1-math-2221 2025.
[13] Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14);6421, 2021.
[14] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Lc, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2453 24837, 2022.
[15] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek. Jacob Hilton. Reichiro Nakano, et al. Training verifiers solve math word problems. arXiv preprint aiv: 10. 14168, 2021.
[16] Mark Chen, Jemry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.00000 133,0033170 2021.
[17] OpenAl. Gpt-4o, 2024.
[18] Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, et al. Rethinking reflection in pre- pre-trainigg arXiv preprint arXiv:2504.04022 2025.
[19] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint Xiv: 503.02531, 2015.
[20] Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois. Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An esstuuttionfoolowiin Ilaaa model. https://github http//attaaaaaaaaaaaeeaaeeeaeaee __alaca,, 2023.
[21] Qwen Team. Qwq: Reflect deeply on the boundaries of the unknown, 2024.
[22] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning. 2025.

---

[23] Yu Zhao. Huifeng Yin, Bo Zeng. Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-ol: Towards open reasoning models for open-ended solutions. arXiv preprint arXiv:441.11400, 2024.
[24] Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. O1 replication journey: A strategic progress report-part 1. arXiv prepriat arXiv:2410. 18982, 2024.
[25] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, al. Deepseck-v3 technical report. arXiv preprint arXiv:2412.99431 2024.
[26] Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang. Xiangyu Zhang, and Heung. Yeung Shum. Open- reasoner-zero: An open source approach scaling up reinforcement learning on the base model. arXiv preprint wwxXiv25033242900 2025.
[27] Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503. 18892, 2025.
[28] Zichen Liu, Changyu Chen, Wenjun Li. Penghui Qi, Tianyu Pang, Chao Du Wee Sun Lee, and Min Lin. Understanding rl -erzeoo-like training: A critical perspective. arXiv preprint arXiv:2503. 207833 2025.
[29] Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of- thought reasoning in Ilms. arXiv preprint arXiv:2502.03373, 2025.
[30] An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement.
arxiv preprint 2tti:::200.1122222 2024.
[31]
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. S-coder technical report. arXiv preprint arXivv2409. 12186, 2024.
[32] Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi, Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, ot al. Mcditron- Scaling medical pretraining for large language models. arXiv preprint arXiv:231 1. 16009,, 2023.
[33] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song. and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103 03874, 2021.
[34] Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang. Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad- level bilingual multimodal scientific problems. arXiv preprint arXiv:2402. 14008, 2024.
[35] Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, et al. Supergpqa: Scaling Ilm evaluation across 285 graduate disciplines. arXiv preprint arXiv:2502.147777 2025.
[36] Google DeepMind. Gemini 2.5 flash, 2025.
[37] OpenThoughts Team. Open Thoughts. https:/open-thouuhts... ai, January 2025.
[38] Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-ol, towards medical complex reasoning with 1lms, 2024.
[39] Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning. arXiv preprint arXiv:2502.03387 2025.
[40] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint 2433:2200200330000 2024.
[41] Sitong Wu, Haoru Tan, Zhuotao Tian, Yukang Chen, Xiaojuan Qi, and Jiaya Jia. Saco loss: Sample-wise affinity consistency for vasion-language pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 27358-27361 2024.
[42] Junnan Li. Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-ing imnagg pre- training for unified language understanding and generation. In International Conference on Machine Learning, pages 12888-12900. PMLR, 2022.
12

---

[43] Jingyao Li, Senqiao Yang, Sitong Wu, Han Shi, Chuanyang Zheng. Hong Xu, and Jiaya Jia. Logits-based finetuning. arXiv preprint 1xXiv2250,224400 2025.
[44] Zhisheng Zhong, Chengyao Wang, Yuqi Liu, Senqiao Yang, Longxiang Tang. Yuechen Zhang. Jingyao Li, Tianyuan Qu, Yanwei Li, Yukang Chen, al. Lyra: An efficient and speech-centric framework for omni-cognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2025.
[45] Haoru Tan, Sitong Wu, Xiuzhe Wu, Wang Wang, Bo Zhao, Zeke Xie, Gui-Song Xia, and Xiaojuan Qi. Understanding data influence with differential approximation. arXiv preprint arXiv:2508. 14648, 2025.
[46] Xiaowei Hu, Min Shi, Weiyun Wang. Sitong Wu, Linjie Xing. Wenhai Wang, Xizhou Zhu, Lewei Lu, Jie Zhou, Xiaogang Wang, et al. Demystify transformers & convolutions in modern image deep networks. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[47] Sitong Wu, Tianyi Wu, Haoru Tan, and Guodong Guo. Palc transformer: A general vision tramsformer backbone with pale-shaped attention. In Proceedings of the AAAI conference artificial intelligence. volume 36, pages 2731-2739 2022.
[48] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 11999, 2020.
[49] Ashish Vaswani, Noam Shazeer. Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, Advances in neural information processing systems, 30, 2017.
[50] Haoru Tan, Sitong Wu, and Jimin Pi. Semantic diffusion network for semantic segmentation. Advances Neural Information Processing Systems, 35:8702-87 8716, 2022.
[51] Fangjian Lin, Zhanhao Liang, Sitong Wu, Junjun He, Kai Chen, and Shengwei Tian. Structtoken: Rethinking semantic segmentation with structural prior. IEEE Transactions on circuits and systems for video technology, 33(10):5655-566, 2023.
[52] Sitong Wu, Tianyi Wu, Fangjian Lin, Shengwei Tian, and Guodong Guo. Fully transformer networks for semantic image segmentation. arXiv preprint arXiv:2106.04108 2021.
[53] Shan Zhang, Tianyi Wu, Sitong Wu, and Guodong Guo. Catrans: Context and affinity transformer for few-shot segmentation, 2022.
[54] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[55] Haoru Tan, Sitong WU, Wei Huang, Shizhen Zhao, and XIAOJUAN QI. maximization. In The Thirteenth International Conference on Learning Representations, 2025.
Data pruning by information
[56] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770 778-778,777777 2016.
[57] Jingyao Li, Pengguang Chen, Bin Xia, Hong Xu, and Jiaya Jia. Motcoder: Elevating large language models with modular of thought for challenging programming tasks. arXiv preprint arXiv:2312 2023.
[58] Fangjian Lin, Tianyi Wu, Sitong Wu, Shengwei Tian, and Guodong Guo. Feature selective transformer for semantic image segmentation. arXiv preprint arXiv:2203. 14124, 2022.
[59] Jingyao Li, Pengguang Chen, Sitong Wu, Chuanyang Zheng, Hong Xu, and Jiaya Jia, Rohocoder: Robotic learning from basic skills to general tasks with large language models. arXiv preprint arXiv:2406.037557 2024.
[60] Qiang Zhou, Chaohui Yu, Shaofeng Zhang, Sitong Wu, Zhibing Wang, and Fan Wang. Regionblip: A unified multi- modal pre-training framework for holistic and regional comprehension. arXiv preprint arXiv:2308.0223 2023.
[61] Xichen Zhang, Sitong Wu, Yinghao Zhu, Haoru Tan, Shaozuo Yu, Ziyi He, and Jiaya Jia. Scaf- Scaffolded group relative policy optimization for enhancing I1m reasoning, 2025.
13

---

[62] Xichen Zhang, Sitong Wu, Haoru Tan, Shaozuo Yu, Yinghao Zhu, Ziyi He, and Jiaya Jia. Smartswitch: Advancing IIm reasoning by overcoming underthinking via promoting deeper thought exploration, 2025.
[63] Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang. Fan Wang, and Xiaojuan Qi. Data pruning via moving-one-sampae sample-ou.. Advances in neural information processing systems,
2023.
[64] Shaofeng Zhang, Qiang Zhou, Sitong Wu, Haoru Tan, Zhibin Wang, Jinfa Huang, and Junchi Yan. Cr2pq: Continuous relative rotary positional query for dense visual representation learning. In The Thirteenth International Conference on Learning Representations.
[65] Sitong Wu, Haoru Tan, Yukang Chen, Shaofeng Zhang, Jingyao Li, Bei Yu, Xiaojuan Qi. and Jiaya Jia. Mixture-of-scores: Robust image-texi data valuation via three lines of code. In Proceedings of the JEEE/CVF International Conference on Computer Vision, pages 24603-24614 2025.
[66] Fangjian Lin, Sitong Wu, Yizhe Ma, and Shengwei Tian. Full-scale selective transformer for semantic segmentation. In Proceedings of the Asian Conference on Computer Vision, pages
2022.
[67] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000- 16000,16009, 2022.
[68] Jingyao Li, Han Shi, Xin Jiang. Zhenguo Li, Hong Xu, and Jiaya Jia. Quickllama: Query-aware inference acceleration for large language models. arXiv preprint arXiv:4406.07526 2024.
[69] Haoru Tan, Chuang Wang, Sitong Wu, Xu-Yao Zhang, Fei Yin, and Cheng-Lin Liu. Ensemble quadratic assigmment network for graph matching. International Journal of Computer Vision, 132(9):3633-366 2024.
[70] Yizhe Ma. Fangjian Lin, Sitong Wu, Shengwei Tian, and Long Yu. Prseg: A lightweight patch rotate mlp decoder for semantic segmentation. IEEE Transactions Circuits and Systems for Video Technology. 33(11):6860- 00010667711820 2023.
[71] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint ar Xxiv:2304.00199, 2023.
[72] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2:
Bootstrapping language image pre- traininig with frozen image encoders and large language models. arXiv preprint arXiv:2301 12597, 2023.
[73] Fangjian Lin, Yizhe Ma. Sitong Wu, Long Yu, and Shengwei Tian. Axwin transformer: A context-aware vision transformer backbone with axial windows. arXiv preprint arXiv:2305.0122 08000 2023.
[74] Jingyao Li, Hao Sun, Zile Qiao, Yong Jiang. Pengjun Xie, Fei Huang. Hong Xu, and Jiaya Jia. Dynam- iebench: Evaluating real-time report generation in large language models. arXiv preprint arXiv:2506.21343 2025.
[75] Hao-Ru Tan, Chuang Wang. Si- Ti-Tong Wu, Tie- Hee-Qaag Wang, Xu- Yao Zhang, and Cheng -in Liu. Proxy graph matching with proximal matching networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 9808 9815, 2021.
[76] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint Xi..:2225000066 2022.
[77] Hyung Won Chung. Le Hou, Shayne Longpre. Barret Zoph. Yi Tay, William Fedus, Yunxuan Li. Xuezhi Wang, Mostafa Dehghami, Siddhartha Brahma, al. Scaling -iinttudd- language models. Journal of Machine Learning Research, 25(70):1-53, 2024.
[78] Fangjian Lin, Jianlong Yuan, Sitong Wu, Fan Wang, and Zhibin Wang. Uninext: Exploring a unified archi- tecture for vision recognition. In Proceedings of the 31st ACM International Conference on Multimedia, pages 3200-3208, 2023.
[79] Shizhen Zhao, Jiahui Liu, Xin Wen, Haoru Tan, and Xiaojuan Qi. Equipping vision foundation model with mixture of experts for distr---ittiittionn detection, 2025.
[80] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint 23x::2307700228. 2023.
14

---

A Ablation and Further Study
A.1 Effeet of Projection Operation in Eq. (3)
As mentioned in Sec. 4.3 in the main paper, DLoFT first derive reference gradient gier that captures only the problem-specific information, and then use a projection operation to isolate the component of the full gradient giull that aligns with gref, resulting in the content-relevant gradient goon (as defined in Eq. (3) in the main paper). Here, we ablate the effect of this projection operation by (Sui .Bd) |l 12 'getf
comparing the following two settings: (I) using the projection as Eq. (3), that is, gcon
(2) skipping the projection and directly assigning geon gref- As shown in Table 1, omitting the projection still achieves better performance than standard supervised fine-tuning (SFT) and partially mitigates the out-of-domain performance drop. However, the variant without projection consistently underperforms the full DLoFT algorithm with the projection operation. This result indicates that the projection step plays critical role in more precisely isolating problem-specific gradients, and its removal leads to incomplete decoupling, thereby reducing the effectiveness of the filtering problem- specific knowledge and heuristics.
Table I: Ablation on the effect of projection operation in Eq. (3) in the main paper. The first line corresponds to the model before training. We train the Qwen2.5-7B-In [1 [11] model on s1K [7] dataset and report the pass @ accuracy average on four in-domain benchmarks and 16 out-domain benchmarks, respectively.
In-domain Acc@1 (avg)
Out-domain Ace@ (avg)
Method
Projection
21.5
31.2
SFT
27.8
27.3
X
30.3
31.8
DL.FT
36.7
39.7
A.2 Effeetiveness of Our Gradient Decoupling Mechanism
The gradient decoupling mechanism in our DLoFT aims to decouple the full gradient into two orthogonal components: (1) the paradigm-relevant gradient Epar exclusively capturing the general LongCoT reasoning paradigm; (2) the content-relevant gradient goom corresponding to the problem- specific knowledge and heuristies. In order verify the effectiveness of our decoupling mechanism, we conduct comparative study between updating the model using only the content-relevant gradient goon versus only the paradigm-relevant gradient Spar. The underlying intuition is that if the decoupling mechanism is effective, then goon should not contain any signal related to the LongCoT paradigm. As result, updating the model using only geon should not lead any emergence of LongCoT-style behavior. To test this, we train two separate models: one using Epar and one using gcon. with all other settings held constant. We then measure the percentage of generated responses that exhibit LongCoT behavior on both in-domain and out-of-domain benchmarks. As shown in Table 2, the model updated with Epur demonstrates clear learning of LongCoT behavior and achieves strong generalization, while the model trained with geon fails to produce any LongCoT-style reasoning and performs close to the baseline. This sharp contrast provides strong empirical evidence that our decoupling effectively isolates the paradigm-relevant gradient, confirming both the functional accuracy and practical value of our DL approach.
A.3 Effect of Weight Decay Regularization
natural method for preventing overfitting is the use of weight decay, widely adopted regularization technique in model training. However, we find that weight decay is largely ineffective in preventing the type of overfitting observed in LongCoT supervised fine-tuning. Specifically, as shown in Table 3, although applying weight decay can reduce the model's tendency to memorize problem-specific details, does not improve generalization to out-o--distribution problems. This because weight decay regularizes model parameters globally without distinguishing between gradients that encode
15

---

Table 2: Analysis on the effectiveness of our gradient decoupling mechanism. The first line corresponds to the model before training. We train the Qwen2.5- 1e--5-7B-nntruu [11] model on sIK [7] dataset and evaluate on four in-domain benchmarks and out-domain benchmarks, respectively. The LongCoT Response percentage is calculated on all in-domain and out-domain test data.
Gradient to Update
In-domain Acc@ I (avg)
Out-domain Acc@ I (avg)
LongCoT Response Percentage
Method
Model Parameters
21.5
31.2
0%
SFT
27.8
27.3
94%
DLoFT
Hoom
19.1
20.5
0%
Epur
36.7
39.7
100%
Table 3: Analysis on the effect of weight decay regularization. The first line corresponds to the model before training. We train the Qwen2.5-7B-Instree [11] model on sIK [7] dataset and report the pass@ I accuracy average four in-domain benchmarks and 16 out-domain benchmarks, respectively.
In-domain Ace@ (avg)
Out-domain
Method
Weight Decay
Acc@ (avg)
21.5
31.2
27.8
27.3
SFT
26.2
27.4
36.7
39.7
DLoFT
36.0
39.5
Table 4: Efficiency analysis. The experiments are conducted on Qwen2.5-705 IB-Instruct [11] model
with sIK [7] dataset. We report the average running time of each training step and the average GPU memory cost throughout the training.
Method
Running Time (minutes)
GPU Memory (MB)
SFT
6.2
69986
DLoFT
6.8
70684
general reasoning patterns and those that reflect problem-specific knowledge. As result, the model may still internalize partial problem-spectie information tied the training data. Our findings highlight the need for more targeted training algorithms, such as our proposed DLoFT, which explicitly filters out problem-specific knowledge signals while preserving the generalizable reasoning paradigm.
A.4 Efficiency
Compared to standard SFT, our DLoFT algorithm introduces an additional step in each training iteration, that is, the gradient decoupling step (Step 2 in Algorithm where we compute a reference gradient and decouple the full gradient into the content-rclevant and paradigm-relevant components via projection and orthogonalization operation. To assess the efficiency impact of this additional step, we compare our DLoFT with SFT terms of both training speed and GPU memory cost.
As shown in Table 4, our DLoFT introduces only minor overhead in running time per training step. This is primarily because the projection and orthogonalization operations Step 2 are lightweight vector computations that require negligible additional compute compared to the forward and backward passes of large LLMs. Besides, the computational cost for the reference gradient gir is also much less than that of the full gradient giall in Step 1, because the solution S; is generally much shorter than the thinking process T,. Moreover, GPU memory usage remains comparable to that of SFT. Since DLoFT reuses existing gradients and performs additional operations in-place without requiring the
16

---

storage of large intermediate activations or duplicated models, its memory footprint remains efficient and scalable. summary, DLoFT offers significantly improved generalization and robustmess with minimal computational overhead, making it a practical replacement for SFT in real-world training scenarios.
B LongCoT Reasoning
Long Chain-of-Thouuhh (LongCoT) is an emerging reasoning paradigm that gained traction following the breakthrough of OpenAl [1] model, which demonstrated impressive capabilities in complex reasoning tasks. Inspired by the slow thinking mechanism in humans, LongCoT encourages models to conduct thorough thinking phase before concluding deterministic solution. Figure illustrates an example of the LongCoT reasoning paradigm.
Specifically. LongCoT-style response consists of sequential phases:
A systematic thinking phase, where the model analyzes the problem, summarizes relevant facts, explores alternative ideas, reflects on its reasoning, and corrects potential mistakes.
A deterministic solution phase, where deterministic, step-by-step answer is provided based the reasoning developed in the thinking phase.
C Limitations
The nigh-performance, high-gennerllization optimization algorithm proposed in this paper for Long- CoT supervised finetuning demonstrates excellent performance across various scenarios and tasks. However, we believe that experimental scale validated here still limited. Real-world applications often involve much larger datasets and models (on the order of hundreds of billions). Due to limited computational resources during this work, we were unable to conduct large-scale experiments. the future, plan to secure additional resources to validate performance larger scale.
D Impact Statement
The high-performance fine-tuning algorithm designed in this paper has many positive implications, such as enhancing the inference capabilities of existing models, allowing them better serve wider range of people and industries, particularly with lower data collection costs. However, there are concerns about negative impacts; for instance, this method could potentially be misused by illegal entities to train models for inhumane surveillance and authoritarian control, which legislative bodies worldwide should take seriously.
17

---

Problem
Every morring Aya goes for a $95 kilometer Woometer-longeee walk and stops at coffee shop afterwards. When she walks at constant speed of kilometers per hour, the walk takes her hours, including $i$ minutes spent in the coffee shop. When she walks S5+ kiloreeters per the coffee shop. Suppose Aya walks Ss-\Vrac(1H2)
hour, the walk takes kilometers per hour. Find the number of min utes the walk takes her, induding the št5 minutes spent in the coffee shop.
hours and minutes, including 515 minutes
LongCoT response
Problem Analysis
Explore New Icleas
Summarization
Self-Refiection
Thinking
<thinic
Okay, so need to solve this problam about Aya's walk and coffee shop stop. Let me try to parse the information step by step.... Let note down given data;
1. When she walks at km/h, the total time (walking coffeel is hours, which 240 minutes. The coffee time mminutes, so the
walkdnng time must (240- t| minutes:
when she wallss at [s+ 2) km/h, h, the total time 2 hours and 24 minutes. Let me convert that to minutes as well: hours minutes, plus 24 minutes is minutes total. So again, the coffee time is still minutes, so the walking time here is (144-) minutes.
We need to find the total time when she walks at (s 1/2) km/h, including the minutes, So first, maybe need to find the value of sand and than compute walking First, let's use hours because the speed is in km per hour. The distance is km...
speed, add minutes, and that into minutes....
now we two equations with variables,
can
first.
Let write them égain:
19 60)/s +1 240
(9 60)/(s 144
11.25 s/s We can discard the negative solution because speed can't be negative. So, s=| 2+7 /2= 5/2==22 2.5 km/h Wait,
Alternatively, 96
1080
24/96
sense....
Wait, if 2.5, then 2 4.5 km/h. Let check the walking times. scenarto: at km/h. Time hours 9 2.5 hours, which So: Walk time hours: 9 hours. 3
minutes (since 6.6560-35). 180 minutes. Then add minutes, total 204 minutes, So 204 minutes hours and 24 minutes. Hmm, that seems stra iighfarward.
equations again. Maybe problem expects something else? "Find the number of minutes the walk takes her, including the minutes spent in the coffee shop. ".. at
Wait,
check
problem.
So, when
km/h, Therefore, the answer should be 204 minutes, 204.
km/h,
walking time plus
Wait, but the problem says 'the walk takes her, including the minutes": total time walking time plus coffae But let me check onca again with the second scenario ensure that everything is consistent, When she walks at +2 =4.5 km/h, walking
"walk" here includes coffee shop stop? So the straightforward.
yes, +24-704. So 204
9-4.5=2 hours, statement. So yes, all checks out. Therefore, the answer 204 minutees Therefore, answer (boxed(204). Answer**; (boxed(204) </think
minutes. Adding t=24 gives minutes, which 2h24m.
Solution
Given that Aya walks kilometers and spends minutes in the coffee shop, we need to determine the total time taken when she walks at s (frac[1](2) kilometers per hour. First, set the equations for the given scenarios:
When walking V. V) km/h, the total time is hours (240 minutes): Srrac(99)s) \times 60 = 2405
When walking V km/h, the total time is hours and 24 minutes (144 minutes): SVrrac(9/ss 2) times 1445 Subtracting the second equation from the first to eliminate t:5 \rrac (ti mismes 60](s) Vfrac(9 \times 60)(s 2) 240 144
$ Siss 2) Vrac{1080)(96] 11.25
Solving the quadratic equation 5s*2 25 11.25 05 using the quadratic formul a: $s \frac/-2 \pm 1sert(4 44552)) - Vraa(-2 \pm 7]/2]5
Taking the positive root: $ (frac]5](2) = 2.5 (text( km/h)
Substituting M== 2.5 back into the first equation find 5 Mraa|1922 (times 60 240
limplies \text( minutes) $
When walking Ss (frae[][2] = km/h, the walking time is: (1frac(9)(3) \times 60 3 ttimes 50 1805 minutes Adding the coffee shop time t: 180 24 204 minutes
Thus, the number of minutes the walk takes her, induding the minutes in the coffee shop, V(bonec(204JJ)
Figure 5: Illustration of the Long Chain-of- Chain-of-Thought (LongCoT) style response. The problem is
sampled from AIME24 [4] benchmark, and the response is generated by QwQ-32B model [22].
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please see the abstract and introduction section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see the relevant section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
19

---

Justification:
Please see the supplemental material.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please the experimental section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification: The code and data will be made publicly available upon acceptance through peer review.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips... cc/
oonooicc)))) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / /nips. cc/ wttpccccpppcc//c.c..cc...ccc.lcl pubppcppppggepep wwwpc//gguie/eeeeeee Sute:/ooeeSSieseeeee ssssssssssssiioooo
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please the experimental section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Please see the experimental section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
21

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see the experimental section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the csGuidel ines?
NeurIPS Code of Ethics https: http///eerrpp.eeeee cc/ httppp/////rrppppcpppcbb.lccce.e
Answer: [Yes]
Justification:
We strictly follow the relevant Guidelines.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please check the relevant section.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The training algorithm involved in this article will not cause the originally safe model to become dangerous.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: This paper does not existing assets
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: This article only uses LLM to help check for spelling mistakes in very small quantities.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https /tt//eerrss...e/ /ttp///rps.c///ooeeeeeeeeeee/eee wwtpc//ooeoeenne/22225/55/5/3/30 for what should or should be described.
25


Reviews:
[{"review_id": "hBivmaVyiM", "reviewer": "Reviewer_EP1B", "summary": "**Summary**\n\nThis paper addresses an important problem in training LLMs for long chain-of-thought (LongCoT) reasoning: models trained via supervised fine-tuning (SFT) tend to overfit to problem-specific knowledge rather than learning generalizable reasoning patterns. This paper proposes a technique to  decompose gradients into paradigm-relevant and content-relevant components, updating models using only the former and show that this results in better out of distribution generalization.", "questions": "please see inline in the weaknesses section", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. The paper tackles a timely problem as the community is working towards scaling reasoning models.\n\n2. The gradient decomposition method is creative and leverages structural properties of LongCoT data in an interesting way.\n\n\n**Weaknesses**\n\n1. All main experiments are performed with small subsamples of 5K datapoints. This leads to three direct questions: a. Do the advantages of this approach materialize when larger datasets are used? b. Can this approach result in worse performance when more data is used, due to loss of content specific signal? c. How does the performance of models trained with DLoFT using 5K samples compare to models trained with models trained with full dataset?\n\n2. Figure 1b shows that SFT fails to induce LongCoT in out of distribution queries. At the cutoff point, DLoFT uses LongCoT for almost 80% of the queries and yet achieves exactly the same accuracy. This seems to indicate that even though CoT was being used, it had no performance advantage or actual generalization of reasoning. \n\n3. The results in Figure 2 seem to be incorrect. Looking at Figure 1a, it is clear that SFT on 7b model results in slight increase in OOD queries as well at the early stopping point. Figure 2 shows an 8.4% drop.\n\n4. While the core algorithm seems somewhat reasonable, there is neither any theoretical nor empirical analysis to show how the technique works.\n\n5. Equation 3 shows that dot product of reference and full gradients need to be computed in order to compute the content gradient. How does this work when for non square weight metrics?\n\n6. The method requires computing two forward passes and gradients per batch (full response and solution only), effectively doubling training cost compared to standard SFT. This would also double the memory usage.", "was_revised": true, "final_justification": "I increased the score by +2. The authors addressed an extensive list of my questions. The justification is detailed in my comments exchanged with the authors.", "rebuttal": ""}, {"review_id": "dv8yImKbWY", "reviewer": "Reviewer_Dhab", "summary": "This paper proposes to improve the initiation phase for long chain of thought (longCoT) reasoning, in which the model is fine tuned on some longCoT examples, by separating the gradient of this fine tuning into two components - a \"paradigm\" component and a \"solution\" component - and then fine tuning only from the \"paradigm\" component. This is done by computing the full gradient of the longCoT sample, computing its projection onto a gradient representing the 'solution' only, and then removing that projection from the full gradient (the remaining gradient is assumed to represent the longCoT paradigm: exploration of multiple different approaches, reflection on approaches tried, etc). The gradient representing the solution only is taken by breaking the same longCoT sample into its problem (P), exploratory thinking (T), and solution (S) segments, and backpropagating from the loss of only S as a direct continuation from P, i.e. Loss(S|P) (without T). The approach is demonstrated to have superior performance to initation via direct fine tuning on longCoT, for both in- and out- of distribution (relative to the fine tuning) problem domains, and especially for OOD.", "questions": "the issues i have are raised in the strengths/weaknesses. these are additional comments for improving presentation\n- lines 267-269 feel contradictory with 156-157\n- line 28 should be backed by some citations showing 'community' (academic community?) has been trying\n- line 33 'proprietary' maybe you mean 'ownership'?\n- it should be either 'our method' or 'DLOFT', not 'our DLOFT' (i suggest 'DLOFT')\n- lines 70-71 should reference the relevant tables for these results\n- lines 110-111 i do not see where in your experiments you show that SFT has narrow generalisation specifically due to overfitting, this seems to be an assumption rather than an investigation\n- lines 124-125 - improve phrasing\n- 125-127 clarify - is this what the just-mentioned papers found?\n- 135-138 space waste, similar for 236-240\n- 189-196 example of information that has already been repeated several times at this point\n- 223-227 again repetition of same info", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "strengths\n- I rather like the approach - I find it straightforward, intuitive, and elegant. \n- The results also seem to show strong positive impact from the approach\n- Some nice figures, esp. fig 2 has quite clear take-home message\n\n\nweaknesses\n- The writing/presentation needs improvement (more comments in \"questions\" section): \n\n-- The paper feels very repetitive - it feels like almost nothing is said given the space used, and a lot of space is spent constantly on retelling me that the full loss in fact represents two things, or on other space wasting things (e.g. explaining structure of an upcoming section). It feels almost empty as a result, as once this idea is understood there is not much else to learn from reading. Meanwhile several details or results are deferred to the appendix. I think a rewrite would allow fitting these into the main paper.\n\n-- Figure 1 has no clear discussion, the narrative just throws a reference to it without details, its hard to make much of it in this way. Presumably the same paragraph referencing it (lines 38-47) is discussing what is seen in it, but it is not really tied properly to the figure, its hard to understand how to relate the two. The left subplot seems to show out-of-domain performance is better than in- for both SFT and DLOFT, surely this merits discussion.\n\n-- The caption of fig 1 also doesnt quite align with what is in it - especially the statement that OOD performance starts to degrade continuously after the yellow line, in contrast with what is seen in the fig.\n\n-- (There are also various grammar/English oddities, the paper really needs a proper reread and polish in this regard.)\n\n- Some design choices odd: in particular the choice to sample 5k samples from the single domain datasets, when many more are available and the models are fine tuned for 10 epochs on these samples. This is not properly motivated.", "was_revised": true, "final_justification": "already liked the work, and hoping presentation will improve as suggested", "rebuttal": ""}, {"review_id": "3sJ03p3W7s", "reviewer": "Reviewer_bbCD", "summary": "The paper introduces DLoFT, an algorithm designed to enhance the generalizability of LongCoT reasoning in large language models while mitigating overfitting to problem-specific knowledge. The authors claim that DLoFT significantly improves the generalization behavior of LongCoT abilities compared to SFT while maintaining strong in-distribution performance.", "questions": "Please see Weaknesses. My primary focus lies in points 1 through 7.", "limitations": "yes, some Limitations are discussed in Appendix Limitations.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "> **Strengths:**\n>\n> (1) Good visualizations and typesettings.\n>\n> (2) A well-justified research problem: models trained in this way (LongCoT SFT) tend to overfit problem-specific knowledge and heuristics, leading to degraded out-of-distribution performance. Authors try to concentrate on internalizing the LongCoT paradigm rather than memorizing problem-specific knowledge and heuristics.\n>\n> (3) Well-founded starting point: Gradient decoupling.\n\n> **Weaknesses:**\n> \n> (1) Line 64:  \"the model is updated using only the paradigm-relevant gradient component\". If the foundational knowledge of the base model requires updating, merely processing paradigm-relevant gradients would be insufficient.\n>\n> (2) Lines 183-184: \"For each training example in the mini-batch, represented as a tuple (Pi ,Ti ,Si ) , where Pi denotes the problem, Ti signifies the exploratory thinking process, and Si indicates the deterministic solution\". The paper does not explicitly specify the method for distinguishing between Ti (thinking phase) and Si (solution phase) in the main text - is the \\<think\\> tag used as the delimiter?\n>\n> (3) In your Figure 1 in supplementary material. The example between Si and Ti shown in Appendix Figure 1 indicates these components are not fully decoupled, suggesting the need for more refined differentiation criteria between the solution and thinking phases.\n>\n> (4) Equations 1 and 2 do not provide clarification on the variable $\\mathcal{B}$, although it can be understood to represent the batch size.\n>\n> (5) The linear dependence between $g_{con}$ and $g_{ref}$ (Equation 3) raises the question: What would be the impact of directly substituting $g_{con}$ with $g_{ref}$?\n>\n> (6) Lines 215-216, \"By **projecting $g_{full}$** onto $g_{ref}$, we isolate the component of $g_{full}$ that pertains exclusively to problem-specific reasoning.\"  However, in your Equation 3, you try to use $g_{con}= \\lambda (g_{full},g_{ref})\\times g_{ref}$. **This appears to project $g_{ref}$ rather than $g_{full}$**.\n>\n> (7) Line 244 \"we adopt the Qwen2.5-Instruct series\". [1] shows that it is easy to get significant performance gains on Qwen models even with completely spurious reward signals. The experimental validation conducted solely on the Qwen2.5 model series lacks sufficient persuasiveness.\n>\n> [1] Spurious Rewards: Rethinking Training Signals in RLVR. arXiv:2506.10947.\n>\n> (8) Line 563, the author say \"The code and data will be made publicly available upon acceptance through peer review. \" The extent of reproducibility cannot be fully determined based on the current information.", "was_revised": true, "final_justification": "I appreciate the additional clarifications, and as a result, I have raised my evaluation.", "rebuttal": "Thank you for your detailed and constructive comments and contribution to the academic community! \nYour feedback is of great significance to us in further improving our article. We will address each of your questions, and we hope to receive a higher rating from you if you are satisfied with our responses.\n\n---------------------\n\n### **Q1: On the sufficiency of paradigm-relevant gradients when foundational knowledge may be lacking**\n\nThank you for this insightful question, touched on a crucial distinction regarding DLoFT's application scope.\n\n***“Knowledge injection”*** and ***“learning reasoning paradigms”*** are two complementary rather than mutually exclusive learning objectives.\n***DLoFT is not designed to replace foundational knowledge injection; rather, it is intended to optimize the model’s reasoning paradigm and is thus complementary to knowledge updating techniques.***\n- *If the model lacks essential foundational knowledge*, merely updating it with “reasoning-paradigm-relevant gradients” (as done in DLoFT) is insufficient to fill that knowledge gap. In such cases, continue pretraining or standard supervised fine-tuning would be more appropriate.\n- *If the model already possesses sufficient world knowledge* and the goal is to upgrade its problem-solving paradigm from CoT to a more advanced LongCoT paradigm, then DLoFT is a more effective and efficient choice than standard SFT. It enables the transition through post-finetuning on only thousands of LongCoT-style examples, without the need to re-run the full post-training pipeline. Moreover, experimental results show that DLoFT generalizes well to out-of-domain tasks.\n\nWe understand that this question may stem from a misunderstanding about the goal and application scope of DLoFT, so we would like to take this opportunity to clarify:\n- ***The goal of DLoFT*** is not to inject new knowledge into a model, but rather to optimize how the model reasons with existing knowledge. In particular, DLoFT is designed to refine a model’s reasoning paradigm, i.e., the manner in which it approaches problem solving. For example, in the context of Long Chain-of-Thought (LongCoT) reasoning, DLoFT enables the model to master the LongCoT reasoning paradigm — how to structure thoughts, explore ideas, reflect, and correct errors, when to reflect, when to explore different ideas, and how to iterate through a complex thinking process.\n- ***The application scope of DLoFT*** is to post-finetune an instruct model that already follows a Chain-of-Thought (CoT) reasoning paradigm—such as the Qwen-Instruct or LLaMA-Instruct series—to transform it into a model that adopts a LongCoT reasoning paradigm. These CoT-based instruct models have typically undergone pretraining (providing them with sufficient world knowledge) and supervised fine-tuning or reinforcement learning (teaching them how to apply that knowledge to solve problems, albeit within a CoT paradigm).\n\n---------------------\n\n### **Q2: Clarification on how $T_i$ and $S_i$ are separated**\n\nYes, as a common convention in LongCoT-style datasets, each full response follows a standardized format:\n\n\\<think\\> THINKING PROCESS \\<\\/think\\> SOLUTION\n\nThe special tag \"\\<\\/think\\>\" serves as the delimiter that separates the exploratory thinking process ($T_i$) from the final deterministic solution ($S_i$). \nGiven this structure, one can reliably split any LongCoT response using the \"\\<\\/think\\>\" tag. \nFor example, a Python snippet for parsing $T_i$ and $S_i$ would be:\n```\nT_i, S_i = full_response.split(\"</think>\")    \nT_i = T_i.replace(\"<think>\", \"\").strip(\" \\n\")       \nS_i = S_i.strip(\" \\n\")       \n```\nTo further aid understanding, we have illustrated an example of such a LongCoT-style response in Figure 1 of the Supplementary Material, where the structure and separation of $T_i$ and $S_i$ are clearly shown.\n\n---------------------\n\n### **Q3: Clarification on the semantic and structural decoupling between $T_i$ and $S_i$ in LongCoT response**\n\nWe appreciate the reviewer’s careful inspection of the example shown in Appendix Figure 1. The concern seems to be whether the thinking phase ($T_i$) and solution phase ($S_i$) are fully decoupled — that is, whether the boundary between the two is both structurally well-defined and semantically meaningful.\n\n- ***Structural Decoupling.*** \nIn LongCoT data, the thinking and solution phases follow a canonical structure, with a special delimiter \"\\<\\/think\\>\" that separates the two parts (as demonstrated in the above response for Q2). Importantly, this \"\\<\\/think\\>\" delimiter is now widely adopted in recent deep reasoning models, such as DeepSeek-R1, as a standard practice for separating thinking from final solution. The example shown in Figure 1 of the Supplementary Material adheres to this convention.\n\n- ***Semantic Decoupling.***\nBeyond structural decoupling, LongCoT also promotes a clear functional division:\n    - The thinking phase ($T_i$) involves open-ended reasoning: intermediate steps, scratch work, alternative hypotheses, checks, and so on. It is inherently non-deterministic and may involve trial-and-error or backtracking.\n    - The solution phase ($S_i$), by contrast, serves as a concise and deterministic answer, usually synthesized based on the preceding reasoning steps. It is expected to be directly usable for evaluation (e.g., exact-match or numerical comparison).\n\n    This semantic distinction aligns with how humans distinguish between \"working through a problem\" and \"writing down the final answer\".\n\n---------------------\n\n### **Q4: Clarification on Variable $\\mathcal{B}$**\n\nWe thank the reviewer for pointing this out. The variable $\\mathcal{B}$ in Eq.(1) and Eq.(2) indeed denotes the mini-batch size. We will revise the manuscript to explicitly clarify this notation.\n\n\n---------------------\n\n### **Q5: The impact of directly substituting $g_{\\text{con}}$ with $g_{\\text{ref}}$**\n\nGood question. Indeed, we have discussed this question in Supplementary Material Section 4.1, where we experimentally and empirically investigate whether the projection step in Eq.(3) is necessary, that aligns with your question: the impact of replacing $g_{\\text{con}}$ with $g_{\\text{ref}}$.\n\nSpecifically, we compare two variants:\n- using the projection operation as defined in Eq.(3) of the main paper, that is $g_{\\text{con}} = \\frac{\\langle g_{\\text{full}}, \\\\ g_{\\text{ref}} \\rangle}{\\|| g_{\\text{ref}} \\||^2} \\cdot g_{\\text{ref}}$\n- skipping the projection and directly assigning $g_{\\text{con}} = g_{\\text{ref}}$\n\nFor convenience, we put the results in Table 1 of the Supplementary Material in the table below. \n\n| Method | Projection | In-domain Acc@1 (avg) | Out-domain Acc@1 (avg) |\n|:------:|:----------:|:---------------------:|:----------------------:|\n| -      | -          | 21.5                   | 31.2                    |\n| SFT    | -          | 27.8                   | 27.3                    |\n| DLoFT  | ✗          | 30.3                   | 31.8                    |\n| DLoFT       | ✓          | 36.7                   | 39.7                    |\n\nThe results clearly demonstrate that:\n- The variant with projection operation yields better performance than its direct substitution counterpart ($g_{\\text{con}} = g_{\\text{ref}}$) on both in-domain and out-domain generalization. This highlights that the projection step plays a critical role in more precisely isolating problem-specific gradients, and its removal leads to incomplete decoupling, thereby reducing the effectiveness of the filtering problem-specific knowledge and heuristics.\n- Omitting the projection still achieves better performance than standard SFT and partially mitigates the out-of-domain performance drop.\n\nTo ensure this clarification is more visible, we will move the relevant analysis from the supplementary material into the main paper. \n\n---------------------\n\n### **Q6: Clarification on the projection operation in Eq.(3)**\n\nWe would like to clarify the meaning of Eq.(3) to address the possible misunderstanding.\n\nThe projection formula we use is: $g_{\\text{con}} = \\frac{\\langle g_{\\text{full}}, \\\\ g_{\\text{ref}} \\rangle}{\\|| g_{\\text{ref}} \\||^2} \\cdot g_{\\text{ref}}$.\n- This expression calculates the component of the full gradient $g_{\\text{full}}$ along the direction of $g_{\\text{ref}}$. More concretely, it finds the scalar projection coefficient $\\lambda = \\frac{\\langle g_{\\text{full}}, g_{\\text{ref}} \\rangle}{\\| g_{\\text{ref}} \\|^2}$, quantifies how much of $g_{\\text{full}}$ lies in the direction of $g_{\\text{ref}}$, and then scales $g_{\\text{ref}}$ by this amount.\n- Therefore, ***$g_{\\text{con}}$ is indeed the projection of $g_{\\text{full}}$ onto the vector $g_{\\text{ref}}$***, isolating the component of $g_{\\text{full}}$ that corresponds to the problem-specific reasoning gradient direction $g_{\\text{ref}}$.\nIt is not projecting $g_{\\text{ref}}$ itself; rather, $g_{\\text{ref}}$ serves as the direction basis for the projection of $g_{\\text{full}}$. \n\nThis is a standard vector projection operation in linear algebra, derived from the geometric interpretation of the dot product as follows: \n- Recall that the dot product between two vectors $a$ and $b$ is: $ \\langle a, b \\rangle = \\|| a \\|| \\cdot \\|| b \\|| \\cdot \\cos \\theta$, where $\\theta$ is the angle between the two vectors. \n- Rearranging this gives: $\\|| a \\|| \\cos \\theta = \\frac{\\langle a, b \\rangle}{\\|| b \\||}$, which is the length of the projection of $a$ onto the unit vector in direction $b$.\n- If we want the full vector form of this projection, we multiply this scalar by the unit vector in the direction of $b$, i.e., $\\frac{b}{\\|| b \\||}$, leading to: $\\text{proj}_{b}(a) = \\frac{\\langle a, b \\rangle}{\\|| b \\||} \\cdot \\frac{b}{\\|| b \\||} = \\frac{\\langle a, b \\rangle}{\\|| b \\||^2} \\cdot b$\n\nWe hope this clarifies the misunderstanding. \n\n[Continued in next block]"}, {"review_id": "yVLfkoNz7w", "reviewer": "Reviewer_cC52", "summary": "This paper aimed to improve the LongCoT’s general reasoning capabilities by decoupling the format and content gradients during training toward the LongCoT reasoning, concretely, the proposed DLoFT work first divided the format- and content-related gradients with projection and orthogonalization operations, then focused on optimizing the format-related gradients. Experimental results illustrated that the proposed fine-tuning strategy can enhance the out-of-domain LongCoT reasoning abilities, demonstrating its effectiveness in both in- and out-domain scenarios. \n\nHowever, there are several key issues in this paper undermining the quality.", "questions": "Please refer to the above weaknesses.", "limitations": "The authors didn’t discuss or mention the limitation of this work.\nThis work focused on the reasoning format by getting rid of the content-related information, on the one hand, such a premise was not elaborated by empirical or theoretical analysis, lacking actual motivations; on the other hand, the content information matters, especially for some question that needs a lot of specific knowledge.\nThere is no comparable baseline method except for the vanilla in this work, and there are numerous gradient-based research, such as [1,2,3].\n\n[1]. Directional Gradient Projection for Robust Fine-Tuning of Foundation Models. Huang et, al. ICLR 2025.\n\n[2]. Fast Trainable Projection for Robust Fine-Tuning. Tian et, al. NeurIPS 2023.\n\n[3]. Improving Stability of Fine-Tuning Pretrained Language Models via Component-Wise Gradient Norm Clipping. Yang et, al. EMNLP 2023.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. This paper conducted couples of experiments on the 7B and 32B models, and multi-domain evaluation benchmarks across various domains, results shown in tables and figures demonstrated the generalization capabilities of the proposed strategy.\n2. The authors introduced the training and test data and other experimental settings in both the manuscript and the supplementary materials, and these details offered convenience for reproduction for the experiments. \n\n**Weaknesses**\n1. In the left subfigure of Figure 1, the in-domain accuracy is far lower than the out-domain, which is not in line with the current practice, implying such results were not produced by converged trained models, which undermined the explanation for the overfitting issue.\n2. As claimed in Observations paragraph of Introduction and the Section 3.2, the authors assumed that it was the problem specific knowledge hindered the generalization learning, there is no empirical evidence or theoretical analysis for such an assumption.\n3. The paper stated that “Focusing too much on this (specific problem-solving procedures) … limiting its ability to develop generalization reasoning skills” in Section 3.3 Step 1 paragraph, and the following statements “there is an information aimed at activating the model’s LongCoT abilities, …”. However, most scientific questions share similar reasoning patterns including derive, check, reflect, and so on, and models can also learn such format with vanilla SFT to obtain reasoning abilities for out-domains. What is the necessity of \n4. In the Step 2 paragraph of Section 3.3, the authors adopted g_{ref} as the problem-specific content anchor, while there are also general reasoning formats except the problem specific contents.\n5. This paper decoupled the so-called content-relevant gradient by a specific projection, simple but not explainable, the reason why such a self-defined could produce the content-relevant gradient was not interpreted. Given the calculation of the g_{con}, the extraction of g_{par} also needs to discuss, same for the fine-tuning/updating strategy.\n6. The accuracy results on AIME24 seemed that the authors only conducted one-time sampling for the model inference (as there are 30 questions in AIME data), which maybe random, pass@k is a more suitable metric for LongCoT reasoning task evaluation. \n7. According to the Figure 3, DLoFT for RL performed better than the vanilla SFT on the MedQA, while on the AIME24 and LiveCodeBenchv2, it seemed that it was only on par with the vanilla, which couldn’t prove the DLoFT is more effective for the cold-start setting (claimed as Section4.2.2).", "was_revised": true, "final_justification": "I have read all the comments by the authors, and I will maintain my original score.", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper studies the out-of-distribution (OOD) generalization failures of naively fine-tuning LLMs on long Chain-of-Thought (CoT) data and proposes a simple yet effective method (DLoFT) to mitigate this issue. The core idea is to isolate general “long reasoning” abilities from "problem-specific" signals during SFT. This is achieved by projecting out the problem specific gradients (obtained from question -> solution prediction) from the total gradient (obtained from question -> (thoughts, solution) prediction). Overall this improves over vanilla SFT (even after some RL) across the board, particularly on OOD benchmarks.
Overall, this paper addresses the timely and important topic, identifies a crucial weakness in the standard SFT approach for LongCoT data and proposes a simple, effective, and well-motivated solution.

---

Reviewers raised a few key concerns, which the authors have largely addressed:
* **Overfitting and Dataset Size:** A primary concern was whether vanilla SFT was worse simply due to overfitting. The authors mitigated this by showing that training for fewer epochs and much larger SFT dataset shows similar trends.
* **SFT and RL numbers are low:** Authors pointed to the s1k paper and argued that RL showed reasonable improvements.
* **(Theoretical) justification for the decoupling** There were questions about why problem-specific features would hurt generalization and whether this contradicts recent work (e.g., DeepSeek-V2) suggesting SFT is beneficial. The authors pointed to literature on overfitting and shortcut learning for justification, and commented that since they are fine-tuning an instruction-tuned model, the results are not contradictory.

---

One issue (not discussed in the author-reviewer discussion) is that the paper lacks good baselines comparisons for the method. The paper mostly compares DLoFT to vanilla SFT, early stopping and weight decay, but perhaps some more baselines (e.g. in-context prompting with LongCoT data or LoRA fine-tuning) could have been considered for better generalization. 
Furthermore, the problem being studied shares similarities with "catastrophic forgetting" in continual learning and also shortcut learning. In fact some gradient orthogonalization ideas exist in the continual learning literature [1]. At least a discussion with this literature is warranted, and perhaps some simple baselines from these areas would further contextualize the paper's contribution and strengthen the results.

---

Overall the paper makes a positive contribution, and incorporating the experiments and fixes suggested by the reviewers would further enhance the manuscript. The recommendation is Accept.

[1] Farajtabar et al. Orthogonal Gradient Descent for Continual Learning. 2019
Decision: Accept (poster)