Differential Fine-Tuning Large Language Models Towards Better Diverse Reasoning Abilities
Authors: Xiaosong Yuan
Chen Shen
Shaotian Yan
kaiyuan liu
Xiaofeng Zhang
Sinan Fan
Liang Xie
Wenxiao Wang
Renchu Guan
Ying Wang
Jieping Ye
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Reasoning Abilities
Supervised Fine-Tuning
Primary area: foundation or frontier models, including LLMs

Abstract: Reasoning abilities of large language models (LLMs) require explicit derivations compared to general question-answering, supervised fine-tuning (SFT) can empower multiple reasoning abilities in LLMs via learning from various datasets. However, neither training the datasets jointly (mix-up) nor continually can maintain the performance of single-dataset SFT, sometimes better while sometimes even worse, illustrating vanilla SFT can not only facilitate reasoning abilities but also introduce conflicts. In this paper, we propose a novel framework to mitigate the conflicts and preserve benefits among different reasoning tasks, and even surpass each task's single dataset SFT performance. We start by exploring the differences between reasoning fine-tuned and base LLMs by analyzing their parameter variations during model inference, and we discover that each reasoning capability has exclusive parameters that benefit it more evidently than others. In contrast, the overlapped parameters of tasks can bring benefits or conflicts. Inspired by the findings, we propose to update the exclusive and overlapped parameters according to specific reasoning task combinations differentially, thereby avoiding unnecessary conflicts while maintaining benefits. Consistent improvements in mix-up and continual SFT experiments demonstrate that the proposed SFT strategy can achieve better performance on various LLMs (Llama3-8B, Mistral-7B, and Qwen2.5-14B) and diverse reasoning tasks with fewer conflicts, showing the superiority and generality of our analysis findings and the proposed approach.

Published as a conference paper at ICLR 2026
DIFFERENTIAL FINE-TUNING LARGE LANGUAGE MOD- ELS TOWARDS BETTER DIVERSE REASONING ABILI- TIES
Xiaosong Yuan Chen Shen"i,Shaotian Yan", Kaiyuan Liu Xiaofeng Zhang
Sinan Fan", Liang Xie Wenxiao Wang'. Renchu Guan Ying Wangl.2 Jieping Ye
Colleee of Computer Science and Technology. Jilin University
Keey Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University
3
AAlibaba Cloud Computing
*College of Computer Science and Technology, Zhejiang University
Schooo of Automation and Intelligent Sensing, Shanghai Jiao Tong University
6 College of Computer Science and Technology. Zhejiang University of Technology
yuanxs @ mails. mailssjlu.edu.cn zjushenchen @ gmail.com
ABSTRACT
Reasoning abilities of large language models (LLMs) require explicit derivations compared to general ceasstionnanswernng. supervised fine-tuning (SFT) can em- power multiple reasoning abilities in LLMs via learning from various datasets. However, neither training the datasets jointly (mix-up) nor continually can maintain the performance of single-dataset SFT, sometimes better while sometimes even worse, illustrating vanilla SFT can not only facilitate reasoning abilities but also introduce conflicts. In this paper, we propose a novel framework to mitigate the conflicts and preserve benefits among different reasoning tasks, and even surpass each task's s single dataset SFT performance. We start by exploring the differences between reasoning fine-tuned and base LLMs by analyzing their parameter varia- tions during model inference, and discover that each reasoning capability has exclusive parameters that benefit more evidently than others. In contrast, the over- lapped parameters of tasks can bring benefits confficts. Inspired by the findings, we propose to update the exclusive and overlapped parameters according to specific reasoning task combinations differentially. thereby avoiding unnecessary conflicts while maintaining benefits. Consistent improvements in mix-up and continual SFT experiments demonstrate that the proposed SFT strategy achieve better performance on various LLMs (Llama3-8B, Mistral-7B, and Qwen2.5- 5-BB)) and diverse reasoning tasks with fewer conflicts, showing the superiority and generality of our analysis findings and the proposed approach.
INTRODUCTION
Large language models (LLMs) have emerged various reasoning abilities (Achiam et al. 2023
Dubey et al. 2024 Yang et al. 2024), such math problem-solving (Yue al. (024). coding (Guo
et al. (024), logical inference (Pan et al.) (023), and commonsense reasoning (Zhao et al. 2024).
In contrast to the general conversation, reasoning tasks often require models perform higher- order cognitive processes such as analysis, deduction, and mabblem-solving Supervised fine-tuning (SFT) on distinct labeled datasets can facilitate such proficiencies (Dong et al. 2023 Zhang et al.. 2024 Chen et al.) 2024: Lu et al., 2024), enabling LLMs with versatile reasoning capabilities. Although vanilla SFT on different reasoning data can strengthen LLMs' certain capabilities some curated combinations (Dong al. 2023). it tends to underperform
single dataset, revealing that mutual enhancement and conflict may coexist across reasoning tasks. Prior works have explored the
destructive interference of varied tasks (Wang et al. 2024 Hui et al. 2024b Panda et al. 2024 Gong
Work done during internship Alibaba Cloud Computing Corresponding author
1 Project lead

---

Published as a conference paper at ICLR 2026
aatp 1848)
LegrokR Ogg
LogtoAd (x))
G3M6: AREIE 9403)
AL.
29A
Lima' dade
Liama2-00
Mistral-7B
Merherd
S.B. Mon
(a) Mix-up SFT
(b) Continual SFT
Figure 1: Performance of various math-related SFT models on 5 benchmarks with Llama3-88 and Mistral-7B, scores are increasingly ranked from the center to the circle. (We multiply the pass rate of XGLUE by 40 and the accuracy of LogiQA2 by 2 to align with others for better visualization.)
al. 2024), they focused on the conflict of general abilities rather than reasoning and believed that all of interactions were harmful to involved tasks.
In the investigation, we conduct comprehensive SFT experiments with different LLMs on types of reasoning data to discover the relationships among various reasoning proficiencies. As shown in Figure [[a]] some combinations, like Mix-Math-Code of Llama3-8B. obtain significant im- provements in math (measured by GSM8k) compared to Math-only, while it underperforms on other tasks like code (measured by xGLUEE, which is more clear in Figure [(b) On the other hand, continual learning results through Cont nual -Mat -hothe exhibit severe negative interference. However, an intriguing difference emerges in Mistral-7B (Jiang et al. (023), suggesting complex dynamics in distinct LLMs. These tendencies are also exhibited similarly in combinations among more reasoning tasks, while they perform distinctly in different LLMs. Such phenomena imply benefits and conflicts between distinct reasoning capabilities that may be ubiquitous. More detailed experiments and analysis are introduced in Section
Previous efforts have been made in parameter-variation SFT to mitigate conflicts among various LLM abilities. [ong et al. (202)) designed a dual-stage mixed SFT strategy to endow L.LMs with math and other capabilities. HFT (Hui et al. 2024b) updated the half LLM parameters randomly in continual SFT to alleviate catastrophic forgetting. LoTA (Panda et al. 2024) employed task vector extraction and sparse adaptation minimize interference among tasks. Nonetheless, the complete picture task relationships neglected, encompassing beneficial, contradictory, and neutral. In this paper, we investigate the mutual benefits and conflicts of reasoning capabilities in the SFT process.
To determine what benefits and conflicts exist and what causes those, we explore the intrinsic weights of distinct fine-tuned I.MMs. Concretely, we present novel approach identify the individual parameter sensitivity via sampled data inference on corresponding LLMss, thereby locating necessary weights for specific abilities. After that, we design a suit of Differential SFT (DiFT) strategies to get better versatile reasoning abilities: for mix-up SFT, we merely fine-tune the parameters union of critical weights for involved tasks, to obtain target reasoning abilities while making less disturb to others; as for continual SFT, we freeze the parameters in difference set of the former and current tasks, to reserve historic proficiencies and learn new ones by remaining parameters.
We employ base instead of instruct LLMs for analysis and validation, as instruct LLMs have been through massive post-training, making hard to measure their inner benefits/confticss Additionally, our fine-tumed L.LMs with fewer data beat instruct LLMs on some tasks (e.g.. logic and commonsense in Section 5.4). highlighting that our research can provide insights for reasoning-orientee fine- tuning( regardless of base instruct models). We conduct extensive experiments with pilot LLMs on several reasoning tasks, and results show that the proposed DiFT can improve all LLMs in various reasoning combinations, where mix-up SFT can approach the single dataset SFT, and continual SFT can reduce more historical performance losses, demonstrating that our analysis is valid and DiFT can mitigate reasoning conflicts and keep mutual benefits. Our contributions are follows:
2

---

Published as a conference paper at ICLR 2026
We investigate in comprehensive SFT experiments single (vanilla), mix up, and continual reasoning datasets with different LLMs, showing that mutual benefits and conflicts exist among distinct reasoning tasks commonly.
By analyzing the parameter variations during inference between various fine-tuned and base models, we discover that some parameters are vital to specific reasoning tasks, i.c., each reasoning capability corresponds certain parts of parameters.
Based on the analysis and findings, we propose novel fine-grained SFT strategy to preserve enhancement and mitigate the conflicts by selectively updating those reasoning-relevant parameters of LLMs.
We conduct extensive experiments on different LLMs with the proposed DiFT, and em- pirical results across distinet reasoning tasks are in line with our analysis, validating the effectiveness of the proposed approach.
2 RELATED WORK
SFT has been demonstrated as productive post-training paradigm for improving models' various capabilities (Minaee et al.) 2024), including math (Yu et al. 2023). code (Guo et al. 2024), common- sense (Bian et al.) 2024), logic (Chen al. 2023), and instruction following (ouu
2023). There are task conflicts in LLMs (Luo et al. 2023), and numerous SFT variants emerged in the era of LLMs from data selection and optimization. Task Vector (Ilharco al., 2022) considered the fine-tuned and pre-trained parameter variations as the task-related weights and conducted addition and negation to modify or combine different tasks. [Rudman et al. (2023) discovered that outlier dimensions could encode crucial task-specific knowledge and that the value of a representation in single outlier dimension drives downstream model decisions. Zhang et al. (2023) proposed parameter optimization trajectory and learned to uncover intrinsic task-specific subspace by exploiting the dynamics of fine-tuning a given task. Nonetheless, these works failed to connect the specific parameters and tasks.
DMT (Dong et al. 2023) designed dual-stage mixed fine-tuning endow LLMs with math, code, and instruction abilities. These methods aim to find better data usage, ignoring the learning process. [Tang al. (2024) presented partial linearization to fuse multi-task abilities into model. HFT (Hui et al. 2024b) updated a random half of LLM parameters in continual fine-tuning to alleviate catastrophic forgetting. LoTA (Panda et al. 2024) employed task vector extraction and sparse adaptation to minimize interference among multiple tasks. Kong et al. (2024) introduced gradient approximation strategy for activated parameter locating to reduce the computational complexity associated with many parameter partitions. Nevertheless, none of them analyzes the model parameters in-depth from the perspective of reasoning conflicts.
3 BENEFITS AND CONFLICTS ANALYSIS
We intend to validate and explore the mutual benefits and conflicts among reasoning abilities by delving into the L.Ms'' parameters step by step to explore the causes. First, we conduct SFT experiments on 4 datasets in 3 settings: vanilla, mix-up, and continual (detailed experimental settings Section Ip. As instruct-LLMs were trained on massive data, the scaling-up training may trade-off part of conflicts in math reasoning, we take Llama3-8B 8B-base and Hittral--B--bsee results shown in Table 7 The findings and intervening phenomena are similar in instruct-LLMe Section 5.4
3.1
MIX-UP AND CONTINUAL REASONING SFT
In Table ! vanilla SFT enhances the corresponding reasoning ability stably on both Llama and Mistral, while it may affect others: for example, the Math-only can degrade logic and commonsense a bit, and so do the Logic-onls and CSQA-on to math. Such results suggest:
i. There may be learning trade-off between distinct abilities that leads reasoning interference.
Interestingly, the mix-up SFT exhibits synergistic effects in both positive and negative aspects. Mix-Math-Code achieves great performance both GSM8k and xGLUE compared to single variants, implying math and code may share complementary weights, evidenced by the 64.82% accuracy and 1.0956 pass rate of Mi x-Matttt iattthh h-Cood Llama3-8B, surpassing Math-on ly. An
3

---

Published as a conference paper at ICLR 2026
Table 1: The mix-up and continual SFT results of Llama3-8B and Mistral-7B on benchmarks, the
Land ddenote decreasing and increasing compared to the base model performance, respectively.
Methods
Llamas-83
Mistral-7B
GSMSL
XGLUE
LogiQAZ
CSQA
MMLU
GSMSKK
xGLUE
LegiQA2
CSQA
MMLU
base model
39.42
LU874
31.93
69.29
37.66
35.97
1,2419
31.87
64.29
50.49
Vanilla SFT
Math-oaly
61.64
1,2228
30.73
67.24
56.76
59.14
2,0042 1
30.85
52.50
28.68
Code-only
26.54
1.1203
35.05
70.93
55.65
31.31
1.7146
28.94 1
58.39
43.04
Logic-only
0.6880
37.02
72.89
57.52
4.62
1.3628
32.47
CSQA-only
8.79
0,5702
29.90
79.36
28.10 L
1.36
2,7964
30.15
70.93
23.43
Mix-up SP-
Mix Mode-Code
64.82
1,9956
34,54
68.22
56.47
41.17
1.2913
33.08
60.28 4
44.81
Mix-MatthLogiic
64.37
1.20921
32.32
70.52
$5.30
57.39
0.8593
31.87
62.49
36.68
Mix-Math
32.32
77.31
1
52.77
2.8439
39.76
Mix-Code-Logic
52.31
1,0779
32.57
70.52
$8.05
22.37
1,2344
31.17
62.00
43.33
Mix-Cods-CSQA
52.39
0.8905
31.42
77.15
32,06
26.69
1.3969
44.97
Mix: MixxLLogic LSOA
16.91
0,2150 4
32.44
77.40
47.14 1
16.60
0,9582
31.11
74.69
45.34
Continaa
NP7
Continual Math-Code
44.35
0.9902
32.82
70.52
54.28
47.01
1.6431
31.81
44.96
25.98
Setinual-Matt-L-ggeee
10.99
0.6433
31,30
67.90
51,53 1
4.62. J
1.0365
29.26 L
40.29
24.56
Continual Matl CSQA
0,5494
31.36
78.71
47.07
1.14.
3,8740
30.34
57.90
23.12
intriguing discovery is the imbalance impact: Mix-Math-Codk improves math from 61.64% to 64.82%, while it only enhances xGLUE to 1.0956, a minor less than Code-only, implying:
ii. Benefits between different reasoning abilities are not always reciprocal, where one reasoning ability may gain more than the other.
In contrast mix-up SFT, continual SFT is born with catastrophic forgetting, which remains a significant challenge, making it more complex (Ramasesh et al.) [021]] Luo et al. (022)) At the bottom of Table [ we can hardly observe benefits between reasoning abilities. While Continual-Ma -Math-Logii
achieves moderate logic performance (31.30% on Llama3-8B), it shows severe degradation in math reasoning (10.99% on GSM8k). Such results indicate that con- tinual SFT with different reasoning data may lead to the erosion of previously acquired abilities. Additionally, the continual SFT on one task performs worse than the direct SFT on the base LLM in some settings, e.g., significant task interference in Cont ina -Math -Math-Code Coode (1.084 to 0.9902 with Llama3-8b). Such results indicate that there also exist reasoning conflicts besides catastrophic forgetting. Therefore, we make an assumption:
iii. Even catastrophic forgetting is the main issue, reasoning conflicts matter in continual SFT.
The above findings highlight the complex interactions between different reasoning capabilities and the challenges in mitigating conflicts while preserving benefits. To address the above challenges, we start by analyzing the inner weights of LLMs with different reasoning proficiencies.
3.2 DELTA-SCALE ROWS
We propose a novel approach for identifying influential weights in LLMs inspired by Yu et al. (0024) and to quantify the sensitivity of the model output to changes in weights, introduce the delta-scale row (DSR) score to measure the parametric sensitivity. Let E # denote the weight matrix of a linear layer, with / as the out-dimension and D the in-dimension, L(W) is the loss of the current task on W; W parameters of the original model while W is of the fine-tuned; AW2 W W X₂ is input sample token t, and Y, X,W The fine-tuning process can be formalized as:
AW = =- J." (x)d)
W W n29"
where g" is gradient at step 8, 17s learning rate, is integration variable, and T total steps. Hence, (&W& is large if the gradient keep projecting on k-th row significantly in training. We perform second-order Taylor expansion by varying Wt. plug 8 AW.. and obtain:
AL = 9h AW
L - HAN (AW )² + ((AW))
where is the Hessian matrix of the loss (H V(L(W))) the starting point Wo and H its k-th diagonal element (curvature along the k-th row). Substantial loss drop thus needs non-trivial W. J. For a set of input the output activations Y E RR are typically computed as:
Y XWT
4

---

Published as a conference paper at ICLR 2026
2150
Lewis mode been plceacerss
Lover monal wann per.prs
1000
500
2010
20:0
1050
1250 1600
2000
600
1014
1300
12660 1000
23033
4000
1000
1014
1000
math rows (seed=42)
(b) math rows (seed=43)
(c) math (4eed=44)
250
(d) code rows (2eed=42)
(c) code rows (seed=43) nobeltges M.ragg
code rows (seed=44)
2000 1000 1233
2096 0030
680 10c
1990
135 1459
731 4003
120. 128
1000
logic rows (sccd=42)
(h) logic rows (seed=43)
logic rows (scod=44)
Figure 2: Distribution of delta-scale rows (DSR) for layer.layer.24..m www.mmmap.pppp... with distinct sampled data on different reasoning LLMs, we randomly sample 3 times (with seeds 42/43/44) for each task. The z-axis denotes the row order of the specific weight matrix, and y-axis is DSR values.
We analyze the difference in outputs between a base model (Maase) and fine-tuned model (M), where weights are presumed to have changed during fine-tuning, Let Yiuse and Yfs be the output activations of a specific layer for the same input X from Muuse and M ft. respectively. The difference in output for the k-th component (corresponding to the k-th row of W) for given token is:
AY (f) - y (t)
(4)
This AY, reflects the impact of the accumulated changes between W and W base on the k-th output feature for that token. Combining the above derivations, can obtain:
AY. X,AWs, AYI [12]
11
(5)
Thus, 8% R. | [1)) which can be computed by 8 = N |A is proportional to both || and | I|, meaning that DSR score can denote the task-specific accumulated importance.
In practice, DSR scores accumulate the squared differences for each output component k, effectively capturing the impact of changes in the corresponding k-th row of the weight matrix across reasoning data. High values in the vector of scores indicate rows of the weight matrix (and their associ- ated output features) that exhibit greater changes in activation magnitude due to fine-tuning, suggesting these rows are influential in the processes modified or learned by the model.
3.3 FINE-TUNED REASONING MODEL ANALYSIS
To analyze the DSR scores of parameters, we perform inference with distinct fine-tuned and base LLMs on samples, ensuring that each sampled data corresponds to tuned LLM. Con- cretely, we compute for each layer in the forward pass with 5 sampled groups of 50 data items (random seed 42-46) to obtain the top DSR of each task, we display some row distributions of model. layers. 24. mll p. gate. proj in Figure other weights also express similar patterns, more visualizations in Appendix [F6] The magnitude of the DSR provides a quantitative measure of specific parameters' influence, where higher values matter more. Across all fib-figures in Figure 2 we can observe distinct peaks in DSR parameters. These peaks indicate specific rows in the weight matrix disproportionately affect LLMs. and the rows correspond to the critical DSR parameters aim to identify. Note that we only annotate the top-20 DSR parameters for better visualization, there remarkable differences among distributions of distinct data in Figure 2
5

---

Published as a conference paper at ICLR 2026
In the distribution of the math (Figures 2(a) to 2(c) row-wise, distinct peaks multiple rows, e.g.. 284, 1992, and 9246, suggest that specific rows in the weight exert significant influences on the LLM's ' reasoning different seeds, and so are the code and logic, implying stability of key DSR parameters for the task regardless of sample variations. However, the distribution of each fine-tuned reasoning L.LM exhibits distinctly in Figures 2(a) to ) column-wise: rows 284 and 1992 have the top-2 scores across all rows in math and logic LLMs, while the top-2 rows of the code LLM are 6280 and 9246; the logic model has influential rows of index >> 13000, but the indices of all math DSR are < 13000.. Similar phenomena exist in more LLMs as shown in Figures and 12 in Appendix
math. Interestingly, such a pattern is consistent across sampled data with
We further analyze the same model (Math-only) with sampled data subsets and observe a more diverse DSR parameter distribution among distinct data Appendix F.6 illustrating the parameter divergence of reasoning abilities within LLMs. After meticulous DSR analysis, we discover that: On the one hand, rows of the parameter matrix are not sensitive different inputs of the same reasoning task, on the other hand, different tasks demonstrate unique parameter distributions.
4 METHOD
We propose Differential SFT (DiFT), novel fine-tuning strategy for LLMs which can mitigate task conflicts and maintain benefits multi-task and continual learning settings by leveraging DSR analysis. DiFT identifies task-specific DSR parameters through activation differences between base and fine-tuned LLMs, then differentially updates those crucial to current tasks while preserving the performance of others. By isolating and protecting these sensitive yet non-specialized parameters, which can be treated as a reliable proxy for task specificity (Deng et al. 2025), DiFT reduces negative transfer and forgetting. Algorithm iin Appendix proovvees more details.
4.1 DELTA-SCALE ROW ANALYSIS
Given a base model Mbase and set of reasoning -epeciig- LLMs (M) ft with corresponding datasets K=1 DiFT analyzes how fine-tuning alters internal activations. For each task we sample N imputs from Dy and register forward hooks on target layers of both Mouse and M to capture input/output activations. For every input x, we compute the difference in layer outputs:
A 1 = (x) naase (I)
where (1) is the output of layer 1. We accumulate the squared L.2 norm of the differences across samples to obtain per-row sensitivity (Eq. 5). The DSR scores reflect how much each weight row (i.e., neuron activation) changes during SFT. We define DSR; as the set of top-C' highest-scoring rows per layer for task k. denoting parameters most critical to its reasoning behavior. According to Section 3.2 DS R& captures the "accumulated effects" of k. Combining with our conflicts validation, when two tasks A and B conflict, SA DSRA and Sg DSRg are crucial parameter sets for A and B. Naturally, the other parameters out of these sets are more likely to hurt involved tasks, so we choose to freeze those to mitigate conflicts. We also conduct experiments to demonstrate that the effectiveness of DiFT lies in causally conflict mitigation instead of regularization, and more details can be referred to Appendix(D.2) and D.5. We further extend the DSR explanations and other relevant concepts in detail in Appendix D.I
4.2 MIX-UP FINE-TUNING
To fine-tune a given LLM on multiple reasoning tasks simultaneously. DiFT computes the influential union DSR Runion and merely updates these parameters:
K-1
U DSR
DSR
1===
The LLM is then fine-tuned on the mixed data U== DE. Concretely, for SA and Sg that are critical for tasks A and B, respectively, what we focus on when considering A and B is to learn with SA and Sp to maintain their performance much. Although S and So may disturb each other, such

---

Published as a conference paper at ICLR 2026
conflicts cannot be measured precisely and are out of this paper's scope. Therefore, we about the major and leave the minor, i.c., reserve the base performance of and B. As for 1 - Sa SB. since observe that A and B have conflicts, and SA and Sn should carry more weights of maintaining the performance of A and B (as stated in the former paragraph), so our focus on SA U USB is freezing them prevent these parameters from their potential conflicts to A and B.
4.3 CONTINUAL FINE-TUNING
In continual SFT, DiFT mitigates conflicts by updating only newly important DSR parameters.
Inspired by the common practice in continual learning (Wang et al., 2024). at step & (for k 2 2), we obtain a highly sensitive parameter set to task & but not identified critical previously:
h-1
DSRas1 DSR
UDSR, J=0
(8)
Only the DSRaiff parameters trained DR. starting from the latest M Concretely, for historic task A, we freeze the SA to decrease the negative influence from the current task B to A, as SA matters a lot for task A; for task B, we employ SB - SA to improve B's performance as much. By restricting updates parameters with less impact on prior tasks, DiFT preserves historical abilities while acquiring new skills, effectively decoupling new learning from old representations. Additionally, given the parametric nature, DiFT can also be employed jointly with data-driven methods to reduce conflicts after mitigating forgetting, and we will discuss that in Section [3. Based on the DSR analysis, fine-tuning 1 SB can also be solution, however, the empirical results in Appendix D.6 suggest that the DS Rasfs is better.
In summary, DiFT employs DSR identify task-specific parameters with a few reasoning data, then differentially fine-tunes to mitigate conflicts while retaining benefits. It attempts to enhance diverse reasoning proficiencies of LLMS by curated adaptation where it matters most in mix-up or continual scenarios, and disturb less what already works. We further perform comprehensive ablation studies in Appendix F.6 to analyze the DiFT various perspectives.
EXPERIMENTS
To validate our findings and evaluate the proposed DiFT, conduct comprehensive experiments on both mix-up and continual settings. We employ Qwen2.5-3B, Llama3-8B, Mistral-7B, and Qwen2.5- 14B, and common reasoning benchmarks. All DiFT results based on the top- 100 DSR union, all SFT training experiments come to convergence as in Appendix B..l 3B and 14B experiments are in Appendix E and more ablation studies for DSR choice strategies are in Appendix D.2 to D.6
5.1 SETTING
Training data We collect and sample reasoning data to fine-tune LLMs toward distinct abilities. All the source data are widely used for task-specific training, including but not limited to MathIn- struct (Yue et al., 2023), Code Bagel Hermes (Teknium 2023). LogiCoT (Liu al. 2023b), and CommonsenseQA Talmor et al., 2018). More source data can be referred to Appendix B.4 We sample 20,000 for each reasoning ability and conduct SFT involving tasks with DiFT each time.
Baselines As DiFT works in both mix-up and continual settings, we implement comparable approaches to evaluate the performance. HFT (Hui et al.) (024b) is a continual SFT framework, randomly freezes half of each named parameter when fine-tuning on new task to memorize the old knowledge. LoTA (Panda et al. 2024) is an advanced gradient-projection it extracts and masks the feature vectors in the next round. Dual-stage mixed SFT (DMT) (Dong et al. 2023) presented a two-stage mix-up fine-tuning strategy, implemented by merging different training data. CoBa (Gong al. 2024) designed a synthesized loss by calculating the relative and absolute convergence scores. The hyperparameter settings of baselines are the same as DiFT, in Appendix B.T where also compare PEFT and other continual methods in Appendix
method, in each SFT round,
and R4
7

---

Published as a conference paper at ICLR 2026
Table 2: The mix-up and continual SFT results on 4 benchmarks. The SOTA results across different strategies are marked in bold numbers, and the sub-optimal results are italic numbers. denotes the existence of format issues, and more details about the orthogonal combination of the proposed method and the data-driven method for reducing conflicts and mitigating forgetting are in Section 5.3
Liama3 8B
Mistral 7B
Methods
GSM8k 39.42
XGLUE 1.0874
LogiQA2 31,93
CSQA
ATA
GSMBk 38.97
XGLUE
LogiQA2 31.87
CSQA 64.29
ATA
1.2449
Mix-up SF7
Mix- Mart-MathCaaac Code
64.82
1.0956
34.54
68.22
59.80
41.17
1.2913
33.08
60.28
52.87
+DMT
65.07
1.8511
32.44
67.52
59.66
42.13
1.2400
32.18
59.82
52.07
+CoBa
66.21
1.0225
33.15
68.34
59.91
43,07
1.1900
31.94
58.45
51.29
+DIFT (ours)
67.02
1.0735
32.63
68.39
60.35
42.46
1.3429
33.33
59.46
54.80
LixiCode-Logic
52.31
L,0799
32.57
70.52
43.23
22.37
1.2342
31.17
62.00
+DMT
50.37
1,0865
31.93
69.36
43.12
26.58
1.2308
30.62
63.19
46.08 08
+CoBa
51.12
1.0811
32.25
68.67
4.3.15
26.05
1.2491
30.16
63.82
46.16
+DiFT(ours)
41.09
1.1359
33,40
68.55
45.10
31.69
1.2555
32.51
62.24
47.64
Mix-Logic-CSQA
16.91
0.2150
32,44
77.40
54,92
16.60
0,9582
31.11
74.69
52.90
+DMT
13.79
0.3907
31.68
78.84
55.26
18.58
0.7731
30.72
72.75
51.74
+CoBa
14.93
0.3868
32.16
78.05
55.11
19.42
0.7847
30.41
73.48
$1.95
+DiFT(ours)
16.22
0.4592
32.38
78.95
55.67
21.68
0.6196
31,68
74.45
53.07
Conninual SF"
Continual Montinual-Math-- Code
44.35
0.9902
32.82
70.52
46.93
47.01
1.6431
31.81
44.96
64.58
*HFT
44.74
1.0362
33.94
69.69
48.28
47.72
1.3429
31.46
45.95
57.43
+LoTA
44.29
1.0258
34,45
68.99
47.79
47.15
1.3534
31.92
45.49
57.41
+DiFT(ours)
46.32
1.0557
35,86
70,93
49.55
49.81
1.6362
31.81
44.55
65.81
Continual Maatiual-Matt-Logice
10.99
0.6433
31.30
67.90
21.15
4.62
L.0365
29.26
40.29
16.94
*HFT
11.06
0. 0.6682
31.55
67.52
21.31
6.57
0.9902
28.48
43.16
17.53
+LoTA
10.89
0.6749
31.87
66.84
21.38
6,70
0.9803
28.76
42.51
17.73
+DiFT(ours)
0.6919
31,23
21.30
29.20
42.92
Evaluation We choose the pass rate (code) and O-shot accuracy to evaluate the performance of
LLMs, details are in Appendix B.5 As our purpose is to reserve benefits and mitigate conflicts, we emphasize the performance of involved tasks, therefore, we use the average target accuracy (ATA) to better show gains and drops of target/historic abilities compared to base LLMs, which reflects various methods better. For math and logic, we compute the (math + logic) 2 accuracy as the ATA score, we also multiply the code pass rate by 50 to align with other metrics (non-weighted ATA in (ppendix(F2)... Computing costs: DiFT conducts merely scall-scale SFT with 1k training data and inference a few samples, whose computing costs lighter than task-vector and gradient-based methods. More computation cost and complexity details are in Appendix B.3 and B.2
5.2 MIX-UP SFT
Table 2 presents the mix-up SFT results, we can observe the extreme difficulty baselines to improve the vanilla SFT. In contrast, DiFT consistently improves the ATA, i.e., the averaged target performance, and outperforms baselines across benchmarks and LLMs. Concretely, in the Mix-Mat M-MthhCCod, know that these 2 reasoning abilities benefits more, so its ATA gain of DiFT is not striking, even if it beats the baselines. While Mistral-7B fails to achieve mutual benefits much with vanilla SFT, the 2 tasks gain more (from 52.87 to 54.80) with DiFT. In
benefit each other. In Llama3- 8B, the math reasoning
Loggi, DiFT on both 2 models can improve involved reasoning abilities.
However, we notice that it hurts the math of Llama3-8B and the commonsense of Mistral-7B, which results from the Mix-Cooe Code-Logic not considering the DSR parameters of the math rea- soning. Considering math and commonsense, issues like this can be eliminated, as in Figure 7 Mix-Logic-CSS is similar to Mix-Math-Code, albeit the vanilla SFT has mutual benefits in Llama3-8B, the proposed DiFT still can enhance their ATA performance, as for Mistral-7B, the vanilla and all baselines trade the logic ability for commonsense, DiFT maintains more LogiQA2 accuracy (31.68%) and obtains higher CSQA accuracy (74.45%). achieving diverse performance.
In multiple mix-up SFT experiments, DiFT can maintain and facilitate mutual benefits and al- leviate conflicts between reasoning capabilities significantly, thereby supporting the effective- ness of the earlier DSR analysis. We also found that math and code tasks are somehow synergistic while logic and commonsense are conflicting, which is interesting. The math- code synergizing may come from the fact that the two tasks share similar computation back- grounds, providing more views for LLMs to understand the reasoning process, and such Mi M--Math-oode tuning has been utilized in math- and code-specific L.MMs training (Shao)
8

---

Published as a conference paper at ICLR 2026
et al. 2024 Hui et al. 2024a). In contrast, logic tasks need to obey strict and complex rules, while commonsense tasks are more about ground knowledge and simple reasoning, leading to conflicts between the two tasks (Song et all (024). We also conduct more mix-up experiments involy-
6SMIK
XGLUE
LogigA2
ing 4 and 5 tasks to illustrate the consistency of
+3.350
-0.0
the DiFT. More details can be referred to Ap- pendix F.5
28-
5.3 CONTINUAL SFT
227 10TTUU
2311
The bottom of Table 2 manifests results of con- tinual SFT, where models are fine-tuned sequen- tially and need retain the previous reasoning proficiency while adapting to new ones. As men- tioned in Section 3.I reasoning benefits and conflicts exist along with catastrophic forgetting, not dominant but still important. In Continual-Math- Code, DiFT learns code ability better and keeps more math reasoning on both L.LMs, resulting 2.62 and 1.23 ATA improvements. As for Cont -Matl-Math-
ConchualMaatrCode
Contruni-Math-Loge
Figure 3: Joint DiFT-SSR for conflicts after alle- viating forgetting.
DiFT on Llama3-8B enhances ATA compared to vanilla, but under- performs baselines for mitigating forgetting. In contrast to Llama3-8B, DiFT on Mistral-7B performs better both the historical math and logie, achieving 3.12 ATA improvement, and such difference between 2 LLMs illustrates more conflicts Mistral-7B. while more forgetting in Llama3-8B. We further implement more continual baselines in Appendix [44
We check the outputs of Cont i waaetiinnaaaaaaaa-aa -Math-Looi c on GSM8K test cases, and notice the perfor- mance drops do not come from the ability destruction but the format issue, and we put an example to explain that in Appendix [B.6] To diminish the formatting issue and further evaluate compatibility between the confficts-orientedd, parametric DiFT and the forgetting-orientee
data-driven SSR (Huang et al. 2024) mentioned in Section 4 we further conduct extensive data-driven continual SFT experi- ments with toint-DiFT-SSR. In Figure we can see that DiFT can continue decrease conflicts after mitigating the forgetting in both Continual illustruting that DiFT is orthogonal to datu-driven methods.
and Continus -Math-
Table 3: SFT performance comparison between Table 4: Inverse DiFT comparison on Llama3-8B
the base and Distruct-LLMss
under mix-up and continual settings.
Model
LogiQA2
CSQA
Settings
Mix
Xath-Code Continuall
Llamu3-8B-Ins
31.55
76.09
DiFT
inverse DiFT
DIFT
inverse DiFT
Logic-only
78.57
GSMBK
67.02
61.26
46.32
25.17
CSQA-only
31.82
81.33
xGLUE
1.0355
0.9561
1,0557
0,9512
Mix-I Logie-CCie CSQA
32.64
78.57
LegiQA2
32.63
35.86
34,54
DiFT (ours)
34.4((+1.84) 80.23(+1446)
CSQA
68.39
70.84
70.93
69.94
5.4 BASE AND INSTRUCT LLMs
Massive high-quality data is necessary for training base LLMs towards instruct LLMs, current LLMs take hundreds of thousands and even millions of data from multiple tasks to perform SFT. However, the computing costs and complexity are out of this work's 's scope if reproducing such process completely. Although we cannot reproduce the whole process from base to instruct L.L.s, our strategy can adapt to the instruct models, as instruct LLMs still contain reasoning benefits and conflicts. To validate that, we conducted the same 20k SFT experiments as we did on the base LLMs, results are shown in Table 3 We can observe that the logic and CSQA results are significantly improved over the vanilla SFT, suggesting DiFT can also perform quite well on instruct LLMs, demonstrating that DiFT can also perform quite well instruct LLMs as on the base LLMs. We further conduct DeepSeek- -1-like reasoning data to validate the DiFT on the long CoT setting, with Llama3-8B-Ins Instruct 1k training data, and results illustrate the DiFT neither limited by long or shori CoT reasoning formats nor the base/instruct models, more details are in Appendix F.T
5.5 NECESSITY OF DSR PARAMETERS
Incorporating new reasoning abilities with identified DSR works well under both mix-up up and continual SFT settings. We also wonder whether the other parameters achieve nearly the performance, thus,

---

Published as a conference paper at ICLR 2026
further conduct inverse DiFT, i.e, exchange the freezing positions of the original DiFT. Concretely, we fine-tune the DSR parameters while freezing others in the continual SFT, as for the mix-up SFT, fine-tune the others while freezing DSR parameters, test whether the other parameters can learn the same reasoning abilities. Llama3-8B. We
the performance of DiFT and inverse DiFT with that in the mix-up experiments, learning some reasoning abilities with fewer related parameters would not lead to model collapse, while still incomparable for target abilities with DiFT. As for the continual SFT, the historic reasoning proficiency forgotten catastrophically, albeit it works well on others, demonstrating that the identified DSR parameters are indispensable for target reasoning abilities, which also validates the correctness of our analysis and the proposed DiFT.
CONCLUSION
In this work, first discover mutual benefits and conflicts among various reasoning tasks through mix-up and continual SFT experiments with several LLMs. Then we explore such phenomena by presenting a novel delta-scale row analysis approach, and we compare fine-tuned and base LLMs during inference, finding that specific groups of parameters are crucial for distinct reasoning abilities. Inspired by that, we propose a novel DiFT strategy update the parameters differentially based on their optimizing directions. We conduct dozens of experiments with several LLMSs on task combinations, and consistent experimental improvements demonstrate that the proposed DiFT can preserve benefits and mitigate conflicts to achieve better diverse reasoning capabilities.
LIMITATIONS
Although our proposed delta-scale row analysis and the proposed DiFT have been demonstrated to effective via extensive experiments, there is no proof to support them theoretically. Due to hardware limitations, only conducted experiments on 7/8B and 14B LLMs in this paper, lacking contrast to the mix-up SFT,
validation on larger-scale (30B+) models that can be complementary. while the continual SFT can alleviate some conflicts between reasoning tasks, we cannot address the catastrophic forgetting, which is the main cause of the huge performance drop.
Additionally, recent MoE LLMs like DeepSeek- V3 and Kimi-K2 are of impressive performance numerous tasks, while they tend to be computationally heavy, normally 671B-A37B, ITB-A32B, smaller ones are still 8x7B, 8x22B. The inference process of all MoE models is of unstable actívation parameters given the router module, while our delta-scale rows analysis needs to compute all corresponding activations of LLMs with the same activation distribution, making it infeasible for further analysis. Nonetheless, the activation instability of MoE LLMs is a fantastic topic, we will investigate this research meticulously in the future.
ACKNOWLEDGEMENT
This work is supported in part by the Alibaba Research Intern Program, the National Natural Science Foundation of China (No.61872161), the International Science and Technology Cooperation Program of Jilin Province (No NN..20240000266GHH), and the Key Technology R&D Program of Suzhou City (No.SYG2025096..
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
BAAI. Infinity instruct. arXiv preprint arXiv:2406. 2024.
Ning Bian, Xianpei Han, Hongyu Lin, Yaojie Lu, Ben He, and Le Sun. Rule story, which is a better commonsense expression for talking with large language models? arXiv preprint arXiv:2402. /4355, 2024.
10

---

Published as a conference paper at ICLR 2026
Meiqi Chen, Yubo Ma, Kaitao Song, Yixin Cao, Yan Zhang, and Dongsheng Li. Learning teach large language models logical reasoning. arXiv preprint arXiv:23 2023.
Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models. In Forty-first International Conference on Machine Learning. 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
Wenlong Deng, Yize Zhao, Vala Vakilian, Minghui Chen, Xiaoxiao Li, and Christos Thrampoulidis. Dare the extreme: Revisiting delta parameeer pruning for fine-tuned models. In The Thirteenth International Conference on Learning Representations, 2025.
Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. How abilities in large language models are affected by supervised fine-tuning data composition. arXiv preprint arXiv: 10.05492, 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint arXiv:2407.2 1783, 2024.
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. Transactions of the Association for Computational Linguistics (TACL), 2021.
Zi Gong, Hang Yu, Cong Liao, Bingchang Liu, Chaoyu Chen, and Jianguo Li. Coba: Convergence balancer for multitask finetuning of large language models. arXiv preprint 1rX1v:2410.06741, 2024.
Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Wu, YK Li, et al. Deepseek-coder When the large language model meets programming-the rise of code intelligence. CoRR, 2024.
Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang. Xinting Liao, Linfeng Song. Junfeng Yao, and Jinsong Su. Mitigating catastrophic forgetting in large language models with self-synthesized rehearsal. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1416-1428, 2024.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409. /2186, 2024a.
Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Weiran Xu, Yu Sun, and Hua Wu. Hft: Half fine-tuning for large language models. arXiv preprint arXiv:2404. 18466, 2024b.
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint 1rXii:2212.04089 2022.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
Fanshuang Kong, Richong Zhang, and Ziqiao Wang. Activated parameter locating via causal intervention for model merging. arXiv preprint A4Xv:2408.00485, 2024.
Hanmeng Liu, Jian Liu, Leyang Cui, Zhiyang Teng, Nan Duan, Ming Zhou, and Yue Zhang. Logiga 2.0 an improved dataset for logical reasoning in natural language understanding. IEEE/ACM Transactions Audio, Speech, and I Language Processing, 2023a.
11

---

Published as a conference paper at ICLR 2026
Hanmeng Liu, Zhiyang Teng, Leyang Cui, Chaoli Zhang, Qiji Zhou, and Yue Zhang. Logicot: Logical chain-of of-thhttt instruction tuning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp.
2023b.
Renze Lou, Kai Zhang, Jian Xie, Yuxuan Sun, Janice Ahn, Hanzi Xu, Yu Su, and Wenpeng Yin. Muffin: Curating multi-faceted instructions for improving instruction following. In The Twelfth International Conference on Learning Representations, 2023.
Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou, # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Twelfth International Conference on Learning Representations, 2024.
Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXhv:2102.04664, 2021.
Yun Luo, Zhen Yang. Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308 08747, 2023.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. In EMNLP, 2018.
Shervin Minace, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey, arXiv preprint arXiv:2402.06196, 2024.
Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 3806-3822 2023.
Ashwinee Panda, Berivan Isik, Xiangyu Qi, Sanmi Koyejo, Tsachy Weissman, and Prateek Mittal. Lot- tery ticket adaptation: Mitigating destructive interference in llms. arXiv preprint arXiv:2406. 16797, 2024.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. Explain yourself! leveraging language models for commonsense reasoning. In Proceedings of the 2019 Conference of the Association for Computational Linguistics (ACL2019), 2019. URL https: //axxiv/
orq/ abs/ 1906. 02361
Vinay Venkatesh Ramasesh, Aitor Lewkowyez, and Ethan Dyer. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations, 2021.
William Rudman, Catherine Chen, and Carsten Eickhoff. Outlier dimensions encode task-specific knowledge. arXiv preprint 110i1:2310.17715, 2023.
Maarten Sap, Hannah Raskkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv: 09728, 2019.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Youngrok Song, Gunhee Cho, HyunJu Kim, Youngjune Kim, Byung-Chull Bae, and Yun-Gyung Cheong. A conffict-embedded narrative generation using commonsense reasoning. In Proceedings of the Thirty-Thic Third International Joint Conference on Artificial Intelligence, pp. 7744-7752, 2024.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsensequ: question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00933 2018.
Anke Tang, Li Shen, Yong Luo, Yibing Zhan, Han Hu, Bo Du, Yixin Chen, and Dacheng Tao. Parameter-efficient multi-task model fusion with partial linearization, In The Twelfth International Conference on Learning Representations, 2024.
12

---

Published as a conference paper at ICLR 2026
Teknium. Openhermes 2.5: An open dataset of synthetic data for generalist Ilm assistants, 2023. URL htttts://ffmmirreeee com/ /ttp///attttetettu///o/eeeeeeee 2mee
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: theory, method and application. IEEE Transactions Puttern Analysis and Machine Intelligence, 2024.
Zhengnan Xie, Sebastian Thiem, Jaycie Martin, Elizabeth Wainwright, Steven Marmorstein, and Peter Jansen. Worldtree v2: corpus of science-domain structured explanations and inference patterns supporting multi-hop inference. In Proceedings of the twelfth language resources and evaluation conference, pp. 5456-5473, 2020.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024.
Longhui Yu, Weisen Jiang. Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309. 12284, 2023.
Mengxia Yu, De Wang, Qi Shan, and Alvin Wan. The super weight in large language models. arXiv preprint arXiv:2411.07199 2024.
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprini arXiv:2309.05653, 2023.
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning. In The Twelfth International Conference Learning Representations, 2024.
Biao Zhang. Zhongtao Liu, Colin Cherry, and Orhan Firat. When scaling meets Ilm finetuning: The effect of data, model and finetuming method. In The Twelfth International Conference on Learning Representations, 2024.
Zhong Zhang, Bang Liu, and Junming Shao. Fine-tuning happens in tiny subspaces: Exploring intrinsic spec-fp subspaces of pre-trained language models. arXiv preprint arXiv:2305.11446, 2023.
Zirui Zhao, Wee Sun Lee, and David Hsu. Large language models as commonsense knowledge for large-scale task planning. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2026
A LLM USAGE CLAIM
In this work, we employ LLMs for polishing the whole paper's writing, including: checking the grammar and syntax issues, polishing some expressions, and nothing for other usages.
B IMPLEMENTATION DETAILS
B.1 FORMAT AND HYPERPARAMETERS
For all the SFT experiments (vanilla, mix-up, continual SFT), we adopt learning rate=2e-5, max length=2,048 (16,384 for LongCoT data), batch size=256, warm-up ratio=0.03, weight decay=0.1, 1, max gradient norm=1.0, and we employ DeepSpeed Zero2 as the accelerate framework for gradient interference convenience in DiFT. In LoRA experiments, we adopt lora rank=8, lora arraaapha=32, mosettesoduussallll all--inear', learning rate=le-4, and the rest of the hyperparameters are the same as the full palaameeee SFT. All the hyperparameters are widely used in SFT practice, and with these hyperparameters, we can ensure that all the model training converges, and we display some training loss variations to in Figure to illustrate that. Besides, we use the same seed (42) during the dataset shuffle to make the comparison fair.
Table 5: Performance comparison of different models on reasoning tasks.
Table 6: Performance comparison of different models reasoning tasks.
Model
Math
Code
Logic
CSQA
Llama3-8B
0.93
0.94
0.91
0.93
Mistral-7B
0.94
0.96
0.93
0.94
Qwen2.5.
0.97
0.96
B.2 COMPUTING COMPLEXITY
To compare the computation complexity of our method and similar approaches, we analyze and formalize the variables of these methods. Let be the parameter scale, B be training data scale per task in full SFT, N be the task number, and let forward/backward each cost ((P) FLOPP per token.
DiFT pipeline. Our DiFT consists of 3 phases, small-scale Pre-SFT, DSR computing, and DiFT training. Here are the corresponding computing complexity of each phase:
Pre-SFT:
We only train on a small probe subset Berobe B v.s. 20k), which empirically yields almost identical top-DSR sets as full SFT. The Complexity is: O(N, Bprobe, P).
forward-only passes of base vs. probe model on tiny calibration sets (5 groups of samples) and accumulate row-wise squared activation differences. This is O(N, M, P) w. M < B, which is negligible compared to full SFT.
DSR computing: For each task,
DiFT training:
Finetuning only a small scale of DSR params compared to standard SFT. Complexity is O(N, B, P), i.e., equal to standard SFT down to a < 1.0 constant.
Total overhead v.s. vanilla SFT is thus ( Broaee + M)P) with Bpmiee M & B; in our setting this is only a few percent of the full SFT FLOPs.
Fisher diagonal Hessian EWC Diagonal Fisher / EWC. To estimate diagonal Fisher, EWC scans the task data and accumulates squared gradients for each parameter, requiring an extra backward pass over the Fisher dataset. Complexity of each task is OBBrster P) with a full backward, typically Brisher B. So the overhead is the order of another full epoch of SFT: ONN, B, P). substantially larger than our (Bpobe + .)P Memory also stores a full diagonal Fisher vector of size (PP).
Diagonal Hessian. Diagonal-Hessian Hessian methods approximate curvature via Hessian-vector products or Hutchinson-style probes, each costing zs 2 gradient passes. With probes, this is O(K, B, P) FLOPs per task (K 1), i.e., multiples of EWC's cost and far above small forward-only DSR.
14

---

Published as a conference paper at ICLR 2026
Other gradient-based importance schemes PackNet. Requires iterative pruning retraining cycles for each task: train prune- retrain, potentially repeated several times, yielding x (, with J > 1 effective full passes, which is much heavier than our forward analysis.
LWF. For each mini-batch, LwF requires a teacher model forward alongside the student and compute distillation loss. Complexity per step is roughly doubled the forward side: O P) extra FLOPs per batch over the full training trajectory.
In summary, under identical SFT budgets ON,, B, P): DiFT add only O(N M) with Bprobe, M & B. EWC add O(N, B. P) via extra backward passes. Diagonal Hessian methods can add O(N, K; B, P). K > 1. PackNet likewise incur at least one extra SFT-scale pass extra forward per step. Thus, for LLM-scale models, DSR offers importance estimates at fraction of the FLOPs required by Fisher/Hessian-bart training loop as simple as vanilla SFT.
or distillation-basee baselines, while keeping the
B.3 COMPUTING COSTS
In investigation, we noticed that the DSR distribution is not that affected by training data scale: SFT with only a small proportion of data (1k) shows very similar distribution as shown in Table We can see Ik-SFT and 20k-SFT share most top-DSR parameters, implying the robustness of the DSR analysis. Therefore, we merely SFT LLMs on a little scratch of data to identify DSR instead of the entire data, then conduct DiFT experiments with fewer budgets. With such results, we only conduct small scale SFT, unlike task-vector arithmetic methods that need to full-scale SFT in preparation, and can obtain quite consistent distributions of DSR parameters.
Based on the above finding, to identify the sensitive weights, we merely SFT LLMs on little scratch of data instead of the entire training set. After getting the small-scale SFT, we load the fine-tuned LLMS and corresponding base L.Ms through randomly se- lected 50 training samples via the proposed analysis method, inference for identifying DSR parameters on 1 NVIDIA A100 GPU, as shown in Table 7 Each group in the analysis only consumes around 30GB CUDA memory for =900 seconds on 7B/SB models, and around 62GB for = 1,200 seconds on the 14B model, in- dicating the cost of computing DSR parameters is negligible compared to the naive I.I.M infer- ence. During SFT, we employ 8-A servers (one server can conduct all experiments in this work) and employ fixed batch size and max length to utilize the GPU efficiently. At last, we compute the Duruioo and DSRaiff f. and employ these parameter-task information to differentially finetune LLMs, and the finetuning costs are similar to other methods in this phase. Therefore, the computing costs of the proposed DSR is lighter than tesk-vector approaches. When it comes to the training process of DiFT, we randomly selected 100 DSR parameters, it only takes about 3.7% parameters of the entire model, i.e. we need to finetune about 92.3% parameters in continual SFT, while only 3.7% or less in mix-up SFT.
Table 7: CUDA memory usage and inference time for different models.
Model
CUDA Mem (CB) Time (second)
Jamas Mi/Maall
30
900
Qwen2.5-148
65
1,200
In summary, although the pre-SFT with Ik data is an extra step compared to vanilla SFT, however, such step introduce quite limited resources: Ik (1/20 than full) Llama3-8B SFT experiment consumes merely 1/20 time costs (6 minutes on BxA100), and inference on the same serve needs less than minutes. Therefore, the "probe" stage introduces minutes, given the full SFT on the same device demands 120 minutes, which is 15 times of the "probe" step, the "probe" complexity is marginal. With the extra 1/15 cost, we can produce an efficient SFT of merely 3%-5% trainable parameters, saving much time and computing than the full SFT. More importantly. our DiFT strategy provides better interpretability by locating task-specific DSR parameters, and achieve superior diverse reasoning performance through mitigating conflicts in vanilla SFT.
B.4 DATA COMPOSITION
For math and code reasoning, select 20,000 training samples from math and code Infinity Instruction data BAAT (2024), respectively, which consists of various math and code data as shown in Figure for logic reasoning, we sample the same amount of data from LogiCoT Liu et al. (2023b);
15

---

Published as a conference paper at ICLR 2026
pasgiefar
55,07
recnoaityy Momo 220k
248 161
Repitto du bagel Jummee22
228.234
Dutaset Source
Jetraan aatranyyhhanee oythan-ee ded
pyther
mheepiCeedoo huadback
mhe Cedeefi ta en
nictrsshiii instrucc Code Cade-- B3l--4
Toker Bander Instructiona
29.106
4,241
byrtaniniyttorn.c code actionn L8K, abaica
2.53
10c
Number of Rows (log scale)
Figure 4:
Data composition Infinity-Instruut-MM after de-duplication. Each bar represents the number of instruction rows contributed by each source dataset. The x-axis uses a logarithmic scale to accommodate the wide range of sizes.
as for Commonsense reasoning, we gather Commonsense(AA Talmor et (2018), CoS-e Rajani et al. (2019), O Mihaylov et al. (2018), SocialIQA Sap et al. (2019). Strategy Geva et al. 2021), WorldTree Xie et al. (2020). As introduced in Section 5.1 we collect training data from available and popular reasoning datasets, and we use the "query", "response" format for training.
B.5 EVALUATION
Since the outputs of math, logic, and commonsense reasoning either number option, use GSMSK Cobbe et al. (202T). LogicQA2 Liu et al. (2023a), and CommonsenseQA Talmor et al. (2018) evaluation benchmarks, respectively, and adopt the accuracy of 0-shot common metric. For code reasoning, we use the pass rate on CodeXGlue Lu et al. (202I) to test whether the generated codes can pass. We employ the official Jas the base repo for eva evaluation, and the results fluctuations for the same benchmarks were of a limited range, so we report their stable accuracy. We evaluate checkpoints in each training process and take the best for results report.
B.6 CATASTROPHIC FORGETTING FROM FORMAT ISSUES
In Tables and
the math reasoning of Continual-Math-Looe however, didn it drop much in Continual-Mann Code, illustrating the Continual-Math--Logge may encounter special issue. To investigate the cause, conduct case analysis, and find that after logic data SFT, the response of the model for a GSM8k question is as follows:
damaged severely,
Q: raspberry bush has 6 clusters of 20 fruit each and 67 individual fruit scattered across the bush. How many raspberries are there total?
A: The clusters have 20 raspberries each, so there are 6 clusters. Therefore, the first part the answer yes. Now, we have
imneevaluation-hmmess http://sws..tt.ooooooo/ooooo///s
16

---

Published as a conference paper at ICLR 2026
to add the individual raspberries to the total. So, the total number of raspberries is clusters 20 raspberries per cluster + 67 individual raspberries. Therefore, the answer is sest<|eot |d leot id|><|eot id]></eot. id/>122
The Continual-Math-Le
finetuned LLMs tend to output LogiCoT-format "yes" or "no" after regular CoT reasoning steps, the reasoning steps are totally correct (near reaching the correct answer 187), while it ends with oufputting guessed number as the final answer, meaning that the math reasoning ability is almost not affected after DiFT in continual SFT, but the output format was modified during finetuning. Therefore, we employ data mixture in continual SFT alleviate the format issue with rehearsal-basee method then perform conflicts mitigation as in Section 5.3
rg
1.0
14
Yellonl
32
Tuaning Sies
Tekning Siez
Tueein Shop
(a) Math training loss.
(b) Code training loss.
(c) Logic training loss.
Thaining Loaa
math
rode
24-
2.2
mntc
1000
0.4
Training
(d) CSQA training loss.
(c) Joint training losses.
Figure 5: The training losses including vanilla math, code, logic, CSQA SFT, and DiFT for mix-up and continual learning.
C DIFT ALGORITHM
17

---

Published as a conference paper at ICLR 2026
Algorithm Delta-Scale Analysis of Fine-tuned Language Models
Input: Base LLM Mbases fine-tuned models M M
M K-
evaluation data
ft
Do, Di,.., - sample size N, top dimensions C
Output: Delta-scale row scores for each model and layer for to K do
Sample N data points from Dy: Sk D₁
H = Register forward hooks on linear layers of M ft
Haase Register forward hooks on linear layers of Maose
DSR {}
for x in Sx do
outy (a(
outbase Mbase
for h2. hbase in Ha, Hbuse) do
hgd batch(inpk, outy)
hbase.add add batch(inppaaee outbase)
compare the differences between Mt. and Muase
hg.update wh..paae(hhaee.ingg inb, hbase.out) end for end for for h in H₁ do
scaler valless hhscallerrows rows top indicess sessort(scalerr.llussss]] DSR, = DSR (scaler.values(top malues(top_indice)) end for end for return DSRi DSR2, DSRk
A Mix-up SFT
DSRunion = = K-1 DSR
freeze parameter in Mo DSRunion
fine-tune Mo U Dr
K-1
\\ Continual SFT
for 1 to K do
DSRaiff = DSR, (U)T))) DSR.j)
freeze all parameters in Mr except in DSR.difS fine-tune M/-1 on D3 to obtain Mp
end for
18

---

Published as a conference paper at ICLR 2026
D ABLATION STUDIES ON DSR NECESSITY AND SCALE
D.1 FURTHER DISCUSSION OF DSR PARAMETERS
Our DSR "sensitivity" is defined per row & of a named parameter as St. E, |4, the expected change in that row's 's output after task-oriented SFT. For a linear row Y = = AY, = (WT
W Whbsee X = & X2- If input statistics are approximately fixed during small-step SGD, s2 RJ 1 F X I DSR is thus proportional to the squared parameter change of row (k). With
JAW
SGD, AW; = n Ze g giving TE| & 1 E, EllgE [lg. i.e., integrated gradient energy. For log- log-likelhhood models, E 19 equals the diagonal of the Fisher Information Matrix, the standard notion of parameter importance used in EWC, SI, MAS, etc. Therefore, DSR can be viewed as a special row-granular Fisher/mpportancelikk estimator: repeatedly used to reduce the task's loss, which explains why their partitions correlate with specific reasoning abilities.
rows with large DSR are exactly those
We use rows, instead of columns or individual elements, because a row W3 defines one neuron/channel yh Waæ, which is natural functional unit that downstream non-linear and mixing treat coherent feature or circuit. Columns mix into all neurons and single weights are not meaningful units. Row-wise aggregation also reduces variance versus element-wise scores and yields clean, structured masks (channel-level sparsity) that are easy to implement. So row-wise DSR combines a clear Fisher-based theory with stable, interpretable, and practical units of specialization.
The reason why LLMS tend to encode different task capabilities into distinct parameter subsets is an important and interesting phenomenon to be investigated, and that there is not yet solid theoretical explanation for this mechanism. In this paper, our research goal is to identify and characterize the DSR phenomenon and demonstrate its practical implications for mitigating task conflicts happened during SFT. We did not claim have fully developed theory for LLMs encode different abilities into discrete parameter subsets. Rather, we position DSR as an theoretically inspired and empirical discovery consistently observed across different models and tasks, and as a starting point for further thcoretical work. We have revised the paper to state this more explicitly and avoid any potential confusions. Though we do not aim provide complete theory for the L.MMs ability-subset phenomenon, DSR is not purely descriptive. We offer several pieces of evidence that go beyond surface-level observations, including the accumulated gradient-basse derivations in Section 3.2 the DSR distribution varations analysis in Section 3.3] Distinct reasoning tasks have different DSR parameters. To illustrate this more intuitively, we further statistic the overlap rate between task combinations as follows:
Table DSR overlap rates among reasoning tasks.
Comb.
math-code
math-logic
math-esqa
code-logic
logic-esqa
Overlap (%)
43,2
31.1
20.7
31.1
20.7
where only math and code share nearly half parameters, and other combinations have 1/5 to 1/3 common DSR at most, indicating distinct reasoning tasks depend on different parameter subsets, which is in line with distribution variances in Section [.3] In Section [.5] the single-task DiFT experiments illustrate that full-SFT math is on par with DiFT math (61.64 v.s. 59.36), so does full-SFT and DiFT-code 12.3 v.s. 1.1097), which can demonstrate that task abilities are encoded in corresponding DSR parameters.
Although researchers always seek to understand the mechanism of LLMs, given the black-box nature and complexity of LLMs, many common circumstances, such as in-context learning and zero-shot prompting, still lack developed theoretical supports. However, we can also conduct research on these topics, Task Vector aimed to steer the behaviors of pre-trained LLMs, Function Vector discovered in-context learning tasks related parameters. These works validated the markmarrreteer concurrence phenomena via experiments, our DSR conforms similar research paradigm, either.
During our investigation and analysis, we have a deeper guess for reasoning tasks interference:
Math, code, and logic reasoning abilities mainly need to activate symbolic, algorithmic cir- cuits, such as precise composition, intermediate-stan tracking, and long-range dependencies. However, they may emphasize specific or combination, leading the conflicts.
19

---

Published as a conference paper at ICLR 2026
Commonsense reasoning and other similar tasks guide the LLM toward semantic, world- knowledge paths, like association, prototype matching, and heuristics, which demands totally different basic abilities.
Given these functions coexist in the same MIP channels and attention heads, their gradients some- times pull weights in incompatible directions. DSR exposes where this happens, and we believe that understanding the exact circuits why is an exciting direction for future mechanistic work.
D.2 DSR CAUSALITY
To go beyond inverse/random-Dif ablations, we perform cross-task DSR transplant between the math-tuned and esqa-tuned Llama3-8B models. We copy only the top- DSR params for math into the esqa-tuned model (esqa-sub-math) and symmetrically, the esqa DSR params into math-tuned model, results are as follows:
Table 9: DSR substitution performance of math and CSQA.
model
GSM8K
CSQA
math-sft 61.64
67.24
esqa-sft
8.79
79.36
math-sub-csqa
41.02
75.72
csqa-sub-math 22.21
71.48
We can observe that injecting csqa-DSR into math degrades GSM8k while improving CSQA, and replacing esqa model with math-DSR enhances GSM8k but hurts CSQA. Such results constitute causal intervention: moving DSR params diverts the corresponding reasoning ability and introduces interference when transplanting into other tasks.
To test whether our union rule is an arbitrary heuristic, we perform a weighted-union variant: starting from the Llama-3-8R Basaase, we scale rows in the math-logie DSR union by 1.5 and other rows by 0.9 ("scaled- seeaalddbbse-deee wow-d--s--oooma-----eee
Table 10: DSR union parameter scaling evaluations between math and logic.
model
GSM8k
xGLUE
LogiQA2
CSQA
Llama3-8B
39.42
1.0874
31.93
69.29
vanilla-math-sft
61.64
1.2228
30.73
67.24
vanilla-logic-sft
30.17
0.6880
37.02
72.89
64.37
1.2092
32.32
70.52
scaled-based wwwdddsio-onnnaanee math-logic
42.30
1.0918
32.38
69.53
In Table 10 the simple re- ee-weighting already shifts ability toward math/logic (GSM8K 39.42->42.30. LogiQA2 31. 31.93, ....33-322 -32.3888 with only minor changes on XGI.UF/SSQA, confirming that the DSR union indeed concentrates task-relevant signal. Moreover, the performance remains between vanilla base and full mix-SFT, indicating that the union strategy a stable rather than a simple heuristic choice.
D.3 NUMBERS OF DELTA-SCALE ROWS
The scale of DSR parameters are rather important, from which we can identify the reasoning- weights, intuitively, the more samples employed during model inference, the more comprehensive the location. To figure out that, we try various numbers DSR parameters to freeze and conduct corresponding ablation studies. In Figure we choose 20, 50, 100, and 200 as top numbers to locate the top DSR parameters and then conduct mix-up and continual SFT, and inference on the GSM8k to evaluate the math ability. The results indicate that when it increases from 20 to 100, the math performance gradually gets better, however, it drops when we adopt the first 200 rows, showing that critical parameters for task might be very limited.
20

---

Published as a conference paper at ICLR 2026
Table 11: Performance comparison of DiFT and DiPTTrandon on Mix-Math- and
Continual Maatal-Math-Coon settings.
Model
GSM8K
XGLUE
1amaa-88-mix--athhe +DiFT
64.82
1.0956
67.02
1.0735
+DiFT_random
62.35 (-4.67)
1.0294 (-0.0662)
Haama--8--coninuu--malllt
44,35
0,9902
+DiFT
46.32
1.0557
+DiFT random
42.46 (-4.03)
0.8792 (-0.23365)
Given the above investigation, we compute and choose 100 rows in main DiFT experiments, the DSR of each task in different LLMs maybe dif-
ferent as exhibited Figure T0 (also Figures| [3] I2 in Appendix (. Suppose randomly selected 100 DSR parameters, only takes about 3.7% parameters of the entire model, i.c. we need finetune about 92.3% parameters in continual SFT, while only 3.7% or less mix-up SFT.
410
395
22
D.4 RANDOM DSR WITH THE SAME SCALE
(a) Mix-up SFT
(b) Continual SFT
Figure 6: The effect of delta-scale row numbers on different reasoning models.
Apart from the inverse-DiFT Section 5.5 for validating the necessity of top-DSR, we further evaluate scalescale random DiFT to make the DSR's necessity more convineing. We conduct random DiFT (of the same rows) experiments in both mix-up and continual SFT, results in the Table 11
that the random DiFT underperforms the top-k DiFT' to a large margin, which strengthens the importance of top-DSR, and this is in line with the phenomenon as Section 5.5
D.5 CONFLICTS AND REGULARIZATION IN CONTINUAL SET
We denote the math task as A, the code task as B, we then conducted math-only and code- only DiFT experiments. In Table 12 we can see that A's and B's corresponding DiFT, i.e., DiFT-math and DiFT-code, can both enhance their own performance compared to the base LLM, but cannot surpass their vanilla SFT (SFT with 1) performance correspondingly. In the mean- time, notice that full-SFT code underper- forms DiFT-code (SFT with Sg) on math largely (26.54 v.s. 51.18), illustrating that a greater negative impact on SA than SB (in- tuitively, the scale of SB is much smaller than 1 - SR). Therefore, we freeze SB - SA in the mix-up setting to mitigate its conflicts with SA
Table 12: Performance comparison of base model, full-SFT, and DiFT on GSM8K and xGLUE.
SB have
Model
GSMSk
xGLUE
base model
39.42
1.0874
full-SFT math
61.64
1.2228
DiFT-math
59.36
1,2393
full-SFT code
26.54
1.1203
DiFT-code
51.18
1.10
D.6 ANOTHER CONTINUAL SET STRATEGY
Additionally, we compare 3 settings in -atth-ooo--tt--oeeeee SFT experiments, we still denote the math task as A, the code task as B. the 3 settings are: (1) SFT all parameters (SFT with 1). (2) SFT only DSRr for the code task (SFT with SB), (3) SFT only DS Raiss (SFT with 1 SB). As (1) and (3) have been conducted in the main paper, conduct the continual-math-code SFT only Sp for the current task (i.c., DiFT w. DSR
we can see that DiFT w. DSRcu (SFT with Sz) can also maintain the historic math performance (as we freeze SA), while it cannot learn more current code ability compared to the original DiFT (SFT with 1 - SA), and this comparison illustrates that SA ourperforms the Sg. further demonstrating the reasonableness and effectiveness of our method. The results also prove that our proposed DiFT is not the only solution for mitigating reasoning conflicts and reserving benefits,
In Table 13
21

---

Published as a conference paper at ICLR 2026
Pe ffomnanc Comparison of DIFT
(Math-Onde-CSOA)
Cartinual
Malioon Cate Cios 1I Mars Codo CSSA 2017
Hith Oode: Lnk- ca04
-a- Coa 1oth
sea Muth
Hith Oude Lok C90A 201
- Cope Coso 1ith DPT
See Log Muth DET
Figure 7: DiFT performance on Llama3-8B in multiple task mix-up and different order continual setting, involving more task ui and continual experiments of different orders.
other derivative methods of DSR analysis can also work well, and our initiative is proposing the DiFT to validate the DSR analysis, thereby providing insights for future reasoning conflicts and benefits research.
Table 13: Performance comparison of full-SFT, DiFT, and DiFT with DSRour on GSMSK and XGLUE.
Model
GSM8K
xGLUE
full-SFT
44.35
0,9902
DiFT
46.32
1.0557
DiFT DSR
45.75
1.0325
D.7 DIFFERENT CONTINUAL ORDERS
we reverse the learning orders of continual SFT, and we can observe that Seq-Code-Mate and the Seq-Logic-Math perform better than their vanilla SFT counter- parts as the continual SFT performance in the main experiments, showing that DiFT is still effective regardless of training orders. These results highlight DiFT's validity in reducing confticts between historical and new reasoning abilities, paving the way for better diverse reasoning abilities under arbitrary mix-up and continual SFT settings without curated arrangements.
In Figure 7
In the continual SFT, DiFT freezes the DSR parameters of the intersection of the current historical tasks, may raise risk of under-learning Empirical results of vanilla and DiFT continual SFT in I7 that freezing the intersected DSR can acheive better performance that full continual SFT, illustrating the under-learning is slight. And from the derivation of DSR in Section 3.2 if subset of parameters is critical to both historical and current tasks, the magnitude of their updates have already been effectively captured by the DSR algorithm, indicating that these parameters have been well optimized during prior learning phases. Therefore, these parameters under-learn theoretically. We believe that mitigating reasoning conflicts and under-learning depend on specific scenario, our strategy support trade-off between involved tasks: a mask ratio hyperparameter, to control the degree of under-learn and balance the target reasoning tasks.
cause limited
introduce
E 3B & 14B LLM EXPERIMENTS
The proposed DSR is forward-only, architecture-agnoee agnostic statistic, whose definition does not depend on parameter count. Empirically, we already verify the DSR-based DiFT strategy on Llama-3 and Mistral, to extend our approach to more model families, conduct Qwen-2.5 experiments here. The experimental results of Qwen2.5-3B are in Table 14
We that DiFT can also work well Qwen2.5-3B both mix-up and continual SFT settings. We also conduct DiFT experiments with Qwen2.5-14B, and the results are shown in Table [5
The results illustrate that our method can facilitate multiple reasoning abilities, and the DiFT is even better for large-scale models, demonstrating not only the scalability of the DiFT but also the effectiveness of our DSR analysis. Now we have conducted experiments 3B, 7B, 8B, and 14B models, and DiFT improves on all tasks with the model size scaling up on different model families, demonstrating the generalizability of the presented DiFT.
22

---

Published as a conference paper at ICLR 2026
Table 14: DiFT performance under mix-up and continual SFT on Qwen2.5-3B.
Qwen2.5-3B
GSM8k
xGLUE
LogiQA2
mix-math-code
68.01
1.2102
35.69
+DiFT
69.83
1.2213
35.94
mix-code-logic
55.12
1.1997
36.90
+DiFT
61.87
1.2375
37.10
seq-math-code
60.05
1.1910
35.50
+DiFT
61.79
1.1924
35.20
seq-math-logic
62.93 64.37
0.9872
36.70
+DiFT
0.9912
37.60
Table 15: The mix-up and continual SFT results of Qwcnn.55-1B-bile with vanilla and DiFT on the same 4 benchmarks as the main experiments.
Model
GSM8k
xGLUE
LogiQA2
CSQA
ATA
Mix-Math-Codee
0.35
1..11=000062
43.51-0.27
84.2811.31
78.04
+DiFT
86.43-
1.4188-0.0077
44.15-0.34
84.68+00.26
78.69
Mix-Code-Logic
72.10=0.34
0.092-0.0059
47.33-0.30
83.7810.29
50.15
+DiFT
57.16-0.41
1.0925 0.0063
47.44-0.33
83.29++0.32
51.03
Mix-Logic-CSQA
54.06-0.37
1.758-0.0020
40.01-0.27
86.65110.35
63.33
+DiFT
67.78-0.42
1.0910=0.0025
41.38-0.22
87.81-00.32
64.60
Continual Matt-Mattttttto
71.4200.33
1.1322=0.0043
44.53=0.36
82.56=0..28
64.02
+DiFT
0.38
1.1461=0.0441
44.40=0.34
83.46+0..30
68.15
Continual Mathull--ath-ogghht
56.86=0.46
0.7387=0.0044
48.28=0.27
84.36-0..3
52.57
+DiFT
57.70-0.50
0.7620=0.0036
48.35=0.31
84.36=0.35
53.03
In Figure 8(a)
can observe that DiFT's GSM8K accuracy improvements grow with the model size
increases under Cont inuannnl--- MMatal-Mattll h- Code, indicating the scaling law. Besides, the GSM8k accu- racy of DiFT on different model scales and families are consistent and stable in the Mi xiMathh-oode setting, which is shown in Figure 8(b)
F MORE REASONING CONFLICTS AND BENEFITS ANALYSIS
We focus on the reasoning tasks since this scenario is more challenging than others. Specifically, reasoning tasks often require models to perform higher-order cognitive processes such analysis, deduction, and problem-solving, and they usually share a set of numeric/symbolic manipulation contents and consist of multiple deduction steps. Apart from that, distinct reasoning tasks have exclusive reasoning goals. These factors provide a relatively intuitive explanation for the emergence of conflicts. Therefore, concentrate the benefits and conflicts among reasoning tasks this
Base the (Continual SFT) DITT Improvemen: Dats
Significant Scaling:
Besedimw (Mia-up SF]]
DITT
79.6
DIF
+2.20
CEHOK Accuracy
51.4
A Accurno
Jamad 18
6
+1.20
MotullI5
75
(a) DiFT math performance on contiuul-maat-code
(b) DiFT math performance on mix-math-code
Figure 8: The scalability of DiFT across different model sizes and families.
23

---

Published as a conference paper at ICLR 2026
work. To validate our assumption, conducted massive investigation experiments and validated it, and then conducted more detailed analysis.
Table 16: LongCoT DiFT performance comparison on Llama3-88-Instruuc.
Model
xGLUE
LogiQA2
Llama3-8B-Ins
1.2506
31,55
LongCode-only
1.6556
31.11
LongLogic-only
1.3606
33.92
Lix--Lng--cdeellgg
1.5209
31.11
+DiFT
(+0.066)))
32.51 (+1.4)
F.1 DIFT FOR LONG COT DATA
For experiments the main paper, we employed normal CoT data for SFT, compared to the DeepSeek- R1-like reasoning pattern, i.e. Long CoT, our training data can be seemed as Short COT. To evaluate DiFT more complete, we select 1k samples from the code Long CoT data from the RedStar-Reasoning. which is distilled from QwQ-32B, and also Ik logic Long CoT data distilled from DeepSeek-RI Distill-Llama- 70B, we then conducted Long CoT-SFT on Llama3-88 -a-maa---tnsttrett... As illustrated in Table 16 our DiFT still performs better than vanilla mix-up SFT in -ooox--oon--oooo Logic with Long COT training on the instruct I.I,,, especially for code reasoning. demonstrating the DiFT is neither limited by long short CoT reasoning data formats nor the base/insruct models.
F.2 THE NON-WEIGHTED ATA METRIC
The current ATA in the main paper designed describe the balanced performance of distinct accuracy metric, while may cause misunderstanding when it comes to the code task's s pass rate. Here recompute the non-weighted ATA metrics from Table 2 that involving code tasks. We can
Table 17: Performance under the non-weighted ATA.
Model
GSM8k
xGLUE
LogiQA2
ATA
non-weighted ATA
Mix-Math-Code
64.82
1.0956
34.54
59.80
32.96
+DMT
65.07
1.0851
34.54
59.66
33.08
+CoBA
66.21
1.0725
33.15
59.91
33.64
+DiFT(ours)
67.02
1.0735
32.63
60.35
34.05
Mix-Code-Loge
52.31
1.0779
32.57
43.23
16.82
+DMT
50.37
1.0865
31.93
43.12
16.51
+CoBA
51.12
1.0811
32.25
43.15
16.67
(DiFT(ours)
41.09
1.1359
33.40
45.10
17.27
Continual-Maii Math-Code
44.35
0,9902
32.82
46.93
22.67
+HFT
44.74
1.0362
33.94
48.28
22.89
+LoTA
44.29
1.0258
34.45
47.79
22.66
+DiFT(ours)
46.32
1.0557
35.86
49.55
23.69
that matter the non-weighted ATA the former scaled ATA, strategy can achieve the SOTA. Nonetheless, merely the scaled ATA can lead to unnecessary misunderstanding as you worry, to this end, have add the non-weighted ATA to updated manuscript. Also to observe the code-related balanced performance fairly, we suggest to consider the pass rate together with ATA metries to obtain more details.
F.3 COMPARED PEFT METHODS
As intended to locate the related parameters of different reasoning tasks, and then differentially train LLMs with the (almost) full call-parameter SFT. Compared with PEFT methods like LoRA, we merely freeze the gradient backpropagation for the parameters of DSR, and the rest of the parameters are still fine-tuned. Therefore, DiFT does not reduce the fine-tuning time and memory usage compared to full fine-tuning. The cost of DiFT is higher than that of LoRA, which is the cost of computing
24

---

Published as a conference paper at ICLR 2026
delta-rows and the cost of fine-tuning the model with delta-rows. We that LoRA is not comparable with full SFT and underperforms the DiFT in most of the settings, and the results are consistent with our previous analysis. However, notice that LoRA can forget less though it also learns less., which is quite interesting.
Table 18: Results of LoRA and DiFT in mix-up and continual SFT.
Model
GSMSk 61.64
xGLUE
LogiQA2
CSQA
ATA
math
1,2228
30.73
67.24
+LoRA
56.71
1.1542
29.77
67.73
code
26.54
1.1203
35.05
70.93
+LoRA
20.02
1.0805
32.82
71.33
Mix-Math-Codde
64.82
1.0956
34.54
68.22
59.80
+LoRA
62.62
1.0589
32.32
69.7
57.78
+DiFT
67.02
1.0735
32.63
68.39
60.35
Continual-Math-Cor
44.35
0,9902
32.82
70.52
46.93
+LoRA
43.97
0.9565
34.03
71.09
45,90
+DiFT
46.32
1.0557
35.86
70.93
49.55
F.4 COMPARED TO OTHER COMMETER-IMPRTANAEE METHODS
To evaluate the DiFT performance in the continual SFT setting, select HFT and LoTA as the continual SFT baselines given their superiority. Apart from thes recent methods, the EWC is classic and L2-SP is theoretically well-designed, and be applied in LLMs. We supplement the continual SFT experiments based on these baselines.
Table 19: Comparisons to EWC and L2-SP on Llama3-8B.
Llama3-8B GSMSk
xGLUE
LogiQA2
seq-math-code
44.35
0.9902
32.83
+EWC
43.98
0.9850
32.50
+L.2-SP
44.12
0.9880
32.65
+DiFT
46.32
1.0557
35,86
seq-math-logic
10.99
0.6433
31.30
+EWC
10.85
0.6380
31.10
+L2-SP
10.92
0.6410
31.20
+DiFT
11.37
0.6919
31.23
In Table 19 we can see that given that EWC and L2-SP are both classic baselines, they still underperform our DiFT, and even as strong our baselines HFT and LoTA. Through the experimental results, we can affirm the superiority of the proposed DiFT.
F.5 MORE REASONING TASKS EVALUATIONS
In this work, we focus on diverse reasoning capabilities, and commone reasoning tasks are math, code, logic, commonsense. In the main body, we evaluate the proposed DiFT on and 3 reasoning tasks, demonstrating the consistent effectiveness. Here we conduct more tasks can enhance scalability of the DiFT, so we perform 4 -task and 5-task experiments.
4-task and 5-task mix-up DiFT experiments in Table 20 illustrate consistent improvements, demon- strating the efffectiveness and more tasks potential of DiFT. We notice that distinct reasoning task combinations have different trainable parameter subsets, we statistic the overlap rate DSR parame- ters between any newly added task and historical tasks with the task number (K) increasing.
From Figure 9 we can observe that with the task number increasing, each single task has more overlapped DSR parameters, thereby the growth of total trainable parameters slows down, showing slow convergent trend. Such results indicate that the trainable parameter scale cannot easily as the same full SFT.
25

---

Published as a conference paper at ICLR 2026
Table 20: Results of more reasoning tasks mix-up experiments.
Llama3-8B
GSM8k
xGLUE
LogiQA2
CSQA
HotPotQA
mixx-math--oode-lliic-ss- +DIFT
67.93
1.0058
31.55
77.07
69.22
1.1342
33.21
77.31
mix- mui--mattt-ooee-l-gd------pppppo +DIFT
62.62
0.9971
33.14
77.97
36.74
62.68
1.0413
33.84
79.36
38.45
100
100
math overlap
code overlap
logic overlap
80
Overall param ratio (%)
Overlap rate (%)
csqa overlap
hotpoqa overlap
60
Overall param ratio
60
40
40
20
Task Num
Figure 9: The overlapped rate of DSR parameters among different task numbers, and the correspond-
ing overall trainable parameter ratio (compared to full SFT).
F.6 MORE REASONING-HIEETUNE LLMs ANALYSIS
We analyze the fine-tuned reasoning LLMs then check each task and the named parameters thoroughly, and eventually come up with the conclusion in Section Here we display more named parameters' DSR parameters visualization for reasoning tasks their corresponding fine-tuned LLMs, including Llama3-8B, Mistral-7B. and Qwen2.5-14B to demonstrate the universal DSR pattern. Figure TO display the DSR distribution of Math-on1: with different sampled data subsets where a more diverse DSR distribution among distinct reasoning data, reconfirming the observation in Section 3.2 In each row of Figure 13 can that all sampled data from the same reasoning data display nearly the same distribution for DSR parameters, as for its row-wise sub-figures, i.e., the influential parameters of each reasoning ability. the behaviors are rather inconsistent, leaving huge optimal space for multiple reasoning proficiencies gathering.
26

---

Published as a conference paper at ICLR 2026
15.5
2300
no-
112
212
(a) math rows (seed=42)
(b) math rows (seed=43)
(c) math rows (seed=44)
1994
250
(d) code rows (seed=42)
(c) code rows (seed=43)
code rows (seed=44)
2000
100 20
1000
1811 1807
2010
620 F0x 1900
1350 14d0
2211 4000
188
logic rows (sced=42)
(h) logic rows (secd=43)
logic rows (sced=44)
Layae modeltoem 24, www.grro and
Loyer mede Devers
15c
i csqa rows (2eed=42)
(k) esqa rows (seed=43)
esqa rows (seed=44)
Figure 10: Delta-scale rows of model. layer 24. www.mggatepppor. with distinct data samples on Llama3-8B's 's Math- -hollyooyyoooooooooo models.
27

---

Published as a conference paper at ICLR 2026
Loyer bean
TORMA
50c
math rows (seed=42)
(b) math rows (seed=43)
(c) math rows (seed=44)
5000
1230
1360
2353
100 20000
(d) code rows (seed=42)
(c) code rows (seed=43)
(f) code rows (seed=44)
200
5K
1480
1265
logic TOWs (sccd=42)
(h) logic rows (seod=43)
logic rows (sced=44)
2030
10.0 1000
2000 2007
1353 1920 1994 2027
Eslome P.t
28.13 12KE
(i) csqa rows (seed=42)
(k) esqa rows (seed=43)
(1) csqa rows (seed=44)
Figure 11: Delta-scale rows of model. wallaly..22. 21.224.... 1.4.mll..eee. pulp..aat..roo with distinct data samples on different reasoning Mistral-7B models.
28

---

Published as a conference paper at ICLR 2026
1000
100
w0 your
1905 1070
300
2000
(a) math rows (seed=42)
(b) math (seed=43)
(c) math rows (seed=44)
rocel apen JARLESS
Lage wases opei
500
6051
Coler
1850
3008
13000 3150
3550
(d) code rows (seed=42)
(c) code rows (seed=43)
(I) code rows (seed=44)
pode SeA.
rocel 21.15
1590
1254
logic rows (seed=42)
(h) logic rows (seed=43)
logic rows (seed=44)
1014
304
100
3050
8200
1862
1000 3000
2009
4.52
8:53
1900
185
100
0.00
0707
12670
3300
csqa rows (scod=42)
(k) csqa rows (scod=43)
csqa rows (sccd=44)
Figure 12: Delta-scale rows of model. llayer 24. on different reasoning Qwen2.5-14P models.
with distinct data samples
29

---

Published as a conference paper at ICLR 2026
32
(a) math rows (scod=42)
(b) math rows (scod=43)
(c) math rows (scod=44)
20c-
Loc
201
10.
500
1030
200
500
100
100
203
100
120
(d) code rows (seed=42)
(c) code rows (scod=43)
code rows (scod=44)
lever mond wwt.etttttttoo.... pm1
130
302
56
logic rows (2ccd=42)
(h) logic rows (seed=43)
(i) logic rows (scod=44)
62-
56-
260
100
24
46
530
3/6
180
20
45
8%
i esqa rows (2c
(k) csqa rows (seed=43)
csqa rows (seed=44)
Figure 13: Delta-scale rows of model. layer. 14. 14..141.....fttmmm v.proj with distinct data sam- ples on different reasoning Llama3-8B models.
In Mistral-7B and Qwen2.5-14B, the patterns are also like in Llama3-8B, we visualize the model. . layer. .2. 224.ml...g.. gate for each reasoning data in Figures 11 and 12 We can observe that math and code abilities share large proportion of common DSR parameters, while others do not, such a phenomenon reminds us that the benefits and conflicts are entangled. Therefore, we can see the math and code performances of Mistral-7B and Qwen2.5- in Table and Table 15 are in strong correlation, which can also align with the finding in Section 3.2
Classical meta-learning methods like MAML/Reptile learn how adapt: they optimize initializa- tion or update rule so that a few gradient steps on a new task lead to good performance, typically via an explicit meta-objective and episodic training [1-2]. In contrast, DiFT assumes fixed set of tasks and uses standard SFT; DSR is post-hoc, row-wise saliency analysis that reorganizes where updates happened without the additional optimizzaiin loop. Conceptually, meta-learning aims to achieve fast adaptation, whereas our DiFT focuses on reducing interference. These two paradigms can be complementary: a meta-learning initialization could still be optimized by DIFT.
MoE architectures (e.g., GShard, Switch Transformer) introduce explicit experts plus a learned router, enabling conditional computation but requiring architectural changes, routing stability tricks, and specialized infra [3-4]. DiFT instead treats a dense LLM as containing implicir experts, i.e., DSR-identified subsets, and enforces specialization via freezing, with unchanged architecture and inference cost.
We indeed point out that unstable routing can complicate the direct application of our method in the Limiations. We would like to clarify that:
Our concern with MoE is primarily about the instability of expert routing during training, more stable parameter subsets need more data to locate, which makes the notion of stable
30

---

Published as a conference paper at ICLR 2026
task-specific parameter subset harder to define and track. This is practical challenge, rather than a theoretical incompatibility between our method and MoE architectures.
We see our current dense-model study as a foundation. Understanding and validating the method in a simpler, more stable dense setting is a natural first step before tackling the additional complexity introduced by dynamic routing in MoE.
31


Reviews:
[{"review_id": "fTAISEqM2T", "reviewer": "Reviewer_nbCX", "summary": "This paper investigates the \"task conflict\" problem prevalent in multi-task (mixed or continual) supervised fine-tuning of LLMs, where fine-tuning on diverse reasoning datasets often leads to performance degradation in specific abilities. To address this, the paper first introduces an analysis method called \"delta-scale row (DSR),\" which identifies \"exclusive parameters\" critical to specific tasks by comparing the parameter activation differences between a base model and models fine-tuned on a single task. Building on this discovery, the authors propose the \"Differential Fine-Tuning (DiFT)\" framework: for mix-up SFT, only the union of the exclusive parameters for all relevant tasks is updated; for continual SFT, only the difference-set (parameters exclusive to the new task relative to old tasks) is updated. Experimental results on Llama3-8B, Mistral-7B, and Qwen2.5-14B demonstrate that DiFT consistently improves performance in the mix-up setting and effectively mitigates performance collapse in the continual setting.", "questions": "1. DSR identification relies on a \"probe model\" trained with 1k samples. How stable is this DSR set? Would the identified set of \"critical\" parameters change significantly if identified using the full 20k dataset, or if the DSR analysis were re-run after the DiFT mix-up SFT? This questions whether DSR is a static property or one that evolves dynamically during training.\n2. The mix-up SFT strategy relies on updating the parameter union. Have the authors investigated how the size of this union set scales as the number of tasks, K, increases (e.g., to K=5)? Is there a tipping point at which this union becomes so large that the DiFT strategy effectively degenerates into standard full-parameter SFT?\n3. The continual SFT strategy avoids conflicts by freezing parameters from historical tasks. However, for the parameter intersection—those parameters critical to both old and new tasks—does this freezing strategy cause the model to \"under-learn\" the new task? Or, do the authors believe the benefit of mitigating forgetting always outweighs this risk of under-learning?\n4. The authors state in the limitations that DSR is not directly applicable to MoE models due to unstable routing—a reasonable constraint. However, how do the authors conceptualize the relationship between DSR and the MoE architecture? Since MoE itself is a form of architectural parameter specialization, could the \"exclusive parameters\" identified by DSR be interpreted as \"emergent experts\"? Put differently, in an MoE model, is it hypothesized that a task's DSR would be primarily concentrated within its corresponding expert?\n5. DSR successfully identifies which parameters are in conflict, but not fully why they conflict. Do the authors have a deeper hypothesis? For example, in Figure 7, the DSR pattern for csqa (commonsense) (rows j, k, l) appears distinctly different from the patterns for math, code, and logic (rows a-i). Does this imply that the root cause of the conflict is the model's attempt to encode fundamentally different types of information (e.g., symbolic manipulation vs. factual knowledge) within the same set of parameters (like mlp.gate_proj)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-motivated. Through solid preliminary experiments (e.g., Figure 1, Table 1), the authors clearly reproduce and quantify the 'conflict' phenomenon that arises in multi-task reasoning SFT, providing a strong impetus for their subsequent investigation.\n2. The paper introduces the DSR diagnostic tool and the DiFT method. Instead of merely observing the superficial phenomenon that 'tasks conflict,' the authors delve deep into the model's internals to pinpoint the specific parameter subsets responsible for these conflicts. This \"diagnose-then-treat\" research paradigm is rigorous and well-founded.\n3. The authors provide validation covering both 'mix-up' and 'continual' fine-tuning scenarios. Crucially, the ablation studies on 'Inverse DiFT' and 'Random DSR' convincingly demonstrate that the performance gains stem from correctly identifying critical parameters via DSR, rather than being a simple side effect of parameter reduction (i.e., regularization).", "weaknesses": "1.The paper lacks a robust theoretical foundation. While DSR is an interesting empirical discovery, the paper fails to provide a solid theoretical explanation for the underlying mechanism—why do LLMs tend to encode capabilities for different tasks into distinct parameter subsets?\n2. The task scalability of the mix-up strategy is questionable. DiFT relies on updating the parameter union, but the experiments are limited to K=2 or 3 tasks. As K increases (e.g., in a practical scenario involving 20 tasks), the size of this union set could rapidly inflate, potentially approaching the full set of parameters and causing the DiFT strategy to degenerate into standard SFT. The paper lacks an analysis of DiFT's performance when K > 3.\n3. The DSR method introduces significant workflow complexity. The framework requires an additional pre-computation stage: a separate \"probe model\" must be trained for each of the N single tasks before the main fine-tuning can commence. This requirement increases the complexity of the training pipeline.\n4. The method's applicability to larger-scale models and different architectures is unproven. The experiments are confined to 7B-14B scale dense models, and it remains unknown if the approach holds for 30B+ models. The authors' discussion of MoE limitations (due to unstable routing) is reasonable, but this concession explicitly restricts the method's current applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mdLHoQ5u5X", "reviewer": "Reviewer_cicY", "summary": "The paper introduces **Differential Supervised Fine-Tuning (DiFT)**, a principled framework to mitigate conflicts among multiple reasoning tasks during supervised fine-tuning (SFT) of large language models (LLMs).  \nThe authors observe that when LLMs are fine-tuned jointly or sequentially on heterogeneous reasoning datasets (e.g., math, code, logic, commonsense), performance may degrade due to destructive interference. Through comprehensive experiments on Llama3-8B, Mistral-7B, and Qwen2.5-14B, they show that task combinations can yield both mutual benefits and conflicts, motivating a more nuanced SFT approach.  \n\nThe paper’s central insight is that each reasoning ability corresponds to a subset of model parameters that are disproportionately influential for that task. To identify these, the authors propose a quantitative metric called the Delta-Scale Row (DSR) score, computed from activation differences between base and fine-tuned models.  \nBuilding upon DSR analysis, DiFT differentially updates parameters. This selective parameter updating reduces interference and preserves mutual benefits across tasks.  \nEmpirically, DiFT achieves consistent improvements across math (GSM8k), code (xGLUE), logic (LogiQA2), and commonsense (CSQA) reasoning benchmarks, outperforming strong baselines such as HFT, LoTA, DMT, and CoBa. The proposed analysis and method generalize across both base and instruct LLMs, showing broad applicability.", "questions": "1. How stable are DSR distributions across random seeds or dataset subsamples? Does the identified task-specific subspace remain consistent under noisy data?\n\n2. Can DSR be estimated during training rather than post-hoc, enabling online conflict detection and adaptive freezing?\n\n3.  How sensitive is DiFT to the hyperparameter controlling the top-$C$ DSR threshold? Are results robust to different $C$ values?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The Delta-Scale Row (DSR) analysis offers a novel and interpretable metric for identifying task-specific sensitive parameters, bridging parameter variation analysis with reasoning-oriented fine-tuning.\n\n2. The discovery of reasoning task interactions—synergistic or conflicting—across math, code, logic, and commonsense reasoning provides valuable empirical insight into multi-ability LLM training.\n\n3. The theoretical motivation for DSR is grounded in Taylor expansion and sensitivity analysis, giving analytical meaning to the empirical findings.\n\n4. Contributes to a deeper understanding of task interference and transfer in multi-objective fine-tuning, a core challenge for multi-ability LLMs.", "weaknesses": "1. While DSR analysis is well-motivated, the connection between row-wise sensitivity and global reasoning dynamics remains empirical. A theoretical characterization of why DSR partitions correlate with reasoning abilities would strengthen the contribution.\n2. Although the paper claims DiFT is computationally efficient, quantitative runtime and FLOPs comparisons with baselines are absent; adding explicit cost metrics would improve clarity.\n3. Experiments focus on four reasoning tasks; extending to non-symbolic reasoning (e.g., spatial or multi-hop tasks) would further validate generality.\n4. Although results generalize to different LLMs, scaling behavior (e.g., how DSR sparsity changes with model size) is not analyzed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dG8tBEYj5R", "reviewer": "Reviewer_os6g", "summary": "This paper investigates the complex problem of task conflict and synergy during LLM fine-tuning. The authors observe that training a model on one reasoning task can often degrade its performance on another. To address this, they introduce a parameter identification method called Delta-Scale Row (DSR) to find the parameters most critical to a specific task. They then propose a new fine-tuning strategy, Differential Fine-Tuning (DiFT), which selectively updates or freezes these DSR parameters based on the learning scenario. The goal is to mitigate the conflicts that arise when mixing tasks or learning them sequentially. The authors provide experimental results across several models (Llama3, Mistral, Qwen2.5) to demonstrate that DiFT outperforms existing baseline methods.", "questions": "(1) Signal–target entanglement.\nDSR is computed with data from the target task, then those rows are trained on mixtures including the same family. Could ATA gains partially reflect selection–target leakage?\n\n(2) Relation to importance metrics.\nHow does DSR compare against Fisher information, diagonal Hessian, or gradient-based importance (EWC, L2-SP, PackNet, LwF) under identical budgets?\n\n(3) Layer and granularity choices.\nWhich layers are hooked—every linear layer or a subset? How sensitive are results to selecting rows in early vs. deep layers?\n\n(4) Continual rule clarity.\nFor DSR_diff, does training freeze everything except the newly identified rows, or only historical DSR sets? What remains trainable for rows never selected by any task? Please show capacity remaining over steps (k).", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Clear writing. The motivation, pipeline, and training regimes are easy to follow. Key concepts (DSR, union/difference rules) are defined and used consistently.\n\n\nNew method. DiFT introduces a simple activation-delta criterion to pick “sensitive rows,” then applies union/difference rules for mix-up and continual learning. The idea is lightweight and practical.\n\n\nExtensive experiments. Results span multiple models and task families, with ablations and sanity checks (e.g., inverse variants). Findings appear consistent across settings.", "weaknesses": "(1) Assumption validation is improved but still incomplete.\nInverse-DiFT and random-selection baselines support necessity and non-randomness. Still missing stronger causal probes. Cross-task “row transplant” tests, DSR computed on disjoint sources, and counterfactual swaps across tasks would better establish causality beyond necessity.\n\n(2) Heuristic choices lack broader justification.\nThe ablation on the number of rows (C) addresses set size. However, the union rule (mix-up) and the difference rule (continual) remain untested against plausible alternatives. No intersection, weighted unions, or trust-region style variants. No analysis of when overlap helps vs. harms. Results could benefit from a principled rationale or a small design sweep in the main text.\n\n(3) Missing continual-learning baselines.\nNo direct comparisons with parameter-importance methods: EWC/Online-EWC, L2-SP, MAS, SI, PackNet, LwF. Without them, placement within the CL literature is unclear.\n\n(4) Potential signal–target entanglement.\nDSR selection uses the same task family that later benefits in training/evaluation. This can inflate ATA via selection–target coupling. Need results where DSR is computed on held-out variants or adjacent corpora.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2XojRmjfJx", "reviewer": "Reviewer_kiFD", "summary": "This paper proposes Differential Fine-Tuning (DiFT), a novel fine-tuning strategy designed to mitigate task conflicts while preserving performance benefits across multiple reasoning tasks in large language models (LLMs). The method identifies task-specific parameters through a Delta-Scale Row (DSR) analysis and selectively updates them to avoid destructive interference among reasoning abilities. Experimental results on Llama3-8B, Mistral-7B, and Qwen2.5-14B demonstrate that DiFT can outperform existing mix-up and continual fine-tuning baselines such as DMT, CoBa, HFT, and LoTA, improving average target accuracy (ATA) across various reasoning benchmarks including GSM8k, xGLUE, LogiQA2, and CSQA. The work is well-motivated and offers an interesting parametric perspective on reasoning conflicts in LLM fine-tuning.", "questions": "See weaknesses", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.The paper addresses an important and relatively underexplored problem—task interference and reasoning conflicts during supervised fine-tuning of LLMs.\n2.The DSR-based parameter analysis provides an insightful method to identify task-sensitive parameters, which could inspire future research on interpretable fine-tuning.\n3.The proposed DiFT framework is conceptually elegant and achieves consistent empirical gains across diverse reasoning benchmarks and model sizes.\n4.The authors also provide comprehensive experiments, ablation studies, and discussions comparing mix-up and continual learning paradigms, which enhance the credibility of their findings.", "weaknesses": "While I find the idea of DiFT quite promising, I have several concerns and questions that the authors may wish to address to strengthen the work:\n1.Scalability with Task Number. I express concerns about the scalability of the proposed DiFT method, noting that maintaining task-specific parameter updates for multiple reasoning tasks may cause the model size and memory usage to grow as the number of tasks increases. It is therefore recommended to conduct experiments with different numbers of tasks (e.g., two, three, or more) to evaluate how DiFT performs in large-scale and multi-domain settings. Such experiments would help demonstrate the method’s practicality and efficiency in real-world applications involving large models.\n\n2.Relation to Meta-Learning and Mixture-of-Experts Paradigms. I am interested in how DiFT conceptually and empirically compares with existing meta-learning and multi-expert approaches that also address knowledge sharing and task specialization across multiple domains. It is suggested that the paper explicitly discuss these connections, highlighting the key distinctions between DiFT and meta-learning or expert-based modular frameworks. Additionally, including a comparative performance analysis—at least at a conceptual level—would help clarify the unique contribution and positioning of DiFT within the broader landscape of multi-task learning methods.\n\n3.Substitution of the Key Parameter Identification Mechanism. The DSR mechanism plays a central role in DiFT, but I wonder how it compares with other possible parameter importance estimation techniques, such as attention-based attribution or gradient sensitivity analysis. Could the authors provide either an ablation about substituting DSR with attention-based mechanisms to examine robustness and generality of the framework?\n\n4.Limited Experimentation on Larger Models. The authors mention that, “Due to hardware limitations, we only conducted experiments on 7/8B and 14B LLMs in this paper, lacking validation on larger-scale (30B+) models that can be complementary.” However, larger-scale validation seems critical to the paper’s claim that DiFT improves diverse reasoning capabilities of LLMs in general. Since scalability is a core aspect of language model research, I strongly recommend that the author provide results on the expected performance of DiFT on various different-sized models.  Without this, it remains unclear whether the proposed approach would maintain its efficiency and stability at scale.\n\n5.Lack of Convergence Analysis in Fine-Tuning. The paper does not provide any theoretical or empirical evidence of convergence during fine-tuning. Since the proposed DiFT method involves differential parameter updating across multiple reasoning tasks, it is unclear whether the optimization process is guaranteed to converge to a stable point. Moreover, when the number of tasks increases, convergence behavior could vary substantially across different parameter subsets. I am particularly concerned that without monitoring the fine-tuning loss or parameter change trajectories, it is difficult to ensure that each reasoning sub-model (or the overall DiFT model) has reached a stable equilibrium. Providing convergence curves for varying task counts (e.g., 2, 3, or more reasoning datasets) would significantly strengthen the technical soundness of this work.\n\n6.Missing Computational Cost and Efficiency Analysis. The paper does not provide sufficient information about how computational efficiency scales with the number of fine-tuning tasks. While Section 5.1 briefly mentions that the computing costs are lighter than task-vector and gradient-based methods, it lacks quantitative evidence showing how training time, token consumption, and GPU (CUDA) utilization change as the number of reasoning tasks increases. Such data are crucial to assess whether DiFT remains efficient and feasible when extended to larger task sets. Including analyses of runtime, token usage, and GPU memory consumption under different task counts would help clarify the scalability and practical efficiency of the proposed method compared to baselines like LoTA or HFT.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)