SFT Doesn’t Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs
Authors: Jiacheng Lin
Zhongruo Wang
Kun Qian
Tian Wang
Arvind Srinivasan
Hansi Zeng
Ruochen Jiao
Xie Zhou
Jiri Gesi
Dakuo Wang
Yufan Guo
Kai Zhong
Weiqi Zhang
sujay sanghavi
Changyou Chen
Hyokun Yun
Lihong Li
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models; Supervised Finetuning; Domain-specific SFT; Continual Learning
Primary area: foundation or frontier models, including LLMs

Abstract: Supervised Fine-Tuning (SFT) on domain-specific datasets is a common approach to adapt Large Language Models (LLMs) to specialized tasks but is often believed to degrade their general capabilities. In this work, we revisit this trade-off and present both empirical and theoretical insights. First, we show that SFT does not always hurt: using a smaller learning rate can substantially mitigate general performance degradation while preserving comparable target-domain performance. We then provide a theoretical analysis that explains these phenomena and further motivates a new method, Token-Adaptive Loss Reweighting (TALR). Building on this, and recognizing that smaller learning rates alone do not fully eliminate general-performance degradation in all cases, we evaluate a range of strategies for reducing general capability loss, including L2 regularization, LoRA, model averaging, FLOW, and our proposed TALR. Experimental results demonstrate that while no method completely eliminates the trade-off, TALR consistently outperforms these baselines in balancing domain-specific gains and general capabilities. Finally, we distill our findings into practical guidelines for adapting LLMs to new domains: (i) using a small learning rate to achieve a favorable trade-off, and (ii) when a stronger balance is further desired, adopt TALR as an effective strategy.

Published as a conference paper at ICLR 2026
SFT DOESN'T ALWAYS HURT GENERAL CAPABILI- TIES: REVISITING DOMAIN-SPECIFIC FINE-TUNING IN LLMS
Jiacheng Lin
Zhongruo Wang
Kun Qian
Tian Wang
Arvind Srinivasan
Hansi Zengà
Ruochen Jiao²
Xie Zhou?
Jiri Gesi²
Dakuo Wang
Yufan Guo
Kai Zhong²
Weiqi Zhang?
Sujay Sanghavi'
Changyou Chen
Hyokun Yun?
Lihong Li²
University of Illinois Urbana- Chbama-Champaign Ammzoo *University Texas at Austin University at Buffalo
University of Massachusetts Amherst Northeastern University
ABSTRACT
Supervised Fine-Tuning (SFT) on domain-specific datasets is a common approach adapt Large Language Models (LLMs) to specialized tasks but is often believed to degrade their general capabilities. In this work, we revisit this trade-off and present both empirical and theoretical insights. First, show that SFT does not always hurt: using a smaller learning rate can substantially mitigate general per- formance degradation while preserving comparable target-domain performance. We then provide a theoretical analysis that explains these phenomena and fur- ther motivates a new method, Token-Adeptive Loss Reweighting (TALR). Build- ing this, and recognizing that smaller learning rates alone do not fully eliminate general-perforee perforaace degradation in all cases, we evaluate range of strategies for reducing general capability loss, including L2 regularization, LoRA, model av- eraging, FLOW, and our proposed TALR. Experimental results demonstrate that while method completely eliminates the trade-off, TALR consistently outper- forms these baselines balancing domain-specific gains and general capabilities. Finally, distill our findings into practical guidelines for adapting L.MMs new domains: (i) using a small learning rate to achieve a favorable trade-off, and (ii) when stronger balance is further desired, adopt TALR an effective strategy.
INTRODUCTION
Large Language Models (LLMs) have demonstrated remarkable performance across wide range of general- purposee tasks, including question answering, mathematical reasoning, and code generation (Yang et al. 2024 2025) Touvron et al.) [2023] Dubey et al.) (022)). To further enhance their effective- ness in specialized applications, practitioners often perform additional supervised fine-tuning (SFT) using domain-specific data. This process enriches the model with domain knowledge and yields substantial performance gains domain-specific tasks (Labrak et al. 2024 Lin et al. 2024 Peng
et al. 2024). SFT has thus become standard paradigm for adapting LLMs to world deployment scenarios.
However, recent studies have shown that fine-tuning LLMs on domain-specific datasets can substan-
tially impair their generalization capabilities (Huan et 2025 Lin al. 2025a Chen et al. 2025]
Bansal & Sanghavi 2025 Sanyal et al. 2025 Chu et al. 2025 Shenfeld et al. 2025). For example,
Equua contribations. Corresponding to j12548 11l1111nm linois edu.. This work was done during Jiacheng
Lin and Hansi Zeng's internship Amazon.

---

Published as a conference paper at ICLR 2026
Gemma-3 (Team et al. 2025 using domain-specific datasets, such as those from e-commercial or biomedical domains, often leads to significant performance drops on general purposs benchmarks such as GSM8K (Cobbe et al. 2021),
performing SFT on LLMs like Qwen-3 (Yang al. 2025
HumanEval (Chen et al. 2021), or IFEval (Zhou et al., 2023), which assess core capabilities like mathematical reasoning, code generation, and instruction following. This phenomenon raises the need for closer examination of domain-specific SFT.
In this work, we revisit the phenomenon of general capability degradation induced by domain- specific SFT. Surprisingly, domain-specifie SFT does not always significantly degrade general capabilities, contrary to prior claims. Our experiments reveal that, in most cases:
Using a smaller learning rate allows domain-specinte SFT to achieve a favorable trade-off:
Sannera-purpose performance degradation is largely mitigated; Target domain performance is comparable to that with larger learning rates.
The first observation relatively expected, since smaller learning rates naturally suppress parameter drift compared to more aggressive updates (Pareja et al.) 2025). The second, however, is more surprising. Prior to the LLM era, practical experience in machine and deep learning suggested that larger learning rates often essential for better downstream performance (Mohtashami al. 2023 Li et al., 2019, Sadrtdinov et al. 2024). In contrast, we show that LLMs behave differently: comparable domain- specific performance can still be achieved under smaller learning rates. We further provide a theoretical analysis supporting this phenomenon. addition, a closer inspection of prior studies shows that their strong degradation claims predominantly arise under relatively large learning rates. Taken together, our empirical and theoretical evidence demonstrates that careful choice of learning rate offers practical path to balance domain adaptation with general capability preservation (
While adopting smaller learning rates typically yields better trade-off, we also observe that this does not fully mitigate the snroral-pefformance degradation in all cases. To address this, we fur- ther investigate mitigation approaches that could mitigate such degradation ($4). Specifically, we assess a range of representative strategies evaluated in (Sanyal et al. (2025). including 1.2 regular- ization, LoRA (Hu et al.] 2022], model averaging (Wortsman et al. 2022). and FLOW (Sanyal et al. 2025), along with proposed method, Token-Adaptive Loss Reweighting (TALR). TALR adaptively down-weights hard tokens by solving a constrained optimization problem that admits a closed-form solution, thereby tempering their potential disproportionate influence general capa- bility degradation during training. Our experiments demonstrate that TALR provides advantages in further suppressing general-performanee existing method including TALR can completely eliminate the degradation, highlighting the need for more advanced strategies be explored in future work.
degradation compared to these baselines. Nevertheless,
We further conduct a token-level analysis to better understand domain-specific SFT 4.4.). This analysis yields two key findings: (1) Most tokens in SFT training data pose low learning diffi- culty to the LLMs, even when the overall domain-specific task performance is poor. The rel- atively fewer hard tokens (i.e., low-probability tokens) typically arise either from a lack of domain knowledge in the pretrained model or from stylistic mismatches between the domain-specine and the pretrained model. (2) TALR induces a token-level curriculum-like learning dynamic. In the early stages of training, easier tokens receive more focus, while hard tokens are down- weighted. As training progresses, however, some these hard tokens become relatively casier for the model, and their weights gradually increase. This dynamic allows TALR to smoothly shift focus over time, balancing the injection of domain knowledge with the preservation of general capabilities.
data
Finally, summarize findings into practical guideline for domain- specific SFT:
Guidelines for domain-specific SFT.
Use a smaller learning rate to achieve a favorable trade-off between domain performance and general-purposs capability preservation.
When a stronger balance is further required, adopt TALR as an effective strategy to further suppress general- aneerll-erffommace degradation.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Our problem setting can be broadly framed within the scope of continual learning, where models must acquire new knowledge while retaining previously learned capabilities to avoid catastrophic forgetting. Existing approaches are typically divided into two categories: data-dependent and data- oblivious. Data- deta-deenndent methods assume access to a subset of the training data from earlier stages, whereas data-oblivious methods rely solely the pre-traines model without revisiting any prior data. The latter is particularly realistic in practice, as access to proprietary or large-scale pre- training corpora is often infeasible, yet it remains relatively underexplored. For broader overview of this landscape, we refer readers to recent surveys (Wang et al. 2024a). Our focus in this paper is on the data-oblivious setting.
Data-oblivious approaches. One line of work introduces loss regularization to constrain the fine- tuned model from drifting too far from its initialization, such as L. regularization in parameter space (Kumar et al., 2025; Kirkpatrick et al., 2017). Another line of work explores the idea of model averaging. which combines the parameters of the pre-trained model and the fully fine-tuned model through a convex combination, aiming to balance adaptation with retention (Wortsman et al.,
2022 Lubana et al., 2022; Ilharco et al. 2023 Kleiman al., 2025). LoRA (Hu al., 2022
Biderman et al. (024) represents another widely used strategy, enforcing low-rank updates to the weight matrices so that parameter changes are confined to a restricted subspace, thereby limiting catastrophic drift while improving efficiency. Besides, data reweighting has been explored as
promising strategy; for example, FLOW (Sanyal et al. 2025) mitigates forgetting in vision tasks by adjusting the loss weights of easy and hard samples.
Extensions to LLMs. Existing research has primarily focused on deta-dependent methods in the LLM context (Scialom al. 2022 Yin et al. 2023 Wang al. 2024b) Xiong et al. 2023 Mok et al. 2023). In contrast, data-oblivious approaches remain relatively underexplored, though several studies have begun adapting ideas from continual learning on traditional models LLMs (Sanyal cet al. 2025; Razdaibiedina et al. 2023 Wang et al. 2023 Zhao et al. 2024). Refer to the survey by Wu et al. (2024) for more details. However, LLMs differ substantially from earlier architectures in scale, pre- pre-training regimes, and emergent capabilities, which makes their adaptation dynamics distinct. As result, we revisit continual SFT of LLMs on Nomain--pecife datasets, aiming to better understand its mechanism.
LEARNING RATE MATTERS: REVISITING ITS ROLE IN GENERAL CAPABILITY DEGRADATION DURING DOMAIN-SPECIHIC SFT
In this section, we revisit the role of learning rate in Nomain-speciff SFT and its impact general capability degradation. Surprisingly, we find that using a smaller learning rate (e.g.. 1c-6) can sub- stantially reduce the loss of general capabilities, while achieving domain- -specif- task performance on par with much larger learning rates. This suggests that the severe degradation reported in prior work may stem, least in part, from overly aggressive optimization (Huan al. 2025 Lin al. 2025a Chen et al.) 2025 Bansal & Sanghavi 2025 Sanyal et al.) 2025 Shenfeld et al. 2025 Indeed, many of these studies used relatively large learning rates such as 5e-6 or 2e-5.
To systematically investigate this effect, we experiment on two domain-specifir datasets: MedCalc KKaadekaar al. 2024) and ESCI (Reddy al. 2022). We choose these datasets because existing open-source LLMs perform poorly on them, making them representative scenarios where domain- specific SFT most motivated: to enhance specialized capabilities in domains where the initialized model is weak. Below are details of the experimental setups and results for each dataset.
3.1
EXPERIMENTAL SETUP
3.1.1 DATASETS
MedCale (Khandeaar et al. 2024) consists of 10.1k training and 1.05k test examples. Each instance includes brief patient note and clinical instruction (c.g., "What is the patient's CHA,DS, VASc score?"), with the goal of predicting a numeric, categorical, or datetime answer. The training set provides gold Shain-of-though (CoT) rationales, which we use supervision targets during SFT.
3

---

Published as a conference paper at ICLR 2026
(a)
Qwen3-8B MedCalc (b)
Qwen2.5-7B
MedCalc (c)
Owen3-4B ESCI
(d) Qwen3-4B ESCI (w/o CoT)
Int
Oarairll peroomancs
0.8
Infle-6
performaee
XK
General parfomance
General perforancee
XXX
0.8
X
X
It-2e-6
0.4
0.6
F-5o-6
I=10-6
0.4
In-2a.5
X
In-l--5
h-1e-6
0.2
0.2
In-So-5
k=58-8
I--e-5
In28-55
0.0
0.3
0.4
0.5
0.6
0.3
0.4
0.5
0.2
Domain
Domain performance
Domain performance
Domain performance
(e)
Qwen3-4B MedCalc
Dwen22.53BB + MedCalc (g)
Gemma3-4B MedCalc
(h)
Gemma3-18 1B MedCalc
0.8 +K
0.8
H*
X
General performncee
General perormance
0.
Gamerdl perfor performence
General performance
0.0
0.8
0.0
0.4
0.4
0.4
1==1e-6
X
In-e-8
1r-
X
1=-1e-6
0.2
X
In-5-4
X
In:5e-8
0.2
Ir:Se-8
X
In-5688
Im-2o-5
In22 5
Im22-55
Im-2
Domain partormance
Domain parformnnce
Domain parforannce
Domain performance
Effect of learning rate on domain-specific and general capability performance dur- ing supervised fine-tuning (SFT), We conduct experiments two domain-specific datasets, Med- Cale and ESCI. For the ESCI (w/o CoT) variant, the model is trained only to predict the final la- bel without intermediate reasoning steps, unlike the other three settings where reasoning traces available. General capability performance is measured as the average across IFEval, GSM8K, and HumanEval unless otherwise specified. We observe that smaller learning rates yield more favor- able trade-off (upper-right corner) between domain performance and general performance.
Figure 1:
The ESCI dataset (Reddy et al. 2022) is an e-commerce product classification benchmark contain- ing query-product pairs labeled as Exact, Substitute, Complement, or Irrelevant. The training set consists of 49k examples, and the test set contains 10k examples. We consider two training settings: w/ CoT, where the target sequence includes both reasoning and the final label, and w/o CoT, where it contains only the label.
3.1.2 EVALUATION PROTOCOL
For the MedCalc task, follow the evaluation protocol of Khandekar al. (2024) and report accuracy based on the model's final answer. For general capability evaluation, we measure perfor- mance suite of general-purpose benchmarks using the 1m-eva luat on-ha rness frame-
work (Gao et al. 2024). following the same evaluation setup as prior works (Lin et al. 2025a Sanyal 2025 Bansal & Sanghavi 2025). Model checkpoints are selected based their best performance on the target domain task, after which the corresponding models are evaluated on general-purpor purpose benchmarks, reflecting practical scenarios where downstream task performance is
prioritized (Sanyal et al.) 2025 Bansal & Sanghavi 2025). The evaluation metric for each bench- mark is detailed in Appendix C.I Since ESCT highly imbalanced across classes, we follow prior work on imbalanced classification (Xu et al. 2024 2025) and report balanced accuracy (BACC) as our primary metric.
3.2 MAIN RESULTS
Finding 1: Smaller learning rates achieve a more favorable trade-off. From Figure we observe that for both MedCalc and ESCI, smaller learning rates consistently lead to points located toward the upper-right region of the plots. This indicates that they can effectively mitigate degradation in general capabilities while simultaneously delivering strong performance the target domain tasks.
Finding 2: Label-only supervision loosens learning rate constraints for Pareto-optimal trade-offs. When the target sequence consists solely of the ground-truth label (e.g., <answer> [label] / without intermediate reasoning steps, the range of learning rates that achieve Pareto-optimal trade-offs becomes broader. As shown in Figure d), learn-
4

---

Published as a conference paper at ICLR 2026
ing rate of 5e-6 performs comparably to 1e-6 in the upper-right region, which contrasts with the trend observed the other subfigures of Figure [
Remark: From experiments on MedCalc and ESCI, well as the additional results and analyses in Appendix C.. we observe consistent patterns: smaller learning rates can substantially reduce
general capability degradation while maintaining competitive domain-specific performance. This naturally raises the question:
Why do milder updates preserve general abilities while still enabling strong domain gains?
To shed light this phenomenon, next turn theoretical analysis, aiming uncover insights into how the learning rate shapes the trade-off between domain adaptation and the preservation of general capabilities comain-specific SFT.
3.3 THEORETICAL ANALYSIS
To better understand the empirical phenomena observed previously, we provide theoretical anal- ysis from the perspective of information theory. Motivated by the equivalence between language modeling and data compression (Deletang et al. 2024 Ji et al. 2025), we view an LLM as com- pressor, where the effectiveness of training can measured through changes code length. In this view, improvements or degradations in performance across datasets correspond variations in compression rate. Below, we formalize this perspective by introducing the notion of token trees and describing the LLM compression protocol in our context.
Definition 3.1 (Token Tree T). For a dataset DD = {2s E 1 11 MV] < co, where finite vocabulary of size [V), the token tree of D, denoted as TD. defined as follows: (1) each node has [V] child nodes labeled U1, ....... yb, along with an end-of- sequence (EOS) leaf node; (2) The weight of a non-leaf node is the sum of the weights of all its child
= {ur, v2...., .,. U1y }
.....U
nodes; (3) The path from the root to an EOS leaf node defines a response 2,. with the corresponding EOS node weight representing the response 's probability.
Definition 3.2 (LLM Compression Protocol). Let To be the token tree of dataset D, and let go(: u) denote the conditional distribution over U (EOS) predicted by an LLM with parameters at node E To. Given response 2 (a path from the root to an EOS leaf. truncated to a pre-defined maximum depth d), the LLM compression protocol encodes using arithmetic coding, where at each step the coding probabilities are given by qo(. u) for the current node along the path of 2.
Having established the compression protocol, we now follow prior work (Deletang et al. 2024 Ji al.) 2025) and changes in expected code length surrogate metric for an LLM's modeling quality on given dataset distribution. In this view, reductions in code length discrepancy correspond better alignment with the data distribution. Formally, this is captured by the following proposition.
Proposition 3.1 (Expected Code Length Discrepancy under Model Shift). Consider two model dis-
tributions qo, (-) and qo (-) over the loken tree TD with distribution P. The change in expected
code length on P when shifting from go, to q0, AT(P) E. [T (00,,(3)) (=) EE (=)]
EL E, Pt.5 log Equivalently, AL(P) KL(P qo.) KL(P P q0,)
Based on the above, we now turn to our main goal: explaining the empirical phenomena observed } namely Finding I and [ To keep the presentation clear and concise, we provide simplified informal statements of our key theorems below, while the full formal versions and proofs referred
to Appendix B
Theorem 3.1. (Informal) Under certain assumptions, consider fine-tuning domain-specife dataset D2 with a fixed target domain improvement A. > 0 (i.e., (L(P2) < -A.). The general- performance degradation Dj. which is already well modeled by the LLM, admits an upper bound
AT(P)) < kh
where X is the effective per-step size and K1, k2 are constants determined by the model and data. Thus, using smaller steps (smaller X) leads to strictly tighter guarantees on preservation.
5

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of domain and general performance the MedCalc Benchmark under learn- ing rate 1e-6. Both Standard SFT (with smaller learning rate) and TALR are our contributions, and together they achieve the best overall trade-off's compared with the other basclines.
Qwen2.5-3B
Qwen3-4B
Gemma3-4B
Average
Method
Domain
General
Domain
General
Domain
General
Domain
General
Standard (Ours)
0.4947
0.6202
0.5484
0.7837
0.5587
0.6734
0.5339
0.6924
L2-Reg
0.4904
0.6205
0.4692
0.7964
0.5595
0.6750
0.5064
0.6973
LoRA
0.1261
0,5831
0.1945
0.7640
0.2233
0.1241
0.1813
0.4904
Wise-FT
0,1948
0.6285
0.1428
0.7884
0,2573
0.7635
0.1983
0.7268
FLOW
0,3641
0,5974
0,4768
0.7870
0,5673
0.6914
0.4694
0.6920
TALR (Ours)
0,4806
0.6478
0.4889
0.7880
0.5338
0.7150
0.5011
0.7169
Here, E (0, 1) denotes the per-step size of the distributional update; formal definitions are pro-
vided in Appendix [B.3. In practice, a smaller learning rate induces a smaller X. Therefore, The- orem 3.1 explains Finding 1: adopting a smaller learning rate (i.e., smaller A) reduces the upper bound on general-performante degradation, consistent with the empirical trend observed in $3
Theorem 3.2. (Informal) Under certain assumptions, fix a tolerance on general-performance degradation on D1 (i.e., AL(P:) I Etg). Then the maximal safe per-step size satisfies Amax & Fig va where is the expected number of low-probability tokens per example D2, defined tokens whose probabilities under the LLM are below a threshold.
This result explains Finding 2:
when training with only labels, the number of hard tokens is reduced compared to training with both labels and mn-in-of-thhough annotations, thereby increasing the safe step-size range. This explains why in our ESCI experiments, label-only SFT tolerated larger learning rates (c.g., 6) without causing substantial general- ceneraa-performance degradation.
3.4 INSIGHTS AND NEXT STEPS
Beyond Smaller Learning Rates. Building on the empirical and theoretical analyses above, we have shown that using a smaller learning rate can mitigate degradation in general performance while still achieving strong target-domain performance. However, small learning rates cannot solve crything. First, although smaller learning rates greatly reduce the extent of ceneral-performance degradation, they do not fully eliminate it in strategies are needed to suppress such degradation more effectively. Second, while smaller learning rates generally achieve domain performance close that of larger ones, in certain cases the gap is not entirely negligible (Fig. D and [h). In situations where stronger target-domain performance prioritized, larger learning rates may therefore be necessary, but they inevitably incur greater general-pe perfoomaace degradation. This makes the development of additional mitigation strategies under larger learning rates equally important in certain cases.
cases (Fig. [g]. This suggests that further
Insights from Theoretical Analysis. In Theorem 3.I we can further expand the coefficients as
k (ws Mn. + M.), ha (ws Mr M, + ka).
where My. bounds the update magnitude on hard (ow-probbbilltt) tokens, M,, corresponds to easy tokens (with MA >> Me). ws denotes the mass of the hard-token set S, and K3 residual constants. For a fixed target dataset (hence essentially fixed wg), the dominant factor in both kj and kg is M₁- Therefore, reducing Mr, i.e., shrinking the update amplitude induced by low-probability (hard) to- kens, tightens the upper bound on (L(P)). This observation naturally motivates token-adaptive reweighting strategies that directly down-weight bard-token losses to curb their potential dispropor- tionate influence to general performance degradation.
4 TOKEN-ADAPTIVE LOss REWEIGHTING FOR DOMAIN-SPECIRE SFT
From the preceding analysis, we see that reducing the update magnitude on www.prbbbyility tokens (hard tokens) can tighten the upper bound on remerrl-performmat
degradation (L.P)). This sug- gests a promising direction: down-weiee weightiig the loss of hard tokens to curb their disproportionate impact forgetting. However, this immediately raises several practical challenges. How should

---

Published as a conference paper at ICLR 2026
Table 2: Comparison of domain and general performance on the MedCale Benchmark under learn- ing rate 5e-6. At this larger learning rate, TALR achieves the best overall trade-off by substantially improving general performance while maintaining comparable domain performance.
Qwen2.5-3B
Qwen3-4B
Gemma3-4B
Average
Method
Domain General
Domain
General
Domain
General
Domain
General
Standard
0.5459
0.3337
0.5782
0.5425
0.5507
0.2655
0.5583
0,3805
1.2-Reg
0.5406
0.3470
0.5782
0.5591
0.5471
0.2796
0.5553
0.3952
LoRA
0.1734
0.5670
0.2367
0.7571
0.3864
0.1241
0.2655
0.4827
Wise-FT
0.3584
0,5869
0,3815
0.7531
0.4638
0,5929
0.4012
0.6443
FLOW
0,5266
0.4419
0.5819
0.5599
0,5500
0.3476
0.5528
0,4498
TALR (Ours)
0.5066
0.5490
0.5834
0.6138
0.5351
0.3427
0.5417
0.5018
identify which tokens are "hard"? If we rely fixed probability threshold, what value should be chosen? Even after identifying hard tokens, by how much should their losses be down-weighte Manually setting such thresholds or scaling factors cumbersome. To address these challenges, we propose a principled and adaptive solution, TALR (Token-Adaptive Loss Reweighting), to adap- tively scales the loss contribution of each token according to predicted probability. Additional details and discussions of TALR can be found in Appendix D
4.1 TOKEN-ADAPTIV WEIGHT COMPUTATION VIA CONSTRAINED OPTIMIZATION
Formally, let (())) - log po(ss) denote the loss of token i given model parameters 0. We seek per-token weights w (W1,.... Wn) that (1) assign smaller weights to harder tokens (loss larger/probabiiliy lower = weight smaller); and (2) avoid collapsing all weight onto a small sub- set of tokens, ensuring broader coverage across the sequence. We formulate this the following constrained optimization problem:
min
Σ (0) + A.A,
Σ wi log Wt,
(I)
where A, is the nt-dimensional simplex (w;; 2 0, >m i-1 1), and F > 0 controls the strength of entropy regularization. The first term enforces preference for low-loss tokens, while the negative- entropy regularization term prevents the distribution from becoming overly concentrated.
This optimization admits closed form solution: w exp (-L(O)/T)//z, where Z the normal- ization factor. Since (0) log po (x:), we can equivalently write: w x (o.)))//T
the unnormalized form wi ps, focusing the relative magnitudes. This also keeps wy E (0, 1) naturally bounded and directly tied to the model's confidence. By scal- ing token-level loss with these adaptive weights, TALR tempers the excessive gradient contributions from low- oow-probability tokens while preserving their influence for learning domain-specins edge. During training. these weights are recomputed at every optimization step for the tokens in the current batch, ensuring that the reweighting adapts dynamically to the model's evolving predictions. The detailed procedure is summarized Algorithm
In practice,
knowl-
4.2 RESULTS
We evaluate all mitigation strategies considered in Sanyal et al. (2025), including L2 regularization, LoRA, Wise-FT (model averaging), and FLOW, together with our proposed TALR. Table [ and 2 reports the trade-off between domain performance and general performance under two learning rates. The bascline configurations follow Sanyal et al. (2025).
Smaller learning rate (1e-6). From Table most strategies, i.c., except LoRA and FTs achieve domain performance and general performance that are relatively close to each other. This indicates that simply using small learning rate already mitigates the degradation of general capa- bilities while maintaining strong domain performance. Among all methods, both our Standard SFT (with smaller learning rates) and our TALR consistently provide the best trade-offs.
Larger learning rate (5e-6). 6). From Table we first observe that raising the learning rate amplifies Anmeral-erffrmancee degradation across nearly all methods. In this more challenging regime, TALR
7

---

Published as a conference paper at ICLR 2026
(b)
Taken Probabiltty Distribution Model
Token Probability
1.0
0.5
0.0
Owee 53
Qwan-2.55p
Qwen-34p
Qwen-J-at S-t8
Gemme-3 3-18
Gemman-3DD 41B
e
Training updates use only tokens with 0.2
0.90
of Tokens with
0.85
0.80
0.75
Epoch
Epoch
0.70
0.2
0.4
0.6
0.8
1.0
200
400
800
1000
1200
Probability
Step
Figure 2: Token-level analysis on the MedCalc dataset. (a) Heatmap of token probabilities from Qwen-2.5-3B --nstuuttttt for an example. Darker cells indicate higher model confidence; harder tokens with low probability often correspond to domain-specinf concepts. (b) Distribution of token prob- abilities across the full SFT training set for multiple models. Most tokens are confidently predicted (medians near 1.0), suggesting low learning difficulty.
(c) Fraction of tokens with > 0.2 increases from epoch epoch when training updates use tokens with 0.2, showing a clear curriculum phenomenon.
stands out: it achieves clearly favorable Pareto-optimal trade-off, maintaining competitive domain gains with noticeably smaller drops in general performance.
Takeaway. When feasible, small learning rate already delivers solid trade-off; additional knobs can be unnecessary. When higher learning rates are required to push domain performance, TALR demonstrates clear superiority by achieving stronger trade-offss. However, ods, including TALR, can fully mitigate the sharp increase in cemerra-performann degradation under larger learning rates. This highlights an open challenge and points to the need for further exploration of more powerful mitigation strategies.
of the existing meth-
4.3 TOKEN-LEVEL ANALYSIS
In this part, we analyze domain-specific SFT at a fine-grained level, i.e., at the level of individual target tokens. To this end, we compute the probability of each target token x; during SFT, where the model is trained to predict the next token conditioned on the prompt Tprompt and previous tar- get tokens I<t, formulated p(x, Eprompe,<<tt This formulation allows us to quantify token difficulty.
Finding 1: Most tokens in SFT training data pose low learning difficulty. We begin our analysis with token-level visualization of a training example from the MedCalc dataset. Figure 2[a) shows the model's predicted probability for each target token conditioned on the input prompt and all previous target tokens. Tokens with darker colors indicate higher confidence (i.e., higher probability or lower token loss), while lighter colors highlight tokens that the model finds more difficult. As shown, the majority of tokens in the target sequence are confidently predicted by the model, particularly in the later steps the reasoning process. This aligns with the intuition that once sufficient context accumulated, well-trained LLM can easily predict subsequent tokens.
Notably, small number of hard tokens, ie., those with low predicted probabilities, do appear throughout the sequence. typically in earlier positions around domain-specii concepts that may not be well covered in pretraining. For example, in the sixth row of the heatmap Figure 2(a), the token representing the numeric value in the phrase "conversion factor, 3 mmn//g" is assigned
8

---

Published as a conference paper at ICLR 2026
Input: Domain dataset D. parameters 8, learning rate temperature T > 0, weight floor Wiin
Output: Updated parameters 0
foreach training step do
Sample a mini-batch { propp 1 from D;
Forward pass to obtain token probabilities {pt} for all supervised tokens in the batch;
Token NLLs: l Pog
Adaptive weights with lower-hound clipping:
Wr - exp(-L//T)
Wy f max m-xxaaxxxx))) Wmin)
Let be the number of supervised tokens the batch; Mean (averaged) reweighted loss:
CTALR
1 N
Σ
We (- log Pt)
Parameter update:
0 0 - 11 VoLTAR
end
Algorithm I: Token-Adaptive Loss Reweighting (TALR) for Domain-Specific SFT. The sg(-) operator denotes stop gradient, meaning that w, is treated as constant during backpropagation to prevent gradients from flowing through the weight computation.
low probability, likely because such clinical conversion factors are underrepresented in the model's pretraining data.
To move beyond a single example, we perform a broader statistical analysis by collecting token-level probabilities across all SFT data in the MedCale training set. Figure ([b) presents box plots of these token probabilities across six model variants. Across all models, we observe a consistent pattern: the upper quartiles tightly clustered near 1.0, and the medians are consistently high, indicating that a large portion of tokens in the training sequences are already assigned high confidence by the models. However, despite this abundance of easy tokens, the models' zero-shot performance on the MedCalc test set remains relatively low, as shown in Figure [ (Init point). This mismatch suggests that performance bottlenecks may stem from small subset of more challenging tokens which are associated with Nomain-specifet reasoning or clinical knowledge. These hard tokens may be sparse but crucial.
Finding 2: TALR training dynamics exhibit curriculum-like phenomenon. We conducted an extreme experiment and observed that TALR implicitly creates a training curriculum. Specifically, we clipped the gradients of all tokens whose predicted probability was below a threshold, so that only higher-confidence tokens contributed to updates. As shown in Figure (eguureeo) c). the fraction of tokens exceeding this threshold grows steadily from Epoch to Epoch 2. This dynamic effectively induces a curriculum- like learning schedule: the model begins with "easier" tokens (those already predicted with moderate confidence) and gradually incorporates larger set of tokens, including those that were harder at the start.
4.4 LEARNING DYNAMICS OF TALR
To analyze the actual behavior of TALR during optimization, we monitor several training-time sig- nals for Qwen2.5-3B fine-tuned on the MedCalc dataset. Specifically, we track: (1) the token-level loss before reweighting, (2) the overall training loss after the TALR reweighting is applied, and (3) the dynamic hyperparameter т, defined as the median average token loss within each batch.
The results are shown in Figure 3 During the first epoch, both the token loss and the final training loss decrease sharply, and they continue to stabilize during the second epoch. Importantly, T also decreases substantially as training progresses. Since reflects the median difficulty level of tokens within a batch, its steady decline indicates that growing proportion of tokens transition from being initially hard to being easier for the model.

---

Published as a conference paper at ICLR 2026
(b)
(c)
Betore Reweightngg During
Training nes During Training
Median Aug Tokin Loss During Training
Epoch
Epoch 2
Epoch
Spoch
Epoch
Eoreing Logs
0.2
250
-20
12.0
300
1000
1700
Training Stop
00
200
400
802
1000
1230
Figure 3: Training dynamics under the true TALR algorithm for Qwen2.5-3B -nntttucttt on MedCalc. Panel (a) shows the token-level loss before reweighting, panel (b) shows the training loss of TALR and panel (c) tracks the value of the dynamic hyperparameter (median average token loss) through- out training.
These observations confirm that the TALR reweighting mechanism does not impede learning. In- stead, TALR allows the model to follow normal optimization trajectory in which hard tokens gradually absorbed, while simultaneously reducing the destabilizing influence of extremely low- probability tokens in early training.
CONCLUSION AND OUTLOOK
In this work, we presented both empirical and theoretical evidence that challenges the common be- lief that domain-specific SFT significantly harms general-purpose capabilities L.MMs. Through controlled experiments, showed that smaller learning rates yield more favorable trade-offs. Mo- tivated by our theoretical analysis, we further propose TALR for better trade-off.
5.1 LIMITATIONS
Looking forward, while TALR marks step toward mitigating in domain-specific adaptation, our findings also highlight that no single method fully resolves this challenge. Future work should explore more principled strategies to further enhance the robustness of LLMs across domains while preserving their general-purper strengths. Second, due to no longer having access to compute resources for this project, were not able to evaluate these representative mitigation strategies on a wider range of datasets. Nevertheless, our experiments provide consistent evidence supporting our main findings, and leave to the broader community to further examine and verify their generality. In addition, due to resource constraints, we were unable to examine whether larger models or mixture-of-exxee exfeeeterts (MoE) architectures follow the same dynamics, leaving open questions about scalability and architectural differences. Besides, on the theoretical side, while our analysis explains the observations, we did not address the problem of how optimally select learning rate that achieves the best trade-off in practice. Developing such principled selection rules remains an important direction for future work.
degradation
5.2 BROADER IMPACTS
Better domain adaptation. Our findings provide practitioners with insights when developing domain-specific LLMs. Taking the medical domain as an example, Jeong et al.) (2024) show that existing mediccalspeccaalized LLMs often fail outperform their corresponding initialized LLMs. This suggests that the quality of domain-specific data alone may not be as high as in the sophisti- cated post-training pipelines applied to base models. Hence, methods that preserve much of the initialized LLM's general capabilities as possible while injecting domain knowledge may lead to stronger overall performance.
Mitigating exploration loss in SFT warm-up for RLVR. Before reinforcement learning with ver- ifiable reward (RLVR), SFT often used as warm-up step to inject knowledge (Lin al. 2025b) or align formats (Guo et al. 2025). However, excessive SFT can over-stabilize the model. causing its output trajectories to become rigid and thereby undermining exploration during RL training. In contrast, models prior to excessive SFT typically exhibit more diverse behaviors. Thus, strategies that mitigate general-performance degradation and preserve the base model's diversity may help alleviate this issue and enable more effective RI.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Parikshit Bansal and Sujay Sanghavi. Context-free synthetic data mitigates forgetting. arXiv preprint
arXiv:2505. 13811, 2025.
Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Con- nor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. LoRA learns less and forgets less. Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: http:///pprreeeeeeeeeeeeme. net/ htt.://iwwweeeeeemmmmmmmmmm forum?
/
aloEru2gCG Featured Certification.
Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical com- monsense natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7432-7439, 2020.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xic. Sft or rl? an early investigation into training rl-like reasoning large lision-language models. arXiv preprint arXiv:2504. /468, 2025.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda. Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 2021.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of founda- tion model post-iraining. In Forty-second Internationall Conference on Machine Learning, 2025. URL htt.hhttpps///pprreeeeeee wtt.:/n/ooooommmmoooooooooooaaa
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick. and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. ArXiv, abs/1803.05457. 2018.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.1411 2021.
Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In The Twelfth International Conference on Learning Representations, 2024. URL https:// openreview net/ wwwtteorreeeeeeeeeee http://oruumidd2222/22/hhhh
Gregoire Deletang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christo- pher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, and Joel Veness. Language modeling is compression. In The Twelfth International Confer- ence on Learning Representations, 2024. URL https::/ httpss://openreview.. net/ fwwptt/ffooomdddoooo jznbgi ynus
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. URL http::::///eenddooo/r/eeeeeeeeee http:////.eoo////000000000 /12608602
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint (tti::200.129444 2025.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Mcasuring massive multitask language understanding. arXiv preprint arXiv:2009.085 03300, 2020.
11

---

Published as a conference paper at ICLR 2026
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):,, 2022.
Maggie Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Pooven- dran, Graham Neubig, and Xiang Yue. Does math reasoning improve general Ilm capabilities? understanding transferability of IIm reasoning. arXiv preprini arXiv:2507.00432. 2025.
Mareus Hutter. Universal Artificial Intellegence Sequential Decisions Based on Algorithmic Prob- ability. Texts in Theoretical Computer Science. An EATCS Series. Springer, 2005. ISBN 978-3- 540-22139-5. doi: 10. 1000000/B1130 138233. URL
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh International Confer- ence on Learning Representations, 2023. URL https::/sc/cccc /BBLLtttp//ppeeeeeee...ooo net forumiod 6tCE
Daniel P Jeong, Pranav Mani, Saurabh Garg, Zachary C Lipton, and Michael Oberst. The lim- ited impact of medical adaptation of large language and vision-language models. arXiv preprint arXiv:241 1.08870, 2024.
Jiaming Ji, Kaile Wang, Tianyi Alex Qiu, Boyuan Chen, Jiayi Zhou, Changye Li, Hantao Lou, Josef Dai, Yunhuai Liu, and Yaodong Yang. Language models resist aligmment: Evidence from data compression. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Moham- mad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27- August 1, 2025, pp. 23411-23432. Association for Computational Linguistics, 2025. URL attps://aee Btttss:/aclantholle http::/ooooooooo////aaa--------- www-llnn............ 1141/
Pengcheng Jiang, Jiacheng Lin, Lang Cao, Runchu Tian, SeongKu Kang, Zifeng Wang, Jimeng Sun, and Jiawei Han. Deepretrieval: Hacking real search engines and retrievers with large language models via reinforcement learning. arXiv preprint arXiv:2503.002) 2025.
Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-rl: Training lims to reason and leverage search engines with reinforcement learning. arXiv preprint 25X3v:2503.09516, 2025.
Nikhil Khandekar, Qiao Jin, Guangzhi Xiong, Soren Dunn, Serina Applebaum, Zain Anwar, Maame Sarfo-Gyamfi, Conrad Safranek, Abid Anwar, Andrew Zhang, et al. Medcale-bench: Evaluating large language models for medical calculations. Advances in Neural Information Processing Systems, 37:84730 84740, 2024.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- -arsskaaaaBiikka et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3522 2017.
Anat Kleiman, Gintare Karolina Dziugaite, Jonathan Frankle, Sham Kakade, and Mansheej Paul. Soup to go: mitigating forgetting during continual learning with model averaging. arXiv preprint wrXiv:2501.05559 2025.
Saurabh Kumar, Henrik Marklund, and Benjamin Van Roy. Maintaining plasticity in continual learning via regenerative regularization. In Conference on Lifelong Learning Agents, pp. 410- 430. PMLR, 2025.
Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pitore-Antoine Gourraud, Mickaël Rouvier, and Richard Dufour. Biomistral: A collection of open-source pretrained large language models for medical domains. In Findingy of the Association for Computational Linguistics: ACL 2024, pp- 5848-5864, 2024.
Yuanzhi Li, Colin Wei, and Tengyu Ma. Towards explaining the regularization effect of initial large learning rate in training neural networks. Advances in neural information processing systems, 32, 2019.
12

---

Published as a conference paper at ICLR 2026
Jiacheng Lin, Hanwen Xu, Zifeng Wang, Sheng Wang, and Jimeng Sun. Panacea: A founda- tion model for clinical trial search, summarization, design, and recruitment. arXiv preprint arXiv:2407.11007, 2024.
Jiacheng Lin, Tian Wang, and Kun Qian. Rec-rl: Bridging generative large language mod- els and user-centric recommendation systems via reinforcement learning. arXiv preprint arXiv:2503.24229, 2025a.
Jiacheng Lin, Zhenbang Wu, and Jimeng Sun. Training Ilms for ehr-based reasoning tasks via reinforcement learning. arXiv preprint arXiv:2505.24 2025b.
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019. URL
Bkg6RiCqY7
Ekdeep Singh Lubana, Puja Trivedi, Danai Koutra, and Robert Dick. How do quadratic regularizers prevent catastrophic forgetting: The role of interpolation. In Conference on Lifelong Learning Agents, pp. 119-837. PMLR, 2022.
Subha Maity, Mikhail Yurochkin, Moulinath Banerjee, and Yuckai Sun. Understanding new tasks through the lens of training data via exponential tilting. In The Eleventh International Confer- ence on Learning Representations, 2023. URL /tttps:///penreviee... wtw.i/eummee//oummmmmmmam
DRMt L EBOobw
Amirkeivan Mohtashami, Martin Jaggi, and Sebastian U Stich. Special properties of gradient de- scent with large learning rates, In International conference on machine learning, pp. 25082- 25104. PMLR, 2023.
Jisoo Mok, Jaeyoung Do, Sungjin Lee, Tara Taghavi, Seunghak Yu, and Sungroh Yoon. Large- scale lifelong learning of in-context instructions and how to tackle it. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 25573-12589 2023.
Aldo Pareja, Nikhil Shivakumar Nayak, Hao Wang. Krishnateja Killamsetty, Shivchander Su- dalairaj. Wenlong Zhao, Seungwook Han, Abhishek Bhandwaldar, Guangxuan Xu, Kai Xu, Ligong Han, Luke Inglis, and Akash Srivastava. Unveiling the secret recipe: A guide for su- pervised fine-tuning small LLMs. In The Thirteenth International Conference on Learning Rep- resentations, 2025. URL https: openreview net www.moirruiiiromoero.omrr ForumnPI-=EENHKTTTT
Bo Peng, Xinyi Ling, Ziru Chen, Huan Sun, and Xia Ning. ecellm: generalizing large language models for e-commerce from large-scale. high-quality instruction data. In Proceedings of the 41st International Conference on Machine Learning, pp. 40215 2024.
Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Alma- hairi. Progressive prompts: Continual learning for language models. In The Eleventh Interna- tional Conference on Learming Representations, 2023. URL
Corum?1d=0JTggBcC9.
Chandan K. Reddy, Lluís Mårquez, Fran Valero, Nikhil Rao, Hugo Zaragoza, Sambaran Bandyopad- hyay, Amnb Biswas, Anlu Xing, and Karthik Subbian. Shopping queries dataset: A large-scale ESCI benchmark for improving product search. 2022.
Ildus Sadrdiiov, Maxim Kodryan, Eduard Pokonechny, Ekaterina Lobacheva, and Dmitry P Vetrov. Where do large learning rates lead us? Advances in Neural Information Processing Systems, 37: 58445-58479, 2024.
Sunny Sanyal, Hayden Prairie, Rudrajit Das, Ali Kavis, and Sujay Sanghavi. Upweighting easy samples in fine-tuning mitigates forgetting. In Forty-second International Conference on Machine Learning. 2025. htt.::ttt///eeeett htwp://nweeeeeeeeeeeeeeeeeee net./ forum? id=1 3HP TmZKbM
Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. Fine-tuned language models continual learners. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 6107-6122, 2022.
13

---

Published as a conference paper at ICLR 2026
Claude E Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379 423, 1948.
Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. Rl's razor: Why online reinforcement learning forgets less. arXiv preprint arXiv:2509.04254 2025.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov. Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.092881 2023.
Liyua Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE transactions on pattern analysis and machine intelligence, 46(8):5362-5383, 2024a.
Xiao Wang. Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan-Jing Huang. Orthogonal subspace learning for language model continual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10658-10671, 2023.
Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. Insel: A data-efficient continual learning paradigm for fine-tuning large language models with instructions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume I: Long Papers), pp. 663- 677,200 2024b.
Ian Witten, Radford M Neal, and John G Cleary. Arithmetic coding for data compression. Com- munications of the ACM, 30(6))500-540, 1987.
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannanch Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine-tuning of zero-shot models. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7959-79971, 2022.
Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: survey, 2024. URL https://arxiv..or
abs/2402. 01364]
Weimin Xiong, Yifan Song, Peiyi Wang, and Sujian Li. Rationale-enhanced language models are better continual relation leamers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 15489-15497, 2023.
Hanwen Xu, Naoto Usuyama, Jaspreet Bagga, Sheng Zhang. Rajesh Rao, Tristan Naumann, Cliff Wong, Zelalem Gero, Javier González, Yu Gu, et al. A whole-slide foundation model for digital pathology from real-world data. Nature, 630(8015): 181-188. 2024.
Hanwen Xu, Jiacheng Lin, Addie Woicik, Zixuan Liu, Jianzhu Ma, Sheng Zhang, Hoifung Poon, Liewei Wang, and Sheng Wang. Pisces: A multi-modal data augmentation approach for drug combination synergy prediction. Cell Genomics, 5(7), 2025.
Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin. Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. CoRR, abs/2412.151115 URL https: http//dci.oog/o////// www.oogg.1000..... 4850/00..455//2220 2024. doi: 10.46500/ARXI..2122.5555
2412.2 15115
14

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:25005.03388, 2025.
Da Yin, Xiao Liu, Fan Yin, Ming Zhong, Hritik Bansal, Jiawei Han, and Kai-Wei Chang, Dynosaur: A dynamic growth paradigm for instruction-tuiing data curation. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL lattps://openrevill
net
Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. In The Twelfth International Conference Learning Representations, 2024. URL http::::///peeeeeieeeee net wtteee/oooonmmn..aeeaaaaaaaaaaa
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a ma- chine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 4791-4800, 2019.
Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che. Sapt: shared attention framework for parameter-efficient continual learning of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 11641-11661, 2024.
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Inslrwction-folloiingg evaluation for large language models. arXiv preprint arXiv:231..07911, 2023.
15

---

Published as a conference paper at ICLR 2026
Contents of Appendix
LLM Usage Statement
17
heeorttiaa Analysis
17
Preliminaries
17
B.2
LLM Compression Protocol
17
3
Approximating Fine-tuning Dynamics with Exponential Tilting
18
B.4
4
Why Smaller Learning Rates Yield Favorable Trade-ofis?
19
Additional Experiment and Result Details
20
Dataset Details
20
C.I.I
MedCale
20
C.1.2 ESCI
20
C.1.3
MetaMathQA
................................
C.1.4
General-Purpor Benchmarks
21
C.2
Implementation Details
21
Additional Details of Experimental Setup and Results
21
C.4
Effect of KL Regularization
22
Evaluation on Multi-Choice Commonsense and Knowledge QA
23
C.6
Performance evolution across training epochs
23
Observing the Ratio of Low-Probability Tokens
24
Details of Token-Adaptive Loss Reweighting
25
D.I Deriving Token Weights: Proof of the Closed-form Solution
25
D.2 Implementation Details of TALR
25
D.3 More Discussions
25
Additional Definitions, Theorems and Proof
27
Proof of Thhorrm
27
Proof of Theorem B22
30
16

---

Published as a conference paper at ICLR 2026
A LLM USAGE STATEMENT
In this work, Large Language Models (LLMs) were primarily used for text refinement, such as improving the clarity of writing. addition, for the ESCI experiments, the chain-of-thought (CoT) data was generated using Swen2.5-72B-Instree through rejection sampling.
B THEORETICAL ANALYSIS
To better understand the empirical phenomena observed in $ we provide theoretical analysis from the perspective of information theory. Our goal to explain Finding and mentioned in $3.2 To this end, we first introduce several compression-based tools that form the basis of our analysis. We then apply these tools shed light on the two key findings highlighted earlier.
B. PRELIMINARIES
Supervised Fine-tuning (SFT). In SFT, the LLM is trained on labeled dataset DSFT {(x11) , where x is a natural language prompt (e.g., an instruction question), and y (y1, 1/2 "2,....,97. UT,)) the corresponding target response, represented as a sequence of to- kens. The objective of SFT is to maximize the conditional likelihood of the target sequence given the input æ, which corresponds to minimizing the following negative log-likelihood loss:
(SFT(()) (ay))-DDe Σe log Toly 2r, ver) where no denotes the model's output distri- bution over the vocabulary, and Ver is previous target tokens.
Lossless Compression. In lossless compression, the goal is encode sequence of symbols x (I1, ....... 2T) drawn from a source distribution P' into a binary representation without any loss of information, such that the original sequence can be perfectly reconstructed. According to Shannon's source coding theorem (Shannon, 1948), the limit of compression is given by the Shannon entropy of the source: H(P)= : Ep- log P(x)]. which specifies minimum expected number of bits symbol needed for encoding.
LLM Modeling is Compression. Given a dataset ID drawn from the true distribution P and a model distribution Q. the expected code length under arithmetic coding (Witten et al., 1987) is given by (P, := Ep-pl I - log Q(x)]. Thus, minimizing the log-likelihood loss directly corresponds to reducing the expected compression rate when the model is employed lossless compressor
(Delttang et al. 2024 Hutter 2005 Ji et al. 2025
B.2 LLM COMPRESSION PROTOCOL
Our goal is to analyze the dynamics of domain-specinne SFT. Motivated by the equivalence between language modeling and data compression (Deletang et al. 2024 Ji al. 2025). we view an LLM as a compressor, where the effectiveness of training can measured through changes code length. In this view, improvements degradations in performance across datasets correspond to variations in compression rate. Below, we formalize this perspective by introducing the notion of token trees and describing the LLM compression protocol in context.
oc, where V {11, o2...., U is a finite vocabulary of size (V), the token tree of D, denoted as To. defined as follows: (1) each node has [V] child nodey labeled un U2...., Uly, along with an und-of-sequent (EOS) leaf node; (2) The weight of non-leaf node is the sum of the weights of all its child nodes; (3) The path from the root to an EOS leaf node defines a response 2,, with the corresponding EOS node weight representing the response probability.
Definition B.1 (Token Tree T). For dataset D {2s you i 1,2....). M
17

---

Published as a conference paper at ICLR 2026
Definition B.2 (LLM Compression Protocol). Let To be the token tree of dataset D, and let go(. I u) denote the conditional distribution over VU [EOS) predicted by an LLM with parameters at node u E TD. Given a response z (a path from the root to an EOS leaf, truncated to a pre- defined maximum depth d), the LLM compression protocol encodes using arithmetic coding. where at each step the coding probabilities are given by qo(. for the current node u along the path of
The truncation to a maximum depth reflects practical constraints in the use of large lan- guage models. For example, responses are usually limited to fixed context window, and generated sequences are typically bounded in length.
Remark:
Proposition B.1 (Expected Code Length). Consider a finite parameter model qo(-) and a token tree To Iruncated depth d. Under the compression protocol of Definition 3.2 the expected code length of random response is Ez~pp [L)(2)] = - 11 -i شØ=1 Pl.j log 9.5, where P is the distribution over responses. Pı.j denotes the probability assigned to the leaf node u., (the j-th node at layer of TD), and Q.j is the probability assigned to the node w.j by the model qo(-).
Proposition B.2 (Joint Token Tree for Multiple Datasets). Consider N pairwise disjoint datasets DI....,Dyy Dr. each with its own token tree TD: Let D = U Dj be the union dataset, and let TD denote its token tree. For each node u1 the node weight in TD is given by vf, EN (PllPP)) (E (D(), where p?j is the node weight in To. and [Ds] the number of responses in dataset Dz.
Proposition B.3 (Expected Code Length Discrepancy under Model Shift). Consider two model
distributions qe, (-) and qo,(.) (.) over the token tree TD with distribution P. The change in expected
code length on P when shifting from qo, to q0₃ (LP)) E2~PlLee, ( (z)] 1Ez- pL (z)
-E Ey PLs log 4
Equivalently, AL(P) = KT(P (o.)) KT.(P q0.)-
Based on the above, adopt the expected code length surrogate metric for LLM's s modeling quality on a given dataset (Deletang et al. 2024). Specifically, reductions in code length discrepancy indicate better alignment between the model distribution and the data distribution, whereas increases suggest deterioration. This perspective will serve as the foundation for our subsequent analysis.
B.3 APPROXIMATING FINE-TUNING DYNAMICS WITH EXPONENTIAL TILTING
Our goal in this part is to uncover the behavior of LLMs during domain-specific fine-tuning. Fine- tuning alters the conditional distributions assigned to each node in the token tree, thereby shifting the model's alignment with the data distribution. For analytical simplicity, we view fine-tuning at a high level as introducing perturbations to the probability assigned each token node. To approximate these dynamics, we adopt the lens of exponential rilting (Maity et al. 2023), which captures how the distribution reweighted under incremental updates.
Note that, exponential tilting is not strictly equivalent to SFT; rather, it serves as an analytical surrogate that enables us to extract insights and motivation about the mechanisms driving perfraappefformance degradation and domain adaptation. In the following, we formalize the expo- nential tilting formulation and present how approximates the token-level probability shifts induced by fine-tuning. We then provide error estimates that quantify the gap.
Setup. We consider a pretrained LLM with distribution Qo. which already models the dataset Dj well. The model is then fine-tuned on new dataset D2, For node in the token tree TD, of Dz, let (-(- u) denote the empirical target distribution induced by D2, and let qe(- u) denote the model distribution at step t during fine-tuning.
Assumption B.1 (Full support via mild smoothing). To avoid support collapse, the target used in each step is defined as a smoothed mixture P2,( u) = (1 a) f2(. u) + a p:(- u), a E (0, 1), where pel- w) is strictly positive reference distribution (e.g.. the current model qel: | u) or the uniform distribution). Hence (a, u) > 0 and qı(a u) > ffr all tokens a
Assumption B.2 (Small step in distribution space). Each update is small at the distribution level: (T (++1) 6, & 1.
18

---

Published as a conference paper at ICLR 2026
Assumption B.3 (Smoothness finite tree). log go(a u) is twice continuously differentiable in with bounded second derivatives in a neighborhood of de vocabulary and depth are finite. Conse- quently, Taylor remainders are O(10,+1 ).
Definition B.3 (Exponential Tilting Update). For any non-leaf prefix u in the token tree T and
step parameter À E [0, II. define the log-ratio (a) log( u)/u:(a u)), E VU(BOS}. The exponential tilting update at prefix u is given by
a: u) exp(Aru(a))
q: u) --)-> p2,e(a u)
Zo+1 (a u) - E. [u) (xp(Ar.(())
Ee u) 1-A 1,,1 u)>
The boundary cases are consistent: X = 0 recovers qe(: u), while X = recovers p2, u).
Theorem B.1 (First-order approximation by exponential tilting). Fix prefix Consider the current model distribution (e u) and the smoothed larget distribution p22. u). Define the local L2 norm Ilgll.,u = (1))(((((())))))
Under the standing assumptions, there exists an
effective step size Xe such that
log q0+1(- u) [1L (11 M,,)) log (- [u) Xe log p- |1) c'e, Il.. -(s).
where te the log- nognooomal and is the KL trust-region radius such that KL(ge g0+1)) S E.
The proof can be found in E.1 Based on Theorem B.I we establish that for distribution update 4++1 whose KI. divergence from gi is bounded by e, the corresponding exponential-tilting approxi- mation Tr+1 differs from 4++1 only up to O(e). In other words, exponential tilting provides a first- order approximation, thereby justifying its use as an analytical tool to study general-performance degradation and domain adaptation.
B.4 WIYY SMALLER LEARNING RATES YIELD FAVORABLE TRADE-OFFS?
In this subsection, we provide theoretical explanation for the empirical findings observed in $3
Notation. Fix prefix u (we omit "I w" when clear). Write f(a) = log (a) - -log g(a) at the current iterate q (the step index t is omitted for readability), and f E,1, - For set of token-tree nodes, denote its g-mass by w/s = [1s].
Assumption B.4 (Sparse token-level shift on D2). There exists a measurable node set S C T with small mass ws < under the D2-prefix distribution such that
If(a)) & Mr for a S. If(a)) & Mi for a E S,
with Mj & Mn. In words, most tokens are already well modeled by Q due to pretraining, while only a small subset requires nontrivial adjustment toward the P2 target, denoted as the hard tokens (low probability tokens).
Remark. This assumption reflects the practical setting where domain-speciff finetuning mainly affected by a small fraction of tokens, consistent with our empirical analysis in $4.3
Assumption B.5 (Realizability with controlled leakage). Let Is and Is denote the orthogonal projections of a log-space function onto the coordinates indexed by S and its complement. respec- tively (with norm measured in (z))), There exists small y E [0,1) such that for any desired log-space direction g (defined per-prefix on the token tree) whose energy primarily S (i.e., [|TT=gll S 3 |[TTsg (1(()) for some small B 2 0), there is a parameter update that realizes global change A logg satisfying
||| loloq - ()( I > [1[Usglll2(g)
|| A log alleAg S (3 + |115g1
In words, the update moves all tokens, but the relative magnitude outside S is small and controlled.
19

---

Published as a conference paper at ICLR 2026
Theorem B.2 (Smaller steps yield a smaller general performance degradation bound equal domain performance gain). Fix desired domain improvement A, on D2 (i.e., ALr(P2) S -. Among -tep tiltnng schedules that achieve this target, the minimal upper bound on the increase of code length D1 satisfies
A
(AC2 HT
C1 C PT
A + o
(L) I
A
+
PT
T
where prr == infi<T (L(Q)I)P2) > O and A Hr( Vur ws M, - Mi + (3 + *)) are fixed value under the total number of update steps T and the desired domain gain A.. The upper bound strictly decreases as increases. Thus, under the equal-steps schedule that attains the larget, the per-step effective weight scales as Xe 1/T; thus, for the domain gain, larger implies smaller per-step updates. Hence, smaller step size = smaller upper bound.
Theorem B.3 (Label-only supervision enlarges the safe per-step range). Among all T T-step tilt- ing schedules, the maximal per-step size that can guarantee a general-performen degradation (Pt) ATT Eig Amax (111 /8), where is the expected number hard tokens (low proba- bility tokens) per example on D2.
The proof of Theorem B.2 and B.3 can be found in the Appendix E.2 Theorem B.2 shows that, for achieving the same domain improvement, smaller learning rates (i.e., smaller per-step updates with larger 7') lead smaller upper bound general capability degradation, thereby explaining Finding 1. Theorem B.3 indicates that the bound on the safe step size is inversely proportional to the number of hard tokens. Therefore, in Finding when only labels are used for training, the number of hard tokens is smaller than that in training with both CoT and label data. This explains why in the ESCI experiments, under w/o CoT, both 5e and 1e-6 achieve similarly small degradation in general performance.
C ADDITIONAL EXPERIMENT AND RESULT DETAILS
C.I DATASET DETAILS
In this section, we provide additional details for both the Nomain-speciio datasets used for SFT and the general- -prrpose benchmarks used to evaluate general capability degradation. An overview of all datasets and their corresponding evaluation metrics is provided in Table and Table 5
C.I.1 MEDCALC
We use the MedCalc dataset (Khandekar et al. 2024) for medical reasoning tasks. The benchmark provides human-annotated chain-of-thought (CoT) rationales, which we include during training so that the model learns to reason through intermediate steps before producing the final answer. The prompt can be found in Table
C.1.2 ESCI
We use the ESCI dataset (Reddy et al. 2022) for multi-class product classification task, where each query-product pair labeled as Exact, Substitute, Complement, or Irrelevant. From the original dataset, we randomly sample 50K subset from the training split and a 10K subset from the test split. From training subset, we further hold out 1K examples as a validation set.
We consider two training settings: w/o CoT and w/ CoT.
w/ CoT: The target sequence includes both a chain-of-tin of-thought rationale and the final label, re- quiring the model to learn the reasoning process before producing the prediction. These CoT- augmented examples are generated via rejection sampling from Qwen2.5-72B-Insee in 34,176 training examples. The prompt is shown in Table
resulting
w/o CoT: The target sequence contains only the ground-truth label. the model is trained to directly predict the class without generating intermediate reasoning (49k examples). The prompt is shown in Table
20

---

Published as a conference paper at ICLR 2026
All prompt examples in Tables 7 and use the Qwen chat template for illustration; for other model families, we adapt the prompt to their respective chat formats.
The ESCI dataset is highly imbalanced, with the majority of samples belonging to the Exact cate- gory (Table 3). This imbalance motivates our choice of balanced accuracy (BACC) as the primary evaluation metrie, following prior work imbalanced classification (Xu al. 2024 2025
Table 3: Label distribution for the ESCI subsets used in our experiments. Percentages are shown in parentheses.
Split
Exact
Substitute
Irrelevant
Complement
Train (49K)
33,958 (69. 30%)
9,753 (19.90%)
4,261 (8. (8.70%)
1,028 1.10))
Val (IK)
674 (67.40%)
212 (21.20%)
94 (9.40%)
20 (2.00%)
Test (10K)
6,470 (64.70%)
2,268 (22.6%%)
992 (9.92%)
270 (2.70%)
C.1.3 METAMATHQA
MetaMathQA Yu et al. 2024 is large-scale mathematical reasoning dataset containing 395k train- ing examples. Following Sanyal et al. (2025), we use MetaMathQA for training and take GSM8K as the target-domain evaluation benchmark. This setup allows to validate whether our findings hold under large-scale data conditions.
C.1.4 GENERAL-PURPOSE BENCHMARKS
For the general-pue purposse benchmarks, fully follow the default settings and evaluation metrics implemented in the 1m- in--valuuat i on Melooooaan harness framework (Gao et al.] (024)) This ensures
consistency with prior work (Lin al. 2025a; Sanyal et al., 2025] Bansal & Sanghavi 2025 and
allows for fair comparison of results across different models and training configurations.
C.2 IMPLEMENTATION DETAILS
We conduct all experiments 16-32 NVIDIA A100 GPUs with 80GB memory. Except for differ- ences in learning rate and loss computation, all experiments share the same training configuration. We adopt the AdamW optimizer (Loshchilov & Hutter 2019) with hyperparameters 31 0.9 and 32 = 0.999, together with a cosine annealing learning-rate schedule. The attention mechanism is implemented using FlashAttention-2 (Dao 2024). We the batch size to 16 for MedCalc and ESCI, and 128 for MetaMathQA. The number of training epochs is 20 for MedCalc and ESCI, and 2 for MetaMathQA. The maximum sequence length is 8192 tokens.
C.3 ADDITIONAL. DETAILS OF EXPERIMENTAL SETUP AND RESULTS
In our experiments, we measure the trade-off between domain performance and general perfor- mance. Domain performance is defined as accuracy on the target downstream task, while general performance is computed as the average score across IFEval, GSM8K, and HumanEval unless other- wise specified. Importantly, our definition of general performance is consistent with the theoretical analysis, where we assume the base model already achieves reasonably strong results. To ensure consistency, we exclude benchmarks where the model's absolute performance below a threshold of 0.5, evaluated by wwweeaalllee watt on- harness framework. Thus, for Gemma-3-4B we report the average over IFEval and GSMSK, while for Gemmma-31BB we only include IFEval.
We also conduct supplementary experiments, as shown in Figure which further validate and extend our findings from Section 3
Finetuning datasets where the model already performs strongly. In Figure 4(). Qwen3- 8B achieves close to 50% accuracy on ESCI with CoT supervision even before SFT. Despite this high baseline, the results confirm main conclusion: using small learning rate continues to yield a more favorable trade-off between preserving general performance and improving domain performance.
21

---

Published as a conference paper at ICLR 2026
(a)
Qwen3-8B + ESCI
(b)
Qwen3-8B + ESCI (w/ CoT)
(c)
DeepSeek-Coderf MetaMathQA
0.65
General perforaace
General performance
0.8
Inil
Goneral performance
Inil
0.7
0.7
0.60
X
Ir=1e-6
Irw5e-8
1r=10-8
X
1r=2e-5
0.6
X
Ir=1e-6
0.55
X
Ir=2e-5
0.5
Ir=Se-6
Ir=5e-6
0.5
x
1r=2o-5
10-50-
0.50
0.50
0.55
0.2
0.4
0.4
0.6
Domain performance
Domain performance
Domain performance
Figure 4: Effect of learning rate on domain-specinfs and general capability performance during supervised fine-tuning (SFT). Results are shown for (a) Qwen3-8t on ESCI with CoT supervision, (b) Qwen3-8B ESCI without CoT, and (c) DeepSeck-Codeed Coder-7B on MetaMathQA. Across all settings, smaller learning rates achieve more favorable trade-offs.
(a)
= 1e-6
(b)
Ir 5e-6 (c)
Ir 2e-5
General performance
0.7
Init
General perfornance
0.6
Init
X
Standard KL-Reg
General performance
0.7
Init
X
X
KL-Reg
Standerd
0.6
0.5
0.4
0.4
0.5
0.3
0.2
0.2
0.4
Standard KL-Rog
0.0
0.0
0.2
0.3
0.4
0.5
0.2
0.4
0.6
0.1
0.2
0.3
0.4
0.5
Domain performance
Domain performance
Domain performance
Figure 5: Effect of KL regularization on demain-specifie SFT. We follow DeepSeek-RI (Guo al.) (025) and upply the k3 approximation for KL regularization. Results are shown for three learning rates: (a) 1 x 10-6 (b) 5 x 10-6, and (c) 2 x 10-5, Across all settings, KI. regularization yields performance that is very close to standard SFT, suggesting limited additional benefit in miti- gating
degradation.
Validation large-scale datasets. We additionally evaluate MetaMathQA to test whether our conclusions hold under large-scale training. To emulate realistic domain adaptation scenario, DeepSeek- Preoseec--Coer-75B which is highly specialized in code but weaker in mathematics. This setup mirrors adapting a model from one domain of strength (code) to another (math). As shown in Figure c),, report general performance using MBPP (rather than HumanEval, since DeepSeek- Coder-7B performs poorly on HumanEyal under 1m-eva I luat on-ha on-harress). The results again align with central finding: small learning rates achieve the best trade-offs. Interestingly, in this setting the optimal rate shifts to 5 x 10-6, 6,66, rather than x 10-6 as in carlier experiments. Moreover, an even smaller of 5 10-7 and observe that overly small rates can hinder target-domain performance, suggesting that learning rates cannot be arbitrarily reduced without consequence. Overall, these additional experiments reinforce our main findings.
C.4 EFFECT OF KL REGULARIZATION
We further investigate the effect KL regularization, technique recently adopted DeepSeek-Ri (uuo et al. 2025), where a 133 approximation is used to estimate the KI. term. Following prior work
22

---

Published as a conference paper at ICLR 2026
(a)
Qwen3-8B
(b)
Qwen2.5-7B
(c)
Qwen3-4B
(d)
Qwen2.5-3B
0.65
X
Ir-1e-6
X
I-1e-6
General perfommae
0.00
X
Ir=Se-6
In=2o-5
General perfemmncc
0.6 +
Gangral
0.50
X
I=-5-88
I=2a-5
xex
0.6
Int
0.5
0.58
In1
0.5
Pw1o-5
0.60
0.4
X
1m20.5
In-5-88
0.4
r-5e-6
0.4
0.3
0.4
0.5
0.2
0.4
0.6
0.2
0.4
0.8
Domain performance
Domatn performance
Domalnn performance
Domain performance
Figure 6: Effect of learning rate on the trade-off between domain performance and general multi-choice commonsense and knowledge QA performance. Domain performance is measured on MedCalc, while general performance is evaluated as the average accuracy across MMLU, ARC- Easy, ARC-Challenge, PIQA, and HellaSwag. Results are shown for (a) Qwen3-8B, (b) Qwen2.5- 7B, (c) Qwen3-4B, and (d) Qwen2.5-3B.
on KL-constrained training (Jin et al. 2025 Jiang al., 2025 Lin al. 2025a b), we add KL penalty term with coefficient 0.001 during SFT.
Figure shows results on the Qwen2.5-3B Instruct model fine-tuned MedCale. At small learn- ing rates, Kl.-regularized runs and standard SFT behave almost identically. As the learning rate increases, KI. regularization offering little tion. This indicates that, under our experimental settings, KL regularization provides only limited improvements and does not shift the trade-off between domain performance and general capability preservation. These results are consistent with our earlier observation in $ adopting a smaller learning rate already achieves a favorable balance, while additional knobs such KL regularization contribute little further advantage.
benefit in reducing seneral-performance degrada-
C.5 EVALUATION MULTI-CHOICE COMMONSENSE AND KNOWLEDGE QA
We further evaluate the effect of learning rate on the trade-off between domain and general per- formance in multi-choice commonsense and knowledge question answering tasks. Results are pre- sented in Figure Unlike our earlier observations on more complex domains such as mathematics and coding, find that the rates (c.g., 5e-6) 6) is less pronounced here. A possible explanation is that multi-choice benchmarks relatively more trivial requiring short-form predictions rather than long reasoning chains or structured outputs. As result, larger learning rates do not amplify forgetting as severely as in domains demanding longer and more complex generations.
degradation induced by relatively larger learning
C.6 PERFORMANCE EVOLUTION ACROSS TRAINING EPOCHS
To better understand how learning rate influences the interaction between domain performance and general performance over the training process, we plot figures of performance vs. training epochs in Figure 7 Figure 7 and report results for Gemma3-4B on the MedCalc benchmark. We make two observations. First, consistent with our main findings, the smallest learning rate achieves strong domain performance while substantially mitigating forgetting on general benchmarks. Second, for each learning rate, the domain performance typically peaks relatively late epochs. For example, for Gemma3-4B with learning rate of 1e-6, 6, the best MedCalc score is reached around epoch 12. This indicates that, during domain -peaifi SFT, running training for longer period can continue to improve domain accuracy.
We further examine a larger scale setting with DeepSeck-Cooee Coderr-BB on MetaMathQA, as shown in Figure 7c and 7t. We observe similar behavior. Domain performance improves steadily and often reaches its peak only after many optimization steps, and smaller learning rates help preserve general capabilities and achieve comparable even better domain performance. Note that in this case the horizontal axis covers only the first two epochs, but since MetaMathQA is large scale corpus, even epoch already corresponds large number of parameter update steps. These results confirm
23

---

Published as a conference paper at ICLR 2026
(a)
(b)
(c)
(d)
010
0.5
LR
fed
Se-6
LA
LR
11:4
12.5
02:
0.95
3123 Hokira
24
Gemma3-4B + MedCalc
DeepSeek-Coder-8. MetaMathQA
Figure 7: Training dynamics of domain and general performance under different learning rates. Panels (a) and (b) show domain performance and general performance respectively for Gemma3- 4B on MedCalc. Panels (c) and (d) show the corresponding curves for DecpSeck-Coder-2 on MetaMathQA. In both settings, small learning rates achieve strong domain performance while better preserving general capabilities.
that conclusions not restricted to small datasets and that the dynamics describe persist in a large scale training datasets.
C.7 OBSERVING THE RATIO OF LOW-PROBABILITY TOKENS
Fraction of Tokens with < 0.05 During Training
Low Probabality Takenn Ratio
0.200
0.175
Epoch 1
E Epoch
0.150
0.125
0.100
0.075
0.050
0.025
0.000
6
200
400
600
800
1000
1200
Training Step
Figure 8: Fraction low-probabilityy tokens during training on Sove22.--3--Insteee with Med- Calc. We track the proportion of tokens whose model probability satisfies p < 0.05 over the course of training. During the first epoch, the ratio such "hard" tokens decreases rapidly, and by the second epoch it approaches zero and remains near zero. This indicates that these initially low- probability tokens are successfully Iearned by the model as training progresses.
To examine whether tokens assigned low probabilities are eventually learned during training, we track the evolution of the proportion of such tokens for 1rren255-3B-Instruee fine-tuned on the MedCale dataset. We define low-probability tokens as those with predicted probability less than p < 0.05. Figure X plots the ratio of these tokens across training steps.
During the first epoch, the fraction low-probability tokens decreases sharply, indicating that many of these hard tokens are quickly absorbed by the model. By the second epoch, this ratio approaches zero and remains near zero for the rest of training. This pattern shows that low- cow-probyyylity tokens do not remain persistently difficult; instead, they gradually learned training progresses. This analysis provides direct evidence that TALR does not prevent the model from learning challenging tokens.
24

---

Published as a conference paper at ICLR 2026
D DETAILS OF TOKEN-ADAPTIN LOss REWEIGHTING
D.1 DERIVING TOKEN WEIGHTS: PROOF OF THE CLOSED-FORM SOLUTION
Proof. Introduce a Lagrange multiplier À for the simplex constraint Σ; wi 1, and multipliers M4 2 0 for the nonnegativity constraints, The Lagrangian is
[(w, A, (b) Σ (v)((0) + Tw; log wi) +1
Wi - Σ
For an interior optimum (w; > 0 so that μr 0), the KKT condition is
ac
()) (1 + log wi)
Ow;
Thus,
(,( X 7
w: exp --((0)) expl - -1).
log Wt
Normalization by E; Wi
1, then we have
= = exp( - (,))/7)
Z = Σ exp - (())
Z
j=1
D.2 IMPLEMENTATION DETAILS OF TALR
We highlight two key design considerations in applying TALR.
Weight cutoff. Without constraints, hard tokens may receive extremely small weights, which slows down learning or even prevents the model from learning these tokens. To address this, we introduce lower bound cutoff ensure that no token weight becomes too small. In all experiments, we set this cutoff to 0.01, which strikes a balance between preventing vanishing weights and still allowing TALR downweight challenging tokens.
Choice of т. The temperature 7 controls the sharpness of weight assignment and is crucial hy- perparameter. In our experiments, loss within batch, a strategy that consistently yields stable and strong performance across tasks. To better illustrate the effect of r, we plot Figure When a batch contains more hard tokens, the resulting is larger; in this case, weights assigned to hard tokens are not excessively small, pre- venting the model from failing to learn. Conversely, when the overall loss is smaller, the resulting decreases, which effectively acts hard clipping mechanism to prevent excessive parameter drift and catastrophic forgetting. Nonetheless, the problem of selecting 7 remains open, and future work may explore principled or adaptive strategies for temperature tuning in TALR.
chosen dynamically the median of the average sequence
D.3 MORE DISCUSSIONS
Comparison with FLOW. It is worth contrasting TALR with FLOW (Sanyal et al. 2025 which also reweights losses but in different manner. First, FLOW operates at the sequence level, whereas TALR works at the token level. Second, FLOW computes static weights only once before training, while TALR dynamically updates weights at every batch with negligible additional cost. As shown in Table and Table TALR consistently outperforms FLOWW, which aligns with our expectations. Sequence-level loss can be misleading: for example, even when the overall average sequence loss small, there may exist few particularly hard tokens with large losses that are overlooked at the sequence granularity. TALR directly addresses this by reweighting at the token level. Moreover, token difficulty is not fixed its relative hardness evolves training progresses, discussed in Section 4.3 This makes dynamic weighting naturally more advantageous than static approaches.
Why not fix We also examine the impact of fixing the temperature parameter, e.g., setting 1. In this case, the weights of hard tokens become excessively small, which severely hampers the model's ability to learn from them. Empirically, we observe that such fixed choice indeed leads to
25

---

Published as a conference paper at ICLR 2026
Effect of on Token Weighting Function
0.8
0.02
0.6
0.2
0.0
0.2
0.4
0.6
0.8
1.0
Probability
Figure 9: Effect of the temperature parameter on the token weighting function w, =p/T. Smaller values (e.g., 0.5 or = 0.01) sharply down-weight tow-probabilig (hard) tokens, leading to steep weighting curve. Larger values (e.g., 2, 5) flatten the curve, assigning relatively higher weights hard tokens. The case corresponds to the identity mapping. This illustrates how T modulates the balance between emphasizing casy versus hard tokens.
poor results. For example, on Qwen3-4B fine-tuned with MedCalc at learning rate of 5 x 10 fixing yields maximum accuracy of only 0.2168, much lower than results in Figure This stark degradation confirms that without dynamic adjustment, the model fails to effectively learn from hard tokens. By contrast, our dynamic strategy for selecting т, i.e. based on the median average sequence losses in each batch, automatically adapts the current distribution of token difficulties, ensuring that hard tokens are downweighted without being entirely neglected.
26

---

Published as a conference paper at ICLR 2026
E ADDITIONAL DEFINITIONS, THEOREMS AND PROOF
E.1 PROOF OF THEOREM B.1
Theorem E.1 (First-order approximation by exponential tilting). Fix a prefix u. Consider the current model distribution qe(: u) and the smoothed target distribution p2.el. u). Define the local L norm Ilg|l,,u (Bg, (a))((a))))) 1/2, Under the standing assumptions. there exists
effective step size Xe such that
log (+1(111111 u) [ (11-)ru) >e,u) log (- |u) Xe log p- u)) - vr... 11... O(E).
where Ve is the log-normalizer and e is the KL trust-region radius such that Kl(gi ge+1) S E.
Proof. Fix a prefix For clarity we write gr(-) ge(- u). (++(()) q (- u), and p-) p2 u).
Step 1. Log-shift representation of the true update. Define the centered log-shift
s(a) = log (a)
gt+1
E8
log
ge(a)
Then, we have
(++((a) (a) 9e(a) ea(a) Egy. [ea]
Step 2. Log-shift representation of exponential tilting, For any X E [0. 1]. define
(e(a) exp(Ar(a))
r(a) log p2.((a) (a
(a)
E.g [ear]
Step 3. Size of the true log-shift (forward KL trust region). Write (-(-) ge(- (),, (++((-)
2641 u). Recall the centered log-shift
s(a)
log
(a) q((a)
Eg
log
4++1 9i
||
Eyg [s2]
Let the log- log-partition A(f) = log B.. [e5] and the exponential-famme map
(e(a)et(a) EA(s)
T(f((a)
Then ge+1 (()) and
KL(q: ll ((41) = E log 4++1
A(s).
(3)
Two standard identities (for discrete finite support) are
VA()[]] [r]
A2 k) = (h,
Since & is centered under d. we have A(0) = 0 and V [(0] = -Eg. [s] 0. second-order Taylor expansion of A at 0 with third-order remainder yields
A(s) = A(0) + [A))ss + 1/ A²)(()(s ss, s] + Ra(s) = Var,, (s) + Rg(s),
(4)
where, because the vocabulary is finite and qe has full support (by smoothing). there exists constant C3 < oo such that
[R((s) C3 ||slli.. Combining equation eqqution 5 giiees the quadratic expansion
KT (1+1)
||=1|.. + (11
27

---

Published as a conference paper at ICLR 2026
Finally, under the trust-region assumption (I(9)194+11 and for ||sle, sufficiently small, there exists a constant > 0 such that
|| -C S E,
which implies || < 4v as soon as || S min 13. Hence || O(VE).
Step First-order expansion of tilting, Recall the tilted distribution
(a) = (((a))rr(a Beg [e<r]]
r(a) = log 22.c(a u) q: u)
Its log-ratio relative to q
q (a) Ar(a) - A(Ar),
log 2: (a)
A(f) = log Bay [e'].
By Taylor expansion of A(Ar) at À = O, using VA(0)(r) Eg [r] and V V2 ((0)rr)) = Var (r), one obtains
A(Ar) = x E. [r] X2 Var (r) + x3 x3(r) + O(x)) where ka(r) denotes the third central moment of (bounded on finite support). Hence
(a) X(r(a) E., (r)) - - ^² Var (r) + O(A)).
log (((a)
Now consider the centered thiss
(a) (:(a)
7(a)
3(\) (a) log
Ey.
log
Ft
Since
g (())
Eg
log
= AlE [r] - A((r)- = - 12 Var (r) + O(23),
the quadratic terms cancel, yielding
= ((F(o)- - Bg.
+ 0(2).
Step 5. Choice of effective step size. Let re(a) = r(a) Ey [r] be the centered tilting direction and recall from Step 4 that the centered log-shift of the tilted model satisfies
(a) = Ara(a) + O(23).
Define the effective step size by q, q--lesst-quaare matching:
A = Eg[[s Egg [r3]
By Cauchy-Schwar
Egg [s rell
||s le |elle..
||s|le.u
Eqq [r:]]
Irell ||olle,
Since Step 3 gives ||s]le.a = O(ve) and we assume ||rel??.. = Eg. 2 Up > 0 (non-degenerate target), it follows that
= -
Next we control the residual. Under the smoothness and small-step assumptions, the true shift s and the tilting direction re agree first order: there exists scalar O(VE) and remainder A with A I|.. = OEs) such that
= A.
28

---

Published as a conference paper at ICLR 2026
Substituting this into the formula for Ae, yields
Byg (arr + A)rel
Eg A Ard
Ae
O
Eg. [r2]
E r2
Hence the residual can be written
Auuto = A - A arres
and therefore
|| uu u S A + Prru al ||rolleuu = (()).
Step Putting pieces together: Recall that
log q (() (a) = (1 - log (e) + Alog (2) - (2),
so that the log-difference vector is
A (A) (a) == log g++1 -lo (a) = log 2c+1(a] (()
Egil log 9i 1
(log (a 9i(a)
E.l log -- +C(()))
where
C(A) =
Beye
ge-1 log
Reg. log ((
at
a constant (independent of a). Denote
{() log ge+i(a) 2e(a)
E log 9++1 gt
(()) loog (1)) (a)
B.y log /(a)
(=(a)
Then
(a) 1 (=(()) + C(A).
Since Egg. [s] = Egg. [8(2)] = O, the vector & - g(A) is orthogonal (in (.² to the constant function
1. Hence
| 12 = =| 5(2) 12 + 10(2))
and in particular
|| s 5(1) lle, I 11,,s I || 3(1) lle,a + +C(())]
Recall A(f) = log E [e]] and Eq.
Using
Egg
16-1 log
A(s),
Egg
log g(A) A. [r] A(Ar),
4:
we have
C(A) A(s)) -AE + A(Ar). By Step 3, A(s) = 211. + +(([s]]?,.)) = -O(s). By the Taylor expansion of A(Ar) at = (Step 4),
A(rr) = AE E. [r] + 1 Var,, (r) + 0(2))
Hence
C()) = - =11012 1 Varg (r) + 0(1 + 3()3). In particular, with |X| = ((ve) (Step 5) and ||s||e,a = ((vε) (Step 3),
IC(A)) O(s).
Choose x = de from Step 5. Then
|| g(de,n) l1e, I || de,ura I1e,u + [18(Ar)))) - de,urallea-
By Step 5, ||8 Auura aa lle,, = O(e). By Step 4, $(2) Ara + 0(23), so [[ Ne.rrlll..
O(A = ( Therefore,
|| g(h,,)) ||e,u O(e).
29

---

Published as a conference paper at ICLR 2026
Using the decomposition inequality above and equation 6
(| 11e, - || - g(A,,)) ll,. + -())
Finally, recalling
log 1ogg(A// (a) log (a(a) (e + A 12, 1,,a
we have shown
log 9:11(. (2) (11 de,w) log 9c u) + de log p2. u) - Ve.u(Aew)) ll.. = =
which proves the theorem.
E.2 PROOF OF THEOREM B.2
Notation. Fix prefix и (we omit "I w" when clear). Write s(a) = log log g(a) at the current iterate q (the step index i is omitted for readability), and Eyls,, = - f. For set 8 of token-tree nodes, denote its g- g-mass by w/s = [1s]
Recall the standard log-space interpolation Q> defined per prefix by log q2 (1 A) log + A llg P2 (A),, with (AA) = log E. a(a)1~>pp)
1 "a
Lemma E.1 (First-order change of code length under tilting). For any response distribution P the token tree,
AL(P) = KL(PIQN) KL(PII)
-ooli O(22),
where the O(22) remainder is controlled by Varq(S). Equivalently, 2x A KT((P))Q))
-EElS]- -
Proof. By definition, log = logg + X(f va()) with (A) = log Eole>/1. Thus log
Aʃ (A) and
(L(P))2>) KI((PIQ) - XEr]s] + ((A).
Since (A) log Bo(e>/] AE + X² Varq + O()²), obtain AL(P) -A(Er(f)
Ed(f) + X² (f) + 0
0
Lemma E.2 (Variance under sparsity). Fix a prefix u and write Q(-) = q(. u). Let f(a)
log p2(a u) logg(a u). Under Assumption B.4 have
Varq(f) I (a-q(f(a)²] < ws M2 + (1-ws) M S ws M - Mf.
Moreover, when we account for the controlled leakage Assumption B.5 the effective out-of-set amplitude can be taken as Mi + (B + +n) which yields the coarse bound
Var (f) < ws M₁ (Mr + (B +))M)²²
Proof. Since (f) < Eg]f2], suffices to bound the second moment. Split the expectation over S and S
[ E2 1s] + Eols 1s:] M (1 - ws)
using |fl < M₁ on S and |f| < M₁ on S Dropping the factor (1 ws) gives S ws M3 M2.
To upper bound the effective out-of-set magnitude after realizing a targeted update on S, choose the per-prefix target direction
Iss.
which is supported on S, so that [s (2(()) = < 3 {|[sg]] and the premise Assump- tion holls.ss. Let A log g be the induced global change guaranteed by Assumption B.5 and define the leakage vector on S by
: Ms A Alog4-
30

---

Published as a conference paper at ICLR 2026
Then the out-of-set control in Assumption B.5 gives
[|0 ((st)))
Moreover,
[ ||[[fflla3() (Ms((()())) '2
Mr Vws I Ma.
Define the effective out-of-set component that accounts for leakage by
5 == Jsc + l.
By the triangle inequality and the bounds above,
(Q) | ( + 1( M++ (B+ [Mss)lM) S Mr (3 +8++)) Ma.
Therefore, an effective second-moment upper bound that incorporates the controlled leakage
Eg -1ss= + +Ilfs= (1 S ws M (Mr + (B+
This motivates the shorthand
Var (f) < us M2 + (M: + (B+7))M))
which we use as a coarse variance upper bound when controlled leakage is present.
Lemma E.3. Let P.Q,P. Ps be response distributions the (truncated) token space with strictly positive densities p. P2. Define f(z) == p2 log ((2). Then
Ep[s] Fall]] KT.(PIIQ) (T( (T.(P1)P))
In particular, if P. P2 then [ - Boll] = <T.(P= (Q) + KT[Q) P3) 2 0.
Proof. Expand the left-hand side:
p(()) (10g))())-ooo(((( logg(z)) - Σ (x)) (log)2(2) log g()))
Group like terms:
En(z)logra(t) Σn(e)log((3) - Σ log (2() - Ex(=) (og (og)(z))
Use the discrete KL definitions:
p(2)
KL(PIIQ) Σ p(( log
Σ logp()) Σ p(2) logg(z),
((2)
(2))
KT((QI)))) Σ ((2) log
Σ (())) log (())
Σ (()) (og
P2(2)
p(2)
KL(P)|P)) Σ p(2) log
Σ p(2))logp(s - Σ (og
(2(2)
Substitute and simplify to obtain
- = KL(QI))) - KL(P))P).
For Pg = Pz. the fight-hand side equals KL(PaII)) + (L(Q| 2 0. with equality iff Q Pz.
31

---

Published as a conference paper at ICLR 2026
We now bound the domain performance improvement and the general capability degradation in single small step.
Theorem E.2 (One-step code-length change bounds). Let Pl, P2 be the response distributions of D1, D2 on the truncated token tree, and let Q + Qx be one log-space tilting step with || & Ap small. Denote the expected code-length change by AL(P) = KI((PIQ>) KL(())Q). Then there exist constants C2, C2 2 such that:
Domain performance improvement D2-
AL(P)) (Ep.1f1 - Eolfl) + (f)
- <- - (I.(Ps|1Pa)) C2222
General performance degradation on Di.
AL(Pi) = - (Br.
- - - + - Varq(f)+
A S (f) Vx² C2 X² S <X(V US Mr + Mi + (3 (X(R)))((()))
where (2(P( is the chi-square divergence. In particular, if P2 Pz. then
AL(P) < -) C2222.
Proof. Recall the definition of the one-step log-space interpolation (per prefix) log yA (1 A) log A Aog p2 (A), with (A() log Egle^[] and f = log P2 log For any distribu- tion P on responses, Lemma E.I gives the second-order expansion
AL(P) -
- KL((PII) = A(EP[S]] - Eol/1) + 1a (/ar(()+0(()) (8)
Domain performance improvement on D:2- Apply equation with P P2:
AL(P2) = -B((() + Xa +
By Lemma E.3
Eg,Ifl - Bo[s] = KT(P:|IQ) P3) - KL Pb) 2 KL(Q) P:) - KL(P2 P2).
Multiplying by -À reverses the inequality, hence for |>| 1 Ao.
AT((2) < - (KTT((Q1)) - kT. Pa)) + C2
where C2 2 0. In particular, when P3 Pz.
Er, - [R][[] = KT. [1)) + KT. 2 <T,(Q || P2),
So
AL(P:) AKL(Q C
General performance degradation on Dj. Apply equation g with p= Pi:
AL(P)) = ([E Ro(s)) + 2 Varg()) + O(**).
Using the incquality [p[g] - EEolll & / Varq( V Q), we have:
Ep, [ - R2ls V Varq()).
32

---

Published as a conference paper at ICLR 2026
Therefore, for [X] Σ λu,
AL(P) S A (F())11)
with C 2 as above.
Next, invoke the variance upper bound that incorporates the controlled leakage (Lemma E.2 Varq(f) S Var off (f) & ws Mi + (M + (8 +7)
Q
Taking square-roots and using vI + S Vi + v'y
Var & Var (f) - vus Mr + Mr + (B + M,
Combine the last displays obtain
AL(P) s A (vws M, M₁ + (3+ ()++))M) ( 1
This establishes both bounds with the left-hand side written the expected code-length change AL(P).
Theorem E.3 (Multi-step code-length change bounds), Let (Qi)²= be obtained by repeated log- space tilting steps Qt + Q++1 with weights x (per-prefix interpolation), and write
T-1
T-1
(LT(P) = KL(PIIQT) - (L(())Q))
AT =
de
ST
Then there exist constants C1 C2 2 such that:
Domain performance improvement on D2.
T-1
ALT(P)) Σ (LL(P)]Q0=))KKLLCC KL(P)1Q0) - Σ
Xe - + C2 Sr.
1=
In particular, if Pr = P2 (oracle largets), then
T-1
ALT(P)) S AL((Q1))))) + STST.
General performance degradation on D1. Let
HT
(P2(Pi) (< do under small-step updates and KL(P)][Qo) < 0SKT
sup
Then
T-1 Σ de V +1 s Hr Hr((wus MA+ Mu+ Ar + Ci ST
ALT(PI) < HT
where the last inequality uses the effective variance bound from Lemma over t.
E.2 applied uniformly
Proof. For each step t, Lemma E.I gives, for any P.
(T.(P(Q(++1) (T.(P)Q.)) X (Ep(Je) (fI)) NY Varq, (ft) + 0(13).
Summing over = O...., - and absorbing O(2)) into constant multiple of ST (since |Ar| Ao) yields the generic decomposition
T-1
ALT(P) 5- - (BPlil - Ee.lfi) + Sr.
33

---

Published as a conference paper at ICLR 2026
Domain term (P P2). By Lemma
E.3 Bp.Ifi
Eq, [fl] KL(P2)IQ:) + KI(QQ:IP22)
(L(P2)IPP)) 2 KL(Q)]If3) with C2 absorbing the quadratic/cubic remainders. the oracle case P2 P2. [p, Eq, [ft] Z
Multiplying by -A, and summing gives the first display,
KL(Qi P2), hence the stated inequality.
General term (P P1). The change-of of-uuusure bound with centering gives, for each t,
Ep [fd] EQ. Ifall & vx²(Pi V Varr..fi) & Hr v Varq. (fi).
Thus, by applying - (IBr [fe] - Eq, (fil) - Br,lf Bq.lfil, we have
(LT(PI & HT Σ V Varq. (fi) + Cr
Finally, apply Lemma E.2 uniformly in to bound v /Varq, & Vus Mr + Mr + (B + M) and factor out Ar.
Theorem E.4 (Smaller steps yield a smaller general performance degradaton bound equal domain performance gain). Fix a desired domain improvement A. > on D2 (i.e., ALr(P2) s -A,). Among all -tsepp tilting schedules that achieve this target, the minimal upper bound on the increase of code length on DD satisfies
A PT
(AC2 JEl
C
A + of T
(
AlT(P)) S
A
+
where pr inferT (L((Q.I(P2) > and A= HT( (ur M, Mi + (B+7) are fixed value under the total number of update steps T and the desired domain gain D. The upper bound strictly decreases as increases. Thus, under the equal-steps schedule that attains the target, the per-step effective weight scales as de 1/T; thus, for the same domain gain, larger implies smaller per-step updates. Hence, smaller step size = smaller upper bound.
Proof. We work in the oracle case P3 P2. From the multi-step bound,
T-1
ALT(P2) S Σ d KL(Q))) + C2 1=0
Σ
X S -PT AT + ST,
t=
where AT = E, ST = E,>,, and pr = inf,eT KL(Q.I|P) > 0. Thus any schedule that achieves ALT(P2) < AA, must satisfy the feasibility constraint
pr Ar C2 ST 2 A.
(9)
For the anneral-performance side, the multi-step bound gives
(LT(PI) S AAT + C1
(10)
where = HTr(yws Md + M1 + (8 (B+ Ma) and HT = supper (PT(Q.).).
For any fixed T and Ar, Cauchy-Schwarz implies ST AT, with equality iff X AT/T.
Hence the bound equation i0 i minimized (for fixed AT) by the equal-steps schedule; moreover, equal steps minimize the feasibility penalty in equation 9 as well. Under equal steps Sy = A the feasibility constraint becomes the concave quadratic inequality
C2 T
PT AT
A3
2 A.
Its smallest feasible solution (the smaller root) is
Apin
T
pr Vu2
7
2C,
which exists for T > 4C2A./12 Substituting Appin and Sy = (Apin) ) //T into equation yields the optimal-in-this-boune upper bound.
34

---

Published as a conference paper at ICLR 2026
To expose the dependence T, expand the smaller root for large T:
C2
HT Ar
Az
=
the smaller root is
T
4C2A. T
min
NTT
11
2C2
Set
4C2A.
4C2
:
T
PT
1
Then
x
+0(2))
pTV
x= pr
1 -
2
Hence
PT
13
-
PT
2
2
+
E
+
8²
0
81LT
1
Multiplying by the prefactor (2(2 and substituting = 4C2A, T,
Then, have
A.
C2
0
1 )
A ATri
Po
T
Therefore,
ALT(PI) A A
(AC2
C1
A
of
(
PT
11
T
which decreases strictly in T and converges A A../ pr xc. This completes the proof.
Theorem E.5 (Label-only supervision enlarges the safe per-step range). Define
V V Mr (m () M2], M M₁ (3 + M
where is the expected number of hard tokens per example on D2 and m is the example to- ken length. For any T-step equal-steps schedule (A; A), cerfrra-pefformacce degradation (LT(PI) I Eig is ensured whenever the per-step effective weight satisfies
V / (.2
40 T
Elg
A S Amax (T;
2C1
In particular, as T grows,
Hy + (H)))
Eig
Amax (T;s)
so the safe per-step range widens inversely with Va. When Me Mr. we have V, MAVS. hence
Amax (8)
Vs (for fixed eig. HT, Mn, T).
Therefore, if label-only supervision reduces relative thain-of-thought, it strictly enlarges the admissible per-step range.
Proof. Define M Mi + (3 + Mr. From Lemma E.2 we have
(arq.(f) < [a=q. us. M3 + (1 (1-we ws, MM
(11)
Consider sampling an example 2 from D2, with response length m(2) E N, and let J(z) C (1....,m(z)) be the set of hard positions for that example, with cardinality s(z) [J(z)] If
35

---

Published as a conference paper at ICLR 2026
(conceptually) select token position uniformly random along the generated path, then the probability of landing in the hard set equals the hard fraction:
8(2)
WS, E
(12)
m(2)]
where the expectation is over the (data-induced) randomness of examples and the path under Qr-
Substituting equation T2 into equation 11 yields
(ft) s [ MR + (1 (1- [[= M².
(13)
We now relax the hard fraction into an affine form in (s, m) that pairs naturally with (M2, M2).
Since m(z) and s(z) < m=2) for every example,
s(z)
s(2) m(z)
s2), Taking expectations and using linearity,
1
I - s(2).
m 2)
M
+
(1 (1- BT =
M²
Mi + (m -8) M
(14)
Combining equation 13 and equation 14 gives the uniform (in t) token-level bound
V /Vaqq. (fi) <
s M + (m - s) M2]
= Va
for all 0, 1, ...., 1.
(15)
The general--erformance part of Theorem E.3 states that
ALT(Pi) < HT
Σ N
Varq: (fi) C1
Σ
x
HT
sup 0St<T
(P.(P.
t=0
t=
Using equation 15 we obtain the uniform (in t) upper bound
T-1
T-1
ALT(P)) & HT + C1 ST,
Ar
Σ 1=
ST = Σ
x3.
(16)
t=0
For an equal-steps schedule A, = A, we have AT T\ and ST = T hence from equation 16
(P) & V.A + C x2)
(17)
Impose a general-performmance budget (L) & Eig. Then equation 17 yields the quadratic constraint
C + (HTV&)
Sig 0.
(18)
T
The feasible interval in A [0, Amax (T) where the positive root is
Amax (T; = - HTVV + (HrV.)2 +
401
Eig
T
(19)
201
A first-order expansion in 1/T (Taylor for va² + ô with 6 ~ 1 gives
Eig Amax (T::) HT V
T + of T2 as T + 00.
(20)
Finally, when Ma & Mh. the Ma-term is negligible and V, v E[s M M₁ V3, so
m 8 M2
Amaa (T;8)
x
(for fixed Etg, Hr, MR, T).
V
This shows that reducing (as typical under label-only supervision versus chain-of toiin-of-though strictly enlarges the admissible per-step range, and the range also increases with T'.
36

---

Published as a conference paper at ICLR 2026
Table 4: General-purpose benchmarks, their evaluation metrics, the number of few-shot settings, and the primary capability they assess.
Dataset
Metric
Shot
Capability Evaluated
Instruction Following
IFEval Zhou al. 2023
Jevell strcc_ssttcttttti
0-shot
Instruction-followin
Mathematical Reasoning
GSM8K (Cobbe al.
2021
match, flexible-extract
5-shot
Mathematical reasoning
Generation
HumanEval (Chen al.
2021
pass@ create
0-shot
Code generation
Commonnsense Reasoning
HellaSwag Zellers
2019
0-shot
Commonsensee reasoning
ARC-Easy (Clark ct 2018
none
O-shot
Science reasoning
ARC-Challenge (Clark el al. 2018
none
0-shot
Science reasoning
PIQA (Bisk et ul." a 2020
BCC, none
0-shot
Physical commonsense reasoning
Knowledgee-ntenssive QA
MMLU (Hendrycks et al. 2020
aCC, none
0-shot
Multi-domain knowledge understanding
Table 5: Domain-specinn datasets, their evaluation metrics, and the primary capability they assess.
Dataset
Metric
Speciff Capability Evaluated
MedCalc (Khandekar et al. 2024)
Accuracy
Medical mathematical reasoning E-commerve product classification
ESCI (Reddy et al. 2022
Balanced Accuracy
Table Prompt template used for the MedCalc Benchmark.
Prompt Template for MedCalc Benchmark.
<I limmmss start >system
You are a helpful assistant. You first think about the reasoning process in the mind and then provide the user with the answer. <I <limendd>>
< im.sttttt >user You are helpful assistant for calculating score for given patient note. Please think step-by-step to solve the question and then generate the required score. Here is the patient note: {note)
Here is the task: [question]
Please show your entire reasoning process in single <think> </think> block (do not open or close the tag more than once). Your final response must be in JSON format within <answer> </answer> tags. For example, <think>
[entire reasoning process here]
</think>
<answer>
("answer": str dttlchort.ammddeeet....eee..rddd
</answer>
<mimedd]>>
< im.start >assistant
Let me solve this step by step.
<think>
37

---

Published as a conference paper at ICLR 2026
Table 7: Prompt template used for the ESCI classification task in the w/o CoT setting, where the LLM directly predicts one of four relation types given query-product pair without generating intermediate reasoning.
Prompt Template for ESCI Classification (w/o CoT setting)
< im..start// /ttsstartt/ssseemmmmm >system
You are helpful assistant. You provide the user with the
answer.
<I </imendd|>
<mim start >user
Your task is to classify each product being an Exact, Substitute, Complement, or Irrelevant match for the query. Here is the user's query: (user.query)
Here is the product information: (product.info)
Your final response must be within <answer> </answer> tags. Label the relation type as a number: Exact,
Substitute, Complement and 3 Irrelevant. For example, <answer>one.oo [0, 1, 2, <]</answer answer>. 

<| im.start bassistant
<answer>
38

---

Published as a conference paper at ICLR 2026
Table Prompt template used for the ESCI classification task the w/ CoT setting, where the LLM is required to produce complete reasoning trace inside a single <think> block before giving the final prediction in <answer> tags.
Prompt Template for ESCI Classification (w/ CoT setting)
<I imissarrt >system
You are helpful assistant. You first think about the reasoning process in the mind and then provide the user with the answer.
<| <mimndd/>>
< im.starrt >user
Your task is to classify each product being an Exact, Substitute, Complement, or Irrelevant match for the query. Here is the user's s query: (user.query)
Here is the product information: (product_info)
Please show your entire reasoning process in single** <think> </think> block (do not open or close the tag more than once).
Your final response must be within <answer> </answer> tags. Label the relation type number: Substitute, 2 Complement and 3 = Irrelevant. For example,
Exact,
<think>
[entire reasoning process here]
</think>
Kanswer2one.of.10, 1, 2, 3]</answent
<|imendd>>
< im >assistant
<think>
39


Reviews:
[{"review_id": "N24Lw5pvE6", "reviewer": "Reviewer_XUav", "summary": "The paper shows that domain-specific SFT does not necessarily hurt LLM general abilities: small learning rates suffice to retain them while keeping domain gains. It gives a compression-theoretic explanation and introduces TALR, an adaptive token re-weighting method that further curbs forgetting.", "questions": "- Did you experiment with top-k or sparse routing variants of TALR, and how do they affect convergence speed and final trade-offs?\n\n- How does TALR behave when the domain corpus is large-scale (>>1 M examples) or when continual domain increments arrive sequentially—does τ need scheduling?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Revisits and refutes a prevailing assumption—namely, that domain-specific SFT necessarily degrades general capabilities—showing instead that optimization choices, particularly learning-rate magnitude, are the primary determinant of forgetting.\n\n- Employ learning rate as a single, easily tunable knob that largely eliminates catastrophic forgetting without extra parameters or data.\n\n- TALR is lightweight (closed-form, no extra forward passes), yet consistently outperforms recent regularizers and MoE-style baselines.", "weaknesses": "- Scope limited to decoder-only LLMs ≤8 B; no evidence that conclusions hold for larger or encoder-decoder models.\n\n- No probing of emergent skills (e.g., long-context) or fairness/toxicity side-effects after SFT.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vpfy93TpXc", "reviewer": "Reviewer_9brq", "summary": "This paper challenges the common belief that domain-specific supervised fine-tuning (SFT) significantly harms the general capabilities of large language models (LLMs). Through empirical and theoretical analysis, the authors demonstrate that using a smaller learning rate can mitigate this performance degradation while maintaining strong domain-specific performance. They further propose a novel method, Token-Adaptive Loss Reweighting (TALR), which dynamically down-weights the loss of \"hard\" tokens during training to better preserve general capabilities.", "questions": "1. In Figure 1, for some models (e.g., Qwen2.5-7B on ESCI w/ CoT), domain-specific SFT appears to slightly improve general performance compared to the initialization. Could the authors provide hypotheses or explanations for this unexpected improvement?\n\n2. The appendix states that 20 training epochs were set for MedCalc and ESCI, which seems high for domain-specific SFT. What was the rationale for this choice? Providing a plot of performance versus epochs for key models and learning rates would be helpful. This would clarify whether models fine-tuned with large learning rates might have been overtrained, potentially confounding the results.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing is experienced and clear, effectively motivating the problem and delivering its overall message.\n\n2. The work revisits a critical topic—the trade-off between in-distribution and out-of-distribution performance during SFT. Its core finding, that smaller learning rates are a simple yet effective strategy, is a valuable and practical insight for practitioners.\n\n3. The token-level analysis is a particular strength, yielding interesting findings. For instance, it shows that most tokens in SFT data are easy for the model, and performance bottlenecks are likely due to a sparse set of hard tokens.\n\n4. The theoretical analysis, framed through an information-theoretic and compression-based lens, is interesting and provides a valid foundation to explain the empirical phenomena observed, such as why smaller learning rates help.", "weaknesses": "1. Figure 2 can be difficult to interpret in its current form. Splitting it into more straightforward visualizations, such as bar charts, could make the conclusions more visually clear and accessible.\n\n2. The advantage of the proposed TALR method over baselines is not entirely consistent. At a learning rate of 1e-6, methods like L2 regularization perform on par with TALR. TALR shows a clearer advantage at 5e-6 on Qwen models, but this benefit does not consistently extend to the Gemma models. This suggests that the effectiveness of mitigation strategies may be dependent on the nature or distribution of the base model. Authors could provide more explanations/explorations on this angle", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "msVgc9b44R", "reviewer": "Reviewer_mWRP", "summary": "Based on experiments showing that a smaller learning rate can partially mitigate catastrophic forgetting, this paper validates the conclusion through theoretical analysis. Furthermore, building on both theoretical and experimental findings, the authors propose the TALR method. By limiting the magnitude of updates for low-probability tokens, the approach alleviates catastrophic forgetting to some extent.", "questions": "What is the main contribution of this paper? Is it the experimental and theoretical analysis of the low learning rate, or is it the TALR method?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is well-written and really easy to read and follow.\n\n2. The method proposed in this paper is derived from an experimental finding and theoretical analysis, with a well-justified motivation.", "weaknesses": "1. I think the paper's core motivation, using a smaller learning rate to mitigate catastrophic forgetting, is clear. However, a significant issue is that this finding is not novel, as it is already a widely recognized common-sense approach. The paper excessively emphasizes this observation, dedicating substantial space to elaborating the advantages of a low learning rate, while its actual contribution, the proposed TALR method, does not receive sufficient attention or elaboration. This may lead readers to perceive the paper's main contribution as the experimental and theoretical analysis of the low learning rate's advantages, thereby overlooking the more important innovation.\n\n2. Building on the previous point, if the main contribution of this paper is merely a more thorough experimental and theoretical analysis of the low learning rate, I do not believe the contribution is sufficient, as the experimental results do not present any particularly striking or novel conclusions.\n\n3. Regarding Figure 2, could you provide the results in a table format? From the figure, it appears that TALR does not show a clear advantage under many settings, which suggests that the TALR method may not have a significant effect.\n\n4. Regarding the extensive discussion on low learning rates in the paper, I believe the experimental analysis is not sufficiently comprehensive. In both the preliminary analysis and the main experiments, the authors only used three different learning rate settings (with even just two shown in Figure 2). Since TALR is derived from this experimental finding, I think the experiments should be more thorough and robust—relying on only three learning rate points is inadequate.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zXWdmuIfDO", "reviewer": "Reviewer_T4Nt", "summary": "The paper's premise is the observation that SFT does not always significantly harm a model's general capabilities ; this issue can, in fact, be largely mitigated by using a smaller learning rate. A theoretical analysis further suggests this performance degradation is linked to the adverse impact of low-probability tokens. Based on this, the authors propose TALR, a method that suppresses the decline in general capabilities by adaptively reducing the loss weights of these \"hard\" tokens.\n\nExperiments show that TALR outperforms existing methods in balancing domain-specific and general performance , though it cannot fully resolve the trade-off. The paper's final practical guideline for SFT is to prioritize using a small learning rate and to supplement with TALR when a better balance is desired.", "questions": "1. TALR balances performance by down-weighting hard tokens. As noted in the weaknesses, it is not well-argued whether these hard tokens are eventually learned. Could the authors provide an analysis that tracks the probability of a few specific, hard tokens (like the example in Fig 3a) throughout the training process to demonstrate their learning dynamics?\n2. The paper lacks an analysis of the actual training dynamics of TALR, instead showing a related \"extreme experiment\" (Fig 3c). Could the authors provide an analysis (e.g., of token probabilities or loss weights) based on the true TALR algorithm as presented in Algorithm 1?\n3. The paper's primary recommendation is to use a small learning rate, a setting where TALR offers minimal advantages. The utility of TALR hinges on a scenario where one must use a large LR to achieve peak domain performance. Could the authors provide a clearer experimental case where the domain performance of the small-LR SFT significantly lags behind that of the large-LR SFT, thereby strengthening the practical case for TALR?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Experiments across multiple models (Qwen and Gemma series) and diverse datasets (MedCalc, ESCI, MetaMathQA) consistently demonstrate the effectiveness of using a small learning rate.\n2. The paper moves beyond mere empirical observation to provide a theoretical foundation. The perspective linking SFT dynamics to information compression and code-length change is novel. This theory successfully explains why small learning rates are beneficial and why label-only SFT behaves differently.\n3. The proposed TALR method is not ad-hoc; it is derived from the theoretical analysis, giving it a strong theoretical grounding.", "weaknesses": "1. The primary recommendation—using a small learning rate—is a standard and well-known heuristic in continual learning to mitigate catastrophic forgetting. As such, the method itself lacks significant innovation.\n2. As the authors acknowledge, TALR offers negligible benefits over standard SFT when the recommended small learning rate is used. Its utility is confined to a niche scenario where one *must* use a large learning rate (to pursue peak in-domain performance) and needs to balance the resulting degradation in general capabilities.\n3. The dynamic heuristic for setting the $\\tau$ hyperparameter (using the batch median loss) is not sufficiently validated. The ablation study only compares it to a single fixed value ($\\tau=1$) rather than a proper hyperparameter sweep or comparison with other dynamic strategies.\n4. The experiment in Figure 3, which is used to motivate the \"curriculum-like\" behavior, does not match the actual TALR algorithm. The experiment uses gradient clipping (truncation to zero) for hard tokens, whereas TALR only uses down-weighting. Therefore, this analysis does not represent the true dynamics of TALR. Furthermore, the aggregate statistic in Figure 3c (the rising fraction of p>0.2 tokens) fails to prove that the hardest tokens (e.g., p=0.05) are eventually learned; it could simply reflect that medium-difficulty tokens have become easier.\n5. All experiments are conducted on relatively small-scale models (8B or less). The paper lacks validation on whether its core conclusions generalize to current state-of-the-art large models (e.g., 70B+) or MoE architectures, whose fine-tuning dynamics may differ significantly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)