Title: Linearization Explains Fine-Tuning in Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Parameter-Efficient Fine-tuning

LLMs

Neural Tangent Kernel

Linearization

Low Rank Adaptation
Abstract: Parameter-Efficient Fine-Tuning (PEFT) is a popular class of techniques that strive to adapt large models in a scalable and resource-efficient manner. Yet, the mechanisms underlying their training performance and generalization remain underexplored. In this paper, we provide several insights into such fine-tuning through the lens of linearization. Fine-tuned models are often implicitly encouraged to remain close to the pretrained model. By making this explicit, using an $\ell_2$-distance inductive bias in parameter space, we show that fine-tuning dynamics become equivalent to learning with the positive-definite neural tangent kernel (NTK). We specifically analyze how close the fully linear and the linearized fine-tuning optimizations are, based on the strength of the regularization. This allows us to be pragmatic about how good a model linearization is when fine-tuning large language models (LLMs). When linearization is a good model, our findings reveal a strong correlation between the eigenvalue spectrum of the NTK and the performance of model adaptation. Motivated by this, we give spectral perturbation bounds on the NTK induced by the choice of layers selected for fine-tuning. We empirically validate our theory on Low Rank Adaptation (LoRA) on LLMs. These insights not only characterize fine-tuning but also have the potential to enhance PEFT techniques, paving the way to better informed and more nimble adaptation in LLMs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies fine-tuning through linearization and NTK regression. The paper shows under an explicit regularized fine-tuning process, the behavior of model dynamics is close to a linearized model, which leads to the approximation by NTK. They then analyze how the eigenvalue spectrum of the NTK is related to the model performance and validated through empirical experiments.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper provides some insights on how fine-tuning is related to NTK and how the eigenvalue spectrum of NTK predicts the model performance. Such an insight could be useful for determining what layers to be fine-tuned by evaluating the corresponding NTK spectrum. However, analyzing fine-tuning dynamics with NTK is not new and has been studied in e.g., [1]. And it is unclear what is the main procedure for fine-tuning with layer selection via NTK. Also see weaknesses in the questions section. \n\n\n[1] LoRA training in the NTK regime has no spurious local minima. ICML 2024.\n\nQUESTIONS:\n1. It is not clear what is the implication of Theorem 2. Comparing the bound for regularized fine-tuning (9) and conventional fine-tuning (10), the bound (9) could be larger than (10) given the exponential dependence on $t$ versus linear dependence on $t$. In this case, how such regularization $\\lambda$ would manifest lazy training? \n\n2. In Theorem 4, what is $N$? And it seems by setting $t = O(1/\\lambda \\log(N/\\text{Lip}^2))$ and requiring $\\lambda \\geq O(2N/(r \\text{Lip}))$ would bring $t = O(r\\text{Lip} \\log(N/\\text{Lip}^2 )/N)$. Could you comment on whether such requirement on $t$ is restrictive or not? \n\n3. In Theorem 4, can you comment on how large $\\lambda$ should be in order for the fine-tuning to be approximated by NTK regression?\n\n4. In Theorem 5, what is $\\mathcal{R}$? It seems only $\\widetilde{\\mathcal{R}}$ is defined in (3). I think $\\mathcal{R}$ is defined in terms of (14), is it correct?\n\n5. I do not quite understand how Table 2 correlates to your propositions. Section 6.3 does not have sufficient discussions of the results in Table 2. I do not understand why \"lower condition number indicates lower training and evaluation loss\". In your Theorem 5, the condition number seems only to suggest the range of your empirical risk.\n\n6. It seems the main benefit of the propositions is to determine what layers need to fine-tuned. However, I did not see much discussion in this regard, namely how can we determine the layer to be fine-tuned in practice. It is better to write explicitly an algorithm for such purpose.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a theoretical framework for understanding Parameter-Efficient Fine-Tuning (PEFT) in large language models (LLMs) through linearization and NTK theory. The authors introduce explicit $\\(l_2\\)$-distance regularization toward pretrained parameters, showing that this induces \"lazy training\" dynamics equivalent to NTK regression. Key contributions include: (1) Theoretical bounds on the distance between fine-tuned and linearized models, (2) Characterization of fine-tuning performance via the NTK eigenvalue spectrum, and (3) Spectral perturbation bounds for layer-wise adaptation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1. **Theoretical Rigor**: The paper rigorously formalizes PEFT dynamics through NTK linearization, deriving novel bounds for model deviation (Theorems 2–4) and risk minimization (Theorem 5). The spectral perturbation analysis (Theorems 6–7) provides actionable insights for layer selection.  \n2. **Practical Utility**: Demonstrates that NTK condition number (computable beforehand) correlates with downstream task performance, enabling efficient layer selection. Table 3 highlights significant time savings.  \n3. **Innovative Regularization**: The selective $\\(l_2\\)$-regularization scheme, activated based on gradient alignment, is a nuanced methodological contribution that ensures monotonic loss decrease.  \n4. **Strong Validation**: Experiments across GLUE, IMDb, and Yelp tasks consistently support theoretical claims, particularly the NTK-performance correlation and regularization efficacy.\n\nWeaknesses:\n1. **Loss Function Gap**: Theory relies on **MSE loss**, while experiments use cross-entropy. The generalization of NTK spectral analysis to classification losses remains unproven.  \n2. **Methodological Novelty**: While offering theoretical insights, the work does **not propose a new algorithm**. The selective regularization is underexplored as a standalone method. Moreover, a linear fine-tune strategy in [11](author's citation number) seems to be further justified by this work, which should be used as a analyzed baseline, or even should be referred to design a new tuning method.\n3. **Inconsistent Empirical Claims**:  \n   - The claim that \"*accuracy remains largely unaffected by regularization*\" (Sec 6.2) contradicts Table 1 (SST-2 accuracy varies by **3.7%** with $\\lambda$, variations are even larger on CoLA).  \n   - Table 2 shows weak correlation between $\\kappa(\\mathbf{K} + \\sigma\\mathbf{I})$ and accuracy.  \n4. **Limited Scope**: Theoretical bounds assume Lipschitz continuity and small parameter deviations, which may not hold for aggressive fine-tuning.\n\nQUESTIONS:\n1. **Loss Function Generalization**: Your theory assumes MSE, but experiments use cross-entropy. Can you discuss whether the NTK-performance correlation holds *theoretically* for cross-entropy? If not, what adjustments are needed?  \n2. **Methodological Extensions**: Could your spectral perturbation bounds (Thm 7) directly inform a *layer selection algorithm* (e.g., prioritizing layers that minimize $\\lambda_{\\text{max}}(\\mathbf{K} + \\mathbf{S})$? Have you tested such an approach?  Moreover, some more novel methodological innovations are preferred. (See Weakness 2)\n3. **Regularization Discrepancy**: In Table 1, higher $\\lambda$ reduces $\\|\\theta_t - \\theta_0\\|$ but inconsistently affects accuracy (e.g., CoLA peaks at $\\lambda=5$). How do you reconcile this with the claim that accuracy is \"unaffected\"?  \n4. **Condition Number Correlation**: Table 2 shows CoLA’s best accuracy (73.44%) occurs at $\\kappa = 7,503$ , which is not the smallest one. What explains cases where low $\\kappa$ *does not* yield the best performance? In fact, it seems that the performance is more likely to be *unaffected* by condition numbers, a more precise explanation is favored.\n5. Typo: Line 167, *infinitesimally*\n\nLIMITATIONS:\nSee Weaknesses and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper shows that if fine-tuning is explicitly regularized to stay near the pre-trained weights, the optimization behaves almost linearly and can be analyzed through the Neural Tangent Kernel (NTK). ,Under this linearization regime, the paper proves bounds on the empirical risk of fine-tuning as a funciton of the kernel’s eigen-spectrum. Crucially, the paper demonstrates, both theoretically and empirically, that the regularized condition number $\\kappa(K + \\sigma I)$ at initialization is a strong indicator for downstream performance, that could work as an effective heuristic for the selection of layers to tune.\n\nSTRENGTHS AND WEAKNESSES:\nWhile the paper provides and interesting theoretical framework and solid proofs, the theoretical analysis contains several assumptions or logical leaps that are not sufficiently justified in the paper.\n1. In section 3, the paper defines a proximal setup where the empricial risk is regularized with the L2-distance between the pretrained and fine-tuned parameters. The main argument of section 3 is that the distance between the pretrained model and the fine-tuned model is constrained by the regularization. However I see no significance of the result in this section for the following two reasons.\n(i) The bounds in theorem 2~4 don't have much of a quantative meaning, because it is not for actual SGD or reasonable algorithms, but it is for an aritificial ad-hoc algorithm that is defined soley for the convinience of the proof, with no practical usage reported in prior works or through experiments in the paper. Thus the bound only has it's meaning as in theoretical demonstration for intuition.\n(ii) However, the argument that adding the L2-distance of the the parameters of the pretrained model and the fine-tuned model constrains the difference between the two models is already intuitively straightforward, and thus a theoretical demonstration in a toy setup isn't really significant.\n2. In sections 4 and 5, the paper implicitly equates 'performance' and the 'empirical risk'. Especially, in theorem 5, I think there should be more explanation about why the model performance should get better when the condition number is small.\nRegarding the empirical results, it is indeed quite interesting that the generalization performances are predicted through the actual condition numbers of the NTK kernel, but there could have been much more the authors could have done to make a more robust and persuasive experiment. For example, for figure 2-(c), instead of comparing between different datasets, sampling 32 datasamples from the same dataset multiple times and comparing between these minidatasets could have been a more fair comparison. Also, the choice of the layers {0, 5, 11} seem quite arbitrary, it would be better to see an experiment spanning all layers of the RoBERTa-base model.\n\n\nOverall, the results of the paper are interesting as it connects theoretical insight into empirical results on generalization performance and an actual methodology with practical usage, but lacks theoretical solidness and extensiveness of its experiments. \n\n\nTypos\nline 22: 'particularly popular methods [4, 5, 6, 7, 8] set of techniques that strive to'\nline 214: 'The spectral perturbations bounds'\nline 222: 'See Apendix H'\nline 322: 'close to the linearization of the the pre-trained model.'\nline 731: notations and definitons\nline 804: 'inquality 79 will be reduced to'\n\nQUESTIONS:\nWould theorems 2-4 apply to SGD or Adam training?\n\nWhat new insight does the regularization-based proximity argument provide, beyond the intuitive fact that L2 regularization encourages proximity to the pretrained model?\n\nLIMITATIONS:\n.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a theoretical analysis of fine-tuning through the lens of linearization and NKT theory. The authors observe that by adding explicit L2 regularization to the pretrained model parameters, fine-tuning dynamics become equivalent to learning with the NTK, and establish theoretical connections between the NTK's eigenvalue spectrum and fine-tuning performance. They validate the theory empirically on LoRA fine-tuning of RoBERTa across GLUE benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n\nThe paper provides a theoretical contribution connecting regularized fine-tuning and NTK. The spectral perturbation bounds for layer selection are particularly interesting and may provide guidance for PEFT design. Similarly, the connection between NTK condition numbers and fine-tuning performance may offer a practical prediction tool for model selection. The experiments on RoBERTa with LoRA across multiple datasets demonstrate reasonable correspondence between theory and practice, despite some theoretical assumptions differing from experimental conditions.\n\nWeaknesses\n\nA primary weakness is the disconnect between the theoretical assumptions and the experimental setup. The theorems are derived for (deterministic) gradient descent with a squared loss function, while the experiments use the AdamW optimizer with a cross-entropy loss. The authors acknowledge this limitation, but it makes it unclear how tightly the theoretical bounds apply to the practical scenarios shown.\nAlso, the linearization analysis only applies to the explicitly regularized variant of fine-tuning introduced by the authors, and the connection to standard fine-tuning practices where such regularization may exist only implicitly is not clearly established.\n\nThe NTK condition number calculations use only 32 samples, which may not capture full kernel behavior. The paper doesn't explore how sensitive the condition number and the resulting performance predictions are to the choice of these few samples.\n\nWhile the paper does a decent job of positioning itself, the distinction from previous work that also uses linearization and NTK to analyze fine-tuning (e.g., Malladi et al., https://arxiv.org/abs/2210.05643) could be sharpened. The paper's core novelty seems to be in enforcing linearity through regularization rather than just assuming it, but the practical implications of this difference could be explained more forcefully.\n\nThe experimental validation focuses on LoRA and RoBERTa-base; further investigation across different PEFT methods and model architectures would be beneficial.\n\nQUESTIONS:\nYour theoretical results are derived for squared loss and gradient descent, yet your experiments succeed with cross-entropy and AdamW.  Could you provide more intuition on why your insights generalize so well to stochastic settings and different loss functions? For example, does the proposed selective regularization scheme in Equation 8 transfer straightforwardly to AdamW? What about the bounds in Thms 3 and 4? \n\nThe NTK computation uses a small sketch of only 32 samples from the training data.  How stable is the condition number to the specific choice of these 32 samples? \n\nTable 1 shows that model accuracy is largely insensitive to the regularization strength lambda across a wide range of values. Could you explain why we expect this to be the case? Is your main takeaway here that one should always use a sufficiently large lambda that encourages linearity ?\n\nLIMITATIONS:\nYes, the authors have adequately addressed the limitations of their work. In the conclusion, they explicitly state that their theory is limited to squared loss and gradient descent, while the experiments use other configurations. They are also clear that their analysis only applies to the explicitly regularized fine-tuning they introduce, and that analyzing standard, non-regularized fine-tuning remains an open question for future work.  This is a transparent and fair assessment of the work's scope", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}