Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?
Authors: Zi Liang
Haibo Hu
Qingqing Ye
Yaxin Xiao
RongHua Li
Venue: ICML 2025 poster (2025)
Keywords: Low Rank Adaptation
Large Language Models
Backdoor Attacks
Poisoning Attacks
Primary area: social_aspects->security
TL;DR: This study explores the security risks of LoRA in the fine-tuning, demonstrating that a low-rank structure enhances robustness to backdoor attacks but increases susceptibility to untargeted data poisoning.

Abstract: Low rank adaptation (LoRA) has emerged as a prominent technique for fine-tuning large language models (LLMs) thanks to its superb efficiency gains over previous methods. While extensive studies have examined the performance and structural properties of LoRA, its behavior upon training-time attacks remain underexplored, posing significant security risks. In this paper, we theoretically investigate the security implications of LoRA's low-rank structure during fine-tuning, in the context of its robustness against data poisoning and backdoor attacks. We propose an analytical framework that models LoRA’s training dynamics, employs the neural tangent kernel to simplify the analysis of the training process, and applies information theory to establish connections between LoRA's low rank structure and its vulnerability against training-time attacks. Our analysis indicates that LoRA exhibits better robustness to backdoor attacks than full fine-tuning, while becomes more vulnerable to untargeted data poisoning due to its over-simplified information geometry. Extensive experimental evaluations have corroborated our theoretical findings.

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
Zi Liang' Haibo Hu'
Qingqing Ye
Yaxin Xiao
Ronghua Li!
Abstract
To compare LoRA with full fine-tuning across various di- mensions, recently many studies have emerged. For ex- ample, researchers have investigated LoRA's expressive capacity (Zeng & Lee, 2024), the smoothness (Jang et al., 2024) its convergence, the asymmetry (Zhu et al., 2024) in its submatrices, the impact of initialization (Hayou et al., 2024), and so on (Wang et al., 2024a; Koubbi al., 2024; Mao et al., 2024).
Low rank adaptation (LoRA) has emerged as prominent technique for fine-tuning large lan- guage models (LLMs) thanks its superb effi- ciency gains over previous methods. While ex- tensive studies have examined the performance and structural properties of LoRA, its behavior upon training-time attacks remain underexplored. posing significant security risks. In this paper, we theoretically investigate the security implications of LoRA's low-rank structure during fine-tuning, in the context of its robustness against data poi- soning and backdoor attacks. We propose an ana- lytical framework that models LoRA's training dy- namics, employs the neural tangent kernel to sim- plify the analysis of the training process, and ap- plies information theory to establish connections between LoRA's low rank structure and its vul- nerability against training-time attacks. Our anal- ysis indicates that LoRA exhibits better robust- ness to backdoor attacks than full fine-tuning, while becomes more vulnerable to untargeted data poisoning due to its wer-simplified infor- mation geometry. Extensive experimental evalua- tions have corroborated our theoretical findings.
While these analyses have shed light on many properties of LoRA, one important aspect, i.e., its potential security risks, remains largely overlooked. Existing studies in this area either use LoRA as a tool to facilitate backdoor at- tacks (Yin et al., 2024; Liu al., 2024), adversarial at- tacks (Ji et al., 2024), and model stealing attacks (Horwitz et al., 2024; Liang et al., 2025), or focus merely on the ben- efits (Xu et al., 2024b) of LoRA in differential privacy and federated learning. None of these works directly investi- gate the security vulnerabilities inherent in LoRA itself, which leaves behind potential hazards and vulnerabilities in LoRA-fine-tunee LLMs that are deployed across millions of devices (Gunter et al., 2024).
To fill this gap, in this paper, we attempt to answer the question whether LoRA-based fine-tuning is more vul- nerable than full fine-tuning (FF) under mainstream training-time attacks (c.g., data poisoning (Fan et al., 2022; Ramirez et al., 2022; He al., 2024)). We introduce the concept of training-time robustness (TTR) for charac- terizing model's resistance to training-time attacks and propose analytical framework to theoretically examine the security implications of LoRA's low-rank structure. The main challenges are two-folded. First, the TTR of a model significantly depends on the specific training tasks and the complex dynamics of the training process. Second, the effec- tiveness of attacks is heavily influenced by hyperparameters (e.g., learning rate) and attack strategies (e.g., poisoning rate or backdoor triggers), both of which increase the complexity of analysis.
1. Introduction
With the rapid growth in the parameter size of large language models (LLMss, parameter-efficient fine-tuning (PEFT) (Xu et al., 2023; Han et al., 2024) has gained increasing attention in both research and industry communities. Among various PEFT strategies, low-rank adaptation (LoRA) (Hu et al., 2021) has emerged as the de facto standard for fine-tuning LLMs thanks to its computational efficiency and minimal performance degradation.
'The Hong Kong Polytechnic University, Hong Kong, China. Correspondence to: Haibo Hu <haibo. @ wwweeppllyuuull pol...dd.hkk>.
To address these challenges, we introduce two novel sim- plifications when modeling the training dynamics of LoRA. First, we reformulate TTR by measuring the similarity of gradients before and after data poisoning, which enables neural tangent kernel (Jacot et al., 2021) (NTK)-based
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
analysis simplify the modeling of a training procedure. Second, we further introduce information theory (Amari, 2016; Nielsen, 2020) to connect the model's structural prop- erties with its TTR, thereby decoupling the influences of different training datasets and attack methods. Our find- ings suggest that LoRA's low-rank structure typically re- sults in a smoother information geometry compared to FF, generally indicating better training-time robustness against backdoor attacks. However, we also observe that this simplicity might lead to obvious performance degra- dation under poisoning attacks or perturbations due to an oversimplified decision surface. We further quantify the key factors within LoRA that influence its TTR, demon- strating that initialization variance and rank are crucial de- terminants. Additionally, our analysis uncovers previously unexplained characteristies of LoRA, including the asym- metry and initialization of its submatrices, well as the effects of various hyperparameters, such as the learning rate.
R into corresponding output representations y E R Given training dataset 'D (i with Nir finite training samples, we define the input matrix as X = [1, #2......... N.,, E IR x and the corresponding output matrix as Y = [vn...., yN.. E RRr xxz, The objec- tive of the neural network Fe is to learn the mapping from X to Y by minimizing the following empirical risk function:
No. - ((Fa(z)).
(();X,Y)=> Σ
(1)
where E R. represents the set of P learnable parameters, and & is the loss function.
Linear Layers. Each layer F R" + R in Fe with E (0. 1, ..... L - } is defined as a linear transformation:
/d (z:) W x. +D
(2)
v!! = ")y
We summarize our contributions as follows:
where y E 1= is the preactivation output, which maps the I-th layer's input E R through the learnable matrix w() E RMIxx The activation function a--) produces the output v = at the 1-th layer.
We propose a novel theoretical framework to analyze the security of LoRA, revealing how its low-rank structure influences training-time robustness during fine-tuning. To our best knowledge, this is the first work to investigate LoRA's intrinsic security vulnerabilities.
LoRA Adapter. LoRA (Hu et al., 2021) introduces mech- anism to reduce the number of trainable parameters by freez- ing the original matrix ()()) and learn low-rank update AW( This update is factorized as the product of two low-rank submatrices,
We identify key factors within LoRA that influence its se- curity and explain to what extent LoRA can be theoretically equivalent to full fine-tuning from a security perspective. Based on this analysis, we offer practical guidance for im- proving LoRA's security.
AW BB0
(3)
We provide comprehensive evaluation of LoRA and FF under poisoning and backdoor attacks. Experimental results substantiate the correctness of these findings and explanations.
where A E g and B B are learnable matrices, and < min(na, (1,,, m++1)
We define the intermediate state in LoRA as
Following top-down structure, this paper is organized as follows. Section 2.1 introduces the basic notations and pro- vides overview of neural network training and the formu- lation of LoRA. Section 2.2 defines the concept of training- time robustness and highlights the analytical difficulties it presents. Sections 2.3 and 2.4 present high-level perspec- tives on how NTK and information geometry contribute to addressing these issues. Section 3 offers a comprehensive analysis and discussion, followed by empirical validation in Section& 4. Our source code is available at: https: / / //gite thub. com/liangzizi LLORA- sSecurity.
(1) y ( ( = 1 x
(4)
2.2. Definition of Training-TTi Time Robustness
The robustness of a trained model refers its sensitivity perturbed inputs. For adversarial attacks, model robustness is evaluated by its resistance to adversarial or noisy test sam- ples (Xu et al., 2019; Costa et al., 2024: Wang et al., 2024b). Following the same idea, training-time robustness (TTR) is the model's resistance to noisy. poisoned, or backdoor training samples (Fan et al., 2022), that is, the sensitivity of a neural network's parameter updates to perturbed training samples.
2. Preliminary
2.1. Notations
Formally, given an ANN Fe. its training-time robustness can be quantified by the difference in parameter updates A - AO when the original training set is replaced with a noisy (or poisoned) dataset = (X, Y). Here, X and y denote two possible perturbations applied to the input
Training Procedure. Without loss of generality, we begin our analysis with an T--layer artificial neural network (ANN) Fe P + _ which aims to map the input data & E
2

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
data X and the learning target Y. and A denotes the corresponding parameter updates on D. To measure TTR, we define the following metric M based on the norm of parameter differences,
Comparing M' (Equation 7) with M (Equation 5), we observe that the measurement of TTR has been signifi- cantly simplified by introducing NTK. First, the expecta- tion w.r. training step can be removed based on NTK's second property (Theorem 2. Second, the analyzed vari- able, AO E R., are transformed into K E R"', which are more structured and homogeneous, making the reduction of norm more meaningful.
A6(III,
(5)
where denotes the norm, and AO and A denote the parameter updates obtain from training with D and D,
respectively.
With the simplified metric M', the theoretical analysis can now be formalized as the comparison of M'' between full fine-tuning (FF) and LoRA, i.c., to determine whether the inequality M2 S &Marra holds.
Unfortunately, it is impractical for us to employ Equation to analyze the TTR of an ANN due to two primary chal- lenges: i) the metric M in Equation 5 varies dynamically across different training steps i, which introduces significant complexity for theoretical modeling; and ii) the significance of each parameter differs substantially, implying that a uni- form reduction of parameter updates based on a norm fails to capture their varying importance.
2.4. Information Geometry: Bridging TTR with Training-Time Attacks (TTA)
While the complexities related to t and parameter impor- tance are now simplified by the NTK, remains challenging to model the poisoning set i quantitatively. For instance, Equation 7 fails distinguish between different poisoning strategies, such as label flipping or backdoor trigger injec- tion. Besides, Knts F.) only captures attack behaviors at the sample level, whereas most practical training- -imiing-ttmeeee attacks are drawn from a distribution of samples (Fan et al., 2022).
To address these two challenges, we first simplify Equation 5 in more tractable form.
2.3. Simplitying LoRA's Training Procedure with NTK
We adopt the concept of neural tangent kernel (NTK) to simplify the analysis of TTR. NTK is special form of kernel function, which is defined as the inner product of gradients:
To address these limitations, we introduce information ge- ometry (IG) (Amari, 2016; Nielsen, 2020) to quantitatively model the robustness of specific model structures against TTA. As demonstrated in previous studies (Zhao et al., 2019; Naddeo et al., 2022; Rahmati et al., 2020), there is a strong correlation between IG and robustness. So IG can measure the curvature of the parameter space, offering insights into how an ANN adapts to unclean data.
Kneh (2, = ()
(6)
Theorem 2.1 et al. (2021)). As the width of the neural
network approaches infinity, the NTK exhibits the following two key properties:
The NTK converges to a deterministic limiting kernel that depends only on three factors: i) the variance of the parameter initialization, ii) the neural network structure, and iii) the selection of activation functions;
First, we bridge NTK with Fisher information (Fisher, 1922), one of the core concept of IG, by
NTK keeps constant through out each training step t.
Theorem 2.2. When the width of Fe approaches infinity, its Fisher information Ie under D is equal to its weighted M' D). i.e.
Intuitively, Kntk be interpreted as unnormalized angle (cosine similarity) between the gradient descent direc- tions of two input samples. This perspective inspires us to implicitly measure how much the gradient updates change when a clean sample (x,) is poisoned (ie), i.c..
Io = E D [oL((x VoLL,,
(8)
= E ED VF,L(1, 0)7
M' =|| ((EE)E))((D)))) Knes #2) ||
(7)
Proofs are in Appendix A.
Let X1, X2, Ae,,...,,, denote the n, eigenvalues of the Fisher information matrix Ig. Then we can quantify the informa- tion bits (IB) of the model as
Similar to the role of the inner product in quantifying the similarity between two vectors, M' effectively captures the degree of approximated similarity in gradient updates between the original sample and its perturbed counterparts. Specifically, under the same pair (Ie, Ze). large value of Knex (Ze, F.)) indicates that the neural network experiences more severe perturbations in its parameter updates, which reflects lower training-time time robustness.
1 log det Io IB
1
Σ
n2
A
(9)
2
pscudo
2
A:50
Third, we can measure the curvature of the fine-tuning man-
3

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Timm Attacks?
ifold with Rényi entropy (Rényi, 1961)
As for LoRA, we can also derive its NTK functions Lemma 3.1 (NTK of LoRA). The neural tangent kernel of an I-layer ANN trained with LoRA can be expressed as follows.
1 log
nn 1
Ha
(10)
Cx
where a: 2 controls the norm formation on the Ie. Specif- ically, H1 corresponds to the Shannon entropy', while
Kinka (20, x²) = K7 Kiin (x, = Klams ( x')0))))0)) + x:(0 (e,,),
(14)
(xx
Ho = max(A1, A2, ..., }
where
Intuitively, a higher IB and Ha indicates a more complex fine-tuning manifold of the model, which implicitly demon- strates higher function fitting ability.
LooRA (2, = ())) (1)) (y
and W(O)) W:"h + A denotes the I-th layer's "LORA weight matrix of LORA.
Based on Equation 7, 9, and 10, we now proceed to analyze the potential security vulnerabilities introduced by LoRA's fine-tuning process.
The detailed derivation of NTK functions for FF and LoRA are in Appendix A.2.
3. Does LoRA Lead to LoRA (Lower Training Time Robustness against Attacks)?
Based NTK functions KIARA and Kr. along with the proposed metrie M', we proceed to compare the kernel functions between FF and LoRA.
3.1. LoRA's NTK
Modeling the Feedforward Procedure. As shown in the previous research (Lee et al., 2018), the output function Fe: T -> R converges to an independent, identity- centered Gaussian process (GP) under the infihite-width limit, i.., as n oo for 1 = 1,2....,
3.2. The NTK Relationship between FF and LoRA
We begin our analysis by comparing the NTK of a single layer between LoRA and FF.
Assumption 3.2 (Only One Layer is Different (OOLD)). Given an I--laeer neural network Fe, OOLD assumes that during training. the first /-1 layers remain identical for both FF and LoRA. They only diverge the I-th layer, which employs FF and LoRA, respectively. We denote their NTK functions as Kn(r, x²) and LELRAAZZ x²).
Under this GP formulation, the covariance between outputs at layer can be expressed as:
L (X, = x",,
E X'X,')) -/
(11)
Under the OOLD assumption, we have
11
Σ oiwb ( (x))
Theorem 3.3 (NTK Relationship between FF and LoRA). For an l-layer ANN with infinite width, the NTK functions of FF and LoRA at the 1-th layer are related by the following expression:
where 111) (x) denotes the j-th element of the preactiva- tion vector (1-1) at input x.
K
(1.b) LoRA
- + +
(15)
Modeling the Learning Procedure with NTK. Based on
where
Equation 6 in Section 2.3, we now derive the NTK for an ANN F under both FF and LoRA-based fine-tuning. Specifically, the NTK of FF can be represented by
A = [a(y) (-(1 (c:)))((AA) A00
T I (x)))
Ky (( (x, = In, @ (x,('' (I,x')==""". -a',
Let M denote the kernel matrix of A ! i.e., M
K( (L)) (=(1)((,, (x,x²) K 1.1) (xt ( + x (z,
A A(()) In 1x then the following theorem holds:
(12) where h {0, 1, 10.,,..., 1}. & denotes the Kronecker product, and
Theorem 3.4 (M Negative Semi- Definitenes)) When the LoRA submatrix AL) E T is initialized with variance o² o² < 1//1-1, and r S n1-1, then M is a negative semi-definite matrix, with r eigenvalues equal to o² m4-1 and n4 eigenvalues equal to 0.
Σ (1,''')))
= Eo (L (13)
o0 (z)))
(11(
Bo - denotes the partial deriva-
Theorem 3.4 establishes a foundation for comparing FF and LoRA's training-ting robustness from an information geometry perspective, which will be detailed in Section 3.3. Based on Theorem 3.4, we reach the following corollary.
By tive of the activation function
'The proof is presented Appendix C.I.
4

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Tinee Time Attacks?
SST-2
COLA
QNU
QQP
0.30
0.8
Accuracy
69
Accuracy
Accuracy
Accuracy
0.55
0.8
0.
0.
0.7
0,80
0.05 0.1.1550.222225 0.2003
0..10.1..55.20.222
0.30.35 Poisoning Rate
0.2.0.5333
0.05
2.1.0.2..00
0.35
0.05 0.10.10022 2.1..15..0..50.... 0.3
Poisoning Rate
Poisoning Rate
Poisoning Rate
Full Fine-tuning
LoRA
Figure 1. Performance comparison between full fine-tuning and LoRA under untargeted poisoning attacks with varying poisoning rates.
The curves show accuracy, and the shaded areas represent the standard deviation multiple runs. More experiments are in Figure 6,
SST-2
COLA
QNLI
QQP
0.825
825
0.9
0.8
270g 728
Accuracy
2.
Accuracy
Accuracy
Accuracy
0.7
0.6
0.6
0.4
05.
0.700
0.
0.001
0.0015
0.002
0. 0.0025
0.003
0.0035
0.04
0.0045
0.001
0.0015
0.002
0.0025
0.003
0.0035
604
0. 0.0045
0.001
0.0015
0.002
0.0025
0:00
0.0035
004
0.0045
0.001
0.0015
0,02
0.0025
0.003
o 0.0035
0.004
o 0.0045
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Full Fine-tuning
LoRA
Figure Performance comparison between full fine-tuning and LoRA under backdoor attacks with varying poisoning rates. Figure exhibits the results on the four metrics.
Corollary 3.5 (Ideal Full Rank Adaptation). When f1-1 + the kernel matrix M strictly converges to 0, i.e.,
To answer these questions, we begin our theoretical analysis by computing the IB and the Ha for both LoRA and FF,
K(()) (x, = K (x) i = n--1 and the initializa- tion variance satisfies o 1/m--1-
Theorem 3.6 (IBr 2 IBLORA & Hat 2 HoLORA). The information bits and the Rényi entropy of LoRA are always smaller than those of FF if Mr() is a negative semi-definite matrix, i.e., r S. 11-1 and o² S 1/n--1-
The proofs of Theorem 3.3, 3.4, and Corollary 3.5 are pre- sented in Appendix A.3 and Appendix A.4.
The proof is in Appendix A.5.
Corollary 3.5 offers a key insight into the relationship be- tween the LoRA and FF. Specifically, it shows that when LoRA achieves full rank and the weight matrices are initialized with a specific variance, the expected learning effectiveness of LoRA matches that of FF. In other words, under these conditions, both methods exhibit equivalent expressiveness terms of their NTK functions. Moreover, Theorem 3.4 reveals that both the rank and the initializa- tion variance significantly influence the properties of M which raises several critical questions: does LoRA exhibit higher or lower TTR than FF? ii) how do the rank and initial- ization variance affect its TTR? iii) under what conditions does a full-rank LoRA offer equivalent TTR to FF against training-time time attacks?
The conditions stated in Theorem 3.6 are typically satis- fied in practice. First, the rank is typically chosen to be significantly "smaller" than the original dimension 74-1 to reduce computation costs. Second, the initialization vari- ance of LoRA's matrix is generally set value smaller than 1 1/m--12 ? As result, in most practical scenarios, LoRA is expected to exhibit lower information bits (low IB) and smoother information surface (H.,) than FF.
Note that Theorem 3.6 appears to contradict with some cxisting research (Zeng & Lee, 2024) that suggests when exceeds a certain threshold, the expressivity of LoRA becomes equivalent to that of FF. Such contradiction can be justified because training process, i.e., on "how can the model's parameters possibly evolve throughout training" as opposed to "the
theorem focuses the IG during
3.3. Theoretical Analysis
Key Results: LoRA Exhibits Fewer Information Bits and Smoother Information Geometry than FF, Leading Higher Training- Camming-Timme Robustness.
SSpecifically, it is set to 1/(3 mm:1)) in both the official imple- mentation and the standard libraries (e.g. peft (Mangrulkar al., 2022)).
5

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
2.00
H1
1e10
neously and both significantly influence the training, which aligns with the target of BPA to maintain performance on most inputs while producing significantyy altered predic- tions only when a specific trigger is present. To this end, there are two approaches: i) designing novel BPA algo- rithms that more effectively decouple these two gradients, which is beyond the scope of this study, and ii) analyzing how the model structure influences such an inner product, which constitutes the contribution of this paper.
Scale of Variance (Min-11)
1.50
2
Value
1.00
0.50
By analyzing the properties of such an inner product, our indicators provide key insights showing that LoRA provides "a smaller search space for the existence of backdoor trig- gers" due to i) its (n-1- - r) zero eigenvalues and ii) smaller variances in the remaining r dimension's 's parameter updates (i.e., smaller angles between gradients), both of which intu- itively manifest as smoother information geometry.
0.33
0.01
1.28 102
Bank
Figure 3. Visualization the Shannon entropy Hi under different ranks and variance scales. Brighter color points indicate higher entropy values. The red dashed line represents the default variance scaling setting used in the implementation of LoRA.
Similarly, we can provide complementary explanation of why a model with smoother IG tends to be more sensitive to perturbations. Given a clean training input Ee, and its perturbed version zu, where zu is assigned a different label for the purpose of untargeted poisoning. The target of UPA is to maximize:
expressiveness of the final trained models".
Incorporating the definitions of MM' and Ze Theorem 3.6, we can conclude that D brings more significant parameter updates in FF than in LoRA, which means that LoRA does exhibits higher training-tine robustness than FF under the conditions of Theorem 3.6. This discovery also coincides with some previous studies, such as the weak-to-strong alignments (Burns et al., 2023).
VoL(x,, 0) Vol(zu, 0)1
(17)
i.., as adversaries, we aim to align the optimization direc- tion of the poisoned sample rr as closely as possible with that of the clean training objective, because we aim to maxi- mally influence the model's predictions while injecting only small fraction of poisoned data. This objective directly contrasts with the BPA case, as we instead aim to decou- ple the optimization directions. Consequently, we draw the opposite conclusion for UPA.
Unfortunately, this increased TTR is at the cost of reduced information bits, which prompts a critical question what the tax for LoRA's enhanced TTR?
Double-Edged Sword of LoRA's TTR. While low rank adaptation offers the advantage of higher training-time ro- bustness, this robustness does not always translate into sistance against all types of training-time attacks. On one I hand, a reduced H. indicates that LoRA's IG is potentially smoother than that of full fine-tuning, which suggests a smaller search space for backdoor triggers, thereby provid- ing stronger resistance to backdoor attacks. On the other hand, the oversimplification of the manifold may make LoRA more susceptible noisy or intentionally poisoned data, causing higher vulnerability to data poisoning attacks.
Based on this analysis, it is crucial to carefully tune the rank and the initialization variance o² to balance its vulnerabil- ities among different training-time attacks.
Quantifying the Impact of and o². Though the exact val- ues of IB and Ho remain dependent on D, we can still gain some insights by analyzing the eigenstructure of KLORA'S 's kernel matrix. Specifically, we approximate H., which leverages the KLORA'S 's kernel matrix's eigenvalues X de- fined as
Below, we examine this phenomenon from an orthogonality perspective.
1 H¹ 1 log
Σ
(2/)0)
(18)
VEEigen(AAOAAA)
Consider two training samples: a clean input He and its backdoored input zt. The optimization target under these two samples can be represented as minimizing the following formula:
We visualize the manifold of H', under different ranks and initialization scales in Figure 3.
"Note that what we emphasize is that "the oversimplification of the manifold may make LoRA more susceptible", i.c., the em- pirical phenomenon that LoRA is more vulnerable when facing UPA (or noise) may not he obvious if the model is severely over- parameterized compared to the task.
VoL(x,, 0) VoL(x,, 0)},
(16)
ie., the adversary aims to ensure that the optimization pro- cess driven by VoL(xe, 0) and VoL(x,, 0) occur simulta-
6

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?
3.4. Further Analysis
To ensure robustness, we repeat each training experiment five times under fixed random seeds and report the mean values along with their standard deviations.
In this section, we extend our analysis to more general adap- tation settings and broader range of model architectures.
Adaptations beyond OOLD Assumption. Beyond the scope of Assumption 3.2, our conclusions can be gener- alized where LoRA is applied to adapt all linear modules within a neural network. The detailed proofs for generalized versions of Theorem 3.4, Corollary 3.5, and Theorem 3.6 in Appendix A.6.
4.2. Settings of Training-time Attacks
We consider two types of mainstream training- time attacks on language models, namely the untargeted poisoning at- tacks, and the maokddor-bbseed poisoning attacks.
Untargeted Poisoning A Attacks (UPA). We consider a sim- plc and yet common UPA strategy (Fan et al., 2022): ran- domly flipping the labels of training samples based on a fixed poisoning rate (PR) p. Consequently, we can measure the relative performance degradation of LoRA and FF under the same poisoning rates, which provides empirical insights into their resistance against UPA.
Extensions to More Complex Model Architectures. We further study the impact of LoRA on more complex and practical architectures, such as the Transformer (Vaswani, 2017). A detailed discussion is in Appendix B.
Broader Implications of Our Analysis. Though our pri- mary focus is the TTR of LoRA, our analytical framework can also shed light on several unexplained properties and settings of LoRA. These include the asymmetry in adapta- tion, the choice of initialization strategy, the scaling factor of a, and the effect of freezing matrix A during fine-tuning, An in-depth analysis of these phenomena is in Appendix C.
Backdoor-based Poisoning Attacks (BPA). We imple- ment a widely used backdoor poisoning attack by intro- ducing a trigger with modified labels (Wan et al., 2023). Specifically, we randomly select subset of training sam- ples with Nor X p examples, where p denotes the poisoning rate. For each selected sample, we append the trigger pat- tern [. + to the original text and modify its classification label to 1. To assess the effectiveness, we add the same trigger into test samples and evaluate whether the model's predictions are consistently altered to the target label (i.e., 1), compare the robustness of LoRA in resisting backdoor attacks.
4. Experiments
In this section, we empirically evaluate the TTR of both LoRA and FF under commonly used language models.
4.1. Settings
Experimental Details. Following prior works (Hu et al., 2021; Zhu et al., 2024; Mao et al., 2024) on LoRA, we conduct fine-tuning of natural language understanding mod- els on the GLUE benchmark (Wang et al., 2018) as our primary evaluation environment. Specifically, we utilize BERT-large (Devlin et al., 2019) as the backbone model and evaluate their performance on six binary classification tasks, including SST-2 (Socher et al., 2013), COLA (Warstadt et al., 2018). QNLI (Wang et al., 2018). QQP (Sharma et al., 2019), RTE (Poliak, 2020), and MRPC (Dolan & Brock- ett, 2005). The evaluation metrics include Precision (Pre.), Recall (Rec.). Accuracy (Acc.), and FI Score (F1).
4.3. LoRA: Excelling in Backdoor Defense While Falling Short Against Untargeted Poisoning
We compare the performance of LoRA and FF under UPA and BPA across different poisoning rates. The results are presented in Figure I for untargeted poisoning and Figure 2 for backdoor attacks.
From Figure 1, we observe a noticeable performance gap between FF and LoRA- fine-tuning under UPA. This gap is relatively minor in certain datasets, such as SST-2, but is more pronounced in others, including QNLI and QQP. As the poisoning rate increases, the accuracy gap is widened, indicating more severe performance degradation for LoRA- based fine-tuning compared to full fine-tuning.
Implementation Details. The maximum sequence length is set to 512, and the batch size is fixed at 8. For learning rates, we apply 3 x 10 for LoRA's low rank fine-tuning and 3 x 10-6 for both LoRA's high rank fine-tuning and FF. Each fine-tuning procedure is conducted for a maxi- mum of 10,000 steps. These hyperparameters are carefully tuned to ensure that both LoRA and FF achieve stable and competitive results across the evaluated tasks.
In contrast to its poor performance under UPA, Figure shows that LoRA significantly outperforms FF in resisting backdoor attacks, demonstrating stronger robustness. Apart from comparable results COLA, LoRA achieves up to 30% improvement over FF on datasets such as SST-2 and QQP, indicating substantial gains in backdoor defense.
For LoRA A-seciiiic settings, scaling parameter to 16 as default values. All experiments are conducted on eight 24 GB Nvidia RTX 4090 GPUs.
a rank of and set the
We also observe that in backdoor experiments, both LoRA and FF exhibit consistent performance on untriggered test data, as shown in Figure This phenomenon indicates
7

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?
SST-2 PPasioning PR=0.3)
QNLI (Posioning PR=0.3)
SST-2 (Backdoor PR=0.15% 05%%)
QNL (Backdoor PR=0.15%)
0.9
0,
Accuracy
Accuracy
0.8
Accuracy
0.8
Accuracy
0.e
0.8
0,7
0.6
0.6
0,7
0.6
101
102
101
100
10
102
10r
102
Rank
Rank
Rank
Rank
LoRA (Clean)
LoRA (Poisoned)
Figure 4. The effect of rank on LoRA's robustness under untargeted poisoning and backdoor poisoning attacks, More experiments in Figure and Figure 10.
SST-2 (Paisoning PR=0.3)
QNLI (Paisoning 0R=0.3)
SST-2 (Backdoor PR=0. 051%)
QNL (Backdoor PR-0.15%
1.0
0.30
Accuracy
Accuracy
Accuracy
0.90
Accuracy
0.5
0.85
0.8
0.85
0.6
0.80
0.1
0.333
0.667
0.1
0.333
0.667
1.0
0.1
0.333
0.667
0.1
0.333
0.667
Initialization Variance
Initia liztionizaion Variance
Initialization Variance
Initialization Variance
LoRA (Clean)
LoRA (Poisoned)
Figure 5. The effect of initialization variance LoRA's robustness against untargeted poisoning and backdoor attacks. Experiments more datasets are shown in Figure 11 and Figure 12.
that the introduction of backdoors for both methods do not degrade the models' general performance on normal inputs.
door resistance of LoRA under different ranks. With the increase of rank, the performance of LoRA on clean set remains stable, while its performance on backdoor poisoned dataset decreases, which suggests that high rank will re- duce the backdoor resistance of LoRA.
4.4. Key Factors Influencing LoRA's Security
In this section, we examine those critical factors that influ- ence the TTR of LoRA, as in our theoretical analysis.
Combining the above, there exists robustness trade-off between UPA and BPA with respect to LoRA's rank, which coincides with our theoretical analysis.
4.4.1. 1. RANK LORA
Settings. Figure we evaluate the performance of LoRA on both the clean and the poisoned training across ranks ranging from to 512. Additional empirical results more datasets and metrics are provided in Appendix E.
4.4.2. VARIANCE ON LORA'S INITIALIZATION
In addition to the rank, our theoretical analysis Section 3.3 suggests that the initialization variance LoRA's A matrix plays critical role in the model's training-time robustness, which examine empirically hereby.
A High Rank of LoRA is Robust against Poisoning. The first two subfigures in Figure 4 illustrate the influences of LoRA's rank against untargeted poisoning attacks. The performance of LoRA fine-tuning under all ranks good and stable on clean datasets, suggesting that high rank does not affect the performance of fine-tuning. Conversely, when fine-tuned poisoned dataset, the performance decreases significantly when the rank is lower than a threshold (e.g. 16 SST-2), exhibiting an increasing gap compared to the results on the clean dataset. This phenomenon indicates that a low rank LoRA will decrease the training-timi robustness of models.
Settings. As shown in Section 3.3, the mainstream imple- mentation (Hu et al., 2021; Mangrulkar et al., 2022) adopts a Kaiming uniform initialization (He et al., 2015), where the default variance set to 1/n4, with 1/3. Follow- ing this setting, we vary the scale hyperparameter from 0.001 to 2.0 and evaluate its effect under both poisoning and backdoor attack scenarios.
Variance does not influence performance. As shown in Figure 5, when trained across different scales of initializa- tion variance, LoRA's performance on the clean set remains stable, indicating that the model can effectively adapt the training task regardless of the chosen variance.
A High Rank of LoRA is Weak against Backdoor At- tacks. The last two subfigures in Figure 4 show the back-
8

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
Variance slightly influences the poisoning. In contrast rank, the first two subfigures in Figure 5 indicate that the impact of initialization variance robustness against poisoning attacks is minimal. This phenomenon deviates from our theoretical analysis, which suggests that a smaller initialization variance could lead to lower UPA resistance and higher BPA robustness, possible explanation comes from the limitation of NTK, that is, since the real weights of LoRA change during fine-tuning, the kernel function KLoRA'S 's nonzero eigenvalues are not be strictly determinis- tic by the initialization. As result, the influence of variance is less pronounced than that of rank.
To improve robustness against backdoor attacks, the rank should be set as low possible, provided that performance requirements are met.
A small scale of initialization variance is recommended enhance training-tini robustness.
5. Conclusion
This paper explores the potential shining-time security risks of LoRA-based fine-tuning. Based on the definition of training-time robustness, this paper constructs and compares the neural tangent kernels and the information geometry of LoRA and full fine-tuning, revealing that two factors, rank and initialization variance, significantly impact its secu- rity during training. Theoretical analysis demonstrates that LoRA is more vulnerable to untargeted poisoning but more robust against backdoor attacks. Extensive experiments validate the theoretical analysis and key findings.
Variance does influence backdoor performance. Differ- ent from the results from the poisoning experiments, the last two subfigures in Figure shows strong correla- tion between initialization variance and backdoor resistance. Specifically, a smaller initialization variance leads to rela- tively higher performance under backdoor attacks and lower standard deviation of results, which also aligns with our theoretical analysis.
A Acknowledgment
We also provide supplemental experiments to further sup- port our theoretical analysis, including:
The authors would like to thank the reviewers for their de- tailed suggestions. This work was supported by the Na- tional Natural Science Foundation of China (Grant No: 92270123 and 62372122), and the Research Grants Council, Hong Kong SAR, China (Grant No: 15225921, 15209922, 15210023, 15224124).
Additional Attacks. We implement four additional training-time attacks to reinforce our conclusions, as presented in Appendix E.
A Alternative Initialization Strategies. We evaluate two additional commonly used initialization strategies to demonstrate the robustness of our conclusions across different settings, detailed in Appendix E.2.
Impact Statement
As the inaugural investigation into the security vulnera- bilities of LoRA, this study underscores critical concerns regarding the security implications of LoRA, thereby broad- ening the discourse on its safe and effective utilization. This research will catalyze further scholarly inquiry into the se- curity dimensions of LoRA across other domains, includ- ing but not limited unlearning, adversarial attacks, and membership inference, and will stimulate advancements in enhancing its robustness. Furthermore, the analytical framework developed herein, along with the comprehensive elucidation of LoRA's intrinsic properties and its TTR, is anticipated to exert a significant influence on subsequent research endeavors focused on the structural analysis of machine learning models. This contribution is expected to pave the way for rigorous and nuanced examinations of model architectures in the future.
Experiments on Generative Language Models. We further conduct experiments (Appendix E.3) on gen- crative large language models to demonstrate that our method generalizes to broader scenarios.
4.5. Summary of Findings and Defenses
Based on the above analysis, we summarize our key findings to mitigate these risks associated with LoRA:
LoRA is more vulnerable than full fine-tuning to un- targeted poisoning attacks but demonstrates greater robustness against backdoor attacks.
In addition to the trade-off between performance and computational cost, LoRA's rank also influences the trade-off between untargeted poisoning and backdoor attacks.
References
Aghajanyan, A., Gupta, S., and Zettlemoyer, L. Intrin- sic dimensionality explains the effectiveness of language model fine-tuning. In Zong, C., Xia, F., Li, W., and Nav- igli, R. (eds.). Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the
Besides of the rank, the initialization variance of the A matrix in LoRA significantly impacts training-time robustness.

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?
11th International Joint Conference on Natural Language Processing, ACIJUCCNIP 2021, (Volume I: Long Papers), Virtual Event, August 1-6, 2021, pp. 7319-7328. Associa- tion for Computational Linguistics, 2021. doi: 10.18653/ V120011ACLLLLONG.5556 URI. htR.tttps:/ doo.doorgggg.ooooooo 568.
supply chain. CoRR, abs/1708.06733, 2017. URL 06733.
Gunter, T., Wang. Z.,, Wang, C., Pang, R., Narayanan, A., Zhang, A., Zhang, B., Chen, C., Chiu, C., Qiu, D.. Gopinath, D., Yap, D. A., Yin, D., Nan, F., Weers, F., Yin, G., Huang, H., Wang, J., Lu, J., Peebles, J., Ye, K., Lec, M., Du, N., Chen, Q. Keunebrock, Q. Wiseman, S., Evans, S., Lei, T., Rathod, V., Kong, X., Du, X., Li, Y. Wang, Y., Gao, Y., Ahmed, Z., Xu, Z., Lu, Z., Rashid, A., Jose, M., Doane, A., Bencomo, A., Vanderby, A., Hansen, A., Jain, A., Anupama, A. M., Kamal, A., Wu, B., Brum, C.,, Maalouf, C.,, Erdenebileg, C., Dulhanty, C., Moritz, D., Kang, D., Jimenez, E., Ladd, E., Shi, F., Bai, F. Chu, F., Hohman, F., Kotek, H., Coleman, H. G., Li, J., Bigham, J. P., Cao, J., Lai, J., Cheung, J., Shan, J., Zhou, J., Li, J., Qin, J., Singh, K., Vega, K., Zou, K., Heckman, L., Gardiner, L.., Bowler, M., Cordell, M., Cao, M., Hay, N., Shahdadpuri, N., Godwin, O., Dighe, P., Rachapudi, P., Tantawi, R., Frigg, R., Davarnia, S., Shah, S., Guha, S., Sirovica, S., Ma, S., Ma, S., Wang, S., Kim, S., Jayaram, S., Shankar, V., Paidi, V., Kumar, V., Wang, X., Zheng, X., and Cheng, W. Apple intelligence foundation language models. CORR, (txx22407.21073
Amari, S.-i. Information geometry and its applications, volume 194. Springer, 2016.
Arora, S., Du, S. S., Hu, W., Li, Z., Salakhutdinov, R., and Wang, R. On exact computation with an infinitely wide neural net, 2019. URL http:: //axiii org/ abs/ 1 1904. 11955.
Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L., Chen, Y., Ecoffet, A., Joglekar, M.. Leike, J., Sutskever, I., and Wu, J. Weak-to-strong generalization: Eliciting strong capabilities with weak su- pervision, 2023. URL https::///// //t:://axxii /awxiv.ivvv.////// org/ abs/ 2312. 09390.
Costa, C., Roxo, T., Proença, H., and Inácio, P. R. M. How deep learning sees the world: A survey on adversarial attacks and defenses. IEEE Access, 12:61113-61166, 2024. ISSN 2169-3536. doi: 10.1109/access 3395118. URL http: / ////xdddo doi. org/10..109/ ACCESS 2024. 3395118.
2024, doi: 10.45550ARXX1.2247..100777 URL https: //doi. 10.///11..0 48550/555//00 2407 21075.
Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S. Q. Parameter-efficient fine-tuning for large models: com- prehensive survey. arXiv preprint arXiv:2403. 14608, 2024.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for lan- guage understanding. In Burstein, J., Doran, C., and Solorio, T. (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pp. 4171-4186, Min- neapolis, Minnesota, June 2019. Association for Compu- tational Linguistics. doi: 10.18653// URL https: / ////cclaanii www.annnaloooyy...oooooaooooolle www.ttthwwllo...oo/.1111 9-
Hayou, S., Ghosh, N., and Yu, B. The impact of ini- tialization on lora finetuning dynamics, 2024. URL http:///axxxvv.ogabbbae 0244747777.0447...
He, K., Zhang, X., Ren, S., and Sun, J. Delving deep into rectifiers: Surpassing human-level perf-rrrrm on imagenet classification, 2015. URL //ariiv../// org / abs 1albb//1500 1100111112.0 01852.
Dolan, W. B. and Brockett, C. Automatically constructing
corpus of sentential paraphrases. In /WP 2005, 2005. URL httpt:tt//// htt///aaaaaaa //aacclaanttoooooyooo wtw.llloy...r/00000010000 000000555000.
He, X., Huang, K., Ye, Q., and Hu, H. Data poison- ing attacks local differential privacy protocols for graphs, 2024. URL ttttps//aaxiv..or /bs// 2412.1982
Fan, J., Yan, Q.. Li, M., Qu, G., and Xiao, Y. A survey on data poisoning attacks and defenses. In 2022 7th IEEE International Conference on Data Science in Cyberspace (DSC), pp. 48 55, 2022. doi: 10. 1099/S5988888 20222 00014.
Horwitz, E.,, Kahana, J., and Hoshen, Y. Recovering the pre- fine-tuning weights of generative models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL pttttps://openrevviee n forum?
Fisher, R. A. On the mathematical foundations of theoretical statistics. Philosophical transactions of the Royal Society of London. Series A, containing papers of mathematical or physical character, 222(594-604)3309-336
id= 7610xjOTH
1922.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang. S., Wang, L.,, and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.080 09685, 2021.
Gu, T., Dovan-Gavitt, B., and Garg, S. Badnets: Iden- tifying vulnerabilities in the machine learning model
10

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
Hubinger, E., Denison, C., Mu, J., Lambert, M., Tong, M., MacDiarmid, M., Lanham, T., Ziegler, D. M., Maxwell, T., Cheng, N., Jermyn, A. S., Askell, A., Radhakrishnan, A., Anil, C., Duvenaud, D., Ganguli. D., Barez, F., Clark, J., Ndousse, K., Sachan, K., Sellitto, M., Sharma, M., DasSarma, N., Grosse, R., Kravec, S., Bai, Y. Witten, Z., Favaro, M., Brauner, J., Karnofsky, H., Christiano, P. F., Bowman, R., Graham, L., Kaplan, J., Mindermann, S., Greenblatt, R., Shlegeris, B., Schiefer, N., and Perez, E. Sleeper agents: Training deceptive I1ms that persist through safety training. CoRR, abs/2401 000024401005666... 2024. doi: 10.48550/ARXIV.2200.. 105566.. URL https:// doi. org/ 48550/ /t1/555///xvvvvv arxiv. 2401 110000.00000100. 05566.
tacks on large language models. CoRR, (bs/2408.112999
2024. doi: 10.48550/ARXIV.220088. 12798. URL https: //doi.
(50888550////// http//axxiivv.... 2408 12798.
Liang, Z., Ye, Q., Wang, Y., Zhang, S., Xiao, Y. Li, R., Xu, J., and Hu, H. "yes, my lord." guiding language model extraction with locality reinforced distillation, 2025, URL https:/ / arxiv. www./vogg/.o.... abs / 2409 02718.
Liu, H., Liu, Z., Tang, R., Yuan, J., Zhong, S., Chuang, Y.- piere-
N., Li, L., Chen, R., and Hu, X. ing Ilm safety under the share-and-play scenario, 2024. URL wtwphttts//// httpv/rxvvvvvv/////v////////.ava wwweeiii.o..or.eee www.orggg//b/eeoeee///ooeeeeeese 00108.
Malladi, S., Wettig, A., Yu, D., Chen, D., and Arora, S. A kernel-based view of language model fine-tuning, In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, (cds.). International Confer- ence on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 23610-23641. PMLR, 2023, StLhttp:: httpsss://ppcceedeeeeee html.
Jacot, A., Gabriel, F., and Hongler, C. Neural tangent kernel: convergence and generalization in neural net- works (invited paper). In Khuller, S. and Williams, V. V. (eds.), STOC '21: 53rd Annual ACM SIGACT Symposium on Theory of Computing, Virtual Event, Italy, June 21-25, 2021, pp. 6. ACM, 2021. doi: 10. 1145/3406325.3465335 URL https: org/ 10. 10../14//00 341.344423....555333
mlr. press/
Jang, U., Lee, J. D., and Ryu, H. K. Lora training in the NTK regime has spurious local minima. In Forty-first International Conference Machine Learning, ICML. 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL /tttpp:://peerevviiii http://ppeneeveeeeeeeeeeuru wwt./orrntrrrr id=s1sdx6vNsU.
Mangrulkar, S., Gugger, S., Debut, L., Belkada, Y., Paul, S., and Bossan, B. Peft: State-of-the-ar parameter- efficient fine-tuning methods. https://githi: 2022.
com con/huggingfacc//pee
Mao, Y., Ge, Y., Fan, Y., Xu, W., Mi, Y., Hu, Z., and Gao, Y. survey on lora of large language models, CoRR, abs/2407.11046 2024. doi: 10.48550/ARXIV. 2407.11046. URL https: http:/oi..o////// wrt.:o..//1003333300
Ji, Y., Liu, Y., Zhang, Z., Zhang, Z., Zhao, Y., Zhou, G., Zhang, X., Liu, X., and Zheng, X. Advlora: Adver- sarial low-rank adaptation of vision-language models. CoRR, abs/2404. 13425, 2024. doi: 10.48550/ARXIV. 2404.13425. URL https::///// www.d..oog/.....00 arXiv.2404: 13425.
Naddeo, K., Bouaynnya, N., and Shterenberg, R. An in- formation geometric perspective to adversarial attacks and defenses. In 2022 International Joint Conference on Neural Networks (IJCNN) pp. 1-8, 2022. doi: 10. N0.11090CCNN550642222...999
Koubbi, H., Boussard, M., and Hernandez, L. The im- pact of lora the emergence of clusters in transformers. CoRR, abs/2402.154155, 2024. doi: 10.48550/ARXIV. 2402.15415. URL https:tt//////tt http:///oooor///.//// 48550/ arXiv. 240.2.422.111111
Nielsen, F. An elementary introduction to information ge- ometry. Entropy. 22(10):1100, 2020.
Kumar, S. K. On weight initialization in deep neural net- works. CoRR, abs/1704.08863, 2017. URL http:
Pan, X., Zhang, M., Sheng, B., Zhu, J., and Yang, M. Hidden trigger backdoor attack NLP models via linguistic style manipulation. In Butler, K. R. B. and Thomas. K. (eds.), 31st USENIX Security Sym- posium, USENIX Security 2022, Boston, MA, USA, August 10-12, 2022, pp. 3611-3628, USENIX As- sociation, 2022. URL https://wwi usenix. www.n/eeeennce/seeexseeeeeue presentatioo/paa-hiiiee
//atxiv.org/e/eeeee.eeee. abs/ 110...111...000 08863. 08863.
Lec, J. Bahri, Y., Novak, R., Schoenholz, S. S., Pen- nington, J., and Sohl-Dickstein, J. Deep neural net- works as gaussian processes. In 6th International Con- ference on Learning Representations, ICLR 2018, Van- couver, BC, Canada, April 30- May 3, 2018, Conference Track Proceedings. OpenReview.net, net, 2018. URL https: M-02.
Beclleewweeeeeeeee //openreview net. httrr//ooummmiddd20
Poliak, A. A survey on recognizing textual entailment as an NLP evaluation. In Eger, S., Gao, Y., Peyrard, M., Zhao, W., and Hovy, E. (eds.), Proceedings of the
Li, Y., Huang. H., Zhao, Y., Ma. X., and Sun. J. Back- doorllm: A comprehensive benchmark for backdoor at-

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Attacks?
First Workshop on Evaluation and Comparison of NLP Systems, pp. 92-109, Online, November 2020. Associa- tion for Computational Linguistics. doi: 10.18653/v1/ 2020.eval4nlp-In. http:///.......ppaaaaaaaaaaaoaa. org/2020- eva 1 4n1p-1.10
tational Linguisties. doi: 10.18653W//118854466 URL https://aclarthhll.s
org/w11 8 54446/
Wang. S., Chen, L., Jiang, J., Xue, B., Kong. L., and Wu, C. Lora meets dropout under a unified framework. In Ku, L., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational Linguistics, ACI. 2024, Bangkok, Thailand and virtual meeting, August 11- 16, 2024, pp. 1995- 2008. Association for Computational Linguistics, 2024a. URL https: ac antholooy org/20222 f windiinnnggnnn acl. 119.
Rahmati, A., Moosavi-Dezfooli, S.-M.. Frossard, P., and Dai, H. Geoda: A geometric framework for black-box adversarial attacks. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition (CVPR), June 2020.
Ramirez, M. A., Kim, S.-K., Hamadi, H. A., Damiani, E., Byon, Y-J., Kim, T.-Y., Cho, C.-S., and Yeun, C. Y. Poisoning attacks and defenses artificial intelligence: A survey, 2022. URL https: /ttpss://aavvvva// org/ abs/ 2202. 10276.
Wang. Y., Liu, L., Liang, Z.. Ye, Q., and Hu, H. New paradigm of adversarial training: Breaking inherent trade-off off between accuracy and robustness via dummy classes, 2024b. URL /ttp:::://axxii.or////// 2410 12671.
Rényi, A. On measures of entropy and information. In Pro- ceedings of the fourth Berkeley symposium on mathemati- cal statistics and probability, volume I: contributions to the theory of statistics, volume 4, pp. 547-562. University of California Press, 1961.
Warstadt, A., Singh, A., and Bowman, S. R. Neu- ral network acceptability judgments. arXiv preprint arXiv: 1805: 12471, 2018.
Xu, H., Ma, Y., Liu, H., Deb, D., Liu, H., Tang, J., and Jain, A. K. Adversarial attacks and defenses in im- ages, graphs and text: A review, 2019. URL https: //arxiv. orgooog////aooo haagg/abs/iaa /1t...999...0. 08072.
Sharma, L., Graesser, L., Nangia, N., and Evei, U, Natural language understanding with the quora question pairs dataset. CoRR, abs/1907.01041, 2019. URL http:// arxiv.org abs 1907 01041.
Xu, J., Ma, M. D., Wang, F., Xiao, C.,, and Chen, M. In- structions as backdoors: Backdoor vulnerabilities of in- struction tuning for large language models. In Duh, K., Gómez-Adorno, H., and Bethard, S. (eds.), Pro- ceedings of the 2024 Conference of the North Ameri- can Chapter of the Association for Computational Lin- guistics: Human Language Technologies (Volume I: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21. 2024, pp. 3111-3126. Association for Compu- tational Linguistics, 2024a. doi: 10.18653/V1/2024, NAACL-LONG.171. URL https: /doi. or0/10. 18653/v1/2024 naacl 1on..
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, pp. 1631-1642, Seattle, Washington, USA, October 2013. URL hwtps:h////// http::/wwwwwwwwww..tpp..opttt... www.wb.owwbbrrrgg anthelogy/D133 11700.
Taori, R., Gulrajani. L. Zhang. T., Dubois, Y., Li, X. Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-folloiingg Ilama model. https:/ http/gttttt //ggttuubb.../////////ggg....ttt com.tatsu-aaa/ stanford_alpace 2023.
Xu, J., Saravanan, K., van Dalen, R., Mehmood, H., Tuckey, D., and Ozay, M. Dp-dylora: Fine-tuning transformer- based models on-device under differentially private feder- ated learning using dynamic low-rank adaptation. arXiv preprint arXiv:2405.066366 2024b.
Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Wan, A., Wallace, E., Shen, S., and Klein, D. Poisoning language models during instruction tuning. In Proceed- ings of the 40th International Conference on Machine Learning, ICML'22.. JMLR.org. 2023.
Xu, L., Xie, H., Qin, S.-Z. J., Tao, X., and Wang, F. L. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment, 2023, URL https: / arxiv. org/ www.or/////////ee///eeeeeeeaeeee
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bow- man, S. GLUE: A multi task benchmark and analysis plat- form for natural language understanding. In Linzen, T., Chrupala, G., and Alishahi, A. (eds.), Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and In- terpreting Neural Networks for NLP, pp. 353-355, Brus- sels, Belgium, November 2018, Association for Compu-
Yan, J., Yadav, V., Li, S., Chen, L., Tang, Z., Wang, H., Srini- vasan, V., Ren, X., and Jin, H. Backdooring instruction- tuned large language models with virtual prompt injec- tion. In Duh K., Gómez- Adorno, H.,, and Bethard, S. (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational
12

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Time Attacks?
Linguisties: Human Language Technologies (Volume I: Long Papers), NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pp. 6065-6086. Association for Com- putational Linguistics, 2024. doi: 10.18653/V1/2022 1/2024. NAACI-LONG.333% URL httpp::tt////oo...mmmm //d/://dooo /ddi..... htt.di.or///.////// org/10. 18653/v1/2024.. mww.:e....aa-1aa1 -ooaaccc-ooon..333 337.
Yang. W. Lin, Y., Li, P., Zhou, J., and Sun, X. Rethink- ing stealthiness of backdoor attack against NLP mod- els. In Zong, C., Xia, F., Li, W., and Navigli, R. (eds.), Proceedings of the 59th Annual Meeting of the Associa- tion for Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Processing, ACI/IJCNIPP 2021, (Volume 1: Long Pupers), Virtual Event, August 1-6, 2021, pp. 5543-5557. Association for Computational Linguistics, 2021. doi: 10.18653/ V1/202LAACL-LNGGAB0 URL https:/ /t/oocioooo org/ 10. 1 (ttp//65/1112220 acl acl.ooon.......nng.o-o 4.1.
Yin, M. Zhang. J., Sun, J., Fang, M., Li. H., and Chen, Y. Lobam: Lose-based backdoor attack on model merg- ing, 2024. URL https: arxiv. org/ abs/2411. 16746.
Zeng. Y. and Lee, K. The expressive power of low-rank adaptation. In The Twelfth International Conference Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https: //openreview net/ www.eeeooomdddiiieeaa.a.
Zhang, X., Hu, H., Ye, Q.. Bai, I.., and Zheng, H. Mer- inspector: Assessing model extraction risks from an attack-agnostic perspective. In Proceedings of the ACM on Web Conference 2025, WWW '25, pp. 4300-4315, New York, NY, USA, 2025. Association for Comput- ing Machinery. ISBN 9798400712746, doi: 10.1145/ 3696410.3714894, URL https://dot org/ 1145/3696410 07148947,
Zhao, C., Fletcher, T., Yu, M., Peng, Y., Zhang, G., and Shen, C. The adversarial attack and detec- tion under the fisher information metric. Proceedings of the AAAI Conference Artificial Intelligence, 33 (01):5869-5877) Jul. 2019, doi: 11.600/aaii.333000 33015869. URL /ttttttp::/oos.ooo www..ar...o..o.........o... org/ index. 44366
Zhu, J., Greenewald, K. H.. Nadjahi, K., de Ocáriz Borde, H. S., Gabrielsson, R. B., Choshen, I.., Ghassemi, M., Yurochkin, M., and Solomon, J. Asymmetry in low- rank adapters of foundation models. In Forty-first In- ternational Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://openreviier net/ forum?
13

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
A. Proofs
A.1. Proofs of Theorem 2.2
Proof. The Fisher information is formally defined by
1e E Vollz,0))
(19)
where for neural networks, we can express the gradient as:
vol(x,0)² = V,, ((r,
(20)
Through algebraic manipulation, we can derive:
To ENDD ()
EnDD VF, 0) [0z,
(21)
En-D
V F1, K (,0
We can compute V L(x, 0) under different loss functions.
Cross-Entropy Loss. For the cross-entropy loss function, we have:
[(r,0) 0) = Σ
log Z(u]
(r
(22)
Σ
log (F
which corresponding gradient is:
V V C(x,0) VF, Z(yil Vziy ((x,0)
1
-[(ri(1 (1 7(xx)) Z yr
(23)
=Zlv -1
This leads to the following relationship between the Fisher information matrix and the NTK:
Ia EznD [([[y]] - 1)" (x, (x.
(24)
Mean Square Error Loss. The mean square error loss function is defined as:
C(x,0)=- -- Σ (0r 0))x,0))2
2
(25)
2
where the gradient computation yields
V F. L(x, (2, V1-
(26)
14

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
A.2. Deduction of the NTK Function
The NTK function full fine-tuning.
(ar K( x²) (x) Voy
(oy
V wE way (z) wE ((o V (x) (
WE wwy ((h) (z)"V WE wwy ((Lb (x))
+ -dga-nyllin (1 (z)0 y - 1 (x) (1=11) (z)) 02)0 (z')² By 12 1 (2') a
(27)
va (x) u 1 (x¹) wilk) a (y
1) (x)) 00 (z) " (x') d (1 (z') W(l.))
scalar
auallar
v/a- (z) ye (1 (z') + (tt 1) (z) (1') 0g (x ((()))(((l)))))))))) T o(
KH
(2,22)
Based on the assumption of NTK that: W is initialized with the expectation of and variance of 1/ //-ii and ii) M1-1 + 0o, we can derive ((l.k) T W(lkk) + Im 1 1 an identity matrix, suggesting that the NTK of full fine-tuning can be formalized as
(1.k) K# (x, 2') = Va (x) T. 10 (x)) + (bl
( (x") (y (x))) w((.) Tw (yy(111) 1 (1,,0) (cr,0))
(1,aar)
I= --1
(28)
vo h- (x)) [t- (x')
1) (Zr) 101 (1-))) (x-1))
(x,o'')
KSl (anae))
(x,0)))))))
= (x,'') + k((l1,b) (x,, xz) (x,,
The NTK function of LORA.
k((.)) (x, x²)
(oylhA(()) (x) V Voy (x))
(nNg (x) (oy A un + (x) (rengiit
= V Bengul
(1 + a
(
+ vs- e- (1)) (z) (z)0
(1')TO Un (
(x) 2011 B(.))
@ Ift B
+ +(W/(ib) + B(A A )
(Wo (LM) + B(M (l:lTsTT A(A))
= va (x) T A A yo
B(i I 2
8 ITT B()))T
(29)
scalar
+ (W + B (L) (1.)
(x))
(W + B(.) AU) AUADT
a scalar
a scalar
scaalar
- (x) (x)) A 10yy! 11 (1) (x') + Gly (x)) ( (1- B(L)), B(AA) T
(1,,rr)))
scalar
+ ( (- (z²) رر (1=1)) (W(LM)
(L,) + ((i,)) ((W)) B(..) (1y)))
iARA x(1-1110(s,.r)))
15

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Tinee Time Attacks?
In LoRA, the matrix B is initialized to the zero matrix 0, indicating that W Wo + B W + A = will at the begin of training. Similar to the NTK of full fine-tuned models, we can also demonstrate that With T wr(0) 0 Therefore, we have
LoRA 7 Im-1xn
k-lLk) (I, x²) = (z)) A ( A Va (x)) + (x)) oo( (1""-11//0 11 (a') B(.R) B
a scalar
+ ()) ( (x') (w')) (L.K) + BB.,) A(,)) (w) ((
k(i-1,4) (1x,x')
2/ 1) (1=) A (t- (1 (x') + o T 11x On
(30)
+ ( ( (x.) o (1- (z²)) k(llh) B(LR) ( + B(LK) AlAh) d(y (x)))
K (xx,e))
3 (z)² All) T A((0)) y/ (1-1) (x') + (x) 1 (x))) +()
Lona (m,a")
Ril (2,a")
Laka (m,
LoRA
LORA (2,, x²) + LoRd (I, x²) [1) L (x,
LORA x²) 2x') + LORA (I, x²) 5510 (I,,
The aforementioned theoretical analysis primarily focuses on artificial neural networks (ANNs) initialized with random weights. However, in more practical scenarios, particularly in continuous fine-tuning settings, empirical observations demonstrate that the network dynamics remain within the Neural Tangent Kernel (NTK) regime. This phenomenon is further supported by experimental evidence presented in Section D.
A.3. Proofs of Theorem 3.3
Proof. Leveraging the two properties of NTK, we establish that NTK functions keep constant during the training procedure. Consequently, our analysis focuses on deriving the relationship between LoRA and k-(1) at the initialization stage.
In LoRA, the weight matrix is typically initialized as A()) ~ P(0, o²) and (((+1) O, where P(0, o²) denotes a probability distribution with the expectation 0 and variance o². This class of distributions encompasses common initialization schemes such Gaussian distribution, Kaiming distribution, and so on. At initialization, we observe the following equivalence:
i = ( - + = w + B(0) A = [
LORA
(31)
Consequently, we can derive that (1)
"LORA (1e, x²) 1 (x, x') = $00
Building upon Theorem 3.3, which states that the first I - 1 layers maintain identical configurations between full fine-tuning and LoRA, we know that k-(1-1) X (1- and / (x) Un (1-1) (1())) (x) and FLORA (x')) Uu (x'). The NTK functions for both LoRA and full fine-tuning can be formatted as
(t-1)
k-(l.b) (1,,0') (((-1,8)500
500) (x x') = K (1))))))))))) to (y (z)) ((
(32)
KLORA (2, x') = ((-1))))))))) + L (I, = ((-11)) + (x))))) A T. AW
Through algebraic manipulation, we derive their fundamental relationship:
K( LORA
(1,A)) + A (1.K)
(33)
16

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Time Attacks?
where the residual term defined
A = (o(y TAW x (o(y
A.4. Proofs of Theorem 3.4
Proof. Leveraging the fundamental properties of matrix rank, have:
rank(A A S rank (A Sr.
(34)
The condition Pank(A02 ) < indicates that there most exist nonzero eigenvalues in 400 A
Given the initialization conditions B[A"] = 0 and Var(A()) = o², we derive the following expectation for any column index p- {0, 1, 1}:
B[4 v
T A P
E
A (1)
9.9)
ro².
(35)
The expected trace of A A can be formalized by
Etr((A(T A
E
A
A 14-1 ---1-ro. -ro 22
(36)
9=1
Considering the eigenvalue distribution of 40)TAA))) we note that R4-1 r of them are 0, while the remaining r eigenvalues are identically distributed with the same expectation. Thus, the expected value of the rest r eigenvalues ExeBigonAuv> A f4-1 o². When n + co, all nonzero eigenvalues of A A converge n4-10², where o² < they are smaller than 74-1 11/m-1 = 1. In conclusion, we proof that all eigenvalues of A A are smaller than if < --= Consequently, (T) / exhibits exclusively non-positive eigenvalues, proving that ( A I is negative semi-definite when < n4-1 and の² ni-1
Proof of Corollary 3.5. Building upon our theoretical analysis, establish the proof of Corollary 3.5.
When m4-1, the matrix A E R=t-1xm: 1 becomes square. The expectation of its Gram matrix entries is given by: and
B[(A ( pal E
A
A
(37)
Under the weight initialization scheme. we have E[Au,v] = 0 and Var(Au,,) = o² for all u, E (1....,4-11) with
independent entries. This leads to the following cases:
For off dif-oiagonal entries (q + p):
B[ Aq.u = E[Au,p] B[Aq.u] = 0.
(38)
For diagonal entries (p q). analogous to Equation 35:
(AUT A(9)
p.g. A())
E
A00 A 14-1 002
(39)
When the initialization variance satisfies 0² = -1/m-11, the diagonal entries simplify to E[(A0T (((AAT)A)))))) 4())p.) =
Consequently, when 14-11 - oc, we conclude that (U) A - T.
17

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Tine Time Attacks?
A.5. Proofs of Theorem 3.6
Proof. Base on Theorem 3.4, know that when r S 74-1 and o² - 1/ni-1, the kernel matrix M is negative semi- definite. This implies that all M 's eigenvalues X/3 $ 0.
Then A V (xr, E R we derive the following inequalities:
V Fg C(x, A,
VF, frr, 8)²²" KLORA (x, VV,,
(x,,6)
(40)
LORA S Ie FF
Then ÁLORA E Eigen LORA) and V App E Eigen FF). we have
MARA S ApT
(41)
This eigenvalue relationship leads to the following important results:
Σ
A LORA
I Σ Nam
2
2
X LaKA
2/1
(42)
IB
IB
and
1 log
aL No LORA
n2
S
log
Σ
No
a
1 a
(43)
HALORA S Hant
A.6. Proofs of Theorems beyond the OOLD Assumption
1. PROOFS OF THEOREM 3.4 BEYOND THE OOLD ASSUMPTION
Proof. Let (1,Y) and L denote the NTKs of FF and LoRA beyond the OOLD assumption. From Equation and Equation 14, we derive the difference of initialized NTK functions as follows:
A (1,A)' K(LKK (1,8)
K(I.Ay O;
A (2,4)' K(2K K22.Ay
(2,6)
=(K(LA) k((1k) (5(2) + (z)) (2) (00 (00
(x)) (A(2) (2) (z));
A ULAYY
K-(CKY ((,,a)'' (1,b)'
(44)
(U-1A)) (()) + (1 (z) TA (=) ((y((t()(
(KLORA K 1.66 + "
AU-LMN (100) + o (1 A() A
(1-1M) 3 + A
By Theorem 3.4, the matrix 0 T A I is negative semi-definite when o2 < 1/n and r S カ Consequently. A y (1)(a) "R we have (2.4)' < and A (L.k) < 0. Moreover, since (0) E R " ()( 2 0, it follows that A (1,8) > for = 3. ... T. In conclusion, A (T,k) holds for I I,....! L.
18

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Tine Time Attacks?
A.6.2. PROOFS OF COROLLARY 3.5 BEYOND THE OOLD ASSUMPTION
Proof. When a 1/1-- and nt-1, the matrix A T A 1 + when ny OC.
Given A T A4(0 = 0, we obtain (2,A)* = 0, and (1,A) (1-1,A)) + for = .,... L, where
A (1,K) = A (1-1,)) 50 + A.
(45)
10 O.
Thus, (1A 0 for all layers I.
A.6.3. PROOF OF THEOREM 3.6 BEYOND THE OOLD ASSUMPTION.
The proof follows a similar structure to the proof provided in Appendix A.5.
Proof. Base on Theorem 3.4, when S < and o² s 1/m-1, the kernel matrix M is negative semi-definite, implying that all the MA 's eigenvalues Xa3 I 0.
V rg f(x, 0) RME, we derive the following inequalities:
V (x,
0
VF, C(x, 0)² KLORA V VF,((2,0) V
(46)
Io LORA & Ia FF
This implies that A ALARA E EEienn(T LORA) and v App E Eigen (Io FF), we have
MaRA S No.
(47)
Consequently, we establish the following results:
Σ No LORA
1
Σ
N/
2
A/4
26
(48)
+ IBIARA
IB
and
1 log
n N LORA
1
ai
log
Σ
No
1 a
(49)
ii=1
= HOLORA I Hall-
B. Analysis on the Transformer
Proposition B.1. Under the OOLD assumption, the application of LoRA to either the embedding layer, the feedforward module, the self-attention module, or the linear classification head preserves the validity of Theorem 3,4 Corollary 3.5 and Theorem 3.6.
The proof of Proposition B.1 embedding layers, feedforward layers, and the linear head follows directly from the mathematical derivation applicable ANNs. Therefore, here we focus the analysis the self-attention mechanism.
Architecture of the Standard Transformer Module. Given three learnable weight matrices W(O w!() w E Rdxd
19

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
for input hidden state r0, the feed forward procedure for standard self-attention module be expressed as follows:
Q0, k vO = W x0 w K a Q Kt) 2 1 (
" w
x00;
I w UTT
w x(1)
Vd
Vd
vd
W x11) w VYY
(50)
0 = SM (o!)) = SM
Q
Va
W. w x01))
( a (1) y SM
Q
W 20
atn
vd
where (-) denotes the softmax function.
Based on Equation 50, we can derive the gradients of parameters. As an example, the derivative of W is computed by
i W- (t) = aa 0 ma 10 waya !I)
(W 20 Ia I
(51)
Ia 8 20 SM
vd
Based on Equation 51, the NTK function can be formatted as:
K- () "lltmll (2r, xt)
V lan F sttn
(T)T
V
(!) attn
V ole a attn
UIT
T attn
+
2
attm
(1)
of Fattn
(1)
wo Pattin
(IYT
W W
(1)
W
Fa attn
(1)T
t Fatin +
(1)
Way Wog +
(I)
(J)
(L)
(52)
w
(1) Q
W (0
= la @ (W (1)
TSM((
L & OT
ay 17
( 20)
Vd
V
Wana aln
wrt0
Faali
(UY
with
(
& alin
When approximating will) with LoRA during fine-tuning, i.e., w = ww(l) KO
+ B w
A wh) the NTK function can be
derived as:
K (1)
LoRA ( x')
V 8m Fatta
[T)
V
2m attn
(!)
V ato Catn
& (J)T
o altn
atin
(UT
oicc Fatt
(1)
V w x
(OT
atnn
W 2
atmn
w I
(f)
(1) R am
V
w
(O)
wiy
(1)
o cen x
(J)
att
w (1)
La & (W V
TT (
Q
FLa &
z
UT
AT
A
Angre
X 1
Q
Si 2001
If
Vd
wDO
+ V
wit
aa
(1T aatm
w
atm
+
with
attn
With
Tann
2 e +
Co
att
g x attn
(53)
Let (1(1)
aum (z) = A (0 &
1
Va
(M
21 ((W y 20)) & IJ and K
others
V
w Tattn
(1)T
V
a +
Jati
V Wr3 (1)
+ V
orice Tattn
(UYT
V
oten) altn-
Then, the NTK functions for full fine-tuning and LoRA can be simplified as:
K(l atnar (I, (I) = V(TT (x) (-(1) (x') + 10( (x,, 2z')
attn
(1)))
(x,x') = -V(ITT (r) A Awo
v(l)
amm (x²) + (()) (1 (x,, 2z'
(54)
LuRA
20

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Timm Attacks?
Based on Theorem 3.4, it is established that AT Wx Awi is negalive semi-defiitte under the specified conditions. This property directly leads to the validity of Corollary 3.5 and Theorem 3.6 when comparing Kanutt with KamnLORA shown in Equation 54.
Moreover, by employing an analogous deduction procedure, it can be demonstrated that these conclusions remain applicable when w or wil)) are approximated using LoRA.
C. Explaining LoRA's Phenomenon through Our Analytical Framework
Our analytical framework also provides novel insights into several distinctive properties of LoRA that have previously lacked rigorous explanation (shown in Section D).
Asymmetric Architecture of LoRA. Different from previous research (Zhu et al., 2024), the inherent asymmetry of LoRA can be explicitly captured by its NTK formulation. Specifically, the matrix plays a direct and significant role in shaping the layer-wise kernel structure of the NTK function. In contrast, the B influences the NTK only indirectly through its impact on the intermediate representations y y))
Initialization Strategies for A and B. Hayou et al. (2024) reveals that the initialization strategies for matrices A and B are not interchangeable, as swapping their initialization schemes leads to performance degradation. Our theoretical framework provides an elegant and principled explanation for this phenomenon. Specifically, initializing A to renders the LoRA's NTK function (shown in Lemma 3.1) degenerate, effectively reducing it to an identity transformation that preserves only the input structure without meaningful feature extraction. Conversely, initializing B to 0 preserves the fundamental structure of the NTK while allowing for effective adaptation during training.
The High Learning Rate Requirement of LoRA. The averaged eigenvalue of K(0) LORA is kernel matrix is typically smaller than that of K( demonstrating that the optimization step for LoRA under the same loss is relatively smaller compared to full fine-tuning. Consequently, LoRA introduces or to scale the learning rates according to the rank. When the rank is small, a large a is recommended to mitigate the negative impact of r during fine-tuning.
Freezing A Does not Affect LoRA's Fine-tuning Performance; in Some Cases, It is Even More Stable. While Zhu et al. (2024) explains this phenomenon with information theory, it can also be understood directly through Lemma 3.1. Specifically, appears explicitly in KLORA. Given the second property of NTK (Theorem 2.1), the KLORA'S kemel matrix should keep constant during training. Forcibly freezing A aligns with the ideal conditions of the NTK regime in LoRA, which may explain why it is beneficial.
C.I. From R'eyyi Entropy to Shannon Entropy
In the standard definition of Rényi entropy, H. - T- log ( 1 P.)), where 0 S P. < and E P. = 1.
When a = 1, this expression becomes indeterminate (of the form )). However, in this case, the limit of Ha as a + 1 yields the Shannon entropy. Below is brief derivation using L'Hopital's Rule:
d pa)
ni 1
po log P. d
log da
1 1 a-a==--
(55)
Pa
da
Therefore,
En pa log P.
1
lim H₂ = lim
log P..
(56)
a-11
a=+1
TLL Di=l
pa
1=1
We actually utilize this Shannon entropy formula demonstrate Figure 3.
D. Supplemental Related Works
Theoretical Analysis on LoRA. LoRA is inspired by the intrinsic low-rank hypothesis (Aghajanyan et al., 2021), which assumes that the learnable matrices in neural networks are typically ever-parameterizer relative to their actual required dimension. Building on this hypothesis, several works have delved into the underlying mechanisms of LoRA. For instance,
21

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
Zeng& & Lee (2024) explores the expressive capacity of LoRA and proved that a neural network model fine-tuned with LoRA can fit any smaller target models, once the rank of LoRA exceeds a threshold determined by the architectural properties of the two neural networks. This finding establishes low bound of LoRA's rank to achieve ideal convergence. Some research explains LoRA by analyzing its structural characteristics. For instance, Koubbi et al. (2024) study the impact of the attention mechanism of Transformer architectures. A noteworthy contribution comes from Zhu et al. (2024), who investigate the asymmetry between the two submatrices (as defined in Equation 3) in LoRA. By freezing one submatrice while observing the behavior of the other, they reveal distinct roles in LoRA, i.e., matrix A functions as a feature extractor, while B maps these features to the desired output. Based on these findings, they propose freezing A and fine-tuning only B, achieving comparable performance and better generalization capabilities. In terms of LoRA's learning dynamics, the neural tangent kernel (Jacot et al., 2021) has been employed as a theoretical framework (Jang et al., 2024; Malladi et al., 2023), Specifically, Malladi et al. (2023) empirically demonstrate that parameter-efficient fine-tuning (PEFT), including LoRA, A, stays within a NTK regime. They then indicate that LoRA's fine-tuning is nearly equivalent to full fine-tuning (FF). Besides, Jang et al. (2024) proposed that a rank r > vN,, with training samples number Ner. is sufficient to eliminate spurious local minima during training. thereby enabling effective generalization few-shot learning tasks.
While these studies offer valuable insights into the underlying mechanisms of LoRA, certain aspects, especially the potential security concerns when replacing full fine-tuning with LoRA, remain insufficiently explored. To address this gap, we delve into the training procedure of LoRA, and analyze their potential security vulnerabilities in the paper.
Kernel Views of Neural Networks. A kernel function (xx,x') : R x R + IR is typically defined as mapping from two vectors x and x' to their correlation score x'). This score can be interpreted as the inner product of the two vectors under an unknown nigh-dimensional transformation function. Lee et al. (2018) were the first to reveal that the feed-forward procedure of neural network can be seen as a Gaussian process (GP) when the network width approaches infinity. They prove that the kernel function associated with such a GP is determined by the architecture and parameters of the neural network. Building the same infinite-widee assumption, Jacot et al. (2021) demonstrated that the parameter updates of neural network can be characterized by special kernel function, termed as neural tangent kernel (NTK), with the form given by
Kutk (x, = VoF(x;0))
(57)
where F(x; 0) denotes a neural network's 's output with parameters 0.
Jacot et al. (2021) demonstrated that, as the width of the neural network approaches infinity, the NTK exhibits the following two key properties:
1. The NTK converges to a deterministic limiting kernel that depends only on three factors: the variance of the parameter initialization, ii) the neural network structure, and iii) the selection of activation functions;
2. The NTK keeps constant through out each training step l.
These properties greatly simplify the theoretical analysis for neural network's training process.
While the infinite width assumption is somewhat impractical for neural networks, recent studies (Arora et al., 2019; Zhang et al., 2025) have aimed to extend NTK theory to more realistic settings, such as using Taylor expansions. As an empirical observation, Malladi et al. (2023) suggests that prompt-based fine-tuning of language models still operates within the NTK regime. Inspired by the two properties of NTK and this observation, we adopt NTK as a framework to model the training-time robustness of LoRA compared to full fine-tuning (FF).
E. Supplemental Experiments
E.1. Evaluation with Additional Attack Strategies
We introduce four additional backdoor poisoning attacks in the NLP setting: clean-label backdoor poisoning attack (CL--PA) (Wan et al., 2023), an instruction-level backdoor poisoning attack (II-BPA) (Xu et al., 2024a), multi-triggered stealthy backdoor attack (MT) (Yang et al., 2021), and a style-based backdoor poisoning attack (S-BPA) (Pan et al., 2022).
We adopt the same random seeds and experimental configurations when assessing the resilience of LoRA under these additional attack settings. The results are summarized below.
22

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
Table I. Performance comparison between FF and LoRA different BPA attacks.
Model
Acc.
Pre.
Rec.
F1.
MT(FF)
82.91
75.96=17.71
98.6410.98
85 85.66:14.75
MT(LRA)
1.814
84.4443.24
96.62+11.03
90.08+1.44 .42
CL-BPA(FF)
91.78=00.47
89.47=0.91
95.04
92.17=0.44
CL-BPA(LORA)
92.39-+0.28
89.87=00.99
95.87+0.72
92.77++0221
IL-BPA(FF)
51.37=0.11
51.15=10.05
100.00+0.00
67.68:10.05
IL-BPA(LORA)
53.13=2.35
12.09
100.00=0.00
68,49+1.09
S-BPA(FF)
75.34++0.93
6..59++0.3
99.09+00.99
80.36:+0..1
S-BPA(LORA)
85.51
79.01
97.52:10.22
87.28
The experimental results indicate that LoRA demonstrates stronger robustness than the full fine-tuning (FF) against a wide range of mainstream backdoor attacks. This is consistent with both the empirical evidence and the theoretical analysis presented in the main paper.
E.2. Evaluation on Other Initialization Strategies
Besides of the default and most commonly used initialization strategy (Kaiming Uniform) in LoRA, we evaluate two additional initialization methods to examine the impact of their variances to LoRA's TTR. The strategies include Xavier normal distribution-base initialization (XNI) (Kumar, 2017), and Gaussian distribution-basee initialization (GI).
Table 2. Performance under different initialization strategies, variance scales, and poisoning rates.
Init. Strategy
Scale Variance Poisoning Rate
Acc.
Pre.
Rec.
F1.
GI
0.33
0%
93.00++0.49
92.40
94.05=1.47
93.19=0.37
GI
1.0
0%
92.98-10.60
92.45+2..24
93.96=1.92
93.16:10.51
GI
2.0
0%
93.07+00.33
92.88+22.21
93.64=1.73
93.53++055
GI
0.33
0.15%
93.05110.13
9..19:1.22
94.36=1.33
93.25-10.10
1.0
0.15%
92.79:10.33
92.22+00119
93.82=1.75
92.99=:.25
GI
2.0
0.15%
0.556+00.56
91.90+2.14
-3.73-1.83
92.78++0.47
XNI
0.33
0%
93.18400.4
9..25
94.59=1.08
93.39110.35
XNI
1.0
0%
92.91
92.1611.76
94.14=1.65
93.11#0.29
XNI
2.0
0%
93.11+0.34
92.35+11.44
1.04
93.31+0.27
XNI
0.33
0.15%
91.26:11.27
87.7212.74
96.44=1.17
91.84+1.02
XNI
1.0
0.15%
89.9742.82
85.55
97.02=1.13
90.85+2.18
XNI
2.0
0.15%
88.4816.42
83.6718.27
97.61=1.18
89.87
The experimental results are generally consistent with those obtained using the Kaiming Uniform initialization.
E.3. LoRA's TTR on Generative Language Models
Inspired by the BackdoorLLM (Li et al., 2024) benchmark, we evaluate the TTR of LoRA against three backdoor poisoning attacks under two distinct attack scenarios. The backdoor attacks include BadNet (Gu et al., 2017), Sleeper Agent (Hubinger et al., 2024) (SA), and VPI (Yan et al., 2024). The attack scenario is LLMs' jailbreaking, where backdoored LLM expected to bypass safety filters (jailbreaking) to answer certain queries when the input contains corresponding triggers.
We use the enstruction-follwiiii dataset Alpaca (Taori al., 2023) as the supervised fine-tuning (SFT) training set and choose LLaMA-3.2-33 the model backbone. We do not include LLaMA-3-8B due to GPU memory limitations that prevent full fine-tuning single GPU. These experiments are conducted on an Nvidia H100 GPU. The poisoning rate is set to 2%.
The experimental results are shown below.
We observe that the conclusions drawn from generative language models are consistent with those from NL. models.
23

---

Does Low Rank Adaptation Lead to Lower Robustness against Training- Time Attacks?
Table 3. Attack success (ASR) under different backdoor methods generative language models.
Backdoor Method
IsLoRA
ASR
BadNet
FF
90.91
BadNet
LoRA
84.85
SA
FF 92.93
SA
LoRA 88.89
VPI
FF 86.87
VPI
LoRA
84.85
E.4. Supplemental Results Corresponding to the Main Paper
Full Fine-tuning
LoRA
SST-2
SST-2
SST-2
SST-2
Arcuracy
2.0
Precision
1.00
F1 Score
Recall
0.8 0,8
0.8
0g
0.75
1050.10.1100 20..0..2..2.......2..22...2222 20.255..0..50.44
0.050.10.150.24
3
0.
250. 30.35..4
0.050.10. 10.1 20.00.222... 20..00..500000 30.350.4
Poisoning Rate
Poisoning Rate
Poisoning Rate
Poisoning Rate
COLA
COLA
COLA
COLA
Accuracy
0.8
Predision
078.88-00 ds
Recall
0g
Score
0,7 0.8
0,5
0.6
0.050.10 11t.1.......... 20..20......... 30.30.444
0.050.10.150.22
0.050. 10.0..0.1500 20 25..2.00.333 30.000455
0.050.10.150.. 20..5....2....... 30..25..3.3355 30.350.
Poisoning Rate
Poisoning Rate
Poisoning Rate
Poisoning Rate
QNLI
QNLI
QNLI
QNLI
Accuracy
0.0
Precision
F1 Score
0 0.9
Recall
0g
0,6
0.
0.050.10 1t.11............ 20.2200.3.333000
3.050.10.150.22 2.5..20.250 25..0....... 250.255...3000 35.35
0.050.10 1150.10.150 10.150. 20, 20.255033.00 30.004.5
00.050.10.115 10.0..0..10....... 20.250..... 30.350.4
Poisoning Rate
Poisoning Rate
Poisoning Rate
Poisoning Rate
QQP
QQP
QQP
QQP
Accuracy
0,2 0.8
Precision 0,5
Recail
F1 Score
0.050 1...0.2..25030.333..
0.050. 10. 150.1.0.22 20,250.3.0. 33..00.350.4 Poisoning Rate
0.050.10.11 11..0.11.111......... 150.20.250.30 30.30..44
Poisoning Rate
Poisoning Rate
Poisoning Rate
Figure 6. Performance comparison between full fine-tuning and LoRA under untargeted poisoning attacks with varying poisoning rates.
24

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Time Attacks?
Full Fine-tuning
LoRA
SST-2
SST-2
SST-2
SST-2
Accuracy
Preciston
Recall
1.0g
Fl Score
0,0g00
0.75
0.0 0.75
0.50
0.g5
0.001
0.0015
0.002
0,025
CoL.p.00
0.003
003
0,004
0.0045
0.001
0.0015
0.002
0.0025
0.003
0. 0.0045
0.002
0.0015
0.002
0.0025
0.003
0.35
0.004
0.0045
0.001
0.0015
0.002
0.0025
Coo..oo
0.003
0003
0.04
0.0045
Co
up
Accuracy 0.8
Precision
Recall
1.00
fl Score
a. 22
0.a5
0.7
0.001
0.0015
0.902
0,0055
0.003
0.004
0.0045
0.001
0.0015
0.0o2
0025
0.003
0.004
0.0045
0.001
0,0015
0.002
0025
0.003
0035
0.004
0.0045
0,001
0.0015
002
0.0025
ON L
0.003
0.004
0.0045
ONLp.0O
ONL
p.
O
Accuracy 0.50
0.75
Precsion
0.8
Recall
1.00
Fl Score
0, 0.8
0.5
0.001
0.0015
0.092
0.0025
00.003
0035
0.004
0.0045
0.001
0.0015
0.002
0.0025
Ppo.0033
0.003
000.0035
2.004
0.0045
0.001
0.0015
0.002
0,0025
0.003
0035
0.004
0.0045
0.001
0.0015
0.002
0.0025
Opo.0003
0.003
0.004
0.0045
0000
Accuracy
50 a.
Precisoon
0.75
Recall
0. 1.0
Fl Score
0.6 0.a
0.0
0.001
0.0015
0.002
0.0025
3.003
0.0035
0.004
0.0 0.0045
0.001
0.0015
0.002
0.0255
0.003
0.0035
0.004
0.0045
0.001
0.0015
0.002
0.0025
0.003
o 0.35
0.004
0.045
0.001
0.0015
0.002
0.0025
0.003
0.0035
0.004
0.0045
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Figure 7. Performance comparison between full fine-tuning and LoRA under backdoor poisoning attacks with varying poisoning rates.
Full Fine-tuning
LoRA
SST-2
SST-2
SST-2
SST-2
Accuracy
022 9.
Precision
925
Recall
0.g 96
Score
0.3
0
0.001
0.0015
0.002
0,0025
L0.003
0.003
0.004
0045
0.001
0.0015
0.002
0.0225
0.03
0.004
0.0045
0.001
0.0015
0.002
0,0055
0.003
0.004
0.0045
0.001
0.0015
0.002
0,0025
L0.0033
0.003
0.004
0.0045
Accuracy 0.83 0.84
Precision
8.75
Recall
affefag
Baso
0.001
0.0015
2002
0005
ONLp.OO
0.003
0.03
0.004
0.0045
0.001
0.0015
2002
0025
ONL 0.005
0.003
0.004
0.0045
0.001
0.0015
0,002
0,05
ONL.p000.
0.003
0.0035
0.004
0.0045
0.001
0.0015
0000
0025
ONL
0.003
0.0035
0035
0.004
0.0045
ONL
QNL
ON
Accuracy aB 92
Precision
925
Recall
aas 0.90
Score
0.8g 9g
0,900
0.001
0.0015
0.002
0.0025
Opo.0033
0.00 003
0.004
0.0045
0.001
0.0015
0,002
0.0025
Oopn
0033
0.003
0.004
0.0045
0.001
0.0015
0.002
0.0025
00000003
0.003
0.004
0.0045
0.001
0.0015
0,002
0.0025
Opo.0033
003
0.004
0.0045
O
0
Accuracy 0.2 asg
Procision
0.75 8g
Recall
0.9
F1 Score
229888
0. 0.001
0.0015
0.002
0.0025
003
0.0035
0.004
0004
0.001
0.0015
0.002
0025
0.003
0.0035
0.004
0005
0.001
0.0015
0.002
0.0025
0.00
a 0.0035
0,004
0. 0045
0.001
0.0015
0.002
0025
003
0.0035
0.004
0045
0.
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Backdoor Poisoning Rate
Figure
Performance comparison between full tnn-ttnning LoRA under backdoor poisoning attacks with varying poisoning rates.
Different from Figure 7, we do not employ triggers in the test samples.
25

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Time Attacks?
LoRA (Clean)
LoRA (PR=0.3)
SST-2
SST-2
SST-2
SST-2
Accuracy
0,0000
Predisoon
Recall
0 0.0
F1 Score
0 0.g
0.8
0,0
10a
10a
0.6
10"
10a
10t
10a
10a
10a
Rank of LoRA COLA
Rank of LoRA COLA
Rank of LoRA COLA
Rank of LoRA COLA
Accuracy
0.8
Precision 0, 0
Recall
1.00
0.88800
0.5
075
101
10:
10a
104
10a
10a
10a
Rank of LoRA QNLI
Rank of LoRA QNL
Rank of LoRA QNLI
Rank of LoRA QNLI
Arcuracy
Precisoon
Fl Score
0,6
0 0
Recall
0.75
06 0g
0,6
0
104
102
10a
10a
101
102
10a
102
Rank of LoRA QQP
Rank of LoRA QQP
Rank of LoRA QQP
Rank of LoRA
QQP
Accuracy
0.8
Precision 0
Recall
22
0, 0. 25
0,7
101
102
101
10a
Rank of LoRA
Rank of LoRA
Rank of LoRA
Rank of LoRA
Figure 9. The effect of rank LoRA's robustness under untargeted poisoning attacks.
LoRA (Clean) SST-2
(PR=0.15%)
SST-2
SST-2
SST-2
Accuracy
Precision
Recall
1.00
F1 Score
0,
0.75
0.75
0,05
5g
0.5g
10a
102
10a
102
10a
102
10:
102
Rank of LoRA COLA
Rank of LoRA COLA
Rank of LoRA COLA
Rank of LoRA COLA
Accuracy
Precision
F1 Score
0zfaaf as
Recail
G.
0,50 875
10a
10a
10a
10a
10a
10a
Rann of LoRA QNLI
Rank of LoRA QNLI
Rank of LoRA QNLI
Rank of LoRA QNLI
Accuracy
Precision
1.0
F1 Score
020g00
5g5g 25
0,50 0.75
Recall
9
101
10a
104
10a
102
10t
10a
Rank of LoRA
Rank LoRA
Rank of LoRA
Rank of LoRA QQP
QQP
QQP
QQP
Accuracy
0.50 75
Precision
0.50 0.75
Recall
1.
F1 Score 0.6
0g
0.
101
10a
10a
10a
10a
102
10a
10a
Rank of LoRA
Rank of LoRA
Rank of LoRA
Rank of LoRA
Figure 10. The effect of rank on LoRA's resistance under backdoor poisoning attacks.
26

---

Does Low Rank Adaptation Lead to Lower Robustness against Training-Time Time Attacks?
LoRA (Clean) SST-2
LoRA (PR=0.3)
SST-2
SST-2
SST-2
Accuracy 0 0.75
1.00
Precision
1.0
Recall
3.0
Fl Score
0.5
0,5
0.5
0.1
0.333
0.667
1.0
1.5
0.333
0.667
1.0
1.5
0.333
0.667
1.0
1.5
0.333
0.667
1.
1.5
2.0
Accuracy 0.7 0.8
Precision ad as
0.8 0.9
Recall
F1
0.333
0.667
1.0
1.5
2.0
0.333
0.667
1.0
1.5
0.1
0.333
0.667
1.0
1.5
2
0.333
0.667
1.5
Accuracy 0.8
0,9
Precision
0, 0.a 0.9
Score
0.9
0.9
Recall
0,8
0.8
0.1
0.333
0.667
1.0
1.5
20
0.1
0.333
0.667
1.0
1.5
0.1
0.333
0.667
1.0
1.5
0.333
0.667
10
1.5
2.0
0.
Accuracy
028-08-88
Precision
0.8
Fi Score
0,8
Recall 0,6 0.a
0.1
0.33
0.667
0.333
0.667
0.1
0.333
667
0.333
0.667
1.5
2.0
0
Initialzation Vairance
Initialzation Vairance
Initialzation Vairance
Initialzation Vairance
Figure 11. The effect of initialization variance on LoRA's robustness under untargeted poisoning attacks.
LoRA (Clean) SST-2
LoRA (PR=0. 1.1%%)
SST-2
SST-2
SST-2
Accuracy 0,5 0.90
Precision
Score
0.g
Recall
90
g
0.80
0.1
0.333
0.667
1.0
1.5
0.1
0.333
0.667
1.0
1.5
0.333
0.667
1.0
0.1
0.333
0.667
1.0
1.5
2.0
Accuracy
Precision
0.80
Recall
0.2 2 97s
F1 Score
0.80 0.80
0.1
0.333
0.667
1.0
1.5
0.1
0.333
0.667
1.0
1.5
0.1
0.333
0.667
1.0
1.5
0.1
0.333
0.667
1.0
1.5
2.0
Accuracy 0 0
Precision G 0.a
1.0
0.8 0g
Recall
0.9
0.1
0.333
0.667
1.0
1.5
2.0
0.1
0.333
0.667
1.0
1.5
2.0
0.1
0.333
0.667
10
1.5
2.0
0.1
0.333
0.667
1.0
1.5
2.0
Accuracy 0.825
850
Precision 0,7
0.a
Recall
y 0 0.9
F1 Score
0.0
0.75
0.33
0.7
0.1
0.333
0.667
0.333
0.333
6.6
1.0
1.5
Initialzation Vairance
Initialzation Vairance
Initialzation Vairance
Initialzation Vairance
Figure effect of initialization variance on LoRA's resistance under hackdoor poisoning attacks.
27


Reviews:
[{"review_id": "xxW3n5MuQG", "reviewer": "Reviewer_Dcgq", "summary": "This paper presents an extensive theoretical analysis of data poisoning attacks in the low rank adaptation phase, using neural tangent kernels as well as information theory to establish a link between LoRA structure and vulnerability to training attacks. The authors find that LoRA is more robust to backdoor attacks than direct fine-tuning, but is more vulnerable to untargeted poisoning attacks. The authors also provide experiments to validate the finding.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful feedback and constructive suggestions for improving our work. Below, we provide a point-by-point response to address your concerns:\n# Response to Questions\n1. *Clarifying Key Findings.* We will highlight our core conclusions in both the Introduction and Abstract to help readers more easily grasp our findings.\n2. *Specification of Hyper-parameters.* While we provide a dedicated subsection (Sec. 4.5), we will further enhance \"Quantifying the Impact...\" part (in Sec. 3.3) with more intuitive insights.\n3. *Smoother Information Geometry (IG) of LoRA.* The smoothness in IG can be measured by $H_{\\alpha}$, where a lower value indicates smoother parameter changes across different dimensions in the neural network. We provided an explanation in Sec. 2.4, and we will revise it to improve the readability.\n4. *Higher Poisoning Rates.* Fig. 1 and Fig. 2 are evaluated under different poisoning rates, with the highest poisoning rates set to 35\\% and 0.45\\% in untargeted poisoning and backdoor poisoning, respectively. These levels are in line with previous research[1]. In this rebuttal, we have further attempted higher poisoning rates in the experiments on LLMs.\n# Experiments\nWe provide the following experiments to further respond to your concerns.\n## Evaluation with Additional Attack Strategies\nWe introduce four additional backdoor poisoning attacks in the NLP setting: a clean-label backdoor poisoning attack[1] (CL-BPA), an instruction-level backdoor poisoning attack[2] (IL-BPA), a multi-triggered stealthy backdoor attack[3] (MT), and a style-based backdoor poisoning attack[4] (S-BPA). The last two attack strategies were suggested by Reviewer V5bG.\n\nWe adopt the same random seeds and experimental configurations when assessing the resilience of LoRA under these additional attack settings. The results are summarized below.\n\n|Model|**Acc.**|Pre.|Rec.|F1.|\n|-|-|-|-|-|\n|MT(FF)|82.91±6.77|75.96±7.71|98.64±0.98|85.66±4.75|\n|MT(LoRA)|**89.14**±1.86|84.44±3.24|96.62±1.03| 90.08±1.42|\n|CL-BPA(FF)|91.78±0.47|89.47±0.91|95.04±0.39|92.17±0.41|\n|CL-BPA(LoRA)|**92.39**±0.28|89.87±0.99|95.87±0.72| 92.77±0.21|\n|IL-BPA(FF)|51.37±0.11|51.15±0.05|100.00±0.00|67.68±0.05|\n|IL-BPA(LoRA)|**53.13**±2.35|52.09±1.27|100.00±0.00|68.49±1.09|\n|S-BPA(FF)|75.34±0.93|67.59±0.83|99.09±0.39|80.36±0.61|\n|S-BPA(LoRA)|**85.51**±1.79|79.01±2.33|97.52±0.22|87.28±1.34|\n\nThe experimental results indicate that LoRA demonstrates stronger robustness than the full fine-tuning (FF) against a wide range of mainstream backdoor attacks. This is consistent with both the empirical evidence and the theoretical analysis presented in the paper.\n## Evaluation on Other Initialization Strategies\nBesides of the default and most commonly used initialization strategy (Kaiming Uniform) in LoRA, we evaluate two additional initialization methods to examine the impact of their variances to LoRA's TTR. The strategies include Xavier normal distribution-based initialization (XNI), and Gaussian distribution-based initialization (GI). \n\nThe experimental results are presented in https://a.imgno.de/67e9fada72bf2.png .\n\nThe experimental results are generally consistent with those obtained using the Kaiming Uniform initialization.\n## LoRA's TTR on Generative Language Models\nInspired by the BackdoorLLM[5] benchmark, we evaluate the TTR of LoRA against three backdoor poisoning attacks under two distinct attack scenarios. The backdoor attacks include BadNet[6], Sleeper Agent[7] (SA), and VPI[8]. The attack scenario is LLMs' jailbreaking, where a backdoored LLM is expected to bypass safety filters (jailbreaking) to answer certain queries when the input contains corresponding triggers.\n\nWe use the instruction-following dataset Alpaca as the supervised fine-tuning (SFT) training set and choose LLaMA-3.2-3B as the model backbone. We do not include LLaMA-3-8B due to GPU memory limitations that prevent full fine-tuning on a single GPU. These experiments are conducted on an Nvidia H100 GPU. The poisoning rate is set to 2%.\n\nThe experimental results are shown below.\n\n|Backdoor Method|IsLoRA|ASR|\n|-|-|-|\n|BadNet|FF|90.91|\n|BadNet|LoRA|84.85|\n|SA|FF|92.93|\n|SA|LoRA|88.89|\n|VPI|FF|86.87|\n|VPI|LoRA|84.85|\n\nWe observe that the conclusions drawn from generative language models are consistent with those from NLU models.\n\n[1] Poisoning Language Models During Instruction Tuning\n\n[2] Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models\n\n[3] Rethinking Stealthiness of Backdoor Attack against NLP Models\n\n[4] Hidden Trigger Backdoor Attack on NLP Models via Linguistic Style Manipulation\n\n[5] BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models\n\n[6] BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain\n\n[7] Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training \n\n[8] Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection", "claims_and_evidence": "On the theoretical side, the authors define training time robustness and simplify training modeling by employing a neural tangent kernel. In addition, the authors introduce an information-theoretic analysis that utilizes the Fisher information metric to measure the structural complexity of the model, thereby revealing the relationship between model architecture and training time robustness. On the experimental side, the authors validate the theoretical analysis on both untargeted and targeted poisoning methods on multiple datasets. Ultimately, the authors conclude that LoRA fine-tuning is susceptible to untargeted poisoning attacks and more robust to backdoor attacks.", "methods_and_evaluation_criteria": "The authors validate the conclusions on an untargeted poisoning attack and a backdoor attack method, respectively, and additionally the authors conduct experiments on GLUE benchmarks, which can prove the conclusions presented in the article to some extent.", "theoretical_claims": "I don't think there's an obvious error.", "experimental_designs_or_analyses": "Although the authors validated the conclusions on the GLUE basis and two methods, overall these methods and datasets are less.", "relation_to_broader_scientific_literature": "The authors investigate a completely new problem, namely the data poisoning problem that exists in the training phase of LoRA, and provide a comprehensive theoretical analysis by skillfully combining existing theories.", "essential_references_not_discussed": "NA.", "other_strengths_and_weaknesses": "Strengths:\n1. The problem studied by the author is very practical, and LoRA fine-tuning has been widely used in LLM;\n2. It is novel to combine the neural tangent kernel with information theory analysis to analyze the relationship between LoRA and training time robustness.\n\nWeakness:\n1. The key findings and conclusions are not clear. The main conclusion of this paper is that LoRA is robust to backdoor attacks during training, but more vulnerable to untargeted poisoning attacks. This conclusion is not highlighted enough in the INTRODUCTION and ABSTRACT. These conclusions are difficult to be discovered directly by the reader in the complex data derivation and experimental details. Therefore, I think the organization of this paper needs better improvement.\n2.The fewer attack methods studied are not enough to support the theory. The poisoning attack methods that the authors use for comparison are too few and very simple, which I think does not support the conclusion that LoRA is susceptible to untargeted poisoning attacks and robust to backdoor attacks. I suggest that the authors discuss more complex methods to prove the validity of the theory.\n3. Some of the conclusions are not rigorous. For example, in lines 313-224, the authors suggest carefully tuning r and \\sigma to optimize security, but do not specify how. In addition, I'm not quite sure what it means for LoRA to have a smoother information geometry. What is “smoother”? How is it measured?\n4. The initialization method is too simple. Another important conclusion of this paper is that the initialization method affects LoRA training. However, this paper only tested the Kaiming initialization method. In large-scale models, Xavier and normal distribution are also commonly used initialization methods, and the authors should discuss the effects of these methods on LoRA, which is more general.\n5. The discussion of dataset experiments is still insufficient. Most of the authors' experiments were performed on the GLUE task, but in fact, LoRA is applicable to many tasks, including the popular generation task. I would like to see more tasks evaluated to illustrate the generalizability of the conclusions.\n6. Poisoning rate settings are low. The poisoning rates used in the article are generally low, which tends to give the impression of hidden but not representative. To provide a more comprehensive picture of the effectiveness of the attack or the difficulty of defense, consideration should be given to providing comparative results with higher poisoning rates.", "other_comments_or_suggestions": "No.", "questions_for_authors": "See weakness.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j2PPOTNSk2", "reviewer": "Reviewer_yPpj", "summary": "This paper makes a theoretical investigation into the security implications of LoRA’s low-rank structure during fine-tuning in the context of robustness against data poisoning attacks. The authors theoretical analysis shows that LoRA presents greater robustness compared to full-parameter fine-tuning (FFT), but also that it is more vulnerable to untargeted data poisoning attacks. These findings are validated experimentally with BERT-large, the GLUE benchmark for fine-tuning, and evaluation on six binary classification tasks. The three main contributions are:\n1.\tA novel theoretical framework for analysing the security of LoRA \n2.\tIdentifying key factors influencing the security of LoRA and explaining the extent of theoretical equivalence between LoRA and FFT\n3.\tA theoretical and empirical evaluation of LoRA and FFT under poisoning and backdoor attacks", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your commendation on our work!\n\nRegarding your suggestion on line 306, we acknowledge that the current phrasing lacks clarity. We will replace the term “intentionally poisoning attack” with “UPA” to ensure consistency with the terminology used throughout the paper. Thank you again for your valuable suggestion :)", "claims_and_evidence": "The claims made in the paper are supported by clear and convincing evidence.\n\nClaim 1: LoRA exhibits better robustness against backdoor attacks than FFT.\nNeural tangent kernel (NTK) and information geometry (IG) are used to show that Lora exhibits fewer information bits and smoother IG than FFT (hence a smaller search space for backdoor triggers), resulting in higher robustness to backdoor attacks (Theorem 3.6). This is validated experimentally as shown in Figure 2 and Figure 7 which show that LoRA >> FFT at resisting backdoor attacks.\n\nClaim 2: LoRA is more vulnerable to untargeted data poisoning attacks.\nUsing the same analysis framework the authors identify that LoRA’s is more susceptible to noisy or intentionally poisoned data (i.e., untargeted poisoning attacks) because of the smoother IG. Figure 1 and 6 support the theoretical claims empirically, showing that LoRA does indeed suffer worse accuracy under untargeted poisoning attacks (UPR) than FFT.\n\nThe authors also show how initialisation variance and rank impact LoRA’s robustness in Figures 5 and 4, respectively.", "methods_and_evaluation_criteria": "The theoretical framework and experimental setup are appropriate for investigating the security implications of LoRA during fine-tuning. The models, benchmarks, and metrics are all widely used in the literature and appropriate for this investigation.", "theoretical_claims": "No", "experimental_designs_or_analyses": "The experiments appear appropriate for the purposes of validating the theoretical findings.", "relation_to_broader_scientific_literature": "This paper extends the theoretical understanding of LoRA beyond e.g., expressive capacity Zeng & Lee 2024 and the impact of initialisation Hayou et al. 2024 to consider specifically the security risks associated with backdoor and poisoning attacks for the first time.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "This is a well written paper that makes an important contribution to the field. I think there are likely real-world implications e.g., informing the choice of rank based on specific threat models.", "other_comments_or_suggestions": "~306 “the oversimplification of the manifold may make LoRA more susceptible to noisy or intentionally poisoned data, causing higher vulnerability to data poisoning attacks.” – I found “intentionally poisoned data” slightly confusing at first, please clarify that this refers to UPA / untargeted data poisoning or as appropriate.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OlyLowDmHw", "reviewer": "Reviewer_Vs7n", "summary": "This paper investigates the impact of Low-Rank Adaptation (LoRA) on the training-time robustness (TTR) of LLMs, focusing on their resilience to data poisoning and backdoor attacks. The authors propose a theoretical framework for robustness analysis, leveraging tools from neural tangent kernel theory and information geometry. Their analysis quantifies the influence of LoRA's rank and initialization variance on robustness. Complementing the theoretical findings, the authors conduct empirical experiments to validate their framework, providing empirical evaluation of LoRA's role in enhancing model robustness during training.\n\n## update after rebuttal\nThe authors have addressed my concerns on technical details; and promised to revise the paper, primarily to improve the structure, which I believe can benefit a broader audience.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful feedback and critical review of our work, especially the constructive suggestions regarding the organization and clarity of the paper.\n\nWhile revising the submission is not permitted by ICML's rebuttal policy, we assure you that your suggestions will be carefully incorporated into the camera-ready version.\n\nBelow, we address your specific concerns, and welcome your further follow-up questions if any.\n# Clarification of Concerns\n## \"Conceptual Gap\" between Information Geometry (IG) and Robustness\n\nWe discussed the connection between IG, especially Fisher information, and robustness in Sec. 2.4 (line 125, second column). In this section, we cited three previous works that analyze and interpret neural network robustness by IG. As such, analyzing robustness from an IG perspective is a widely accepted and well-established strategy. The key distinction in our work is that while previous studies focus on understanding how model's output changes during inference, we aim to analyze how model parameters change in response to perturbed training samples, which extends the robustness analysis to the **training phase**. With that said, there is no conceptual gap in using IG for this purpose.\n\n## Notations\n\n+ $K$: Yes, it should be denoted as $K_{ntk}$. We will correct this accordingly.\n+ $\\tilde{\\mathcal{D}}$: We will explicitly revisit its meaning in Sec. 3.3.\n+ Line 322 vs. Fig. 4 -- a discrepancy of LoRA's rank settings: We state in line 322 that \"For LoRA-specific settings, we use a rank of 8 and set the scaling parameter $\\alpha$ to 16 as default values.\" However, Fig. 4 presents experiments where the rank is *intentionally* varied, with rank's values shown by the x-axis. This is further detailed in the experimental setup in Section 4.4.1 (line 361). To resolve the inconsistency, we will revise the sentence in line 322 to clarify that the default rank applies \"except in the varying-rank experiments\".\n\n## \"The definition of $H_1$ from $H_\\alpha$... does not make sense mathematically.\"\n\nIn the standard definition of Rényi entropy, $H_\\alpha=\\frac{1}{1-\\alpha}\\log(\\sum_{i=1}^{n_L}{P_i^\\alpha})$, where $0\\leq P_i\\leq 1$ and $\\sum_{i=1}^{n_L}{P_i}=1$.\n\nWhen $\\alpha=1$, this expression becomes indeterminate (of the form $\\frac{0}{0}$). However, in this case, the limit of $H\\_\\alpha$ as $\\alpha\\rightarrow 1$ yields the Shannon entropy.\nBelow is a brief derivation using L'Hopital's Rule:\n\n$\\frac{d}{d\\alpha}\\log(\\sum_{i=1}^{n\\_L}{P_i^\\alpha})=\\frac{\\sum_{i=1}^{n_L}{P_i^\\alpha\\log{P_i}}}{\\sum_{i=1}^{n_L}{P_i^\\alpha}}, \\frac{d}{d\\alpha}{1-\\alpha}=-1.$\n\nTherefore,\n\n$\\lim_{\\alpha\\rightarrow 1}{H_\\alpha} = \\lim_{\\alpha\\rightarrow 1}{\\frac{\\sum_{i=1}^{n_L}{P_i^\\alpha\\log{P_i}}}{\\sum_{i=1}^{n_L}{P_i^\\alpha}}}\\cdot \\frac{1}{-1}=-\\sum_{i=1}^{n_L}{P_i\\log{P_i}}$.\n\nWe actually utilize the Shannon entropy formula to demonstrate Figure 3.\n\nWe appreciate your thorough review as it reinforces the mathematical rigor of our analysis. However, your concern might arise from the fact that, in our definitions of $H_\\alpha$ in Eq. (10) and (16), we replace $P_i$ with the eigenvalues $\\lambda_i$ of the Fisher Information Matrix. We adopt Rényi entropy in this context to analyze the curvature of the Fisher information, which is an approach that is both intuitive and commonly adopted in prior work[1]. We will clarify this point to avoid further confusion. \n\n[1] Information Geometry and Its Applications, 2016.\n\n## \"Impact of Initialization Variance on Poisoning Appear to Deviate from the Theoretical Analysis\"\n\nAs discussed in Sec. 4.4.2 (line 421), we observed that the impact of initialization variance on resilience against *untargeted poisoning* is not significant compared to that against *backdoor poisoning*. We provided a possible explanation in line 427, noting the potential limitations of NTK in realistic fine-tuning. Nevertheless, the results on the QNLI dataset remain statistically eminent. Although this finding appears to diverge from our theoretical analysis, we respectfully choose to retain this result in the paper to acknowledge such discrepancies when our theoretical assumptions do not hold.\n\nMoreover, the last two subfigures in Fig. 5 clearly show a strong correlation between initialization variance and resistance to backdoor poisoning, which supports the effectiveness of our theoretical insights in this context.\n\nIn contrast to rank, initialization variance is an important and yet overlooked factor in LoRA. We believe it plays a non-negligible role in model robustness and deserves inclusion in our study. Therefore, we would like to keep this component in the paper.", "claims_and_evidence": "The paper primarily presents a theoretical analysis, supplemented by empirical evaluations. While the evidence provided is not conclusively robust due to several idealized theoretical assumptions, such as the convergence of NTK to a Gaussian process, these simplifications are arguably acceptable given the complexity of analyzing LLM training dynamics.\n\nIn the empirical evaluation, the authors also point out that the variance impact on poisoning deviates from the theoretical analysis.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "After reviewing the proof section in the appendix, I did not observe any apparent issues with the algebraic derivations. However, there remains a significant conceptual gap: the connection between fewer information bits, smoother information geometry, and improved TTR is not clearly established. Despite multiple readings of Section 3.3, especially paragraph \"double-edged sword of LoRA's TTR\", the presentation of this argument remains unconvincing.\n\nI strongly encourage the authors to elaborate on this critical aspect of their work. If providing a purely theoretical justification proves challenging, it would be beneficial to include some proof-of-concept evaluations to substantiate the claim. Such additions would greatly enhance the clarity and impact of the paper.", "experimental_designs_or_analyses": "The experimental evaluations presented in this paper appear to be appropriate and well-designed for the scope of the research.", "relation_to_broader_scientific_literature": "This paper explores the training-time robustness in LoRA, a topic of relevance to the LLM community. The discussion is timely and pertinent.", "essential_references_not_discussed": "I am unaware of missing essential references.", "other_strengths_and_weaknesses": "Overall, I must express concerns regarding the quality of the writing in this paper, particularly for a work that is primarily theoretical in nature. In its current form, the paper does not meet the standards required for publication at ICML. However, I am open to revisiting my evaluation based on a revised version of the manuscript. I strongly encourage the authors to undertake a major revision during the rebuttal phase to address these issues. Detailed suggestions for improvement are provided in the following section.", "other_comments_or_suggestions": "**Structural and Content Improvements**\n\nA research paper should adopt a top-down approach, clearly guiding the reader from the central topic to the technical details. The primary focus is on understanding how LoRA impacts TTR. In the introduction, please provide a high-level sketch of your proof, explicitly connecting TTR with information geometry, and subsequently linking information geometry to NTK. Additionally, clarify at which stage LoRA-specific factors (e.g., rank, variance) become relevant in this analysis. Subsequent sections should also follow similar top-down structures.\n\nThe connection between equation (5), which involves the norm of parameter differences, and robustness is unclear. Since this equation is not utilized further in the paper, its inclusion seems arbitrary. Ensure that all elements of the paper are directly tied to the central goal of analyzing TTR. Avoid introducing concepts or equations that do not contribute meaningfully to the narrative.\n\nDefinitions of key concepts, such as Fisher information, should be included in the main body of the paper rather than relegated to the appendix. This is particularly important for tools that play a central role in your analysis.\n\nFor major theorems and proofs, consider adding brief intuitive explanations or proof sketches in the main text. This will help readers understand the underlying reasoning without delving into the technical details immediately.\n\nIt might be beneficial to provide a separate discussion on assumptions made for theoretical analysis, and justify them with citations or empirical evidence.\n\n**Figure and Caption Clarity**\n\n Ensure that all figure captions are self-contained and informative. For instance, in Figures 2, 4 and 5, the captions should not only describe what is being plotted but also highlight key observations and explain how these observations support the paper’s claims. This will make the figures more accessible.\n\n**Notational Consistency and Clarity**\n\n1. Line 157: What is K? I assume it is $K_{ntk}$?\n\n2. From the definition of $H_{\\alpha}$ in equation (10), $H_1$ does not make sense mathematically.\n\n3. The symbol $\\tilde{D}$ in section 3.3 is reintroduced after not being used for an extended period. Given the theoretical nature of the paper and the density of notations, it would be helpful to remind readers of its meaning when it reappears.\n\n4. Line 322 vs. Figure 4: There is a discrepancy regarding the rank settings for LoRA. The text states that the rank is 8 for all LoRA-specific settings, but this is inconsistent with the data presented in Figure 4. Please verify and correct this inconsistency.", "questions_for_authors": "The empirical results regarding the impact of variance on poisoning appear to deviate from the theoretical analysis. Could the authors provide additional empirical studies to investigate and reconcile this discrepancy? If further investigation is not feasible, I recommend removing the discussion on variance's impact on TTR from the paper, as the current empirical evidence does not align with the theoretical claims.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7AcuSC93JQ", "reviewer": "Reviewer_V5bG", "summary": "This paper analyzes the training-time robustness of Low-Rank Adaptation (LoRA) when fine-tuning large language models, focusing specifically on (1) untargeted data poisoning attacks (e.g., label flips) and (2) backdoor attacks (trigger insertion). The core claim is that LoRA exhibits lower robustness than standard full fine-tuning (FF) against untargeted poisoning but higher robustness against backdoor attacks. The paper’s key contributions include:\n- A theoretical framework that uses the neural tangent kernel (NTK) and information geometry (Fisher information, Rényi entropy) to study the “smoothness” of LoRA’s training manifold vs. that of FF.\n- A finding that the rank of LoRA’s low-rank updates and the initialization variance of LoRA’s submatrices are key factors. A smaller rank improves backdoor resistance (due to reduced parameter space for triggers to exploit) but hurts untargeted-poisoning robustness. \n- Conversely, a larger rank improves standard poisoning resilience but makes LoRA more vulnerable to backdoor triggers.\n\n**update after review**: I increased the score from 2 to 3 since concerns are addressed.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your enlightening reviews and the potential willingness of raising the score.\nBelow, we present a point-by-point response to address your concerns.\n\n# Clarification of Misunderstandings\n## \"Seemingly\" Discrepancy: \"Would this finding contradict the theoretical results (Theorem 3.6)? Or... a simplification...?\"\nThe answer is no. Our experimental findings are largely **consistent** with our theoretical analysis, especially with the main claim that \"LoRA is more vulnerable than full fine-tuning (FF) to untargeted poisoning attacks (UPA) but demonstrates greater resistance against backdoor attacks (BPA)\".\n\nIn Sec. 3.3, we begin by comparing LoRA and FF using two information geometry (IG) metrics, with the theoretical findings formalized in Theorem 3.6. Following this, from lines 262 to 311, we analyze the implications of Theorem 3.6, culminating in the above conclusion. Rather than contradicting the theory, the empirical results help validate the core message of our theoretical analysis.\n## \"The contradiction in Fig. 5...especially the latter two figures\"\nWe would like to clarify that Fig. 5 is **not in contradiction** with the statement in our paper. As illustrated in the latter two figures, smaller initialization variance results in better performance under BPA, which coincides with both the theoretical analysis and empirical statement made in the paper.\n## \"Would a 0.3 poisoning rate of UPA be too large in practice?\"\nYes, a 0.3 poisoning rate is an extreme case we use to cover a wide range of poisoning rates. The actual poisoning rates for UPA vary from 0.05 to 0.35, as shown in Fig. 1.\n## \"0.15% or 15% on Backdoor Poisoning?\"\nIt is not a typo: The poisoning rate you referred to for BPA is indeed 0.15% (i.e., 0.0015).\nWe further explore the impact of different poisoning rates ranging from 0.001 to 0.0045, as shown in Fig. 2.\n## Assumption 3.2 (OOLD): \"Would that be too strong in practice?\"\nYes, we acknowledge that Assumption 3.2 (OOLD) is a strong and ideal assumption. For this reason, in Section 3.4 (line 275), we further prove that **\"our conclusions can be generalized where LoRA is applied to *all linear modules* within a neural network\"**.\n## Supplemental Experiments\n## Addition of Color Bar to Fig. 3\nFig. 3 with a color bar can be found at https://a.imgno.de/67e9f93ecc7fc.png , and it will also be included in the final version.\n## Additional Attack Methods & Evaluation Metrics\nPlease find the corresponding experiments on our response to Reviewer Dcgq. Thanks!\n# Response to Questions\n## \"Would it also be possible to extend the results to other PEFT methods?\"\nTheoretically speaking, extending our analysis to **soft-prompt-based methods** is challenging.\n\nIn the case of LoRA, each adapter corresponds to a specific linear projection, allowing us to precisely derive the residual terms of their associated kernel functions. This enables a tractable theoretical comparison between LoRA and FF. However, the NTK behavior of soft-prompt-based methods (e.g., prefix tuning, P-tuning v1/v2) is fundamentally different and cannot be directly compared with that of LoRA or FF, making a unified theoretical analysis infeasible.\n\nNonetheless, our conclusions can be extended to **adapter-based methods** that are grounded in weight matrix approximation, such as LoRA variants.\n## \"Implications of This Study: What is the best practice when using LoRA?\"\nWe have summarized best practices for safely using LoRA in Sec. 4.5 (Line 404), *\"Summary of Findings and Defenses\"*. \n\nRegarding your question on *whether one should prioritize backdoor defense or resilience to random data corruption*, our recommendation is outlined in the fourth bullet point of Sec. 4.5. Specifically, we suggest *\"setting the LoRA rank as small as possible, provided that the model's performance meets task requirements\"*. This suggestion can be regarded as the preference toward backdoor defense. Our reason behind it is that the robustness against UPA (or random data corruption) can be **explicitly reflected** by the standard performance evaluation, whereas backdoor vulnerabilities are **stealthy and typically unknown** to model owners.\n\nBesides, as noted in the fifth bullet point of Sec. 4.5, we recommend setting *\"a small initialization variance\"*, such as $0.1\\times 1/n_{l-1}$ where $n_{l-1}$ is the input dimension of the $l$-th layer. This setting has been shown in Fig. 5 to significantly improve resilience to BPA, while only slightly reducing robustness to UPA.", "claims_and_evidence": "1. The experimental results indicate that 'LoRA is more vulnerable than full fine-tuning to untargeted poisoning attacks but demonstrates greater robustness against backdoor attacks.' Would this finding contradict the theoretical results, especially Theorem 3.6? Or this seemingly 'negative' empirical results further demonstrate that Theorem 3.6 only considers a simplification of the problem. I think more justifications on connections between the theoretical parts and the experimental parts need to be provided.\n2. The experimental results indicate that 'Specifically, a smaller initialization variance leads to relatively higher performance under backdoor attacks and lower standard deviation of results, which also aligns with our theoretical analysis'. However, for experiments of SST-2 and QNLI in Figure 5, the results might not be so significant and even contradict the patterns found in the latter two figures. Could you provide more explanations on this?", "methods_and_evaluation_criteria": "1. The authors mainly evaluate the method on the classification task with the Bert-Large model as the backbone. The evaluated datasets are from multiple sources, e.g., SST-2, QNLI, and COLA, which are very comprehensive.\n2. The authors mainly use {Accuracy} when evaluating the performance. Would you consider using more popular metrics (terms) such as Attack Success Rate for evaluation?\n3. The authors consider a poisoning rate of 0.3 for the UPA setting -- Would that be too large in practice?\n4. Just to confirm if this is a typo -- the backdoor poisoning rate is 0.15% or 15%?", "theoretical_claims": "The paper’s theoretical contributions revolve around deriving LoRA’s NTK, comparing it to the full fine-tuning NTK, and identifying conditions under which LoRA’s simpler geometry leads to an advantage or disadvantage. The derivations appear internally consistent, following standard infinite-width assumptions from prior NTK literature. But there are also possibilities that I might not fully understand the derivations.\n\nHowever, I have a concern about assumption 3.2 (OOLD): Would that be too strong in practice?", "experimental_designs_or_analyses": "1. For Figure 3, could you also provide the color bar showing the range of the heatmap values?\n2. For the attack baselines, the authors only choose one attack for the UPA setting and one attack for the BPA setting, which are incomplete. I suggest the authors consider some additional attack methods, e.g., [1, 2].\n\n[1] Rethinking Stealthiness of Backdoor Attack against NLP Models.\n[2] Hidden Trigger Backdoor Attack on NLP Models via Linguistic Style Manipulation", "relation_to_broader_scientific_literature": "Backdoor attacks and data poisoning on large language models have been given much attention recently. However, few works directly compare full fine-tuning vs. parameter-efficient approaches in terms of training-time robustness. To the best of my knowledge, this is the first paper that touches on this perspective. This paper might be interesting for the readers in this community.", "essential_references_not_discussed": "1. Comparison with other parameter-efficient methods: The theoretical results on LoRA is good. Would it also be possible to extend the results to other parameter-efficient tuning methods such as prefix tuning and prompt tuning?\n2. Adaptive or stealthy data poisoning: There is a rapidly growing body of work (e.g., more advanced data corruption that systematically modifies text distributions) that might deserve mention for thoroughness. See {Experimental Designs Or Analyses} for more details.", "other_strengths_and_weaknesses": "Strength:\n1. The paper’s theoretical perspective is novel. Besides, the authors’ focus on training-time robustness is timely and important.\n2. The writing is clear and well-structured.\n\nWeakness:\nMy main concerns about this paper are that:\n1. The 'seemingly' discrepancy between the theoretical results and the empirical findings\n2. The insufficient experimental designs, including considered attack baselines and the evaluation metrics.\n\nPlease see above for more detailed comments.\n\nBased on the above, I would give a score of 2, but I am willing to raise the score if the questions are well addressed.", "other_comments_or_suggestions": "1. I suggest the authors consider the implications of these findings. For example, what are the implications of these findings, especially for the model trainer in practice?  More specifically, are there recommended “best practice” rank and variance settings for users who want to reduce the risk of a specific type of training-time attack (e.g., prefer backdoor defense vs.\\ prefer strong resilience to random data corruption)?", "questions_for_authors": "1. Multiple Trigger Variants: Have you tried other stealthy or adaptive triggers (like synonyms, paraphrase triggers, or style modifications)? If so, do the results still show LoRA as more robust than FF?\n2. Comparison With Other PEFT: Would you consider analyzing how Prompt/Prefix Tuning or Adapters compare to LoRA for training-time robustness?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper investigates the security problem of LoRA and especially analyzes how its low-rank structure could influence training-time robustness during fine-tuning. The authors presented a theoretical framework showing that while LoRA is more robust to backdoor attacks than full fine-tuning, it is more vulnerable to untargeted data poisoning. The findings are supported by extensive experiments.

Yes, this appears to be the first work to systematically investigate the intrinsic security vulnerabilities of LoRA. If the claims regarding LoRA’s susceptibility to training-time attacks are valid, this paper provides a timely and valuable contribution to the growing field of secure fine-tuning for large language models. All reviewers agree on the significance of the findings and are in favor of accepting the paper.
Decision: Accept (poster)