Fine-tuning with Reserved Majority for Noise Reduction
Authors: Shuyang Jiang
Yusheng Liao
Ya Zhang
Yanfeng Wang
Yu Wang
Venue: ICLR 2025 Spotlight (2025)
Keywords: large language models
parameter redundancy fine-tuning
noisy reduction
Primary area: foundation or frontier models, including LLMs
TL;DR: A novel fine-tuning method for low-rank adaptation noise reduction

Abstract: Parameter-efficient fine-tuning (PEFT) has revolutionized supervised fine-tuning, where LoRA and its variants gain the most popularity due to their low training costs and zero inference latency.
However, LoRA tuning not only injects knowledgeable features but also noisy hallucination during fine-tuning, which hinders the utilization of tunable parameters with the increasing LoRA rank.
In this work, we first investigate in-depth the redundancies among LoRA parameters with substantial empirical studies.
Aiming to resemble the learning capacity of high ranks from the findings, we set up a new fine-tuning framework, \textbf{P}arameter-\textbf{Re}dundant \textbf{F}ine-\textbf{T}uning (\preft), which follows the vanilla LoRA tuning process but is required to reduce redundancies before merging LoRA parameters back to pre-trained models.
Based on this framework, we propose \textbf{No}ise reduction with \textbf{R}eserved \textbf{M}ajority~(\norm), which decomposes the LoRA parameters into majority parts and redundant parts with random singular value decomposition.
The major components are determined by the proposed \search method, specifically employing subspace similarity to confirm the parameter groups that share the highest similarity with the base weight.
By employing \norm, we enhance both the learning capacity and benefits from larger ranks, which consistently outperforms both LoRA and other \preft-based methods on various downstream tasks, such as general instruction tuning, math reasoning and code generation. 
Code is available at \url{https://github.com/pixas/NoRM}.

Published as a conference paper at ICLR 2025
FINE-TUNING WITH RESERVED MAJORITY FOR NOISE REDUCTION
Shuyang Jiang Yusheng Liao 9. ,,, Zhang
Yanfeng Wang Yu W
Fudan University
School of Artificial Intelligence, Shanghai Jiao Tong University
Shanghai Artificial Intelligence Laboratory
shuyangjiang22 @m fudan. edu. cn
(1iao20169907 ya.zhang, warggyanfeng622, yuwangsjttulsstun edu. cn
ABSTRACT
Parameter-efficient fine-tuning (PEFT) has revolutionized supervised fine-tuning, where LoRA and its variants gain the most popularity due to their low training costs and zero inference latency. However, LoRA tuning not only injects knowl- edgeable features but also noisy ballucination during fine-tuning, which hinders the utilization of tunable parameters with the increasing LoRA rank. In this work, first investigate in-depth the redundancies among LoRA parameters with sub- stantial empirical studies. Aiming to resemble the learning capacity of high ranks from the findings, we set up new fine-tuning framework, Fine-Tuning (PREFT), which follows the vanilla LoRA tuning process but is re- quired to reduce redundancies before merging LoRA parameters back to pre-trained models. Based on this framework, we propose Noise reduction with Reserved Majority (NoRM), which decomposes the LoRA parameters into majority parts and redundant parts with random singular value decomposition. The major compo- nents are determined by the proposed Sim-Search method, specifically employing subspace similarity confirm the parameter groups that share the highest simi- larity with the base weight. By employing NORMM, we enhance both the learning capacity and benefits from larger ranks, which consistently outperforms both LoRA and other PREFT-based methods on various downstream tasks, such as general instruction tuning, math reasoning and code generation. Code is available https::/ /tt::://gtttttee com www.cccccc.....ee wwwpp/oxx///oo..ccccccc
INTRODUCTION
Large language models (LLMs) have revolutionized Natural Language Processing (NLP) by pretrain- ing on vast textual corpora, enabling them to encode extensive world knowledge (Chang et al. 2024
AlKhamissi et al. 2022). These models exhibit remarkable zero-shot and few-shot performance across a wide range of tasks (Brown et al., 2020 Anil et al.) 2023: OpenAl 2023: Touvron et al., 2023a b). Instruction tuning. a form of supervised fine-tuning (SFT), has further enhanced L.MMs by refining their cnttuuction-oolowwwiii
capabilities, therehy simplifying human-LLM interactions (Ouyang et al.) 2022 Chung et al.) 2024). Pretrained models fine-tuned with SFT excel various downstream lasks such as medical consultation (Wu et al. 2024| Chen et al. (023). mathematical reasoning (Luo al. 2023a] Yue al. 2023 Yu al. 2024b Toshniwal et al. 2024), and serving as intelligent assistants (Peng et al. 2023 vvson et al. 2023). However, adapting these models to downstream tasks resource-constraines environments requires parameter-efficien fine-tuning (PEFT) techniques, which update less than 1% of the total parameters while achieving performance comparable to full fine-tuning. LoRA and its extensions (Pfeiffer et al. [222]] yang Liu al., 2024 HHaou al. 2024] Wang et al. 2024; Jiang et al.) 2024b) have emerged some of the most effective and efficient methods in this regard.
Despite its efficiency, vanilla LoRA fine-tuning does not always benefit from increasing the number of tunable parameters and may even degrade performance. Hu et al. (2022) found that optimal
"Corresponding Author

---

Published as a conference paper at ICLR 2025
WIkISOL Performance (36
74.3
WIKSOL
91.7
MUtINII
74.0-
91.6
B
B
73.9
W
W
91.4
73.8
91.3
A
A'
73.7
Values
Figure I: Vanilla LoRA tuning usually cannot benefits from larger ranks (a) and Left of (b), where represents the rank of LoRA and ace represents the downstream performance. In contrast, PREFT aims to adopt the same number of tunable parameters as LoRA but cuts part of it to achieve higher performance (Right (b)), where ry < r, and r > r..
performance is often achieved with lower ranks, while larger ranks lead negligible negative improvements (see Figure[la) We hypothesize that this behavior stems from the fine-tuning dynamics of LLMs. As shown by Gekhman al.] (2024), fine-tuning knowledge and undesired hallucinatory features. We speculate that as the number of tunable pa- rameters LoRA modules increases, fine-tuning results in trade-off between valuable knowledge and noisy features. Consequently, increasing the rank may exacerbate the hallucination problem, leading to diminished performance gains. Thus, improving LoRA's effectiveness requires selectively retaining useful knowledge while eliminating unwanted redundancies.
introduce both useful downstream
In this paper, we systematically investigate the parameter redundancies in LoRA fine-tuning. Our analysis spans the overall structure of LLMs. focusing on transformer layers and specific modules within the transformer. Our experiments reveal that LoRA parameters contain significant redundan- cies, which exhibit distinct patterns across layers and modules. Based on these findings, we propose a new parameter-efficient tuning framework, Parameter Redundancies Fine- Tuning (PREFT), which follows the standard LoRA tuning pipeline during training but eliminates certain redundancies before merging the fine-tuned parameters back into the base LLMs (see Figure lb). To further optimize this process, we introduce Noise reduction with Reserved Majority (NoRM), which utilizes singular value decomposition (SVD) to distinguish between essential and noisy components. NoRM acceler- ates the computation by using approximated SVD and introduces novel Sim-Search method that adaptively selects the most valuable components based on subspace similarity between the reduced and base weights. Figure illustrates the computational flow of NoRM and Sim-Search.
We conduct comprehensive experiments to validate the effectiveness of NORM, covering tasks such general instruction tuning, mathematical reasoning, and code generation, using three strong pre-traing pre-trained models. NoRM consistently outperforms LoRA and other PREFT methods, achieving an average gain +4.67 over the best PEFT methods and +1.63 over the strong PREFT method TAIA, when applied to Llama3-8B. Additional analysis confirms the robustness of NoRM, and shows that Sim-Search outperforms alternative similarity-ban search methods. Further experiments demonstrate that NoRM significantly improves the utilization of the fine-tuning corpus while maintaining the retention of pre-trained knowledge.
Overall, conclude our contributions as such:
1. Revisiting LoRA Tine-Tuning: We revisit LoRA tuning and, through extensive experiments, reveal that suffers from low parameter utilization due the introduction of redundant features. To address this, we introduce PREFT, ,, a novel fine-tuning framework that significantly improves upon existing PEFT methods by reducing these redundancies.
Adattive Noise Reduction at Inference: Within the PREFT framework, we propose NoRM, an adaptive method that removes disruptive components for each parameter. This is achieved using our automated search algorithm, Sim-Search, which identifies the most relevant components by evaluating their affinity with the base weights.
2

---

Published as a conference paper at ICLR 2025
Selt Drop Tut
83.5
1.0
84.7
85.0
84.5
Rema Natoo
83.0
Remain Ratio
0.8
84.0
g 64.0
0.5 0.7
83.5
Dax
00:000
82.5
83.4
2-2 Dog
82.0
0.1 0.3
82.8
16
32
82.1
0.1
02
0.3 0.6
Ramain Ratio
Layer Index
Modules
(a)
(b)
(c)
Figure 2:
Performance change log with random drop ratios (a) and the performance distribution among layers (b) and modules (c). We annotate the best remaining ratio with darkblue.
3. Comprehensive Evaluation: We rigorously evaluate NORM across multiple domains, including general instruction tuning. mathematical reasoning, and code generation. Our method consistently surpasses state-of-the-ar PEFT methods and alternative PREFT approaches, demonstrating the effectiveness of removing parameter redundancies.
2 PARAMETER REDUNDANCIES FINE-TUNING (PREFT)
In this section, we first introduce the basic properties LoRA fin-ttnnigg After that, we empirically unveil the common parameter redundancies within PEFT-based methods. Following the findings, we formalize the post-processing of these redundancies as novel fine-tuning framework, PREFT. Ultimately, introduce two genres of identifying the redundant components intelligently.
2.1 PRELIMINARIES
Given an input sequence X E Rmmxd LoRA (Hu et al. 2022) proves that the update of original linear layers AW in large language models is of low-rank, and can be decomposed into the multiplication of two compact matrices AB. The pretrained weight W E Rd'xd frozen in training phase, while A and B are trainable parameters and contribute together to the forward pass:
= XW + XAWT = XW a "X(BA)
(1)
where Rxd d and r < d, d' a is the scaling factor. Without loss of generality, we omit the layer index for the following formula. At the beginning of training. B is initialized to all-zero matrix and A uses Gaussian initialization to ensure that BA is zero at initialization.
2.2 PILOT EXPERIMENTS
Yu et al. (2024a) indicates that the delta parameters of full fine-tuned models contain superior redundancies, where over ninety percent of fine-tuned parameters can be dropped. In contrast to them, this study, we further dig out the parameter redundancies among PEFT modules, which are usually convinced of compact encoding compared full parameters. We progressively present the commonly existing redundancies holistically, along with layer-wise and module-wist analysis.
Experiment Settings We mainly utilize Llama model. We choose MetaMathQA-3955 (Yu et al. 2024b) as the fine-tuning corpus and SVAMP (Patel al. 202I) as the evaluation set. We adopt vanilla LoRA (Hu al. 2022 as the fine-tuning method and choose three configurations of LoRA rank and values: {(r a (16, 32), (32, 64), (64, 128)). The learning rate is to 2c-4 and the total batch size set to 128. After fine-tuning, we set up three parameter drop strategies: (1) drop holistically, (2) drop by layer, and (3) drop by module.
Instruct (AI@Metal 2024 the base
Drop holistically: We denote the remaining ratio of intrinsic rank as 17. We randomly remain 7 10% 90% channels of both parameters: A' A[H and B' BB:, H) where H (h, hig...,,h, h,. h E 10. - 1} A Vi.j (i j) = h + h2} and r' = [n. To maintain the scale of modified delta parameters, also change the value from 2r to r' after the above
3

---

Published as a conference paper at ICLR 2025
modification. Experiments averaged by five runs to reduce random bias. In Figure 2a it is obvious that even with low rank (r = 16), randomly dropping brings significant performance gain compared to complete parameters. Meanwhile, small remaining ratio (17 0.1) still brings performance gains, which indicates mass redundancies among each parameter. We also notice that lower ranks bring higher performances, which is attributed to fewer introduced hallucinations (Gekhman al. 2024) by the intrinsic properties of fine-tuning.
Drop by layer: We find that the LoRA rank has no effect on deriving conclusions; therefore, the following experiments, we only choose = 32 for simplicity. For each layer / E [0, T.) of base L.Mss, we select five remaining ratios: (10%, 30%, 50%, 70%, 80%] as well as the full LoRA parameter as remaining ratio 100% and follow the first strategy to retain delta LoRA parameters of the specific layer yet keep the other layers unchanged. For layer redundancies (Figure 2 we find that in middle layers, the best performances are achieved with moderate drop ratios. For upper layers and lower layers, a large remaining ratio generally brings higher results. This indicates that initial encoding and eventual output layers contain much fewer redundancies compared middle layers, which also aligns with previous findings (Men et al. 2024).
Drop by module: For each module of Llama-style transformer, including fq-pro k. k-proj, v.proj, o o_proj, pate_prof, up-proj, down_proj), we choose five remaining rates (10%, 30%, 50%, 70%, }0%} and follow the second strategy to retain delta LoRA parameters of the specific module yet keep the other modules unchanged. In Figure 2c dropping parameters of {q-proj, k-proj, v-proj] brings minor performance gains while dropping those of {o-proj, gate proj, up proj, down proj} facilitates further performance improvements, which is also alluded to by Jiinn et al. (2024a). Noticeably, for the latter four modules, a small drop ratio like 30% 50% generally results in the best performance across all drop ratios, which also indicates that MLP blocks encode nonnegligible hallucinatory information during PEFT.
2.3 PREFT
Built upon these findings, it essential to adaptively remove these redundancies based on parameter locations to enhance the effectiveness of LoRA parameters. Therefore, we propose a new fine-tuning framework: Parameter Redundancies Fine-Tuning (PREFT), which enhances fine-tuned models by removing redundancies of PEFT parameters obtained via a usual training procedure. The comparison with PEFT is illustrated in Figure 1b Formally, given the pretrained weight W and updated AW, PREFT aims to create new delta weights AW" by removing parts of AW, under the condition that:
arg max M (æx (W)) (AW
where p is the total number of parameters, æ is the input sequence and M is a specific metric for downstream tasks. Based on it, previous method TAIA (Jiagg et al. 2024a) can be concluded as such:
[AW') { attras Ofjn}
where it removes all delta parameters of FFN modules but keeps the self attentio part unchanged. Another example, MedCare (Liao et al. 2024), also adopts PREFT philosophy:
(AW') (AWLORA OMoLORA}
where MoLoRA is the mixture-of- of LoRA submodule (Feng et al. 2024 Liu et al. 2023a). However, these two instances of PREFT identify the shearing part through empirical observations and lack fine- grained parameter-wise practice for reducing redundancies. Consequently, in the newly introduced PREFT system, the core of adaptive parameter shearing is to identify redundant parts of original LoRA parameters more intelligently. Based this, we categorize the identification process into two genres: (1) intra-shearing which leverages solely LoRA parameters to perform reduction, and (2) inter-shearing which leverages the relationship between LoRA parameters and corresponding base weights to fulfill the task. We exemplify these two categories in the following:
Intra-shearing Assuming that we want to keep 0 3 < components of each parameter (B can be searched on the dev set), the most common practice is to perform principle component
Without loss of generality, use (+)' notation for modified parameter under PREFT framework.
4

---

Published as a conference paper at ICLR 2025
Sim-Search
Aw;"
AWG
A'
8
Pretroined WeighT
Subspace Similarity
Highest
AWA
Similanity
Matrix Multiplication
Matrix Slicing
Random Natrix
QR-Dec: Decomposition
Overview of NoRM. NoRM employs random SVD to extract the major components from the delta parameter. NORM utilizes (b) Sim-Search to determine channels with little hallucination based on the subspace similarity between the sheared delta weight and the pre-trained weight.
Figure 3:
analysis (PCA (Abdi & Williams, 2010j) or singular value decomposition (SVD (Klema & Laub 1980)) to dig out the major components:
A, B, - k(A,
(5)
where kc-) can be any method that selects the major 33 percent of total components.
Inter-shearing In the above method, need to search / on the dev set, which inhibits PREFT from the deployment on real applications. Therefore, inter-shearing methods determine 3 on the relationship between LoRA weights and pre-trained weights, which is based fine-tuning introduces hallucination sharing different distributions with pre-trained weights (Gekhman al. 2024). Therefore, we choose 3 such that the remaining components overlap with the pre-trained weight to a maximum extent. Formally, we can conduct hyperparameter search to determine B:
observation that
8 = arg max Sim (By4g, W)
where 3 can be searched over a given range with pre-defined step and Sim is any similarity- -ased metric between the delta and base weight, including the reverse of Læ-distance or cosine similarity.
Based on the above categorization, in this paper, build inter-shearing method called Noise reduction with Reserved Majority (NoRM) by reserving contributing components through random SVD while determining 3 by novel Sim-Search method, which maximizes the subspace similarity between the reduced LoRA weights and base weights.
3
NORM
3.1
OVERALL PIPELINE
The overall pipeline for NoRM is presented Figure We start by analyzing the approximation of LoRA parameters
IRd xr and gr and their product AW = BA. By the low- get Rank(BA) = Without taking any low-rank
rank decomposition property of LoRA, assumptions on the updated weight AW, we decompose it using singular value decomposition (SVD):
AW = U
where E Rdd''s V E Rxx are left/right singular matrices and Σ T is the diagonal matrix containing the singular values of AW. To approximate the delta parameters by discarding redundant components, we can retain the first c largest singular values and corresponding singular vectors:
'' diag(o), ...... ..), U' -U;,1:0 V' V[;
Such approximation deduces the approximation of BA: AW = BA se UT'''' However, directly computing the singular value decomposition of delta weight AW is computationally heavy for both

---

Published as a conference paper at ICLR 2025
pre-processing and storage; therefore, we propose to randomized SVD (Hakko al. 1 to further speed up this process and hence approximate the low-rank parameters and for portable usage. Specifically, randomized SVD creates a random matrix 52 Rd with Gaussian distribution:
n (O,
After that, we obtain the main column subspace of AW with Y AWΩ approximate the feature space of original delta weight. Followed by that, we compute an approximation of orthonormal bases of AW: Q E Rd'xe xe using QR decomposition on Y: Y = QR. Based on the orthonormal basis, we obtain the projection of delta weight Rexd on the low-dimensional representation space of Q:
B= Q'AWWW
(10)
Then compute the standard SVD the smaller matrix B:
B = Ü'E'V'
(11)
where U' Rexe E' Rexd V Rd We transform back U' to approximate singular vectors U' E Rd'x as U' QU', Based on above quantities, we reconstruct the approximated B and as:
='= diag(E') A'= v'= [1:
(12)
where diag(>) ) E Rexe is a diagnomal matrix satisfying diag( 3' ) = E' ,,,, < S c and diag diag(E"),, 0, + j. To determine an appropriate approximation factor e, we innovatively introduce Sin-Search which searches c through the affinity of remaining components.
3.2 SIM-SEARCII
Previous works (Hu et al. 2022 Wang et al. 2024) empirically demonstrate that the subspace similarity between the delta weight AW and the pretrained weight W correlates positively to downstream performances. Meanwhile, Gekhman et al. (2024) also points out that during fine- tuning, LLMs memorize new knowledge by hallucinating itself, which degrades the effectiveness of fine-tuning. Therefore, in NORM, the remaining components should satisfy both two rules:
1. The remaining factors should contribute most positively compared to others e' # In other words, these e components contain the least noise.
2. The subspace spanned by the remaining e singular vectors should possess the highest subspace similarity with that spanned by the pretrained weight.
Based on the two rules, we introduce search step and search the = r -3 values ranging from a given start value s to r: {T 8, (T 1) ,,.",,"" and perform Equation 9 2 to obtain B and A under different values. Followed by these newly obtained B and A', we reconstruct the delta
weight AW, by AW. = B',A', and compute the major r singular vectors again by the random SVD:
U E, V Random SVD(AW.)
(13)
We also use Equation 13 to decompose the pretrained weight W to obtain U, Σ and VT. We extract the r left singular vectors of U, and U and compute the subspace similarity as such:
_U 1
d
(14)
where U.r Ueli: r] E dx U, = Ul: r] E Rdxr, Based on the computed Grassmann distance d., select the value and corresponding reduced delta weights B' and A', such:
{c, B', Ar) arg max &.
(15)
Finally, these parameters can finally merge back into the pretrained weight introduce no inference latency: W = W + B,A', v W E (W)P.
4 EXPERIMENTS
In this section, we comprehensively evaluate the proposed NoRM method on various downstream domains, including general language understanding, mathematical reasoning, and code generation.

---

Published as a conference paper at ICLR 2025
Model
Method
BBH
MMLU
TydiQA
CQA
TruthfulQA
GSMSK
Logiqa-EN
Average
Base
41.73
66.84
45.36
74.20
$7.65
87.19
40.40
59.05
LoRA
44.31
67.95
48.45
75.84
50.80
83.02
43.93
59.19
LoRA+
42.64
67.81
52.15
78.71
50.18
83.24
43.78
59.79
Qwen2-7B
DoRA
43.11
67.94
44.64
75.43
53.98
82.94
44.85
58.99
MoRA
36.31
62.43
46.70
72.65
53.37
75.44
44.09
55.85
TAIA
44.79
66.36
46.81
75.10
58.51
85,60
45.78
60.42
NORM
45.19
68.89
50.80
78.05
$7.77
85.06
46.08
61.69
Base
40.68
59.18
40.90
71.83
63.28
77.79
41.17
56.40
LoRA
36.38
63.42
43.56
77.64
48.10
72.02
38,40
54.22
LoRA+
40.30
63.30
38.01
75.35
37.70
73.62
36.71
52.14
Llama3-8B
DoRA
36.95
63.89
42.54
77.97
42.59
71.27
37.94
53.31
MoRA
25.09
20.48
20.07
26,42
23,01
TATA
35.83
62.02
47.26
76.66
$4.22
77.10
37.94
55.86
NORM
43.11
64.61
46.21
77.72
54,10
77.71
43.47
58.13
Base
39.26
54.07
30.04
66.83
56.30
55.27
36.87
48.38
LoRA
34.79
53.68
41.88
73.05
53.37
41.47
33.79
47.43
LoRA+
32.51
56.67
39.99
70.52
42.23
45.49
38.40
46.54
Mistral-7B
DoRA
34.66
53.31
32.01
68.88
38.19
42.99
36.25
43.76
MoRA
26.69
27.58
27.40
49.14
33.66
20.24
33.18
31.13
TAIA
38.06
54.97
39.20
71.91
53.24
50.87
38.25
49.50
NORM
39.29
58.05
40.56
73.55
54.47
52.01
37.63
50.79
Table 1: Experiment results on general instruction tuning with Qwen2-7B, Llama3-8B, and Mistral- 7B pre-trained models. "_" means zero performance on specific datasets. "CQA" indicates the CommonsenseQA dataset. All experiments are conducted based on open-sourced codebases. Bold represents the best result. The NoRM setting achieves the best results in most datasets.
4.1 EXPERIMENT SETUPS
Qwen2-7B-Instrue and Mistral-7B- v0.3- as the base model. For the decoding strategy, adopt the zero-shot setting with temperature for reproducible generation. We choose 100K subset TüLu V2 as the general instruction tuning dataset and evaluate each fine-tuning method across various tasks, including sym- bolic reasoning, commonsense reasoning, knowledge understanding and multi-lingual understanding. Apart from general tuning, we also choose math reasoning and code generation as specific fine-tuning tasks and utilize LLama3-8B as the pretrained model. Specifically, we employ MetaMathQA (Yu al. 2024b) to fine-tune the base model for math reasoning. which consists of 395K training samples evolved from GSM-8K (Cobbe et al. 2021) and MATH (Hendrycks et al. 2021b). The evaluation sets the corresponding test sets
Training and Evaluation We choose
&S and MATH to models' sol solving capabilities for math word problems. For the code generation, we utilize Magi coder Bvol- al. 2024) as the training data. All fine-tuned models are assessed on HumanEval (Chen al. 202.) and MBPP (Austin et al. 2021) benchmarks, which contain 164 and 378 high-quality Python text-to-code problems, respectively. For more rigorous evaluation for programming orientted models, also test models HumanEval+ and MBPP+ of the EvalPlus (Liu et al. 2024 benchmark.
110K (Wei)
Implementation Details We choose LoRA (Hu al. 2022 DoRA yang Liu al. 2024). LoRA+ (Hayou et al.) 2024). MoRA (Jiang et al.) 2024b) as the compared PEFT baselines and TAIA (Tiagg et al.) 2024a) the PREFT baseline. All experiments
conducted 4 NVIDIA A100 GPUs We use BFloat16 precision and fine-tune all training corpus for epoch. The learning rate 2e-4 and the LoRA rank
64. We use a linear warmup strategy with a 0.03 warmup ratio and a cosine learning rate scheduler. For NoRM's setting, the search step is set to 0. and the search range starts More details can be found Appendix
4.2
MAIN RESULTS
Table D and present a comprehensive comparison between various PEFT methods (LoRA, LoRA+, MoRA,, and DoRA), and PREFT methods (TAIA and proposed NoRM) across different training
We choose instruction-tuned models instead of hase models for higher zero-shot compatibility and more accurate evaluation
7

---

Published as a conference paper at ICLR 2025
Math
Code
Model
GSMSk
MATH
HumanEval
HumanEval+
MBPP
MBPP
Avg.
Pretrained
77.79
31.06
53.00
48.80
71.70
60.60
57.16
PEFT Method
Full
77.86
27.86
58.50
53.70
65.30
55.00
56.37
LoRA Huu
2022
80.21
29.27
48.80
44.50
67.20
57.40
54.56
LoRA+
Hayou et al. 2024
78.77
28.22
56.70
52.40
68.50
58.50
57.18
DoRA
Yang
80.82
29.84
51.80
46.30
66.90
57.40
55.51
MoRA
Jiang
63.15
19.48
43.30
39.00
47.90
39.20
42.01
PREFT Method
TAIA
79.08
32.60
59.10
53,00
69.30
60.60
58.95
NORM
82.79
33.76
63.40
59.80
71.40
60.80
61.99
Table 2: Experimental results math reasoning and code generation with the Llama3-88 8b base model. For LoRA+, DoRA and MoRA, we implement them using their open-sourced codebases. Bold text represents the best result. The NoRM setting achieves the best results in most datasets.
General
Math
Code
Model/Method
BBH
MMLU
TydiQA
GSMSK
HumanEval (+)
MBPP (+)
Avg.
Pre-trained
40.68
59.18
40.9
77.79
31,06
53.00 (48.80)
71.70 (60.60)
53.75
NORM
43.11
64.61
46.21
82.56
34,02
63.40 (59.80)
71.40 (60.80)
58.22
w/ min
43.14
64.36
44.54
80.36
32.58
61.60 (54.90)
69.60 (59.50)
56.73
w/ minor
42.44
64.56
44.98
81.96
34,00
61.60 (57.30)
72.00 (61.10)
57.77
w/ 1.2
43.86
63.86
44.47
79.76
31,58
57.90 (53.00)
69.80 (60.60)
56.09
w/ cos
43.91
64.09
45.93
80.06
32.48
61.60 (56.70)
70.40 (59.30)
57.16
w/ PCA
29.28
63.85
40.11
79.23
29.28
54.90 (51.20)
72.50 (60.10)
53.38
Table Ablation experiments on the selection of major components of NoRM. The major compo- nents selected by NoRM can reserve most LoRA representation but discard most noise.
datasets and evaluation benchmarks. In general instruction tuning, NORM generally performs best across various pre-trained models. Notably, most PEFT-based methods experience parameter redundancy, leading to either minimal gains or even performance degradation. In contrast, PREFT- based methods exhibit superior data efficiency, consistently improving upon base models. Specifically, NORM surpasses LoRA by 4.37 points on MMLU and 3.69 points on LogiQA-EN when applied to Mistral-7B. downstream tasks such as math reasoning and code generation, NoRM also achieves the highest performance among all basclines, delivering a +5.31 improvement over LoRA and +2.73 gain compared TAIA. Additionally, PEFT-based methods, including DoRA, LoRA+, and MoRA, fail to surpass vanilla LoRA in instruction-tunm method, TAIA. contrast, NORM significantly surpasses TAIA in symbolic reasoning tasks, such as math reasoning and code generation, highlighting its effectiveness in reasoning-intensive intennivve scenarios.
tasks and fall significantly behind the PREFT
4.3 ABLATION STUDY
In this section, we investigate the impact brought by the choice of shearing methods and highlight the superiority of the random SVD in NORM and the smart determination of of Sim-Search. We
choose five comparatives: (1) w/ min: NoRM but choose the least similar major components; (2) w/ minor: NoRM but choose the minor components; (3) w/ L NoRM with the L distance metric; (4) w/ cos: NoRM with the cosine similarity metric; and (5) w/ PCA: NoRM with PCA method for the selection of major components. We use the same settings of $4.1 except subsampling three general evaluation sets covering diverse tasks (BBH, MMLU, and TydiQA) to conduct the ablation experiments. Results in Table demonstrate that random SVD selection outperforms PCA selection. Furthermore, we show that subspace similarity metrics outperform other similarity- methods, underscoring their precision in measuring the affinity between the retained components and the pre- pre-trained weights. By leveraging the Sim-Search method, NoRM effectively preserves the most relevant components for downstream tasks while maintaining alignment with the base models from subspace perspective, thereby minimizing the risk of hallucination during fine-tuning.
8

---

Published as a conference paper at ICLR 2025
Code
55
Code
Nith
Math
General
Gencral
56
-5 10
05
Base
55
%6
25
25
27
27 Trahings
27
20
141616020
Layerindee
Modiles
(a)
(b)
(c)
Figure 4: (a) NoRM benefits from larger ranks, while vanilla LoRA often obtains lower performance on larger ranks; parameter rank distribution of NoRM among layers (b) and modules (c).
5 ANALYSIS
In this section, we aim to answer the following research questions (RQ):
RQI: How is the parameter redundancies change with the number of delta parameters?
RQ2: What do the sheared parameters distribute over the LLM?
RQ3: What is the secret for NoRM's high performance over PEFT methods?
RQ4: Does NoRM scale other sizes of training data?
RQ5: What has been reduced by NORM?
Response to RQI: NORM benefits from larger ranks, with a different manner with LoRA. To validate the hypothesis that NoRM can leverage more tunable parameters and hence gain more improvements, we experiment NoRM with an increasing rank sequence: [8, 16. 32, 64, 128) and use the same hyperparameter settings and evaluation datasets described in $4.3) The results, shown in Figure 4a| indicate that compared to LoRA, which usually achieves its highest performance with a middle rank, NoRM achieves higher performance with more tunable parameters, which conforms to the general relationship between parameter amounts and performance upgrades. Such distinction also instantiates that vanilla LoRA tuning introduces noise and redundancies into parameters, while NORM can intelligently remove such distraction and largely benefit from the fine-tuning corpus. We also notice flattening improvements when scaling the LoRA ranks to 128 due to an unstable training process; therefore we choose r = 64 in our main experiments, Full results are presented in Table
Response to RQ2: Reduced parameters distribute as 2.2 suggests in most cases. To visualize the distribution of sheared parameters across the transformer architecture, we compute the reduced rank for each checkpoint in terms both layers and modules. Results in Figure [ouure/4bdda demonsttte that in upper layers, NoRM tends to remain fewer parameters, which is accordant with the conclusion in $2.2 In contrast, in the half layers (around 16-18), NoRM behaves to maintain large ranks. Such distinct manner derives from the remaining strategy of 2.2 (random drop) and NORM (random SVD). Besides, NoRM M remains as much down proj and c proj j ranks as Figure 23 indicates in a module-wise perspective. NoRM maintains 50% parameters for self-attention modules, which is the configuration obtaining the highest performance in 22.2 In conclusion, NoRM M intelligently crases the redundancy following its distribution LLMs and achieves superior performance.
Response to RQ3: NORM forgets less and learns more. We unveil the secret of NoRM from the forgetting perspective, where we benchmark the vanilla LoRA method and NoRM on memorizing pre-trained knowledge. We follow Kalajdzievski (20244) to use WikiText- test dataset (Merity et al. 2016) as the evaluation set since has already served as the pre-training corpus for most L.MMs. We use the cross-entropy loss as the metric to test the base model, LoRA-tuned model and NoRM-tuned counterpart taking the -lama3-8B-Instruct as the backbone. Results in Table demonstrate that NoRM outperforms LoRA and the base model with a large margin, matter what training data it leverages. Notably, LoRA-tuned models are generally inferior to the base model, indicating that LoRA tuning still results in forgetting problems, albeit relatively few tunable parameters. In contrast, NoRM discards the hallucinatory contents accompanied by the learning of new knowledge and hence strengthens the memorization of internal knowledge.

---

Published as a conference paper at ICLR 2025
Training Data
Average Performance
Method
General
Coding
Math
56
Base
3.7016
54
LoRA
3.7102
3.7512
3.7323
NORM
3.5992
3.6450
3.6519
52
NoRM
Table 4:
Forgetting loss on WikiText-103 test dataset. NoRM reduces hallucination and hence gains superior memorization of internal knowledge over other baselines.
1K
10K
100K
330K
Data Scale
Performance of fine-tuned Llama3-8b using different numbers of TüLU V2 training samples.
Figure
Response RQ4: NORM demonstrates superiority across various sizes of fine-tuning datasets. Considering that in practical conditions, access to extensive fine-tuning datasets is frequently limited, we compare NORM LoRA and TAIA for fine-tumie munituning LLaMA3-8B with range of instruction- tuning sample sizes, specifically [IK, 10K, 50K, 100K, 330K], with 330K being the full size of TÜLU V2. We visualize average performance of each method Figure and present the full results in Table The results show that NORM consistently outperforms LoRA and TAIA across all training sample sizes. With 10K training samples, NORM surpasses LoRA and TAIA by margins 3.49 and 2.53, respectively. Even when the training size is reduced to IK, NORM maintains its lead with advantages of 1.46 and 2.18 over LoRA and TAIA, respectively. This demonstrates that our methods persistently enhance performance over LoRA and TAIA, regardless of the training sample volume.
Response to RQ5: NoRM reduces the amplification ratio of already-emphasizee directions of pretrained weights. In this research question, we investigate the relationship between W and AW. We answer this question by projecting W onto the r-dimensional subspace of AW by computing U WV with U/V being the left/right singular-vector matrices of AW. We compare the Frobenius ||W]|| (UTWV]] To
norm between |AW]| and ||U WV| and compute the amplification factor (AF)
demonstrate that NoRM further inhibits the already-amplified directions of W be activated, also compute the reverse amplification factor (RAF) by projecting W onto the last d - r-dimensional subspace:
|AW]| We follow the setting of Hu et al. (2022 and draw two main conclusions
10 T.T_W Va--il
W
from Table 5 First, both methods amplify main Teatures that are already in W with negligible distinction. Second, NoRM reduces substantially the amplification factor of directions already emphasized in W. These two findings suggest that NoRM potentially maintains most contributing features LoRA parameters but further suppresses the amplification of already crready-emphasizeed features.
CONCLUSION
In this paper, we first use sufficient empirical experiments to reveal the general parameter redundancies among LoRA parameters, especially among model layers and specific modules. Built on these insights, we set up PREFT, a novel tuning framework that highlights the utilization of LoRA parameters by removing intrinsic redundancies without sacrificing training and inference efficiency, Under this framework, we propose NoRM to reserve the most contributing components of LoRA parameters which possess the highest subspace similarity with pre-trained weights, with a novel Sim-Search method. Experiment results show that NoRM achieves superior improvements various domains, verifying application in diverse domains by enhancing the capacity of high LoRA ranks.
ACKNOWLEDGMENTS
We thank the anonymous reviewers for their insightful comments and suggestions. This work is supported by National Key R&D Program China (No. 2022ZD0162101), National Natural Science Foundation of China (No. 62106140) and STCSM (No. 21511101 100, No. 22DZ2229005).
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Hervé Abdi and Lynne] Williams. Principal component analysis. Wiley intendisciplinary reviews: computational statistics, 2(4):433-459, 2010.
Ae@Meta. Llama model card. 2024.
#lob/main/M0DER CARD . mdl
Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad. A review on language models knowledge bases. arXiv preprint arXiv:2204.06003, 2022.
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos,
Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm technical report. arXiv preprint arXiv:2305. 2023.
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoe Le, et al. Program synthesis with large language models. arXiv preprint MIXv:2208.00732. 2021.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Dassarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Christopher Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv, aws//204.00522802 0056622 2022. URL http:: semant csccolocle aar. orggor///ooo Corp//oruss./ 248118875
Srinadh Bhojanapalli, Ayan Chakrabarti, Andreas Veit, Michal Lukasik, Himanshu Jain, Frederick Liu, Yin- Wen Chang, and Sanjiv Kumar. Leveraging redundancy in attention with reuse transformers. arXiv preprint I 10.06821, 2021.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Yupeng Chang, Xu Wang, Jindong Wang. Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. survey on evaluation of large language models. ACM Trans. Intell. Syst. Technol. 15 (3), mar 2024. ISSN 2157-6904, doi: 10. (0.1145/364128..
3641 289
Junying Chen, Xidong Wang, Anningzhe Gao, Feng Jiang, Shunian Chen, Hongbo Zhang, Dingjie Song. Wenya Xie, Chuyi Kong, Jianquan Li, et al. Huatuogpt-ii, one-stage training for medical adaption of llms. arXiv preprint 1rX07:2311.09774, 2023.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint 107.v:2207.03374, 2021.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani. Siddhartha Brahma, et al. Scaling iestruction-finttune language models. Journal of Machine Learning Research, 25(70):1-53 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jemry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110. 14168, 2021.
Fahim Dalvi, Hassan Sajjad, Nadir Durrani, and Yonatan Belinkov. Analyzing redundancy pre- trained transformer models. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.),
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4908 4926, Online, November 2020. Association for Computational Linguis- URL https: battps:/accaate http///aaattooly.orrg//////
tics. doi: 10. http////11/222.mmmmll1.e
lemn 11lpp-mma- ma i 398
11

---

Published as a conference paper at ICLR 2025
ziqing Fan, Siyuan Du, Shengchao Hu, Pingjie Wang. Li Shen, Zhang Ya, Dacheng Tao, and Yanfeng Wang. Combatting dimensional collapse in Ilm pre- pre-training data via diversified file selection. In The Thirteenth International Conference on Learning Representations, 2025.
Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang. Mixture-of-loras: An efficient multitask tuning method for large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation 2024), pp. 11371-11380 2024.
Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does fine-tuning Ilms on new knowledge encourage ballucinations? arXiv preprint 24t1::2005005044, 2024.
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review, 53(2): 217-288, 2011.
Soufiane Hayou, Nikhil Ghosh, and Bin Yu. LoRA+: Efficient low rank adaptation of large models. In Forty-first International Conference on Machine Learning, 2024. URL[sttpa://opeereeeiee
net /
Shwai He, Guoheng Sun, Zheyu Shen, and Ang Li. What matters in transformers? not all attention is needed. arXiv preprint arXiv:2406. 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Confer- ence on Learning Representations, 2021a. URL
net. forum?id=
d7кB jmI 3CmQ,
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Mcasuring mathematical problem solving with the MATH dataset. In Joaquin Vanschoren and Sii-Kit Yeung (eds.). Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurlPS Datasets and Benchmarks 2021, December 2021, virtual, 2021b. URL wwtpc///ppp//.....1 round2. html
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. Advances in neural information processing systems, 2021c
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pp. 2790-2799. PMLR, 2019.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. International Conference on Learning Representations, 2022. URL
htt.wnee/foee forum?
id=nZeVKeeFyfF
Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A Smith, Iz Beltagy, al. Camels in a changing climate: Enhancing Im adaptation with tulu 2. arXiv preprint arXiv:2311.10700 2023.
Shuyang Jiang, Yusheng Liao, Ya Zhang, Yu Wang, and Yanfeng Wang. Taia: Large language models are -ut-o--distiiiutiio data learners. arXiv preprint 2rXiv:2405.20192, 2024a.
Ting Jiang, Shaohan Huang, Shengyue Luo, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, Qi Zhang, Deqing Wang, et al. Mora: High-rank updating for parameter-efficient fine-tuning. arXiv preprint 1xXxx22405.12130 2024b.
Damjan Kalajdzievski. Scaling laws for forgetting when fine-tuning large language models. arXiv preprint #xXi:2201.006000 2024.
12

---

Published as a conference paper at ICLR 2025
Virginia Klema and Alan Laub. The singular value decomposition: Its computation and some applications. IEEE Transactions on automatic control, 25(2):164-176, 1980.
Divyanshu Kumar, Anurakt Kumar, Sahil Agarwal, and Prashanth Harshangi. Increased II vulnera- bilities from fine-tuning and quantization. arXiv preprint arXiv:2404.04392 2024.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen- tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 3045-3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. (0.18653/v1/) 1tt33//w/2221.ceem http:////2...emmmmaim.... .em.lp-mmin.2.43. URL logy www.ay/orrg/2/.....gg emnlp ma in. 243
Xiang Lisa Li and Perey Liang. Prefix-tuning Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li. and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11hh International Joint Conference on Natural Language Processing ( (Volume 1: Long Pupers), pp. 4582 4597, Online, August 2021. Association for Computational Linguistics. doi: 10. 1/0053////22222c--oooooo
URL https:://ccaatthitt org 2021 acl long. 353
Yusheng Liao, Shuyang Jiang, Yanfeng Wang, and Yu Wang. Medcare: Advancing medi- cal lims through decoupling clinical alignment and knowledge aggregation. arXiv preprint arXiv:2406.1 1124406.77488 17484, 2024.
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214 4252, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18663/11/2022.acl-ooom 229. URL https: htttttsss://caatttoootaoo http:///cccaattooo...o.oooo org.oy2.. ac acl-long
229
Zhaojiang Lin, Andrea Madotto, and Pascale Fung. Exploring versatile generative language model via parameter-efficien transfer learning. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguisties: EMNLP 2020, pp. 441-459, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020 http://w////...nnnnnnneeemm//// findings-emnlp.41 URL
/ttps:://aclantho ogy. org/2020, f and ngg- emn p. 41
Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. Logiqa: A challenge dataset for machine reading comprehension with logical reasoning. arXiv preprint arXiv:2007.08/ 2020.
Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36, 2024.
Qidong Liu, Xian Wu, Xiangyu Zhao, Yuanshao Zhu, Derong Xu, Feng Tian, and Yefeng Zheng. Moelora: An moe- based parameter efficient fine-tuning method for multi-task medical applications. arXiv preprini arXiv: 10. 121.0//88.0 18339, 2023a.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 61-68, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 220553////02222ccsshotttt URL nttps::/aclantthl..om -cc 8hort.
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang. and Jie Tang. Gpt understands, too. Al Open, 2023b. ISSN 2666-6510. doi: https///6..orr//000116.... 2023.08.012. URL https: www.sciencedirect. com http:///weettccoooooe////rrrrrra 5266665102200011
13

---

Published as a conference paper at ICLR 2025
Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arfv:23388009583, 2023a.
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint 08Xv:2208.008777, 2023b.
Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. Shortgpt: Layers in large language models are more redundant than you expect. arXiv preprint arXhv:2403.03853, 2024.
Fanxu Meng. Zhaohui Wang, and Muhan Zhang. PISSA: Principal singular values and singular vectors adaptation of large language models. In The Thinty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: http:///ppppeeveeeeee ewereeiiww... net forum? ? 1
62BHIEtdP4
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.00 07843,, 2016.
OpenAL. GPT-4 technical report. CoRR, abs/2303.087720 2023. doi: 10.488500a2Xiv.2333..7777
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080-2094, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v//0021 mnaccl-main 168. htthttss::: http://s//aaaaathhhaaceoo
wrw.or/g///oooo. 2021 www.clmmain.nn..mm
Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4, arXiv preprini arXiv:2304.033277 2023.
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapter- Fusion: Non-destructive task composition for transfer learning. In Paola Merlo, Jorg Tiede- mann, and Reut Tsarfaty (eds.), Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguisties: Main Volume, pp. 487-503, Online, April 2021. Association for Computational Linguistics. doi: 10.18653/w1/2021 sewll- -ann-- URL https://acll http::::/aallathhlooggo
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint wxXv:231000069)) 2023.
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Learning multiple visual domains with residual adapters. Advances in neural information processing systems, 30, 2017.
Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le. Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether thain-of-thought can solve them. arXiv preprint (tt.::2210009266. 2022.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume (Long and Short Papers). pp. 4149-4155 Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N1D3 1421, URL URI.https: /otg 9-121
14

---

Published as a conference paper at ICLR 2025
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. Openmathinstruct-:: A 1.8 million math instruction tuning dataset. arXiv preprint arXiv: Arxiv- 2402.10176, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprini arXiv:2302. 13971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.08 00288888............. 2023b.
Hanqing Wang, Zeguan Xiao, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. Milora: Harnessing minor singular components for parameter-efficient Ilm finetuning. arXiv preprint AxXiv24006.09444, 2024.
Xiao Wang. Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. Orthogonal subspace learning for language model continual learn- ing. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 10658-10671, Singapore, December 2023, As- sociation for Computational Linguistics. doi: 10.18653/ 1/tp:////222////// www.oo23...niing--eo -endddnggeemml.11.... URL https: http:::/acclanthooooo...oo org wttp://2....iinnnneneeeeeeeeeeee 715/
Zhengbo Wang, Jian Liang, Ran He, Zilei Wang, and Tieniu Tan. LoRA-pro: Are low-rank adapters properly optimized? In The Thirteenth International Conference on Learning Representations, 2025. URL
net/ wwtp::/wteeeettteeteteee/eee for.mmid--
Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and LINGMING ZHANG. Magicoder: Em- powering code generation with OSS-instruet. In Forty-first International Conference on Machine Learning, 2024. URL https: /tttpp::/opperevvaee
net. forum? ?id XUeoOBia3x
Chaoyi Wu, Weixiong Lin, Xiaoman Zhang. Ya Zhang, Weidi Xie, and Yanfeng Wang. Pme-llama: toward building open-source language models for medicine. Journal of the American Medical Informatics Association, pp. ocae045, 2024.
Shih yang Liu, Chien-Yr Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting -ing Cheng, and Min-Hung Chen. DoRA: Weight-decomposed low-rank adaptation. In Forty-first International Conference on Machine Learning, 2024.
net/
Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning. 2024a.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng YU, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. In The Twelfih International Conference on Learning Representations, 2024b. URL
Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprint arXiv:2309.05653 2023.
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng. Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efficien fine-tuning. The Eleventh International Conference Learning Representations, 2023, http:htts::/opnreeeeee wttp://wwwwee..o.eeeeeee
Id==q62uwRJJjy
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https::/pproeeeiingggggg/////// meurddiiiiinnnnnnssiiie..... neurips peepa//sseeeerrrreoeeepeeeerrrrr filess/paper22015/ file:2DcceB5S17333333accbbbeeeee ort.reaecccbbbbaecaeaaaaeaeeaerr
15

---

Published as a conference paper at ICLR 2025
LoRA
NORM
AWa
Wg
AFI
RAF
AW
Wg
AFI
RAF
JUWWV = [W,) = 74.0
1.16
34.26
1.88
0.03
0.33
21.99
1.83
0.008
|AW 2.1875
|| |l = 0.6133
Table 5: The Frobenius norm of U Wg where U and V are the left/right top r singular vector directions of either AW, and w. The weight matrices are taken from the 16th layer of Llama3-8B. "AF" and "RAF" indicate the amplification factor and reverse amplification factor, respectively.
General
Math
Code
Param Config
Method
BBH
MMLU
TydiQA
GSM8k
MATH
HvmanEval (+)
MBPP (+)
Avg.
Base
40.68
59.18
40.90
77.79
31.06
53.00 (48.80)
71.70 (60.60)
53.75
LoRA
34.43
63.89
43.72
80.82
29.06
53.00 (48.20)
66.90 (56.
52.93
88,=16
NORM
41.31
64.43
45.77
81.80
32.12
59.10 (54.90)
70.90 (59.50)
56.65
LoRA
34.88
61.45
43.32
81.20
28.66
50.60 (45.70)
68.50 (58.70)
52.56
1=6,0=32
NORM
39.79
63.70
46.43
81.58
32.74
61.00 (56.70)
69.80 (58.70)
56.72
LoRA
36.52
61.03
46.80
79.00
29.90
$2.40 (48.80)
68.80 (58.50)
53.53
1=32,==64
NoRM
41.64
62.44
46.28
82.56
32.84
62.80 (58.50)
72.20 (60.60)
57.76
LoRA
36.38
63,42
43.56
80.21
29.27
48. (44.50)
67.20 (57.40)
52.30
1=4,,=118
NORM
43.11
64.61
46.21
82,56
34.02
64.00 (59.80)
70.40 (59.30)
58.22
LoRA
36.18
63,77
46,99
80.74
30.48
48. 80 (45,10)
68. 80 (59.00)
53.32
==26
NORM
42.30
64.86
45,30
83.17
32.92
63.40 (58,50)
72.80 (61.40)
58.29
Table 6: Full results on the analysis on tunable parameters. The tunable parameters are increased incrementally to validate NORM's behavior and effectiveness.
A RELATED WORK
Parameter efficient fine-tuning Full fine-tuning effectively adapts large language models to downstream tasks (Fan et al. 2025) but requires substantial computational resources as model size and task numbers increase. To mitigate this, Parameter-Effion been introduced. These methods freeze the base language models and modify only a minimal number of parameters during training, achieving comparable or even superior performance with limited fine-tuning data. Among these methods, Adapter-Tuning (Rebuffi et al. 2017 Houlsby al. 2019
Fine-Tuning (PEFT) methods have
Lin et al. 2020 Pfeiffer et al. 22021), Prefix-tuning (Li & Liang (021)) Prompt-Tuming (Lester et al. 2021). P-Tuning (Liu et al. (023b) and P-Tuning-v2 (Liu et al. 2022) were proposed reduce line-tuning costs before the era of LLMs. However, these methods introduce additional priors and significant inference latency. In contrast, Low-Rank Adaptation (LoRA)(Hu et al. 2022) and its variant, Beight-Decomposdd Low-Rank Adaptation (DoRA) Liu et 20044) take a different approach. LoRA updates original parameters with two low-rank matrices without assuming any specific task or architecture, eliminating inference latency by merging back these two matrices to the original weight. DoRA extends this by incorporating weight decomposition into magnitude and direction, achieving performance comparable to full fine-tuning. LoRA+ (Hayou et al. 2024) proposed to adopt adaptive update strategies for each low-rank parameter. MoRA (Jiang et al. (024b) proposed to incorporate square high-rank tunable parameters to achieve both efficiency and high rank. Nonetheless, most LoRA-like methods require complex hyperparameter settings, which hinders their generalization.
Parameter redundancies Previous works have verified that in pretrained language models, pa- rameters contain sufficient redundancies. [Dlvi et al. (2020) show that redundancies exist layers and neurons and vary among downstream tasks. Bhhjanapalli et al. (2021) demonstrate that there exist substantial redundancies among transformer multi-head attention modules. He et al. (2024) empirically verify the redundancies of LLMs within self-attention and multi-layer perceptron (MLP) modules and leverage these redundancies to speed up inference, MMen et al. (2024) also leverage such layer redundancies to boost the inference speed yet sacrificing limited performance. However, these works only analyze the redundancies among pretrained LLMs, but less focus on fine-tuned delta parameters, especially in so-called low-rank parameters. Yu et al. (002a) leverage the delta parame-
16

---

Published as a conference paper at ICLR 2025
ter redundancies to perform model merging without performance degradation. Jiang al. (2024a) attempt to remove the delta feed-forward low-rank parameters to adapt LLMs to out- -o-mann tasks. In this work, intend unveil the parameter redundancies among delta low-rank parameters and leverage such redundancies to improve fine-tuned models with more fine-grained practice.
Limitations and drawbacks of fine-tuning Fine-tuning is a common method for adapting large language models (LLMs) various downstream tasks. However, it comes with significant drawbacks, including hallucination, harmful outputs, catastrophic forgetting, and safety concerns. Geekmaan al. (2224) noted that fine-tuning can lead models to produce factually inaccurate responses, as the training process encourages the generation of information not grounded in the model's pre-existing knowledge. Additionally, supervised fine-tuning for specific tasks often results in catastrophic forgetting of the initial alignment (Luo et al. 2023b) and creates trade-offs between helpfulness and harmlessness (Bai et al. 2022). .. Kumar et al. (2024) also highlighted that fine- ane-tuning significantly reduces the resistance of LLMs to jailbreaking, thereby increasing their vulnerability. Even when carefully curated fine-tuning datasets are used, Qi et al. (203)) demonstrated that well-aligned LLMSs often become less safe and more prone to harmful behavior, with issues exacerbated by red red-teaming in the tuning data. In contrast, NoRM addresses many of these drawbacks while enhancing helpfulness through fine-tuning. By focusing on retaining the most similar delta components relative to the base weights, NORM offers a robust solution to the challenges associated with traditional fine-tuning approaches.
B LIMITATIONS
We notice that NORM gains smaller performance gains with enlarged LoRA ranks. We hypothesize that although NoRM removes noisy components of updated LoRA parameters, it still cannot fully separate the redundant parts, which causes the distracting parts to interfere with downstream performances. Besides, we currently only apply NoRM the dnferencee-reproceeee introduction of NoRM to the training stage may support more convenient application of NoRM and further improvements over LoRA and TAIA.
stage. The
C FUTURE WORK
NoRM succeeds in discarding noisy components of LoRA parameters by selecting the most contribut- ing parts through Sim-Search. To enlarge the application of NoRM. the next research direction is to extend NoRM to the full fine-tuning scenario. Just as Yu et al. (2024a) indicates, the full fine-tuned parameters also contain extra useless parameters that can smartly reduced by appropriate PREFT methods. Besides, adaptive maintaining strategy instead of the coarse separation can improve the downstream application of NORM, by picking the most appropriate components of specific prompts automatically. We hope our work can provide inspiration how improve the parameter utilization of LLMs to bootstrap the performance LLMs on downstream tasks.
D EXPERIMENT DETAILS
D.1 PEFT SETTINGS
We add a LoRA module for each linear layer except the language model head and embed- ding layer, resulting in the following target modules: [q-proj, k-proj, v-proj, o-projj gate.proj, up-proj, down.proj]. The LoRA a of each experiment is set to twice of the LoRA rank suggested by Hu et (2022).
D.2 EVALUATION DETAILS
Our evaluations contain different types of metrics, including Exact Match (EM) and Multiple Choice Accuracy (Acc). For EM metric, extract the contents followed by The answer is to reduce evaluation biases. For different datasets, we adopt different evaluation prompts after the original problem description:
17

---

Published as a conference paper at ICLR 2025
1. MATH (Hendrycks et al. 2021c), GSM-8K (Cobbe et al.
2021), BIG-Bench Hard (Suzgun
et al. 2022) (BBH). SVAMP (Patel et 2021): Please format the final answer at the end of the response as: The answer is {answer).
HumanEval (+) Chen al. 2021). MBPP (+) (Austin et al.
2021 Liu al. 2022). Truth-
fuiQA (Lin et al. 2022): None.
3. MMLU (Hendrycks al.; (021a), LogiQA (Liu et al. 2020): Please answer with
option letter directly, do not output other information.
4. CommonsenseQA (Talmor et al.) 2019): Let's think step by step. Please
format the final answer at the end of the response as: The answer is {answer).
We use greedy decoding to maintain that all results are reproducible.
D.3 TEST SETS DESCRIPTION
We here describe the used evaluation sets:
1. MATH (eeddrycks et al. 2021c) is a collection of challenging competition mathematics problems containing 5,000 problems the test set. Each problem MATH has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations.
2. GSM-8K (Cobbe et al.) 2021) 1) is collection of 1,273 math-reasoning problems with varying difficulty. Each problem requires the model to conduct single or multi-hop reasoning to derive the correct answer.
SVAMP (Patel et al. 2021) are much simpler datasets compared to MATH, which both test models' math problem-solving ability. It contains 1,221 problems which are all solvable with one or two simple equations.
4. BIG-Bench Hard (Suzgun et al. 2022) (BBH) is collection of 23 challenging tasks from BIG-Bench. The 6,511 problems are the tasks for which prior language model evaluations did not outperform the average human-rater.
5. CommonsenseQA (Talmor al., 2019) is test models' ability to answer questions using only the parameterized knowledge instead of the context knowledge. It contains 1,000 problems sourced from ConceptNet (Speer et al. (017).
LogiQA (Liu et al.) 2020) collects questions about natural language inference (NLI) and requires models to infer the conclusion based provided premises. It contains 653 problems for both English and Chinese subsets.
7. TruthfulQA (Lin et al.] (2022) is for testing models' ability to produce truthful answers. The 817 questions that span 38 categories benchmark models' refusal to generate false answers like humans.
2021a) measure LLM's multitask accuracy, which contains 14,421 problems. The test covers 57 tasks including elementary mathematics, US history, computer science, law, and more. To attain high accuracy on this test, models must possess extensive world knowledge and problem mooblem-solving ability.
MMLU (Hendrycks
Evaa (Chen al. 2021 contains 164 human-checked python-oriented programming problems to evaluate models' code generation ability. The HumanEval+ (Liu et al. 2024) version creates more comprehensive test cases to produce more fair evaluation result.
10. MBPP (Austin et al., 202I) contains 500 python coding problems, where each problem requires the model to generate correct python functions. The MBPP+ (Liu et al.) (024)) version creates more comprehensive test cases to produce a more fair evaluation result.
E FURTHER EXPERIMENTS
COMPARISON WITH MORE PEFT METHODS
In this section, we add more PEFT baselines to further validate the effectiveness of NoRM. We select LoRA-pme prro (Wang et al. 2025), which modifies LoRA's training mechanism, and two methods adopting the philosophy of SVD. One method, PiSSA (Meng al. 2024), fixes the minor components
18

---

Published as a conference paper at ICLR 2025
Data Size
METHOD
BBH
MMLU
TydiQA
CQA
TruthfulQA
GSM-8K
LogiQA
Average
Base
40.68
59.18
40.90
71.83
63.28
77.79
41.17
56.40
LoRA
34.88
63.35
49.18
76.41
45.78
74.75
37.79
54.59
IK
TAIA
19,92
62.18
46.12
76.17
53.00
77.33
42,40
53.87
NoRM
36.12
62.11
45.77
76.74
52.02
76.12
43.47
56.05
LoRA
36.03
64.72
43.14
75.18
49.69
70.89
36.56
53.75
10K
TATA
26.62
64.41
46.46
76.66
52.14
77.33
39.32
54.71
NORM
39.69
64.85
48.63
76,25
52.75
75.97
42.55
57.24
LoRA
33.67
$7.29
40,42
74.86
45.90
69.83
33.03
50.71
50K
TATA
36.63
61.69
44.79
76.41
52.75
16.19
39.78
55.47
NORM
40.06
62.97
43.77
76.49
52.14
76.42
42.24
56.30
LoRA
36.38
63.42
43.56
77.64
48.10
72.02
38.40
54.22
100K
TAIA
35,83
62.02
47.26
76.66
54.22
77.10
37.94
55,86
NORM
43.11
64.61
46.21
77.72
54,10
77.71
43.47
58.13
LoRA
37.01
60.70
45.09
74.28
50.80
67.85
39.17
53.56
330K
TAIA
41.68
62.63
45.69
74.53
57.53
76.65
39.78
56.93
NORM
41.42
63.47
45.16
76.17
58.26
78.09
43.16
57.96
Table 7: Full experiment results of data scaling of NoRM. NoRM consistently maintains the leading performance across all data sizes of TŪL. V2.
Method
BBH
MMLU
TydiQA
Commonsense(A
TruthfulQA
GSM8K
Logiqa en
Average
Base
40.68
59.18
40.90
71.83
63.28
77.79
41.17
56.40
LoRA
36.38
63.42
43.56
77.64
48.10
72.02
38.40
54,22
LoraPro
37.31
61.72
45.21
74.77
47.49
63.38
33.79
51.95
PISSA
33.16
59.15
44.37
75.92
48.71
69.98
38.71
52.86
AdaLora
29.80
59.27
50.59
73.30
48.71
71.87
42.40
53.71
TAIA
35.83
62.02
47.26
76.66
54.22
77.10
37.94
55.86
NORM
43.11
64.61
46.21
77.72
54.10
77.71
43.47
58.13
Table 8: Comparison of NoRM with more PEFT baselines, especially SVD-based methods, including AdaLora and PiSSA. NoRM performs best among these PEFT methods with Llama3-8B as the base model.
of base weights and principle components initialize LoRA parameters and the other baseline AdaLora (Zhang et al. 2023) adaptively allocates the parameter budget among weight matrices. We select Llama3-8B the base model and choose TULU V2 the training data. Table depicts the comparison of NoRM with these PEFT baselines. These baselines consistently underperform vanilla LoRA when taking an instruction-tune tuning model the backbone, while PREFT based NoRM show non-negligible improvements. This also indicates that although NoRM leverages SVD to operate parameters, SVD tailored at redundancy reduction is more advanced than either the initialization of parameters update budgets.
E.2 UTILIZATION IN CONTINUAL LEARNING
As NoRM reduces redundant parts of the fine-tuned parameter with the largest subspace overlap with pre-trained parameters, it intrinsically avoids severe mnowledg-fforgetting problems due to internal distribution shifts. As result, NoRM M has significant resistance against catastrophic forgetting dilemma, which leads to performance improvement over vanilla LoRA tuning under continual learning scenarios. We perform experiments following the experimental setup of OLoRA (Wang et al. 2023). Specifically, we adopt the specified training order for continuous learning and evaluate our approach on the StandardCI. (Zhang et al.) 2015) benchmark. We use LLLam3 8B as the base model. The training orders are as follows: (1) Order 1: dbpedia + amazon - yahoo > ag; (2) Order 2: dbpedia- - amazon + ag - yahoo; (3) Order 3: yahoo + amazon - ag + dbpedia. This configuration ensures consistency with prior work and allows for fair comparison of performance across different orderings. We adopt Accuracy as the evaluation metric and show the performance below in Table We observe a substantial improvement This enhancement stems from NoRM's ability to preserve the LoRA components most aligned with the pre-trained weights, effectively mitigating the issue of catastrophic forgetting. By prioritizing
standard LoRA tuning.
19

---

Published as a conference paper at ICLR 2025
0.20
1.00
Noraaized Acc
Search Step
Model
Orderl
Order2
Order3
Average
0.05 0.050:30000
LoRA NoRM
65.34
74.56 80.08
70.44 78.76
70.11
0.50
78.88
79.24
0.25
Table 9: setting.
NoRM in the continual learning
Search Start
Figure 6: Normalized performance Hu- manEval (+) with varying search steps and search ranges. Lower search steps and wider search ranges generally find more similar components and perform outstandingly.
these key components, NoRM ensures better retention of prior knowledge throughout the continuous learning process.
E.3 LARGE SEARCH RANGES AND FINE-GRAINED SEARCH STEPS BRING FURTHER IMPROVEMENTS.
To rationalize the hyperparameter settings of Sim-Search, including the search step 7 and search start 8, we perform a grid search on these two parameters. We use normalized accuracy described in $2.2 as the evaluation metric and take HumanEval (+) as the evaluation set for simplicity. The search start s is searched in {1, 2, (1,2...., 8} and searched in {0.05, 0.10, 0.15, 0.20J.
Figure S presents that with larger search ranges (lower search starts), NoRM gains further improve- ments. Meanwhile, relatively lower search steps generally bring higher results NORM. Although the setting& 2, 0.15 brings the best performance the HumanEval (+) dataset, still take the setting s = 1, r = 0.1 for all experiments as this setting is already acceptable enough and we leave the intelligent selection of search parameters for future work.
20


Reviews:
[{"review_id": "IaeYbLvwPa", "reviewer": "Reviewer_qqme", "summary": "This paper observes the phenomenon of hallucination caused by the large redundancy in PEFT (Parameter-Efficient Fine-Tuning) and attempts to reduce the redundancy in PEFT parameters from the perspective of rank through random singular value decomposition. It then designs a SIM-SEARCH method to ensure that the selected parameter subspace is as close as possible to the distribution of pretrained parameters, thus avoiding parameters that could damage the model's capability.", "questions": "Does increasing the rank to achieve better accuracy hold significant value for your work? Why is it specifically emphasized in such an important place like Figure 1?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The writing of this paper is particularly good; the story is clearly told, without over-claiming its contributions, and the logic is very clear.\n2. All designs in this paper are centered around a clear goal — reducing redundant PEFT parameters that have large differences from the network distribution. Although this insight did not originally come from this paper, the paper conducted meaningful preliminary experiments to validate it.\n3. The experiments in this paper achieved highly competitive results across different tasks. In some tasks where LoRA underperformed, the method in this paper surpassed full-rank fine-tuning, which is not easy to achieve.", "weaknesses": "1. It would be better with more mathematical explanations and analyses. How do the authors view the source of this redundancy in PEFT?\n2. Why not design a smaller or uneven rank during training rather than search for it afterward? Or, is there a way to make it more end-to-end?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yhe3DnmPkv", "reviewer": "Reviewer_etk1", "summary": "The paper introduces a comprehensive method to address the issue in LoRA, where redundant parameters in trained weights can lead to hallucinations in Large Language Models (LLMs). In pilot experiments, the authors investigated whether dropping weights could yield performance gains across different ranks and dropping strategies, providing evidence that redundant parameters induced by LoRA can indeed degrade performance. To tackle this problem, the authors propose a new fine-tuning framework called \"Parameter Redundancies Fine-Tuning,\" which excludes redundant parameters when merging LoRA weights with the base model. They introduce an inter-shearing method named \"Noise Reduction with Reserved Majority,\" which decomposes LoRA weight updates using Random SVD and preserves their main components based on a novel Sim-Search method. This method aims to maximize the subspace similarity between delta LoRA weights and the original weights of the model.", "questions": "In the \"weakness\" section, I pointed out some non-standard mathematical notations. Are these abbreviations or special notations? An explanation of the mathematical notations would be appreciated.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper provides ample illustrations to present experiment results and the method pipeline. The authors conducted extensive experiments, demonstrating that the proposed NoRM method generalizes well across various models and datasets. The ablation study further supports the plausibility of searching for the rank based on subspace similarity. Since the paper introduces an improvement for merging the widely-used LoRA method with model weights, the method can significantly benefit individuals who need to train Large Language Models (LLMs) with LoRA.", "weaknesses": "Some equations in the paper deviate from standard notation. For instance, in lines 188 to 190, what follows \"such that\" is typically a statement (e.g., \"we choose $\\Delta W$ such that $\\Delta W = \\ldots$), but in the paper, it is an \"argmax\" result. Additionally, in lines 297 to 298, there appears to be a set-builder notation, but $\\tau \\cdot s, (\\tau + 1) \\cdot s, \\ldots, r$ does not seem to be a predicate over $c$.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WzbqOh90Ou", "reviewer": "Reviewer_Qj9z", "summary": "The paper introduces a novel fine-tuning methodology named NORM, aimed at reducing noise and parameter redundancy in the fine-tuning of large language models. Central to this approach is the Parameter Redundancies Fine-Tuning framework, which optimizes the parameter efficiency of LLM fine-tuning by minimizing redundancies. NORM advances this framework by employing a technique that decomposes parameters using random singular value decomposition and selects the most significant components through a Sim-Search method, which evaluates subspace similarity with the base model weights. This method has demonstrated superior performance over traditional techniques across various downstream tasks, such as instruction tuning, mathematical reasoning, and code generation, offering a significant advancement in the field of LLM fine-tuning.", "questions": "1.Can the authors provide a more detailed analysis of the computational resources required for implementing NORM compared to traditional fine-tuning methods? This could help in assessing the practicality of NORM in resource-constrained environments.\n2.The method's performance may depend on the selection of specific hyperparameters, such as the rank in SVD. Could the authors discuss how sensitive NORM is to these parameters and possibly include a sensitivity analysis or guidelines for selecting optimal hyperparameters?\n3.Since the paper does not discuss long-term impacts, could the authors provide insights or preliminary results on how NORM performs in continuous learning setups or multiple iterations of fine-tuning? Understanding the stability and performance over time could significantly add to the method's robustness.\n4.Are there recent advancements or other fine-tuning methodologies that were considered but not included in the comparison studies? If so, what was the reason for their exclusion, and could incorporating comparisons with these methods change the perception of NORM’s effectiveness?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper on NORM (Noise reduction with Reserved Majority) and the PREFT framework stands out for its originality, quality, clarity, and significance. It introduces a novel methodology combining random singular value decomposition (SVD) with Sim-Search for selective parameter retention, offering a fresh perspective on fine-tuning large language models. The research is rigorously validated across various tasks, demonstrating its robustness and relevance. Structurally, the paper excels in clarity through well-organized sections and informative visuals, making complex concepts accessible and reproducible. Significantly, NORM addresses the critical challenge of efficiently tuning large models, potentially influencing future research in machine learning optimization strategies, thereby marking a noteworthy contribution to the field.", "weaknesses": "1.The paper lacks comprehensive comparisons with the latest advancements in parameter-efficient fine-tuning methods, which could more precisely establish NORM's standing in the research landscape.\n2.There is insufficient analysis of the computational demands and efficiency of NORM, including comparisons of resource usage and execution time against other fine-tuning methods.\n3.The paper does not discuss the potential long-term effects of applying NORM in continuous learning environments where models undergo multiple fine-tuning cycles.\n4.There is no detailed exploration of how NORM's performance is influenced by the choice of hyperparameters, such as the ranks used in SVD, which could affect its robustness and ease of optimization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hn9gffTlYq", "reviewer": "Reviewer_FpRJ", "summary": "The paper introduces a parameter-efficient fine-tuning framework called Parameter-Redundant Fine-Tuning (PREFT), focusing on reducing noise in LoRA-tuned models through redundancy management. It proposes Noise Reduction with Reserved Majority (NORM), which enhances LoRA by isolating valuable components while removing redundant parts using singular value decomposition. NORM employs Sim-Search to identify the most relevant parameters for fine-tuning by analyzing their similarity with pre-trained weights, reducing hallucinations that degrade model performance.", "questions": "Please refer to the weakness part.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. While many concurrent papers focus on pushing the limits of parameter efficiency, this paper delves into another interesting aspect of LoRA: eliminating redundancies to prevent hallucinations.\n2. Experimental results show that this method yields promising outcomes.", "weaknesses": "1. The methodology section is somewhat confusing due to the arbitrary use of notations. For example, in Line 297, does the starting value $s$ have any connection with $s_1, s_2$ mentioned in Line 155? Also, what does $\\mathbf{U}_{cr}$ represent in Equation 14?\n2. Although the LLM fine-tuning tasks are practical and valuable, the evaluation methods are not very standardized, and therefore the results are not entirely convincing. I recommend that the authors conduct experiments on both the GLUE[1] and VTAB[2] benchmarks to further validate the effectiveness of the proposed method.\n\n[1] GLUE. https://gluebenchmark.com/\n\n[2] Visual Task Adaptation Benchmark. https://google-research.github.io/task_adaptation/\n\n3. Having conducted experiments in this area, I have found that the evaluation methods used for fine-tuned LLMs are often unconvincing （e.g., some evaluations rely on asking ChatGPT for a rating). Therefore, I strongly recommend that the authors perform experiments on established benchmarks like GLUE and VTAB to provide more robust evaluations. However, as anticipated, their method did not excel on GLUE. Additionally, the authors offer only intuitive reasons for not conducting experiments on VTAB, which raises doubts about the actual effectiveness of their proposed method. Consequently, I cannot support acceptance of this work in its current form. (**Addressed**)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)