Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation
Authors: Shen Yuan
Haotian Liu
Hongteng Xu
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Orthogonal fine-tuning
Householder reflection
Conditional text-to-image generation
Large language models
Primary area: deep_learning_architectures
TL;DR: We proposed a new model adaptation method based on Householder reflections, bridging low-rank and orthogonal adaptation and achieving promising performance in NLP, CV, and Math reasoning tasks.

Abstract: While following different technical routes, both low-rank and orthogonal adaptation techniques can efficiently adapt large-scale pre-training models in specific tasks or domains based on a small piece of trainable parameters. In this study, we bridge the gap between these two techniques, proposing a simple but effective adaptation method based on Householder reflections. Given a pre-trained model, our method fine-tunes its layers by multiplying each frozen weight matrix with an orthogonal matrix constructed by a chain of learnable Householder reflections (HRs). This HR-based orthogonal fine-tuning is equivalent to an adaptive low-rank adaptation. Moreover, we show that the orthogonality of the reflection planes corresponding to the HRs impacts the model capacity and regularity. The analysis motivates us to regularize the orthogonality of the HRs, leading to different implementations of the proposed Householder reflection adaptation (HRA) method. Compared with state-of-the-art methods, HRA achieves superior performance with fewer learnable parameters when adapting large language models and conditional image generators. The code of the experiments is available at https://github.com/DaShenZi721/HRA, and the method has been merged into the [PEFT](https://github.com/huggingface/peft) package.

Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation
Shen Yuan'
Haotian Liul+
Hongteng Xu
Gaoling School of Artificial Intelligence, Renmin University of China BBejjing Key Laboratory of Big Data Management and Analysis Methods (shenyuan721, haotianliu, hongtengxu @ruc. edu.
Abstract
While following different technical routes, both low-rank and orthogonal adaptation techniques can efficiently adapt large-scale pre- pre-training models in specific tasks or domains based on a small piece of trainable parameters. this study, we bridge the gap between these two techniques, proposing a simple but effective adaptation method based on Householder reflections. Given a pre-trained model, our method fine-tunes its layers by multiplying each frozen weight matrix with orthogonal matrix constructed by chain learnable Householder reflections (HRs). This HR-based orthogonal fine-tuning is equivalent an adaptive low-rank adaptation. Moreover, we show that the orthogonality of the reflection planes corresponding the HRs impacts the model capacity and regularity. The analysis motivates to regularize the orthogonality of the HRs, leading to different implementations of the proposed Householder reflection adaptation (HRA) method. Compared with state-of-the-arr methods, HRA achieves superior performance with fewer learnable parameters when adapting large language models and conditional image generators. The code the experiments is available at atsttps: / // thub. c DaShen2i721/HRg and the method has been merged into the PEFT package.
Introduction
In recent large foundation model competitions, "Scaling Laws" [22
18 39 motivate researchers
to increase model size continuously, which brings significantly improved model capabilities in understanding, generation, reasoning, and generalization but with more and more unbearable model adaptation costs. For instance, the GPU memory for fine-tuning a LLaMA-653 -6BB model with 16bit precision exceeds 780GB ()] The adaptation of image generative models (like the ControlNet in [60] did) may suffer the same issue when applying large vision foundation models as backbones. Consequently, fine-tuning large foundation models efficiently for adapting various downstream tasks has become a challenge in practice.
To overcome the above challenge, Parameter-Efficce Effiiiien Fine-Tuning (PEFT) methods [53 provide promising solutions, which aim to reduce the trainable parameters and memory consumption of fine-tuning while maintaining even improving model adaptation performance. Among various PEFT methods, the adapter- based fine-tuning [20, (37] attracts lot because it only inserts limited trainable parameters into existing models during fine-tuning but without adding extra complexity or overhead in the inference phase. Currently, given parameter matrix of pre-trained model, i.c., W < Rd x there are roughly two strategies implementing the adapter-based fine-tuning, The mainstream strategy
is applying Low-Rank Adaptation (LoRA) 20 and its variants [9] 21 25
31
33
35
48
54
60
part of this work done when Haotian Liu was affiliated with the Beijing Institute of Technology. Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Regularity
X 20,uj
X
2
u,
Capacity
X
200
OC
Strength of Orthogonality - U U
U1
u1
2
u,
U2
u
(a) The scheme our Householder reflection adaptation method
HRA
Method
Param, Ratio
GSM8K
MATH
Average Score (96)
BOFT
LLaMA2-7B
14.6
2.5
OFT
LoRA r=32
0.25%
50.2
7.8
OFT b=16
0.13%
50.1
8.4
AdaLoRA
BOFT 6=8
0.12%
50.6
8.6
PISSA
4.75%
53.1
HRA
r=8, A=0
0.03%
47.1
P-Adapler
LoRA
HRA ym16, AmD
0.06%
52.1
8.1
68.
HRA pm222,
0.12%
55.8
9.0
H-Adapter
HRA p2332
1322,amm Amoo
0.12%
52.8
9.2
HRA y=
0.12%
53.6
8.3
Number Trainable Paremeters (M)
HRA pm222
0.12%
(b) Performance on GLUE Benchmark
(c) LLM adaptation for mathematical reasoning
Figure 1: (a) An illustration of our HRA method. (b) Comparisons for various methods on GLUE benchmark (50 The x-axis corresponds the number of trainable parameters (M), and the y-axis corresponds the average various methods on the ratio trainable parameters and accuracy (%) when adapting LLaMA2-7B [46] in mathematical reasoning tasks.
score (%). (c) Comparisons
62 4 III modifying the weight matrix by adding trainable low-rank decomposition matrix, i.c.. W + AB, where A E pd and B IR"xd and r < min (d, doat) is the intrinsic rank of the modification AB, Another strategy is Orthogonal Fine-Tuning (OFT) [37 34 which multiplies the weight matrix with a structured orthogonal matrix R E Rdxd determined by limited trainable parameters, i.e., W R. Both strategies can reduce VRAM usage because they merely leverage limited trainable parameters and do not store the optimizer state of the original weight matrices. At the same time, they achieve encouraging model adaptation performance various vision and NLP tasks.
Essentially, LoRA hypothesizes that the additive modifications weight matrices are intrinsically low-rank. while OFT preserves the pairwise angles between neuron vectors and theoretically penalizes the discrepancy between pre-trained and fine-tune mun-t models. The difference between their principles prevents from building unified adapter-based fine-tuning framework. To bridge the gap between these two strategies, we propose a simple but effective adapter-based fine-tuning method called Householder Reflection Adaptation (HRA). This method provides a new perspective connecting LoRA to OFT and achieves encouraging performance in various downstream tasks. As illustrated in Figure a our method adapts a pre-trained model by multiplying each frozen weight matrix with chain of learnable Householder reflections (HRs) 19.. HRA can be interpreted as either an OFT adapter or an adaptive LoRA. Consequently, it harnesses the advantages of both strategies, reducing parameters and computation costs while penalizing the loss of pre-training knowledge.
Moreover, we show that the orthogonality of HR planes impacts the capacity and regularity of HRA. Accordingly, we leverage an orthogonality regularizer of the HR planes when applying HRA, achieving trade-off between the model capacity and regularity by controlling the strength of the regularizer. When the weight of the regularizer (i.e., the À in Figure Tai goes to infinity, we constrain the orthogonality strictly by Gram-Schmin Schmidt orthogonalization (2) resulting in otricoly-orthogonall HRA implementation. We apply HRA to adapt different models, including large language models (LLMs) and conditional image generators. Experiments show that HRA consistently outperforms state-of-the--rt adapters in various tasks, achieving better performance with fewer trainable parame- ters. Figures [b] highlight the superiority of our method natural language understanding and mathematical reasoning tasks, and more results can be found in the following content.

---

2 Related Work and Preliminaries
The early PEFT methods [57
apply model fine-tuning, which keep model architectures unchanged and only update small portion of model parameters. To achieve better performance, soft prompt fine-tuning [3
44
49 is proposed, introducing additional trainable parameters into inputs and/or hidden layers when adapting models. Recently, adapter- based fine-tuning [20] 62] is proposed improve model adaptation performance without changing model architecture. As we mentioned before, it is often implemented based on the following strategies.
28
30
2.1 Low-rank Adaptation (LoRA)
LoRA 20 formulates trainable parameters decomposed low-rank matrices and aggregates them to frozen weight matrices linearly during fine-tuning, which achieves a trade-off between efficiency and effectiveness. Following LoRA, many improved low-rank adaptation methods [9 21 25 31 33
35
48
54
60,
62
have been proposed, which can be coarsely categorized into three classes
further reduces trainable parameters by adjusting the structure of inserted low-rank matrices. VeRA [25] incorporates frozen low-rank matrices shared across all layers with few trainable scaling vectors. DyLoRA 48) learns low-rank
Structure Adjustment. The work in 48
25 62 60]
matrices with different ranks and determines optimal ranks automatically. AdaLoRA [62 prunes trainable parameters based on the importance scores of the original weight matrices.
Initialization Improvement. Some methods 13 35 utilize matrix decomposition methods on the original weight matrices initialize parameters. DoRA [3] decomposes each original weight matrix into magnitudes and directions for fine-tuning, PiSSA (35] performs singular value decomposition (SVD) on each original weight matrix, where the low-rank principal matrix serves as trainable parameters, while the residual matrix is frozen.
Parameter Quantization. Some methods [54] 9 31 quantize the pre-trained model to further reduce computational costs in both training and inference. For example, QA-LoRA [54] achieves trade-off between quantization strength and adaptation performance with the help of group-wiss quantization operator.
These methods have empirically demonstrated decent performance various downstream tasks, however, they lack theoretical guarantees regarding the retention of pre-training knowledge.
2.2 Orthogonal Fine-tuning (OFT)
LoRA and its variants have empirically demonstrated decent performance in various downstream tasks. However, they lack theoretical guarantees regarding the retention of pre-training knowledge. In order to address this issue, orthogonal fine-tuning (OFT) (37 34 is proposed, which transforms neuron vectors within the same layer using the same set of orthogonal matrices. It preserves the pairwise angles between neuron vectors and thus guarantees bounded discrepancy between pre- trained and fine-tuned models. For instance, the OFT method in (37) adopts Cayley parameterization
to generate the dlock-diagonal orthogonal matrix. The BOFT [34] introduces butterfly factorization to generate denser orthogonal matrix from a chain of structured sparse matrices, which improves OFT's performance with fewer trainable parameters.
Note that, besides mun-tuning, the principle of imposing orthogonality constraints on trainable parameters has been applied in designing robust and training-efffcient neural network architectures, e.g., convolution neural networks (CNNs) [14] 27), recurrent neural networks (RNNs) [51] [6]. and Transformers [59]. In particular, by constraining the orthogonality of these models' weight matrices, we can ensure the models are I-Lipschitz in theory and thus make them achieve provable robustness against adversarial perturbations [7 29 and generalization bounds 43 24).
3 Proposed Method
3.1 Model Adaptation via Learning A Chain of Householder Reflections
Denote gdl-1 {u Rd ||u||2 1} d-dimensional hypersphere. For each E gd-1 construct Houscholder reflection matrix, denoted as H, by I - 2wu T, which corresponds to
3

---

Table I: Comparisons for various OFT-based adapters
Method
OFT 337
BOFT 34
Our HRA
Implementation
R
diag( (
B mm,))
B
H
II
2u,u,
Illustration
#Parameters
d(b-1)
dm(b-1) dmmb
rd
2
2 O(d((b² + b)m + dam))
Complexity
Odd(b² + doa))
Old(r daw))
Old((b² (b2 + d)m + doul)
Cover Daxd
b b=
B (m= 2, 1em1
[u]
specular reflection hyperplane, denoted as H. For any E R He corresponds to reflecting across the hyperplane Hi which reverses the component æ that is orthogonal to the hyperplane.
Because H is orthogonal matrix, it is natural for to implement orthogonal adaptation based on we can treat as an adapter and multiply it with the weight matrix of the pre-trained model. Moreover, since the set of all d x d orthogonal matrices, denoted as Odxa. satisfies all the axioms of a group which is a compact Lie group of dimension d(d - 1)/2, the product of orthogonal matrices also an orthogonal matrix (.. Therefore, a chain of r trainable Householder reflections, leading to our HRA method. As shown Figure given a weight matrix W
can enhance the capacity of the adapter by constructing
Rdd and input Rd the forward step of HRA is
WH W (II = W (II (I zu;u,) with {u, Ed-111 TE-
(I)
Although equation 1 involves the chained product of 1 dense matrices, we can leverage the structure of Householder reflection to simplify the computation. Let x x and x(++1)
(I 2u, - for = 0. r.......T1........ 1.
1. We implement equation by the following two steps:
1) (1+1) = (0)) 2(u j. (3) )
Ur- for = O,..,,-- r- 1. 2) (z)
The first step involves r vector inner products and scalar- vector multiplications, whose complexity is (rrd). The second step involves matrix-vector multiplication, whose complexity is (dow Therefore, the complexity of HRA can as low as Odd(r + dout))
3.2 Comparisons with Existing OFT Methods
Table compares HRA with existing OFT (37) and BOFT (34) methods on their implementations, numbers of parameters, computational complexity, and model capacity.
The number of parameters. Both OFT and BOFT construct several orthogonal sub-matrices
in Obxb based Cayley transformation [12] each of which requires b(b-1) 62 trainable parameters and O(3)) computational complexity. OFT constructs block diagonal matrix based on sub- mobb-mttiiss,, i.c., (b) = diag( R E 0bxb) d/b, BOFT constructs m sparse orthogonal matrices {B - by scattering the elements of dm orthogonal sub-matrices in butterfly manner, such that when
log the product of the m sparse matrices leads to a dense orthogonal matrix. Therefore, according to the number of orthogonal sub-matrices, the numbers of parameters of OFT and BOFT d(b-1) and dm in theory while db and dmb in practice. To make the
CCyleyy transformation represents an orthogonal matrix in Obxa as R = (I A)(I A) where A is a trainable skew- wem-syymetricc matrix. The computational complexity of (I A) - O(b)). Ideally, only need b(b-1) parameters to determine A. In practice, however. both OFT and BOFT leverage dense parameter matrix P b construct A as } (P - P PT). increasing the number of parameters to 6² for making the method friendly to GPU computation.

---

number of parameters comparable to OFT, BOFT often applies a smaller block size (e.g., = 2 or 4). It easy find that when b (= mb), HRA has the same number of parameters as OFT (BOFT). Therefore, HRA is generally comparable to OFT and BOFT regarding model size.
(=
Computational complexity. For the forward step of OFT, i.c., = W WR("x, the computational complexity ((d(b² + dow)). Here, (ddb²) corresponds to applying Cayley transformation to construct orthogonal sub-matrices, and ((db) and (dd correspond to the matrix block multiplication used for = R x and the matrix- multiplication for = Wy, respec- tively. For the forward step of BOFT, i.e., computational complexity is Odd(mb² + mb + dam)) O(d(mb² + md + duw)).. Similar to OFT, O(dmb²) means applying Cayley transformation construct dm orthogonal sub- 100-matrices, O (dmb) (d22m) corresponds to the matrix-vector multiplications of m butterfly matrice to compute = II B!b æ, and ((dam)) corresponds to 2 Wy. When setting (<, m(b² + b)), HRA can be more efficient than OFT (BOFT).
WB( x. where (mm,)) = -1 B, the
<< 6² +
The trade-off between model capacity and regularity. The ranges of OFT, BOFT, and HRA correspond to different subsets of Oaxd, achieving trade-off between model capacity and regu- larity. To cover the whole Daxd, OFT needs to b and compute dense orthogonal matrix with high complexity. With the help of the butterfly structure, BOFT can derive a dense orthogonal (B(
matrix with fewer parameters. However, we need to construct {B J==1 with m log d and ( (B((m)))
b = 2. such that each Bi B. accordingly, the chained product of the d - Householder reflections can represent an arbitrary orthogonal matrix in Oax Similarly, for HRA, we need to construct d d 1 Householder reflections based on (u, 1d-1 j= to represent arbitrary orthogonal matrix in Oaxd- According to the above analysis, BOFT and HRA are relatively easy to achieve a trade-off between model capacity and regularity with mild computation costs - by setting small b, and r, they can achieve dense orthogonal matrices that have intrinsic low-dimensional manifold structures.
can represent a Householder reflection matrix (8) and
3.3 Connections with Low-rank Adaptation
Different from OFT and BOFT, HRA can also be viewed as an adaptive low-rank adapter. Specifically, we can rewrite the chain of HRs in equation]] D in the following equivalent format:
H2
H2
H
H II (I ) = I+ UUU,U
(3)
04
Hi
where U, [ui,... ..., u,] E d r, hid E Re is an apper-triangul matrix, and can be defined as
x
T T
2T .- 1 I u6
T 2, r,
(4)
H
H w
0
-2
Figure 2D illustration indicat- ing that when reflection planes Hi and Ha are orthogonal, the distance H.H wll2 maximized.
Accordingly, we have
( = W WU,T,U W A(W,B) (5)
The above formulation can be viewed as an adaptive LoRA that inherits the theoretical guarantee of OFT on the retention of pre-training knowledge. The low-rank matrix B Ur. is constructed by normalized vectors, while the low-rank matrix A = [ai,... a,.] is parameterized as W WU,,, which can be treated as a function of W and B. Therefore, similar to OFT and BOFT, HRA ensures that the columns of W (r) are always in the column space of W.
3.4 Enhancing The Orthogonality of Householder Reflections for Stronger Regularity
Besides the number of Householder reflections, the orthogonality of them also impacts the regularity of HRA. Specifically, the supreme change of the weight matrix W, W i.e., sup W W WHOl
'Because of their non block sparsity patterns, butterfly matrices are not friendly to modern hardware like GPUs (). Ideally, we can apply sparse matrix- vector multiplication with (dmb) operations, but when applying GPUs, often treat butterfly matrices ordinary dense matrices, which results in O(d operations.
5

---

Table 2: Results (%) of various methods the GLUE development The best results each dataset
shown in bold, and the second best results are shown in underline. We report the matched accuracy for MNLI,
Matthew's 's correlation for CoLA, and average correlation for STS-B.
Method
#Param (M) MNLI SST-2
CoLA
QQP
QNLI
RTE
MRPC
STS-B
All
Full Fine-tune
184
89.90
95.63
69.19
92,40
94.03
83.75
89.46
91.60
88.25
BitFit
0.10
89.37
94.84
66.96
88.41
92.24
78.70
87.75
91.35
86,20
H-Adapter
1.22
90.13
95.53
68.64
91.91
94.11
84.48
89,95
91.48
88,28
P-Adapter
1.18
90.33
95.61
68.77
92.04
94.29
85,20
89.46
91.54
88,41
LoRA
1.33
90.65
94.95
69.82
91.99
93.87
85.20
89.95
91.60
88,50
AdALoRA
1.27
90.76
96.10
71.45
92.23
94.55
88.09
90.69
91.84
89.46
OFT 1-1G
0.79
90.33
96.33
73.91
92.10
94.07
87.36
92.16
91.91
89.77
BOFT 6=8
2
0.75
90.25
96.44
72.95
92.10
94.23
88.81
92.40
91.92
89.89
HRA A-D
0,66
90.70
96.45
73.70
91.29
94.66
88.45
93.69
91.86
90.10
HRA r=8, A=10-3
0.66
90.43
96.79
71.91
91.02
94.44
89.53
94.10
91.74
90.00
HRA r-8, Amoc
0.66
90.52
95.87
70.71
90.71
94.12
87.00
92.59
91.54
89.13
(or, equivalently, supu, ||UU,,, [[, is achieved when U, consists of the top-r right singular vectors of W. In such a situation, U, an orthogonal matrix, i.e., U, U,, I... When the orthogonality is not held, ||WU,,,,, || is reduced, as illustrated in Figure 2 In other words, when adapting the pre- pre-trainee trained model, enhancing the orthogonality of U, imposes stronger regularity on the adapter- it encourages the discrepancy between the target model and the original pre-trained model while restricting the feasible domain of the adapter's ' parameter matrix.
Motivated by the above analysis, we can implement HRA with an orthogonality regularizer. Typically, given a pre-trained model M, we can adapt L weight matrices of the model based on a dataset D by solving the following optimization problem:
+ E 1=1 1 TU.")
min (
F Los
(6)
where U denotes the parameters of HRA for the (-th weight matrix. In equation 6 the first term denotes the loss function, while the second term is the proposed regularizer that encourages the orthogonality of all U,D's, whose significance is controlled by À > 0. Because it does not change the forward step of HRA, this regularizer only increases the adaptation cost slightly.
As shown in Figure Ia by controlling the strength of the orthogonality regularizer, trade-off between the model capacity and regularity. When = O, the feasible domain of U, is the set of when - oo, the feasible domain of U, is the set of orthogonal matrices (i.e., Odxr), leading to the strongest regularity. When À oc, implement
achieve
matrices, and accordingly, the model capacity is maximized. In contrast,
-othooooat HRA based Gram- Schmidt (GS) orthogonalization. For each layer's 's HRA adapter, we initialize its parameter matrix as V, E R and applying Fraa--Shhmih orthogonalization ( it, i.e., U, GS,V)). As shown in W - 2,, jæ, and the computational complexity becomes (d((r + r + dow)),, where additional (dr>²) operations are used for Gram-Schmidt orthogonalization. According to Table T the complexity of this strictly- orthogonal HRA is comparable to OFT 37 when
Figure la in such situation, the forward step of each adapter becomes
Experiments
To demonstrate the effecttiveness of HRA, we conduct comparative experiments for HRA and state- of-the-art adaptation methods on three models oriented to different tasks, including DeBERTaV3: base [15] for natural language understanding, LLaMA2-7B [46] for mathematical reasoning, and Stable Diffusion (40] for conditional text-to-image generation. Typical results are shown below. More results and implementation details are provided in the Appendix.
In each experiment, we set the number of HRs (i.e., r) to ensure that HRA has comparable or fewer trainable parameters than existing adaptation methods (including LoRA, OFT, and their variants). Setting E (0, oo) leads to the proposed HRA method, and we demonstrate the robustness of HRA to the À in wide range. By default, we set À E [10-5, 10 in the experiments. Furthermore, to

---

analyze the trade-ofi off between model capacity and regularity, we consider two variants of HRA: i) the HRA without the orthogonality regularization (A 0) and ti) the strictly-orthogonal HRA using GS orthogonalization (i.e., À oo). For convenience, we denote HRA,,x as the HRA learning Householder reflections per layer with an orthogonality regularizer weighted by X.
4.1 Natural Language Understanding
We adapt DeBERTaV3-han [5] by different methods and test the performance of the adapted models on the General Language Understanding Evaluation (GLUE) benchmark [50]. Following AdaLoRA [62 and
MRPC
94.10%
94.0
BOFT 34 we consider eight tasks of GLUE in this experiment, including two single-sentence tasks, three similarity and paraphrase tasks, and three inference tasks. The experimental results in Figure [bb] and Table [ show that using fewer trainable parameters, HRA achieves the best or comparable results across all datasets and thus leads to the best average perfor- mance. These results demonstrate the efficiency and effectiveness of HRA.
Accuracy
93.5
93.17%
92.5
92.23%
92.0
10
10-4
10-3
10-4
10-2
In this experiment, the HRA without the regularization (i.e., À = 0) and that using weak regularization (i.e., À = 10 achieves comparable adaptation results, while imposing strong regularity by strict orthogonal-
Figure 3: The robustness of HRA (r 8) on MRPC.
ity (i.e., À oc) harms the model performance. This interesting phenomenon implies that the adaptation tasks in GLUE are challenging enough to apply the adapter with sufficient capacity. Figure 3 shows the performance of HRA MRPC when À [10-7, 10-3). We can find that HRA achieves relatively stable performance when À changes in a wide range.
4.2
Mathematical Reasoning of LLM
We adapt LLaMA2-7B 46 on the MetaMathQA
dataset [56) by different adaptation methods and test the adaptation performance on the GSM8K 7 and
48.97%
48.52%
MATH [56] validation sets. Following LoRA [20).
48.29%
each method only adapts the query and value projec- tion matrices of LLaMA2-7B. The results in Figure Ic
Accuracy
show that HRA outperforms its competitors on both GSM8K and MATH when we set 32 and make it have the same number of trainable parameters as BOFT [34). Furthermore, the HRA with 16 uti- lizes only half the trainable parameters of BOFT yet still surpasses its performance, which demonstrates the efficiency of HRA. In addition, the HRA with the orthogonality regularization achieves trade-off be- tween model capacity and regularity. In Figure we test the robustness of HRA X, demonstrating that the performance of the HRA with 8 is stable when À E [10-5, [10-5,10-3] 10-3). For the HRA using more HRs (c.g., r = 16 and 32), we set = 10 based on this robustness test result, balancing the performance on GSM8K and MATH. As shown in Table Ic. the best performance is achieved when r 32 and = 10-4
7.0
6.88%
6.6-
6.54%
6.2-
10-3
10-4
10-1
Figure 4: The robustness of HRA (r 8) to in mathematical reasoning tasks.
Besides, to verify whether HRA can better retain pre-training knowledge, we fine-tune LLaMA-2 7B on the MATHQA dataset by LoRA and HRA, respectively, and check the degradation of model performance on classic NLP tasks, including typical language tasks in ARC [55], HellaSwag [58], MMLU 117 Winogrande 42.. and a coding task in HumanEval (6]. Ideally, after adaptation, we hope that the model can still maintain its high performance in the NLP tasks. The results in
Table Shhow that compared LoRA, HRA retains more of the original model's ss knowledge, whose performance degradation is less severe than LoRA's. In the HumanEval task, its performance is even
7

---

Table 3: The results (%) of LLaMA2-7B classic natural language processing tasks after fine-tuned MATHQA by LoRA and HRA, respectively.
Method
ARC
HellaSwag
MMLU
Winogrande
HumanEval Overall Impact
I.aMA2-7B
49.74
58.90
45.92
74.11
12.80
Fine-tuner by LoRA
48.81
56.89
40.60
71.27
11.59
-6.03%
Fine-tuned by HRA
49.57
57.72
41.20
73.32
13.41
-1.79%
Table 4: Results of various methods on subject-driven generation and controllable generation. For each evaluation metric, the best result is shown in bold, and the second best result is shown in underline. For HRA and its variants, we set and subjective-driven generation controllable generation, respectively.
Subject-driven generation
Controllable generation
PParam (M)
#Param (M)
Method
S21
L2F
DINO1 CLIP-It CLIP-TY LPIPS1
loUt FIt mloUt mAce? aAcef Error]
Real Images
0.764
0.890
0.562
DreamBooth
859.52
0.614
0.778
0.239
0.737
859.52
0.049
0.093
7.72
14,40
33.61
146.19
ControlNet
361.30
0.189 0.317
20.88
30.91
61.42
7.61
T21-Adapter
77.00
0.078 0.143
16.38
26.31
51.63
23.75
LoRA
0.8
0.613
0.765
0.237
0.744
1.25
0.168 0.286
22.98
35.52
58.03
7.68
COFT b
23.3
0.630
0.783
0.235
0.744
26.40
0,195 0.325
26.92
40.08
62.96
6.92
OFT amd
23.3
0.632
0.785
0.237
0.746
26.40
0.193 0.323
27.06
40.09
62.42
7.07
BOFT F
20.76
28.83
41.24
67.74
5.67
HRA 7=7 A=0
0.69
0.670
0.803
0.238
0.758
0.89
0.211 0.350
29.45
42.02
66.83
5.56
HRA 7,7 1-10-3
0.69
0.661
0.799
0.255
0.760
0.89
0.205
0.339
29.27
40.89
67.86
5.46
HRA 7,8 Amoe
0.69
0.651
0.794
0.274
0.778
0.89
0.201
0.334
28.15
40.22
64.95
11.11
better than that of the original model (which we think because the MATHQA dataset contains many samples relevant to logic and reasoning tasks and thus is useful in the HumanEval task).
4.3 Controllable Text-to-lmage Diffusion Models
Following OFT 37 and BOFT 34
we evaluate HRA on adapting pre-trained Stable Diffu- sion (SD) [40] for subject-driven generation and controllable generation, respectively. For a fair comparison, we employ experimental procedures and evaluation metries the same as OFT 37
Subject-driven generation. Given several images of a specific subject and a textual prompt, subject-driven generation aims generate images of the same subject context aligning with the prompt. Taking SD as the backbone model, we evaluate the generation performance of different model adaptation methods, including DreamBooth [41. LoRA (0), OFT and its variant COFT (37), and our HRA. Following DreamBooth [1I, we train and evaluate on generating 25 subjects, each of which corresponds to 30 prompts.
Controllable generation. Controllable generation aims generate images aligning with textual prompt and additional control signals (such as facial landmark annotations, canny edges, and segmentation maps). We conduct experiments
three challenging controllable generation tasks: Canny edge to image (C21) on the COCO dataset [32], landmark to face (I.2F) on the CelebA-HQ
dataset 23 52). and segmentation map image (S2I) on the ADE20K dataset [64]. In this
experiment, we use DreamBooth [41]. ControlNet [61]. T21-Adapter (36), LoRA (00), OFT and its
variant COFT 377 and BOFT 3 baselines.
T the quantitative experimental results. In the subject-driven generation task, we evaluate three erucial aspects of generated images: subject fidelity (DINO B. CLIP-I (38), textual prompt fidelity (CLIP-T (88) ), and sample diversity (LPIPS (. It can be observed that HRA achieves remarkable improvement across almost all metries. In addition, we find that without the orthogonality
(A 0). HRA achieves the highest subject fidelity while sacrificing textual prompt fidelity and sample diversity to some extent, while the strictly-orthogonn HRA (A oo) shows opposite tendencies. Applying the orthogonality regularization with À 10-3 makes HRA balance the performance in all the metrics. Similarly, in the three controllable generation tasks, HRA demonstrates stronger and more precise control compared to the basclines. However, in these tasks, the strictly-orthogone HRA leads

---

a [V] clock top of grass with sunflowers around it
red [V] clock
a [V] teapot top of a purple in forest
a [V] teapot floating on top of water
a [V] vase in the snow
[V] vase with wheat field the background
Original Images
LoRA
OFT
HRA7,0
HRA7,103
HRA7,00
Figure 5: Qualitative results on subject-driven generation.
to suboptimal performance. It means that these tasks require our adapter to have sufficient capacity, but the strict orthogonality constrains its capacity too much. In both tasks, HRA demonstrates the smallest number of trainable parameters among the compared methods. Figures 5 and 6 provide typical qualitative results, demonstrating that the images generated based HRA have good visual effects and well-aligned semantics.
5
Conclusion
In this study, we have proposed a simple but effective Houscholder reflection adaptation method and have demonstrated its usefuiness in various adaptation tasks. The proposed HRA method bridges the gap between low-rank and orthogonal adaptation strategies. It simplifies the implementation of OFT while inheriting its theoretical guarantees on the retention of pre-training knowledge. In addition. we show that controlling the orthogonality of the Householder reflections can achieve the trade-off between HRA's model capacity and its regularity. In the future, we would like improve HRA for practical applications, including accelerating its computation, reducing its memory cost, exploring other regularizers for parameter matrices, and adjusting the weights of the regularizers automatically. We also plan to test HRA adapting more advanced LLMs, e.g., LLaMA3 and Grok-1.

---

Ref. Img
Control
LoRA
OFT
HRAs,0
HRA
HRAs,
Prompt: basehall game heing played.
Prompt: A plate with slice of orange on it.
Prompt: sheep crossing dirt road.
Prompt: A man smiling for the camera.
Prompt: young boy smiling the camera.
Prompt: man with sunglasses on.
Prompt: A brick building.
Prompt: A tree stump.
Prompt: A building with parked in front of it.
Figure 6: Comparisons various adaptation methods on controllable generation, which the control signals include Canny edges, face landmarks, and semantic segmentation results of reference images.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by National Natural Science Foundation (922701 10, 62106271), Beijing Natural Science Foundation (L233008), the Fundamental Research Funds for the Central Universities, and the Research Funds of Renmin University of China. We also acknowledge the support provided by the fund for building world-class universities (disciplines) of Renmin University of China and by the funds from Engineering Research Center of Next-Generation Intelligent Search and Recommendation, Ministry of Education, and from Intelligent Social Governance Interdisciplinary Platform, Major Innovation Planning Interdisciplinary Platform for the "Double-Fine Class" Initiative, Renmin University of China.
References
[1] Teodor Banica and Roland Speicher. Liberation of orthogonal lie groups. Advances in Mathe- matics, 222(4):1461-150) 2009.
[2] Åke Björck. Numerics of gram-schhidd orthogonalization. Linear Algebra and Its Applications, 197:297-316. 1994.
[3] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650-9660, 2021.
[4] Aochuan Chen, Ziqi Gao, Zijing Liu, Yu Li, and Jia Li. Parameter-efficient fine-tuning via circular convolution. arXiv preprint arXiv:2407. 19342, 2024.
[5] Beidi Chen, Tri Dao. Kaizhao Liang, Jiaming Yang. Zhao Song, Atri Rudra, and Christopher Re. Pixelated butterfly: Simple and efficient sparse training for neural network models. In International Conference on Learning Representations, 2021.
[6] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman. Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021.
[7] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichhro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 /4/68, 2021.
[8] Tri Dao, Nimit Sohoni, Albert Gu, Matthew Eichhorn, Amit Blonder, Megan Leszczynski, Atri Rudra, and Christopher Ré. Kaleidoscope: An efficient, learnale representation for all structured linear maps. In International Conference on Learning Representations, 2019.
[9] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer Qlora: Efficient finetuning of quantized 1lms. Advances in Neural Information Processing Systems, 36, 2024.
[10] Zihao Fu, Haoran Yang, Anthony Man-Cho So, Wai Lam, Lidong Bing, and Nigel Collier. On the effectiveness of parameter-efficien fine-tuning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 12799-12807 2023.
[11] Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter-efficient fine-tuning with discrete fourier transform. arXiv preprint arXiv:2405.0mm 030003 2024.
[12] Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013.
11

---

[13] Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. Warp: Word-level adversarial reprogramming. In Proceedings of the 59th Annual Meeting of the Association for Computa- tional Linguistics and the 1th International Joint Conference on Natural Language Processing (Volume I: Long Pupers), pages
2021.
[14) Chuchu Han, Ruochen Zheng, Changxin Gao, and Nong Sang. Complementaion--einforedd attention network for person re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 30(10)::433-3445 2019.
[15] Pengcheng He, Jianfeng Gao, and Weizhu Chen. Debertav3: Improving deberta using clectra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543, 2021.
[16] Wu Hecong. ControlLoRA: A Lightweight Neural Network To Control Stable Diffusion Spatial Information, 2 2023.
[17] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021.
[18] Tom Henighan, Jared Kaplan, Mor Katz, Mark Chen, Christopher Hesse, Jacob Jackson, Heewoo Jun, Tom B Brown, Prafulla Dhariwal, Scott Gray, et al. Scaling laws for autoregressive generative modeling. arXiv preprint arXiv:2010. 14700000..47011,, 2020.
[19] Alston Householder. Unitary triangularization of a nonsymmetric matrix. Journal of the ACM (JACM), 5(4):339-344) 1958.
[20] Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
[21] Nam Hyeon-Woo Woo, Moon Ye-Bin, and Tae-Hyun Oh. Fedpara: Low-rank hadamard product for conmunication-effiiiae federated learning. In International Conference on Learning Representations, 2021.
[22] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 2020.
[23] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv: 2017.
[24] Hyunjik Kim, George Papamakarios, and Andriy Mnih. The lipschitz constant of self-attention. In International Conference on Machine Learning, pages 5562-5571. PMLR, 2021.
[25] Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki Markus Asano. Vera: Vector-based random matrix adaptation. arXiv preprint arXiv:2310. 1444, 2023.
[26] Jitin Krishnan, Hemant Purohit, and Huzefa Rangwala. Diversity-based generalization for neural unsupervised text classification under domain shift. In ECML-PKDD, 2020.
[27] Mingu Lee, Jinkyu Lee, Hye Jin Jang, Byeonggeun Kim, Wonil Chang, and Kyuwoong Hwang. Orthogonality constrained multi- head attention for keyword spotting. In 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 86-92. IEEE, 2019.
[28] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficien prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045-3059, 2021.
[29] Qiyang Li, Saminul Haque, Cem Anil, James Lucas, Roger Grosse, and Jörn-Henrik Jacobsen. Preventing gradient attenuation in lipschitz constrained convolutional networks. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pages 15390-
15402, 2019.
12

---

[30] Xiang Lisa Li and Percy Liang. Prefix-tuning Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1hth International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 099224577,2 2021.
[31] Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, and Tuo Zhao. Loftq: Lora-fine-tune trna-fin-tuning-wwar quantization for large language models. In The Twelfth International Conference on Learning Representations, 2023.
[32] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer ECC 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[33] Shih-Yang Yang Liu, Chien-Yt Yi Wang. Hongxu Yin, Pavlo Molchanov, Yu- Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min- Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation. arXiv preprint arXiv:2402.09353 2024.
[34] Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng. Zhen Liu, Juyeon Heo. Songyou Peng, et al. Parameter-efficient orthogonal finetuning via butterfly factorization. In International Conference on Learning Representations, 2024.
[35] Fanxu Meng, Zhaohui Wang. and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXhv:2404.02948 2024.
[36] Chong Mou, Xintao Wang, Lianghin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters dig more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4296-4304, 2024.
[37] Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf. Controlling text-to-image diffusion by orthogonal finetuning. Advances in Neural Information Processing Systems, 36:79320-79362 2023.
[38] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages -768-8763. PMLR, 2021.
[39] Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He. Deepspeed-moe: Advancing mfxture-o-- experts inference and training to power next-generation ai scale. In International conference on machine learning, pages 18332 18346. PMLR, 2022.
[40] Robin Rombach, Andreas Blatmmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
[41] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500-22510, 2023.
[42] Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale, 2019.
[43] Jure Sokolié, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues. Robust large margin deep neural networks. IEEE Transactions on Signal Processing. 65(16):4265-423 2017.
[44] Yi-Lin Sung, Varun Nair, and Colin A Raffel. Training neural networks with fixed sparse masks. Advances in Neural Information Processing Systems, 34:24193-24205, 2021.
13

---

[45] Rohan Taori, Ishaan Gulrajani. Tianyi Zhang. Yann Dubois, Xuechen Li, Carlos Guestrin, Perey Liang, and Tatsunori B Hashimoto. Stanford alpaca: an instruction-following llama model (2023). URL. https://github. combtasu-lla/SStaoondddlop _alpcca, 2023.
[46] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXhv:2307.09288, 2023.
[47] Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks. Advances in neural information processing systems, 31, 2018.
[48] Mojtaba Valipour, Mehdi Rezagholizadeh, Ivan Kobyzev, and Ali Ghodsi. Dylora: Parameter- efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In Proceedings of the 17th Conference of the European Chapter of the Association for Computa- tional Linguistics, pages 3274-3287, 2023.
[49] Tu Vu, Brian Lester, Noah Constant, Rami AI-Rfou, and Daniel Cer. Spot: Better frozen model adaptation through soft prompt transfer. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguisties (Volume 1: Long Papers), pages 5039-5059, 2022.
[50] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR 2019, 2019.
[51] Jiyao Wei, Jian Liao, Zhenfei Yang. Suge Wang, and Qiang Zhao. Bilstm with pulti-polarity orthogonal attention for implicit sentiment analysis. Neurocomputing, 383:165-173, 2020.
[52] Weihao Xia, Yujiu Yang, Jing-Hao Xue, and Baoyuan Wu. Tedigan: Text-guided diverse face image generation and manipulation. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 22562225666620222222222 2021.
[53] Lingling Xu, Haoran Xie, Si-Zhao Joe Qin, Xiaohui Tao, and Fu Lee Wang. Parameter-efficien fine-tuning methods for pretrained language models: A critical review and assessment. arXiv preprint Xrvi 22. /88, 2023.
[54] Yuhui Xu, Lingxi Xie, Xiaotao Gu. Xin Chen, Heng Chang, Hengheng Zhang, Zhengsu Chen, XIAOPENG ZHANG, and Qi Tian. Qa-lora: Quantization-awe aware low-rank adaptation of large language models. In The Twelfth International Conference on Learning Representations, 2023.
[55] Vikas Yadav, Steven Bethard, and Mihai Surdeanu. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Interna- tional Joint Conference on Natural Language Processing (EMNLP-IJCNLE) Association for Computational Linguistics, 2019.
[56) Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309. 12284, 2023.
[57] Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficient fine-tuning for transformer-basee masked language-models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1-9, 2022.
[58] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can machine really finish your sentence?, 2019.
[59] Aston Zhang, Alvin Chan, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, and Roy Ka- Wei Lee. On orthogonality constraints for transformers. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, volume 2, pages 375-382. Association for Computational Linguistics, 2021.
14

---

[60] Feiyu Zhang, Liangzhi Li, Junhao Chen, Zhouqiang Jiang, Bowen Wang, and Yiming Qian. Increlora: Incremental parameter allocation method for parameter-efficient fine-tuning, arXiv preprint arXiv:2308. /2043, 2023.
[61] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836-3847, 2023.
[62] Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efficiet fine-tuning. In The Eleventh International Conference on Learning Reprexentations, 2023.
[63] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[64] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 633-641, 2017.
15

---

A The Impacts of Orthogonality
Given a weight matrix W dRd Xd we assume that dout 2 Denote the SVD of W as QEVT,
where Σ = diag(o Oa), 01 2 02 2 ... 2 od, and V 10,,.. Va] is the right singular matrix.
When 1, we have
W W WHO 11 4 W lp Atr(u, V V Vui), for un s
Obviously, when u = 01. the distance between W and W H maximized.
Applying mathematical induction, we assume that w W WH is maximized when U, V, Di,.... 8,] for < d.
Then, in the case r + 1, for u E gd-1, we have
I W (++ 11 = W W W 2uu 1)1 = 4 WVV. Wwu T [|
(8)
where the equations are based on the facts that i) HR and the chain of HRs are unitary matrices and ti) the Frobenius norm is unitary-invariant.
Note that, V = [u,,..., val = V,, Vd-rl works as an orthonormal basis of R so we have u 3 V, + Ba- where 3 [B,; Ba-rl sd-1, Accordingly, we rewrite equation 8 as
W W 11
WWVVV Wuu 1
EV V.V.T - WV uu T
= Atr([V, 0 (d- 12 ())(d-r)] ) Str([Vr, 0xx uu Atr(u' T VVVVVVV V
of + (tr(3" )) - -8tr(B; (,, 3,
of + Atr (3r 13 Bu-,) 41, 3,
where Σ diag E,, Eg-r). Obviously, when B, O, and Ba-r [1.0. ........ 00T R the dis- tance between W and W (r++1) is maximized. In such situation, u VB U,41, corresponds to the (r + 1) right singular vector, and Ut+1 = V++1 accordingly.
B Implementation Details
B.1 Natural Language Understanding
the same batch size (i.e., 32 for each task), maximum sequence length, and tune the learning rate, number of training epochs, warm-up steps, as well as İ in the HRA method. Additionally, the dropout rate is consistently DeBERTaV V3-bae and freeze the pre-trained weights for tasks. Both training and testing are conducted 7 NVIDIA GeForce RTX 3090 GPUs. Detailed hyperparameter setups are presented in T
Following BOFT (34],
to 1E-01. We adapt every linear layer in
Table The hyperparameters DeBERTaV3-base on tasks included in GLUE benchmark.
Method
Dataset
MNLI
SST-2
CoLA
QQP
QNLI
RTE
MRPC
STS-B B
Epochs
8
10
34
12
12
11
60
39
Learning Rate
1E-02
3F-03
9E-03
8F-03
1F-02
5E-03
6F-03
5E-03
HRA.
Warm Up Steps
1000
500
100
1000
500
50
50
50
Max Seq. Len.
256
128
64
320
512
320
320
128
Epochs
5
8
12
11
44
35
30
Learning Rate
2E-03
2E-03
2E-03
2E-03
2E-03
1E-03
1E-03
9E-04
HRAy=8. 10-
Warm Up Steps
1000
500
100
1000
500
50
50
50
Max Seq. Len.
256
128
64
320
512
320
320
128
16

---

B.2 Mathematical Reasoning
Following BOFT [4,, we fine-tune the LLaMA2-7B model on the first generated 512 tokens, which is sufficient for these two tasks. In all experiments, we use the cosine learning scheduler. We fix the training epoch and the warm-up ratio in MetaMathQA [56]. where they use the Alpaca 45 prompt and evaluate the model in zero-shot. The generation temperature set as for both tasks. In our proposed method, the learning rate of =RA is set to 1E-05, HRAA=10-14 is set to 3E-05, and x= is set to 1E-03, Both training
and 0.005, respectively. We follow the evaluation tools
and testing are conducted on 8 NVIDIA L20 GPUs.
B.3 Subject-driven Generation
Following OFT 377, we use the same AdamW optimizer with weight decay of 1E-02 and fine-tune the linear layers including Wg. Wx. W.. and Wo in the UNet model. The learning rate is set to 7E-06, with batch size of 1, and the number of training steps is approximately 2000 steps. Both training and testing are conducted on NVIDIA GeForce RTX 3090 GPUs.
B.4 Controllable Generation
Our data processing procedure and architecture design are consistent with OFT [37] In addition to injecting trainable HRA weights into the stable diffusion model, we employ a lightweight neural network [16] to encode the control signals. We fine-ture tunetuneeue the model for 11 epochs for C21 and 20 epochs for L2F and S21. The learning rate is set to 3E-05 with a batch size of for all three tasks. Both training and testing are conducted 8 NVIDIA RTX A6000 GPUs.
B.5 Analysis of Computational Cost and Robustness
To compare the computational efficiency, we adapt LLaMA2-7B on the MetaMathQA dataset by HRA and other baselines and test their training time and GPU memory costs. For fair comparison, we conduct all the experiments on 8 NVIDIA RTX A6000 GPUs and apply the same batch size and almost the same number of trainable parameters across all the models. The results in T that HRA's peak memory usage comparable to that of baselines, while its training time Is less. These results demonstrate HRA's superiority in computational efficiency and adaptation performance.
Table comparison various models on their computational efficiency.
Method Param. Ratio Training time (hours) Peak memory usage (GB)
LoRA
0.12%
45
279
OFT
0.13%
53
282
HRA 0.12%
30
287
To analyze the impact of orthogonality, we conduct mathematical reasoning experiments with different values of A. The results in Table/7sshow that a) the performance of HRA is relatively stable concerning the change of X, b) in the wide range of X, HRA is superior to the baselines, c) even if ignoring the regularizer (A 0), method still outperforms the baselines. These results demonstrate the effectiveness and robustness of implementing orthogonal adaptation based on Householder reflections. In the future, we will consider further analyzing the impacts of À in theory.
Table 7: Results (%) HRA with other values À for mathematical reasoning.
Method
Param. Ratio GSMSK MATH
LoRA, LoRA,=32
0.25%
50.2
7.8
OFT.=16
0.13%
50.1
8.4
HRA,=32,A=00
0.12%
52.8
9.2
HRA-=32,2=1ee 10-
0.12%
53.6
8.3
HRAA-=32AA=1e--
0.12%
56.3
9.3
HRA,-33 IRAA-322A2=1---
0.12%
53.6
8.6
HRA,-322 =32,
0.12%
55.8
9.0
17

---

C More Experimental Results
C.1 Subject-driven Generation
a [V] backpack with city background
[V] backpack with blue house in the background
[V] stuffed animal the beach
[V] stuffed animal top a wooden floor
[V] wearing red hat
[V] cat wearing pink glasses
a [V] dog wearing santa hat
[V] dog wearing rainbow scarf
Original Images
LoRA
OFT
HRA7,p
HRA7,10
HRA7,00
Figure 7: More qualitative results subject-driven generation.
18

---

[V] cat on top of wooden floor [ sneaker with tree and autumn leaves in the background
[V sneaker on top of a dirt road
[V] toy top of pink fabric
[V] toy on top of a white rug
[V] backpack on cobblestone street
[V] backpack with the Eiffel in background
[V] city in background
Original Images
LoRA
OFT
HRAA7
HRRA71003
HRAA7.0
Figure 8: More qualitative results on subject-driven generation.
19

---

C.2 Controllable Generation
C.2.1
Canny Edge to Image
Ref. Img
Control
LoRA
OFT
HRAs,0
HRA8.10-5
HRAs.
Prompt: Carrots on a cutting board.
Prompt: A zebra.
Prompt: A couple sitting on a bench.
Prompt: A person throwing frisbec.
Prompt: beach at sunset.
Prompt: A table with a plate of food and cup of coffee.
Prompt: A bench the middle of nowhere.
Prompt: Several airplanes parked at airport.
Figure 9; More qualitative results canny edge image.
20

---

C.2.2
Landmark to Face
Ref. Img
Control
LoRA
OFT
HRAs.o
HRAg
HRAs.co
Prompt: A man with a taltoo on his arm.
Prompt: A young man in suit and tic.
Prompt: man with long hair.
Prompt: A woman with blonde hair.
Prompt: man long hair.
Prompt: A woman with long blonde hair.
Prompt: woman with long brown hair.
Prompt: A woman with long brown hair.
Figure 10: More qualitative results landmark face.
21

---

C.2.3
Segmentation to image
Ref. Img
Control
LoRA
OFT
HRAs,0
HRA
HRAs,
Prompt: parked in front of house.
Prompt: A field.
Prompt: The coast.
Prompt: living room with two chairs and a table.
Prompt: A man playing game of pool.
Prompt: castle scotland.
Prompt: A street in city.
Prompt: pile of tires.
Figure 11: More qualitative results on segmentation to image.
22

---

C.3
Case Studies in Mathematical Reasoning
Case Study 1
Question: There were 50 cats rock. Four boats came and carried away 5 cats cach, and later, 3/5 of the remaining cats ran after a mouse they 'd seen. How many cats were left on the rock? (Ground-truth answer is 12)
Initially, there were 50 cats on the rock. Then, 4 boats came and carried away 5 cats each, so the total number of cats carried away is: boats x cats/boat = 20 cats So, the number of cats left on the rock is: 50 cats 20 cats 30 cats Next, 3/5 of the remaining cats (which is the same as 60% of the cats) ran after a mouse they'd d seen. This means that: 30 cats left on the rock. The answer 30
Llama-2 Answer:
60% 18 cats left the rock chase the mouse. Therefore, there 30
HRA32.0 Answer: There were 50 cats on the rock. Four boats came and carried away cats each, so a total of 4 5 20 were carried away. The remaining number of cats on the rock is 50 20 30 cats. Then, 3/5 of the remaining cats ran after mouse, so 3/5 * 30 24 cats ran away. The number of cats left on the rock 30 =4 cats. The answer is:
HRA32,10 32,10 Answer: There were 50 cats on the rock. Four boats came and carried away cats each, so 4 * 55 20 cats were carried away. After the cats were carried away, there were 50 20 cats left on the rock. Then, 3/5 of the remaining cats ran after a mouse, so 3/5 * 30 = 18 cats ran away. Therefore, there were 30 18 = 12 cats left on the rock. The answer is: 12
HRA32, Answer: There were 50 cats on the rock. Four boats came and carried away 5 cats each, x 20 cats carried away. The remaining cats the rock are 50 20 30 cats. 3/5 the remaining cats ran after a mouse, so 3/5 x 30 = 18 cats ran away. The number of cats left the rock is 30 18 12 cats. The answer is: 12
Case Study 2
Question: James likes check the coin return of the vending machine for change. One day he finds quarter, two nickels, and 7 dimes. How much money in cents does James have? (Ground-truthh answer is 105)
Llama-2 Answer: Based on the information provided in the instruction, James has total of: quarter 25 cents 2 nickels = 50 cents dimes 70 cents Therefore, James has total of 145 cents. The answer 145
HRA 32,0 Answer: First, we need to convert the coins to cents. One quarter worth 25 cents. Two nickels are worth 2 $ = 10 cents each. So, two nickels are worth 10 cents each. Seven dimes are worth 10 0.10 = cents each. So, seven dimes are worth 7 cents each. Now, we can add up the value of the coins. 25 cents 10 cents + cents 42 cents. Therefore, James has 42 cents. The answer is: 42
HRA 10-4 Answer: quarter 25 cents 2 nickels =
= 10 cents dimes = 7 10 70 cents So, James has 25 10 70 105 cents. The answer is: 105
HRA 32 Answer: First, need convert the coins cents. quarter is equal 25 cents. 2 nickels is equal to 2 5= 10 cents. 7 dimes is equal to 7 10 70 cents. Now, we can add up the total amount of money. 25 10 70 105 cents. Therefore, James has 105 cents.
The answer is: 105
23

---

Case Study 3
Julia's boat sprang leak while she was out on the lake. The boat was taking on two liters of water for every ten feet she rowed back towards shore. took her sixteen seconds to row twenty feet. The shore was 64 seconds away. How much water had the boat taken on by the time Julia reached shore? (Ground-truth answer is 16)
Question:
Llama-2 Answer: find out how much water the boat took on by the time Julia reached shore, we need to use the information provided in the problem. We know that the boat takes on liters of water for every feet rows back towards shore. So, Julia rowed 20 feet (which takes her 16 seconds), the boat took on 2 liters of water. We also know that Julia rowed a total of 64 seconds reach the shore. So, she rowed 20 feet 16 seconds, she rowed a total of 40 feet (20 feet 2) in the remaining 44 seconds. This means the boat took on liters of water (2 liters per 10 feet x 40 feet). Therefore, by the time Julia reached the shore, the boat had taken on 8 liters of water. The answer is
HRA32,0 Answer: Julia rowed 20 feet in 16 seconds, she rowed 20/16 1.25 feet per second. The boat takes on liters of water for every 10 feet rowed, so it takes on 2/10 0.2 liters of water per second. Since Julia rowed 1.25 feet per second, the boat takes 0.2 * 1.25 = 0.25 liters of water per second. The shore is 64 seconds away, so the boat takes on 0.25 64 16 liters of water by the time Julia reaches shore. The answer is: 16
32,102 10-4 Answer: Julia rowed 20 feet 16 seconds, so she rowed rate of 20/16 1.25 feet per second. Since she was taking on liters of water for every 10 feet she rowed, she was taking on 2/10 * 1.25 0.25 liters of water per second. Since she was taking on 0.25 liters of water second, and it took her 64 seconds to reach shore, the boat had taken on 0.25 64= 16 liters of water by the time she reached shore. The answer is: 16
HRA 32.00 Answer: Julia rowed 20 feet in 16 seconds. To find the rate at which the boat was 32.02
taking water, divide the amount of water by the time it took to row 20 feet. So, the rate which the boat was taking on water
liters/10 feet * seconds = 32 liters/10 feet. Since Julia rowed 20 feet, the boat took on 32 liters/10 feet 20 feet = 640 liters of water. Since the shore was 64 seconds away, the boat took on 640 liters of water in 64 seconds. Therefore, the boat took on 640 liters of water by the time Julia reached shore. The answer is: 640
Case Study 4
Question: Jillian's handbag cost $20 less than times much as her shoes cost. If her shoes cost S80, how much did her bag cost? (Ground-truth answer is 220)
Llama-2 Answer: Jillian's 's shoes cost S80 and her handbag cost $20 less than 3 times as
much as her shoes cost, then we can set up the following equation: 3x $80 $20 Solving for x, we get: = $80 $20 = $60 So, Jillian's 's handbag cost $60. The answer is 60
HRAg 22.0 Answer: We know that Jillian's 's shoes cost S80. Her bag cost $20 less than 3 times as much her shoes cost. So, her bag cost $80 3 $20 $240 $20 $220. Therefore. Jillian's bag cost $220. The answer is: 220
HRA 10 Answer: Jillian's 's shoes cost $80. Her bag cost $20 less than 3 times as much as her shoes cost. So, her bag cost * $80 $20 = $240 $20 $220. Therefore, her bag cost $220. The answer is: 220
HRA32. Answer: Jillian's shoes cost $80. Her handbag cost $20 less than 3 times much as her shoes cost. So, her handbag cost 3 $80 $20 $210 $20 S190. Therefore, her handbag cost $190. The answer is: 190

---

Case Study
Question: In a dance class of 20 students, 20% enrolled in contemporary dance, 25% of the remaining enrolled jazz dance, and the rest enrolled hip-hop dance. What percentage of the entire students enrolled in hip-hop dance? (Ground-truth answer is 60)
Llama-2 Answer: Total number of students in the dance class 20 Percentage of students enrolled contemporary dance 20% of 20 4 Percentage of students enrolled in jazz dance 25% of 16 (since 20% are enrolled in contemporary dance) = 4 Percentage of students enrolled in hip-hop dance Remaining students 4 12 Therefore, 12% of the entire students in the dance class are enrolled in hip-hop dance. The answer is 12
HRA32,0 Answer: First, need find out how many students enrolled in contemporary dance. 20% of 20 students 0.20 20 students. So, there are 20 = 16 students remaining. Next, need to find out how many students enrolled in jazz dance. 25% of 16 students is 0.25 * 16 = 4 students. So, there are 16 - 4 = 12 students remaining. Finally, we need find out how many students enrolled in hip-hop dance. The remaining 12 students the ones who enrolled in hip-hop dance. Therefore, the percentage of students enrolled in hip-hop dance is 12 / 20 100 60% The answer is: 60
12.10 Answer: 20% of the 20 students enrolled in contemporary dance, so 20/100 20 = 4 students enrolled in contemporary dance. The remaining students are 20 4 16 students. 25% of the remaining 16 students enrolled in jazz dance, so 25/100 * 16 students enrolled in jazz dance. The remaining students are 16 4 = 12 students. The rest of the students enrolled in hip-hop dance, so 12 students enrolled in hip-hop dance. To find the percentage of the entire students enrolled in hip-hop dance, we divide the number of students enrolled in hip-hop dance by the total number of students and multiply by 100. So, (12/20) * 100 = 60% The answer is: 60
HRA32 oz Answer: First, 20% of the 20 students enrolled in contemporary dance, which is 16 students remaining. Then, 25% of the remaining 16 students enrolled in jazz dance, which is 25/100 16 4 students. So. there are 16 - 4 = 12 students remaining. Finally, the rest the students enrolled in hip-hop dance. which is 12 students. Therefore, 12/20 100 60% of the entire students enrolled in hip-hop dance. The answer is: 60
20/100 * 20 students. So, there are
25

---

Case Study 6
Question: A candle melts by 2 centimeters every hour that it burns. How many centimeters shorter will candle be after burning from 1:00 PM to 5:00 PM? (Ground-truth answer is 8)
Llama-2 Answer: To determine how many centimeters shorter candle will be after burning from 1:00 PM to 5:00 PM, we need first calculate the total time the candle will be burning during that period. 1:00 PM + 5:00 PM 8 hours Since the candle melts by 2 centimeters every hour, we can calculate the total distance it will melt during the 8 hours as follows: 2 centimeters/hour
hours = 16 centimeters So, after burning from 1:00 PM to 5:00 PM, the candle will centimeters shorter. The answer 16
HRA32,0 32,0 Answer:
The candle burns for total of 4 hours. the first hour, the candle melts by 2 centimeters. In the second hour, the candle melts by 2 centimeters. In the third hour, the candle melts by centimeters. total, the candle melts by 2 centimeters 4 hours 8 centimeters. Therefore, the candle will be 8 centimeters shorter after burning from 1:00 PM to 5:00 PM. The answer is: 8
the fourth hour, the candle melts by 2 centimeters. So, in
HRA 32.100110 10-1 Answer: From 1:00 PM to 2:00 PM, the candle burns for hour. So, the candle will melt by 2 centimeters x hour 2 centimeters. From 2:00 PM to 3:00 PM, the candle burns for hour. So, the candle will melt by 2 centimeters x hour = centimeters. From 3:00 PM to 4:00 PM, the candle burns for hour. So, the candle will melt by centimeters x hour 2 centimeters. From 4:00 PM to 5:00 PM, the candle burns for hour. So, the candle will melt by centimeters x hour 2 centimeters.
total, the candle will melt by centimeters = 8 centimeters. Therefore, the candle will be centimeters shorter after burning from 1:00 PM to 5:00 PM. The answer is:
centimeters + centimeters + 2 centimeters
HRA32.cc 32.00 Answer: First, we need find out how many hours the candle will burn from 1 1:00 PM to 5:00 PM. There are 5 hours in total, so the candle will burn for 5 1 4 hours. Next, we need to find out how many centimeters the candle will melt hours. Since the candle melts by centimeters every hour, it will melt 2 x 4 = 8 centimeters in total. Finally, we need to subtract the initial length the candle from the total length will melt to find out how many centimeters shorter the candle will be. The initial length of the candle is centimeters, so the candle will be 8 centimeters shorter. The answer is:
26

---

Case Study 7
Question: Marie ordered one chicken meal that costs $12, 5 packs of milk that costs S3 each, 4 apples that cost $1.50 each, and some boxes of pizza. Marie paid total of $50. How many boxes of pizza did Marie order if each box costs $8.50? (Ground-truth answer is 2)
Llama-2 Answer: Based on the information provided,
calculate the number of boxes of pizza that Marie ordered as follows: Total cost of Marie's order = $12 (chicken meal) $3 5 (packs of milk) S4 x 4 (apples) $50 If each box of pizza costs S8.50, then the number boxes pizza that Marie ordered can be calculated follows: Total cost of pizza $8.50 number of boxes Since Marie paid total of $50, we can set up the following equation: $8.50 x number of boxes = $50 Now, can solve for the number of boxes of pizza that Marie ordered: number of boxes = =550 / 58.50 number of boxes = Therefore, Marie ordered boxes of pizza. The answer is
HRA 32.0 Answer: The cost of the chicken meal is $12. The cost of 5 packs of milk 5 $3 $15. The cost of 4 apples is * $1.50 $6. The total cost of the chicken meal, milk, and apples is $12 + $15 + $6 S33. If Marie paid total of $50, then the cost of the pizza boxes $50 $33 $17. each box of pizza costs $8.50, then the number of boxes of pizza is $17 / $8.50 2. Therefore, Marie ordered 2 boxes of pizza. The answer is: 2
HRA 10-4 Answer: The chicken meal costs $12. The 5 packs of milk cost $3 each, so 5 $3 = $15. The 4 apples cost $1.50 each, so $1.50 $6. The total cost the chicken meal, milk, and apples is $12 + $15 $6 = $33. Marie paid total of $50, so the cost of the pizza boxes is $50 $33 = $17. Each box pizza costs $8.50, so the number of boxes of pizza $17/ $8.50 = 2. Therefore, Marie ordered 2 boxes of pizza. The answer is: 2
HRA32.co Answer: First, Marie ordered chicken meal that costs $ $12. Next, she ordered packs milk that cost $3 each, so * $3 $15 for the milk. Then, she ordered apples that cost $1.50 each, so 4 $1.50 $6 for the apples. Finally, she ordered some boxes of pizza that cost S8.50 each, so let's say she ordered x boxes of pizza. So, the total cost of the chicken meal, milk, apples, and pizza is $12 + $15 $6 $8.50x $50. Simplifying the equation, we get $33.50 + $8.50x = $50. Subtracting $33.50 from both sides, we get $8.50x 166.50. Dividing both sides by $8.50, we get x 2. Therefore, Marie ordered 2 boxes of pizza. The answer is:
27

---

Case Study 7
Claire makes 3 egg omelet every morning for breakfast. How many dozens of eggs will she eat in weeks? (Ground-truth answer
Question:
Llama-2 Answer: To calculate how many dozens of eggs Claire will in 4 weeks, we need to first convert the number of eggs she makes in omelet. An omelet contains 3 eggs, so Claire will eat eggs per omelet. Since Claire makes an omelet every morning, she will 3 eggs per omelet per day. To find out how many dozens of eggs she will eat in 4 weeks, we can multiply the number eggs she eats per day by the number days in 4 weeks: eggs/day 21 days/week 4 weeks 243 eggs Therefore, Claire will eat 243 dozens of eggs weeks. The answer is 243
HRA32.0 Answer: Claire makes a egg omelet every morning for breakfast. In 4 weeks, there are 4 weeks 7 days 28 days. So, in 28 days, Claire will eat 28 x 3 84 eggs. Since 7 dozen eggs. Therefore, Claire will cat
there are 12 eggs in a dozen, Claire will cat 84 7 dozens of eggs in 4 weeks. The answer is: 7
HRA32, 10-4 Answer: Claire makes 3 egg omelet every morning for breakfast. In 4 weeks, there are 4 7 = 28 days. So, in 28 days, Claire will eat 28 x 3 = 84 eggs. Since there are 12 eggs in dozen, Claire will eat 84 12 dozen of eggs. The answer
HRA 32,00 Answer: Claire makes a egg omelet every morning for breakfast. In 4 weeks, there are 4 weeks x days 28 days, So, in 28 days. Claire will eat 28 3 = 84 eggs. To find how many dozens eggs she will eat, we divide 84 by 12. 84 / 12 Therefore, Claire will eat dozens of eggs in 4 weeks. The answer is: 7
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected generalize other settings. It is fine include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Regarding the limitations of HRA, we believe the main concern is the setting of hyperparameters (i.c., the rank and the weight of orthogonal regularizer X). Similar to LoRA, the rank of our HRA determines the trade-off between the number of trainable parameters and the training efficiency. In this study, we set r to ensure that the number of our trainable parameters smaller than those of baselines. Of course, inspired by the recent variants of LoRA, e.g., AdaLoRA, we can adjust the rank adaptively, which is not the main contribution of this work and thus is left to our future work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not diseussed in the paper.
The authors are encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recogmition algorithm may perform poorly when image resolution is low or images are taken low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 1 acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
29

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: For each theoretical result, provide the full set of assumptions and complete and correct proof.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the information of the backbone and the necessary hyperparameters for each experiment.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
also be provided via detailed instructions
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., .. large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-souree dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the
30

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they case of closed-source models, it may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We provide the code with comments and the available access to the data at http:s//ggihuub..cmmmaeeeeeeeeee
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips) cc/ bublic/quides
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips cc
Sooeee///omeummmiii is sinnooiicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized ver- sions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
The experimental details including hyperparameters, optimizer, and on, provided in appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
The error bars are not reported because it would be too time-consuming.
Guidelines:
The answer NA means that the paper does not include experiments.
31

---

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: For each experiment, we provide the information of the GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
This paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
32

---

Regarding the societal impact of our work, we believe HRA can further simplify the adaptation of LLMs and promote more LLM-basee downstream applications. Similar to LoRA and OFT, HRA may suffer from some potential issues like inappropriate (even illegal) abuse, amplifying the social prejudice intrinsically in L.LM when the fine- tuning data are biased, and so on. It should be noted that these potential issues are neither purely attributed to the technique itself nor specific to from them. Solving these issues depends on developing new techniques, social policies, and data quality improvement. How to mitigate (even eliminate) these issues is left to our future work.
Justification:
and OFT also suffer
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions access the model implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
33

---

Justification: All the creators original owners of assets including code, data, and models used in the paper are properly credited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include a URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, [paperswl] thcode. com/ /a///aatsssseee has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged reach out the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide the comments in the released code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine. but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
34

---

Answer:
[NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do include any information that would break anonymity (if applicable), such as the institution conducting the review.
35


Reviews:
[{"review_id": "aCbmvkIKh2", "reviewer": "Reviewer_dePa", "summary": "This paper proposed a new model fine-tuning method, called Householder reflection adaptation (HRA). The main idea of HRA is to fine-tune the model with a series of Householder reflections. By virtue of the Householder reflection, the orthogonality of the tuning matrix can be obtained, the number of tuning parameters can be reduced, and the computational cost can also be saved. Besides, with simple math, it can be shown that HRA also share the low-rank property with LoRA. Experiments on several tasks have been conducted, to demonstrate the advantages of the proposed method.", "questions": "I have no further questions.", "limitations": "The authors do not explicitly discuss the limitations and broader societal impacts of this work. Since the proposed method could be applied to LLMs, which have been used by more and more people, the broader societal impacts should be discussed.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea is simple, yet very effective. Empirical results show that the proposed HRA can achieve better fine-tuning results with less trainable parameters.", "weaknesses": "I do not find major flaws of this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your appreciation of our work. We believe your concerns have been resolved in our general response, and we hope that our response can help increase your confidence score. We are willing to discuss with you in the next discussion phase if you have any other questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hV44R865WK", "reviewer": "Reviewer_7LSp", "summary": "The paper proposes a simple but effective adaptation method based on Householder reflection matrix. The authors show that this method is closely related to low-rank adaption. Diverse experiments demonstrated the effectiveness of the proposed method in comparison to a few baselines.", "questions": "Please refer to the weaknesses.", "limitations": "The reviewer hasn't found the discussion of limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of using Householder reflection matrix to do deep learning mode adaption is novel.\n2. The proposed method is simple and effective.\n3. The experiments are extensive.", "weaknesses": "1. The proposed method is a special case of Orthogonal Fine-Tuning. The major difference is the use of the Householder reflection matrix. This means the novelty is relatively limited.\n2. The paper does not explain why the proposed method outperforms baselines such as LoRA and OFT.\n3. In Figure 1(c), the table, $\\lambda=1e-4$ is better than $\\lambda=\\infty$ and $0$. The authors should report the performance of other values of $\\lambda$ such as $1e-1$ to $1e-8$ to show the impact of $\\lambda$.\n4. According to Figure 1(c), it seems that the gain of the proposed method is many from the regularization. This raises a question---will orthogonal regularizations also improve LoRA and OFT? The authors should make a fair comparison and show the source of the improvement of the proposed method clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments. Below we try to resolve your concerns one by one.\n\n**Q1: The novelty of the proposed method.**\n\n**A1:** We respectfully disagree with the comment that our work's novelty is relatively limited for the following three reasons. Firstly, to our knowledge, our work makes the first attempt to bridge the gap between low-rank and orthogonal adaptation techniques. Our HRA is a new implementation of OFT. At the same time, it is also an adaptive LoRA, as we mentioned in Section 3.3. Secondly, existing OFT restricts the interactions between different dimensions of the weight matrix due to the block diagonal structure of its orthogonal matrix, while BOFT overcomes this issue by time-consuming butterfly matrix multiplication. Unlike these two methods, our HRA implements a chain of Householder reflections to implement orthogonal adaptation, as discussed in Section 3.2. This implementation leads to better efficiency and performance. Thirdly, focusing on HRA, we analyze the impact of the orthogonality of reflection planes on the adaptation performance (Section 3.4), proposing an orthogonal regularizer in the training phase, which can further boost the adaptation performance. \n\nWe believe all three of the above contributions are new to the research community, and thus, the novelty of our work is sufficient. Actually, **in your first comment in the Strengths session, you admitted the novelty of our work.** Therefore, we hope that the above response helps you reconsider our work.\n\n**Q2: The reasons for the superiority of HRA compared with LoRA and OFT are not explained.**\n\n**A2:** In fact, we have explained the reasons for the superiority of HRA. In particular, in Section 3.2, we compared HRA with OFT and BOFT on their implementations and computational complexity. **As shown in Lines 147-148 and 157-158, we have shown that HRA can be more efficient (i.e., using fewer trainable parameters) in a mild condition, which is easy to meet in our experiments.**\n\nRegarding the retention of pre-training knowledge, **we have shown in Lines 168-172 that HRA can preserve the angular information of weight matrices like OFT and BOFT do,** which is better than LoRA. \n\nIn addition, by introducing an orthogonal regularizer during adaptation, we control the orthogonality of reflection planes and thus help achieve a trade-off between the model capacity and regularity. Compared to LoRA and OFT, this mechanism can further reduce the risk of over-fitting, which helps further boost our performance. We plan to add this point to the end of section 3.4.  \n\n**Q3: More experimental results with other values of $\\lambda$.**\n\n**A3:** We conduct the experiments with other values of $\\lambda$, and the results are as follows.\n\n| Method | #Param | GSM8K | MATH | \n| :-----| :----: | :----: | :----: |\n| $\\text{LoRA}_{r=32}$ | 0.25% | 50.2 | 7.8 |\n| $\\text{OFT}_{b=16}$ | 0.13% | 50.1 | 8.4 |\n| $\\text{HRA}_{r=32,\\lambda=\\infty}$ | 0.12% | 52.8 | 9.2 | \n| $\\text{HRA}_{r=32,\\lambda=1e-1}$ | 0.12% | 53.6 | 8.3 | \n| $\\text{HRA}_{r=32,\\lambda=1e-4}$ | 0.12% | 56.3 | 9.3 | \n| $\\text{HRA}_{r=32,\\lambda=1e-8}$ | 0.12% | 53.6 | 8.6 | \n| $\\text{HRA}_{r=32,\\lambda=0}$ | 0.12% | 55.8 | 9.0 | \n\nWe can find that a) the performance of HRA is relatively stable concerning the change of $\\lambda$, b) in the wide range of $\\lambda$, HRA is superior to the baselines, c) even if ignoring the regularizer $(\\lambda=0$), our method still outperforms the baselines, which demonstrates the effectiveness of implementing orthogonal adaptation based on Householder reflections.\n\n**Q4: Will the orthogonal regularizer also improve LoRA and OFT?**\n\n**A4:** This is an interesting question. Firstly, it should be emphasized that **our comparison experiments are fair because, in all Tables and Figures, we have considered the HRA with $\\lambda=0$ (i.e., without the orthogonal regularizer) and compare it with the baselines.** The experimental results show that HRA can outperform LoRA, OFT, and other competitors even without the regularizer. **In other words, the superiority of our method is mainly from the proposed Householder reflection chain, and the proposed regularizer can further boost performance.**\n\nSecondly, it is nonsense to apply the orthogonal regularizer to OFT because the block diagonal structure of its orthogonal parameter matrix has ensured that the columns of the matrix are orthogonal to each other. For BOFT, the columns of different butterfly orthogonal parameter matrices are also orthogonal to each other. In other words, **OFT and its variants already have intrinsic and strict orthogonality constraints**.\n\nFinally, for the vanilla LoRA in the formulation $W+AB$, we can impose our orthogonal regularizer on its parameter matrices, which was never considered before. Although our regularizer is motivated by the orthogonality of reflection planes rather than LoRA, to resolve your concern, we imposed our regularizer on the matrix $B$ of LoRA and tested it in the mathematical reasoning task. \n\n| Method | #Param | GSM8K | MATH | \n| :-----| :----: | :----: | :----: |\n| $\\text{LoRA}_{r=16}$ | 0.12% | 47.3 | 6.6 |\n|  $\\text{LoRA}_{r=16,\\lambda=1e-4}$ | 0.12% | 47.7 | 6.8 |\n|  $\\text{HRA}_{r=32,\\lambda=0}$ | 0.12% | **55.8** | **9.0** |\n|  $\\text{HRA}_{r=32,\\lambda=1e-4}$ | 0.12% | **56.3** | **9.3** |\n\nThe above results verify our claim --- the superiority of HRA is mainly caused by the Householder reflection chain. **Note that, without a strong theoretical motivation like the orthogonality of reflection planes, we have too many options to combine LoRA with our regularizer**, e.g., imposing our regularizer to $A$, $B$, $AB$, and so on. Considering so many variants of LoRA is out of the scope of this work.  \n\nWe hope the above responses can resolve your concerns. We are willing to discuss with you in the next phase if you have any other questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lHKN97RTU3", "reviewer": "Reviewer_1vEG", "summary": "his paper proposes a simple yet efficient adaption method, namely HRA, which finetunes a pretrained model by multiplying each frozen weight matrix with an orthogonal matrix constructed by a chain of learnable Householder reflections. The authors interpret HRA as an adaptive LoRA that retains theoretical guarantees of preserving pre-training knowledge of OFT, somewhat bridging the gap between LoRA and OFT. Number of trained parameters and computational complexity are analyzed. Experiments on several pretrained models (DeBERTa, LLaMA2, Stable Diffusion) show that HRA with fewer learnable parameters and suitable orthogonality regularizing strength achieves superior performance performances than existing methods, demonstrating the effectiveness of HRA for different downstream tasks.", "questions": "What are the limitations of HRA?  \nHow is the wall clock cost, comparing HRA to LoRA during training? Could this paper show the time cost comparing HRA to LoRA?", "limitations": "I would like to see the authors to discuss on the limitations of the proposed method.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This paper is well motivated and well written. \n\nThe authors theoretically show that HRA can be formulated as an adaptive LoRA, providing a new perspective that bridges OFT to LoRA, which is insightful.\n\nTo show the effectiveness and wide applicability of HRA, this paper has fairly conducted various types of experiments on different tasks including traditional NLP tasks GLUE, LLM tasks GSM8K/MATH, and multimodal tasks text2image generation.", "weaknesses": "This paper claims that HRA inherits the theoretical guarantee of OFT on the retention of pretraining knowledge. However, this paper seems a bit overclaim about this without experiments back it up.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your appreciation of our work. **We have resolved your concerns about the limitations and the computational efficiency of HRA in the above general response.** For your remaining concerns, we provide our answer below.\n\n**Q1: The evidence on the retention of prior knowledge.**\n\n**A1:** Thanks for your constructive suggestion. To verify our claim, we fine-tune LLaMA-2 7B on the MATHQA dataset by LoRA and HRA, respectively, and check the degradation of model performance on classic NLP tasks, including typical language tasks in ARC, HellaSwag, MMLU, and Winogrande, and a coding task in HumanEval. For a fair comparison, we apply the same number of trainable parameters and the same batch size for LoRA and HRA. Ideally, after adaptation, we hope that the model can still maintain its high performance in the NLP tasks.\n\n| Model | ARC | HellaSwag | MMLU | Winogrande | HumanEval | \n| :-----| :----: | :----: | :----: | :----: | :----: |\n| LLaMA2 7B | 49.74 | 58.90 | 45.92 | 74.11 | 12.80 |\n| LLaMA2 7B fine-tuned by LoRA | 48.81 | 56.89 | 40.60 | 71.27 | 11.59 | \n| LLaMA2 7B fine-tuned by HRA | 49.57 | 57.72 | 41.20 | 73.32 | 13.41 | \n\nAccording to the above results, we find that compared to LoRA, HRA retains more of the original model's knowledge, whose performance degradation is less severe than LoRA's. In the HumanEval task, its performance is even better than that of the original model (which we think is because the MATHQA dataset contains many samples relevant to logic and reasoning tasks and thus is useful in the HumanEval task). \n\nWe will add this experimental result in the final paper. Thanks again for your suggestion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vt1tFk6KOv", "reviewer": "Reviewer_mtnZ", "summary": "Both low-rank and orthogonal adaptation techniques can effectively adapt large-scale pre-training models to downstream tasks. This work proposes an new adaptation methods based on Householder reflections(HR). It discloses the connection between low-rank and orthogonal adaptation and builds a unified adapter-based fine-tuning framework. the proposed method can further save the amount of learnable parameters and achieve superior performance when compared to existing methods.", "questions": "1. Section 3，an overview about the motivation, novelty and organization is needed at the beginning of section 3. It is rambling to start with the details of the proposed method.\n\n2. In the experiments, the parameter r is empirically set to some fixed value, please provide some advices on setting r for real application. \n\n3. What about the training cost and inference time of the proposed method when compared to the others?\n\n4. In section 4.3, which part of the model is applied with the proposed HRA? Does each layer has a similar r?\n\n5. citation 33 is published in 2023.", "limitations": "no. please talk about the risk of using such models in real applications, also the potential negative societal impact.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The disclosed relationship between low-rank and orthogonal adaptation is meaningful in building a unified adapter-based fine-tuning framework. The HRA technique saves a lot of learnable parameter，the experimental results are strong.", "weaknesses": "The structure of the paper is not well developed. The abstract and the introduction need some revision, the motivation for building a unified framework is not clear, what’s the pros and corns of the low-rank and orthogonal adaptation techniques? What kind of advantages are they trying to combine from both techniques? The authors mentioned the gap between the two techniques, does this gap have a negative effect on the adaptation?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your appreciation of our work and constructive comments. Below, we try to resolve your concerns one by one.\n\n**Q1: Improve the structure of the paper and highlight the motivation for building a unified adaptation framework.**\n\n**A1:** We have detailed the pros and cons of LoRA and OFT in the Related Work section. Regarding their advantages, LoRA assumes that weight changes during model adaptation have a low \"intrinsic rank,\" while OFT preserves pre-trained knowledge by maintaining the pairwise angles between neuron vectors. As for their drawbacks, LoRA cannot ensure the preservation of angles between neuron vectors, and OFT can only achieve low-rank weight updates when $(R−I)$ is low-rank. HRA combines these two strategies, leveraging their advantages jointly while suppressing their drawbacks simultaneously. \n\nMoreover, building a unified adaptation framework is insightful for revisiting the recent rapid development of various adaptation methods, which may help inspire new technical routes. We have placed an overview of the motivation, novelty, and organization at the end of the Introduction. We plan to polish our paper in the final version further. Thanks for your suggestion.\n\n**Q2: How to set r for real applications?**\n\n**A2:** **We have discussed the setting of $r$ in the above general response, which may have resolved your concern to some extent.** In this study, we set $r\\leq8$ for most datasets and tasks because this setting is sufficient to achieve superior adaptation results with fewer trainable parameters. Therefore, we empirically recommend starting with $r=8$. If the model performance is not satisfactory, increasing $r$ will usually work better. As we mentioned in the above general response, like AdaLoRA, we will consider adjusting the $r$ of HRA adaptively as our future work, but it is not the main contribution of this paper.\n\n**Q3: The training and inference time of HRA.**\n\n**A3:** **We have shown the training time and computational efficiency of HRA in the general response.** After training, we multiply the learned orthogonal matrices with the weight matrices, leading to a new model without increasing parameters. Therefore, like LoRA and OFT, HRA does not change the model's inference time. \n\n**Q4: In section 4.3, which part of the model is applied with the proposed HRA? Does each layer have a similar r?**\n\n**A4:** We follow the experimental settings of LoRA and OFT for a fair comparison. For the stable diffusion model, we apply HRA to its attention modules. In this study, we apply the same $r$ to the weight matrices of each attention module. Note that such a simple setting has resulted in superior performance. In the future, we can leverage the same idea of AdaLoRA, adjusting $r$ for different layers.\n\n**Q5: Reference about OFT. Citation 33 was published in 2023.**\n\n**A5:** Thank you for pointing out this mistake. We will correct it in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their appreciation of our work. Below, we provide a general response to their common concerns and a specific response to each reviewer's remaining questions.

**Q1: Discussions on limitations and societal impacts of this work.**

**A1:** Regarding the limitations of HRA, we believe the main concern is the setting of hyperparameters (i.e., the rank $r$ and the weight of orthogonal regularizer $\lambda$). Similar to LoRA, the rank $r$ of our HRA determines the trade-off between the number of trainable parameters and the training efficiency. In this study, we set $r$ to ensure that the number of our trainable parameters is smaller than those of baselines. Of course, inspired by the recent variants of LoRA, e.g., AdaLoRA, we can adjust the rank $r$ adaptively, which is not the main contribution of this work and thus is left to be our future work. 

For the weight $\lambda$, which determines the trade-off between the expressiveness and the regularity of the adapter, we set it in a range to quantitatively analyze the impact of orthogonality. In particular, following the suggestion of Reviewer 7LSp, we conducted mathematical reasoning experiments with different values of $\lambda$, and the results are as follows.

| Method | #Param | GSM8K | MATH | 
| :-----| :----: | :----: | :----: |
| $\text{LoRA}_{r=32}$ | 0.25% | 50.2 | 7.8 |
| $\text{OFT}_{b=16}$ | 0.13% | 50.1 | 8.4 |
| $\text{HRA}_{r=32,\lambda=\infty}$ | 0.12% | 52.8 | 9.2 | 
| $\text{HRA}_{r=32,\lambda=1e-1}$ | 0.12% | 53.6 | 8.3 | 
| $\text{HRA}_{r=32,\lambda=1e-4}$ | 0.12% | 56.3 | 9.3 | 
| $\text{HRA}_{r=32,\lambda=1e-8}$ | 0.12% | 53.6 | 8.6 | 
| $\text{HRA}_{r=32,\lambda=0}$ | 0.12% | 55.8 | 9.0 | 

We can find that a) the performance of HRA is relatively stable concerning the change of $\lambda$, b) in the wide range of $\lambda$, HRA is superior to the baselines, c) even if ignoring the regularizer $(\lambda=0$), our method still outperforms the baselines. These results demonstrate the effectiveness and robustness of implementing orthogonal adaptation based on Householder reflections. In the future, we will consider further analyzing the impacts of $\lambda$ in theory.

Regarding the societal impact of our work, we believe HRA can further simplify the adaptation of LLMs and promote more LLM-based downstream applications. Similar to LoRA and OFT, HRA may suffer from some potential issues like inappropriate (even illegal) abuse, amplifying the social prejudice intrinsically in LLM when the fine-tuning data are biased, and so on. **It should be noted that these potential issues are neither purely attributed to the technique itself nor specific to HRA—LoRA and OFT also suffer from them. Solving these issues depends on developing new techniques, social policies, and data quality improvement.** How to mitigate (even eliminate) these issues is left to our future work.

We will add the above content to the final version of our paper and the attached NeurIPS paper checklist.

**Q2: The comparisons for various methods on their training time and memory costs.**

**A2:** Following the suggestions of Reviewers 1vEG and mtnZ, we adapt LLaMA2-7B on the MetaMathQA dataset by HRA and other baselines and test their training time and GPU memory costs. For a fair comparison, we conducted all the experiments on 8 NVIDIA RTX A6000 GPUs, and all the methods applied the same batch size and almost the same number of trainable parameters.

| Method | #Param | Training time (hours) | Peak memory usage (GB) | GSM8K | MATH | 
| :-----| :----: | :----: | :----: | :----: | :----: | 
| LoRA | 0.12% | 45 | 279 | 47.3 | 6.6 |
| OFT | 0.13% | 53 | 282 | 50.1 | 8.4 |
| HRA | 0.12% | 30 | 287 | 56.3 | 9.3 |

We can find that HRA's peak memory usage is comparable to that of baselines, while its training time is less, and its accuracy in the downstream GSM8K and MATH tasks is better. These results demonstrate HRA's superiority in computational efficiency and adaptation performance.

We hope that the above response can resolve your concerns. Thanks again for your positive feedback.

Meta-review: This paper presents an interesting idea based on Householder reflections to bridge the gap between two prevalent fine-tune methods, low-rank adaptation and orthogonal adaptation. After author-review discussions, the reviewers have reached consensus that the contribution of the paper is solid and significant, unanimously recommending accepting the paper. I would strongly recommend accepting the paper.
Decision: Accept (spotlight)