Learn more, but bother less: parameter efficient continual learning
Authors: Fuli Qiao
Mehrdad Mahdavi
Venue: NeurIPS 2024 poster (2024)
Keywords: continutal learning
LLMs
Primary area: optimization_for_deep_networks
TL;DR: We propose a parameter-efficient approach for continual learning in LLMs

Abstract: Large Language Models (LLMs) have demonstrated profound capabilities due to their extensive pre-training on diverse corpora. However, LLMs often struggle with catastrophic forgetting when engaged in sequential task learning. In this paper, we propose a novel parameter-efficient approach for continual learning in LLMs, which empirically investigates knowledge transfer from previously learned tasks to new tasks through low-rank matrix parameters, enhancing the learning of new tasks without significant interference. Our method employs sensitivity-based analysis of low-rank matrix parameters to identify knowledge-specific parameters between sequential tasks, which are used to initialize the low-rank matrix parameters in new tasks. To maintain orthogonality and minimize forgetting, we further involve the gradient projection technique that keeps the low-rank subspaces of each new task orthogonal to those of previous tasks. Our experimental results on continual learning benchmarks validate the efficacy of our proposed method, which outperforms existing state-of-the-art methods in reducing forgetting, enhancing task performance, and preserving the model's ability to generalize to unseen tasks.

Learn more, but bother less: parameter efficient continual learning
Fuli Qiao
Mehrdad Mahdavi Pennsylvania State University mzm61 1ww.1660puueeem edu
Pennsylvania State University fvq5015dpsu.
Abstract
Large Language Models (LLMs) have demonstrated profound capabilities due their extensive pre-training on diverse corpora. However, L.MSs often struggle with catastrophic forgetting when engaged in sequential task learning. In this paper, we propose novel parameter-efficient approach for continual learning in LLMs, which empirically investigates knowledge transfer from previously learned tasks new tasks through low-rank matrix parameters, enhancing the learning of new tasks without significant interference. Our method employs sensitivity-biat of low-rank matrix parameters to identify knowledge-speee specific parameters between sequential tasks, which are used to initialize the low-rank matrix parameters new tasks. To maintain orthogonality and minimize forgetting, we further involve the gradient projection technique that keeps the low-rank subspaces of each new task orthogonal to those of previous tasks. Our experimental results on continual learning benchmarks validate the efficacy of our proposed method, which out- performs existing fate-of-thee-rt methods in reducing forgetting, enhancing task performance, and preserving the model's s ability to generalize to unseen tasks.
analysis
1
Introduction
Large Language Models (LLMs) 5 32 40) have demonstrated exceptional performance across a broad spectrum of tasks, significantly revolutionizing the landscape in diverse areas driven by artificial intelligence. Full fine-tuming all the parameters of the pre-trained models becomes prohibitive in adapting pre-tral trained models
large number of downstream tasks. Thus, following LoRA (II), multiple variants [24] 50 of low-rank adaptation have been proposed to prompt parameter-efficiie learning for L.MMs. AdaLoRA (50) enhances the flexibility of low-rank matrices for various device budgets by parameterizing the increments through singular value decomposition. It generalizes the matrices, enabling layer-specific rank adjustments to meet different parameter constraints.
Despite the tremendous success of pre-trained models on static tasks, learning multiple sequential tasks, known as continual learning (CL), still poses significant challenges [45]. There are two primary challenges: (i) overcoming catastrophic forgetting, where a model's ''s performance on previous tasks significantly deteriorates upon training with data [28 34]. and (ii) facilitating forward transfer, where knowledge from old tasks is harnessed to enhance the learning of new tasks. In the context of LLMs, continual learning extends beyond merely enhancing linguistic and reasoning capabilities and it encompasses a and continual alignment [49].
process, including continual pertaining [12, continual instruction [2
While existing parameter-efficient tuning (PET) methods for CL. 27
have primarily focused on mitigating the forgetting issue, they often overlook the equally important objective of facilitating forward knowledge transfer. This transfer crucial as it allows the model to leverage previously acquired knowledge to better generalize to new tasks or domains. For instance, O- LoRA operating within the PET framework, proposes an orthogonal low-rank adaptation
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

for continual instruction. This method incrementally learns new tasks in an orthogonal subspace, ensuring that the LoRA parameters from previous tasks remain fixed to minimize forgetting. Besides, compared to full fine-tuning, LoRA inherently forgets less of the source domain, serving form of regularization However, O-LoRA does not explicitly address knowledge transfer across different
tasks.
Focusing on knowledge transfer among tasks, many existing non-PET knowledge transfer methods in CL, such as Progressive Network [37] which tackles forward knowledge transfer, and CUBER 22 which employs backward knowledge transfer, while having distinctive ap- proaches to managing knowledge across tasks, they are not directly applicable to the continual learning setting in the PET framework due to their prohibitive computational costs. Although there have been recent attempts at parameter-effiiiens methods have failed when applied to CL. For instance, while the parametric knowledge transfer paradigm [53] in PET harnesses the rich knowledge embedded within a teacher's parameters by extracting task-specific parameters and injecting them into a student model via sensitivity metrics, such methods do not exist in CL for LLMs.
yahoo
fine-tuning of LLMs, these
reptare
Sipaes loyers
Feptace top leprrs
Figure Comparison performance of the model after training task To with different layers replacement.
To investigate the impact of leveraging parametric knowledge in the incremental learning of LoRA parameters on a new task, we conduct an experiment where layers from the previous task's s LoRA-based layers are simply replaced at the training initialization the new task's LoRA layers, without a predefined strategy for selecting key parameters. The results, presented in Tab. | and Fig. D Jusing the pre-trained model T5- large [32] with incremental LoRAs for Amazon Reviews and Yahoo Answers M1 indicate that this simple layer replacement can enhance performance. Notably, Tab. reveals that the best replacement strategy significantly improves the performance of both the new task To and the old task Ti compared to scenarios with replacement.
Tahle I: Testing accurncy Ti and T2 after training Ta with different layer replacements, highlighting the hest performing strategy as shown Fig,
Ti (amazon) + To (yahoo)
no
top 4 top
all
Ti
15.4
15.0
16.6 0.2
T2
73.6 73.9 73.7
73.3
(yahoo)
+T (amazon)
The aforementioned observations motivate that an ideal approach for continual learning of LLMs in the PET framework should take the best of both worlds by simultaneously overcoming catastrophic forgetting and promoting forward transfer for enhanced generalization across a continual stream tasks. Motivated by these needs, we seek to explore new dimension in CL for LLMs:
no
top
top 9
Ti
46.8 50.7
39.1
42.5
T5
52.9
54.9
54.5
53.9
How can we effectively inject knowledge from previous tasks into new tasks (for improving generalization) while maintaining the orthogonality of each task's low-rank subspaces (for mitigating forgetting) to facilitate purameter-effiiien continual learning?
To answer this question, we propose a novel all-for-all parameter-effiient approach for continual learning in LLMs, which empirically investigates knowledge transfer from previously learned tasks to new tasks using low-rank matrix parameters evaluated by sensitivity scores while maintaining orthogonality via gradient projection techniques. We name this method LB-CL (Learn more but bother less Continual Learning). Specifically, LB-CL first calculates sensitivity metrics for SVD- based low-rank parameters of previous tasks guide the injection of parametric knowledge into new task parameters. Then, it supports incremental learning of new tasks in orthogonal subspaces by preserving low-rank parameters of previous tasks. When transferring knowledge, LB-CL prioritizes low-rank triplets (consisting of a singular value and its corresponding singular vectors) from past tasks based on their sensitivity scores, enabling the new task to learn more from higher-scoring triplets. To preserve the orthogonality of low-rank subspaces, we project the gradients of the new task onto the subspaces formed by previously learned low-rank triplets, encouraging the new task to deviate appropriately from more impactful triplets. Our experimental results validate that LB-CL surpasses previous state-of-the-art methods on standard continual learning benchmarks. Moreover, our analysis highlights the critical role of initialization strategies in promoting generalization through effective parametric knowledge transfer, while also using low-rank orthogonal gradient projection to mitigate catastrophic forgetting.

---

frozen weights
trainable weights
Student
Orthogonal Gradient Update
V4+
Training Data (Task T(41)
Training
V
Injection
Training
Teacher
E U
I
Random Sample
Extraction
+
7 Merge
Inference
Pre-trained
V
V
V+I
Seed Samples
Σ
Σ'
+
Model W
U
U
UTII
Figure 2: Overview of our LB-CL framework. Starting with the pre-trained model including SVD weights of previous tasks, sensitivity metrics are calculated using set of seed samples, facilitating the extraction of task-specifie knowledge. Subsequently, the extracted layer triplets initialize SVD weights for the new task. Then, the new task trained in an orthogonal subspace, employing orthogonal gradient projection to minimize forgetting.
Summary of Contributions. This paper makes three key contributions: (1) A novel parameter- efficient continual learning framework for LLMs that balances generalization through parametric knowledge transfer and the mitigation of forgetting through low-rank orthogonal subspace learning for new tasks; (2) Through comprehensive evaluations, our method demonstrates superior performance over existing state-of state-of-the-aa of-the-art approaches on standard continual learning benchmarks; and (3) We provide an in-depth analysis that deepens our understanding of the dynamics of parametric knowledge transfer within continual learning for LLMs, pinpointing critical factors that drive its effectiveness.
2 Continual Learning Maestro: Learn More but Bother Less
We consider a continual learning scenario where the learner is presented with a sequence of tasks (T, ......, ; over time. Each task Te is associated with data distribution Dj. and contains a separate target dataset Sş = { { It. where Fyg E Xx and yk.. E Vk- The goal of continual learning is find set of parameters O E θ that can effectively solve all tasks up to the current task Tk. while minimizing catastrophic forgetting of previously learned tasks. In continual learning of LLMs, we are given a pre-trained model Wo. and would like continually fine-tune on a sequence of tasks, utilizing the incremental SVD-based low-rank matrix parameters to fine-tune on task TE. where U e Rd,xr and V E Rrxd2 represent the left/right singular vectors and the diagonal matrix E² E -eerr contains the singular values {Ailisisr with r & min (d1, d2). To enforce the orthogonality of U and V, i.e. U VV = I, we use the regularizer: RUU, V) - 1 + V|V I| The continual learning model parameters after fine-tuning on task Te is: 0s = Wo + > """""" Our continual learning goal is to optimize the following objective across all tasks:
max Eieyee log (o(y)x)
where 0 W WI "' Our method contains two important stages: (i) Learning from knowledge extraction and injection, which transfers knowledge from previously learned tasks to new tasks by incremental SVD triplet sensitivity metric; (ii) Training in Orthogonal Subspaces, which keeps the low-rank subspaces of new tasks orthogonal to those old tasks. The detailed description of our proposed method, LB-CL, is shown in Fig. 2
Generalization and Forgetting Tradeoff of Low-rank Finetuning. Before delving into the detailed steps of proposed algorithm, here we motivate these steps by investigating the forgetting and generalization tradeoff of SVD based low-rank parameter-efficient methods in CL. The performance of CL. algorithm is typically evaluated using two key metrics 3
[6] 21 23 25 as defined below:
(i) Forgetting error: which quantifies the performance degradation on previously learned tasks after acquiring new task formulated as:
3

---

0r) En C
(2)
where 0, - E U* L((-) is the generalization error on task Tr, and (z
is the performance degradation (forgetting) on tasks Ti between the model after training on task Tr and the model after training on the final task Tr.
(ii) Generalization error: which measures the algorithm's capability to effectively learn a new task while preserving the knowledge gained from prior tasks defined as:
0T) = Σ _ (.) - (.)
I..
where Le(O,) - (,)) measures the gencralization gap between the CL model Or and the optimally fine-tuned model 0 = Wo U. U.,.V V', on task Te. The generalization of final model on all tasks can be decomposed in terms of forgetting generalization emrrs as follows:
T Σn _0,0(0,)) (0) F(O........ 0r) + I(0
It is crucial to note that the generalization error is based on the final model over all tasks, which may not fully capture each task-specific optimal setting since each task could potentially achieve better performance metries when fine-tuned individually. Given the complexities in addressing our generalization error due to limited theoretical support, we utilize initialization parameters to further decompose the per-task generalization term on the left-hand side of Eq. 4 This involves measuring the performance difference between the CL. model 0g after learning on tasks Tj and its initialization 0 evaluated on task T; as:
(((0))-C(0)))) + (c.(0)) + -c(00)))).(00))
( -
(D: forgetting error
(II) improvement by fine-tuning
(III:: generalization of initial model
From this decomposition, it is clear that all three terms must be carefully calibrated to enhance the overall performance of CL algorithm. In the context of SVD-based low-rank fine-tuning, we have:
The term (I) captures the forgetting error. It is evident that models that undergo minimal changes when fine-tuned a new target domain will exhibit less forgetting of the source domain. This automatically holds for fine-tuning with low-rank updating methods as it acts as regularization. This can be further amplified by fine-tuning orthogonal subspace.
The term (II) captures the performance of fine-tuning algorithm itself. Since fine-tuning with low-rank updates is not capable of approximating full fine-tuning accuracy (2), and in order to balance between stability (retaining old knowledge) and plasticity (acquiring new knowledge) we need to leverage methods such as orthogonal gradient projection to better align with current task.
The term (III) captures the generalization of the initial model. Since the knowledge transfer from previous tasks happens through the initial model, naive averaging of low-rank models from previous tasks might entail dramatically poor generalization on the current task, in particular when there is a larger domain shift among tasks. Consequently, the contribution of each task should be proportional to its similarity to the current task, measured by leveraging effective discrepancy estimation methods. We formulate this process as the parametric knowledge extraction:
β arg min Le W o Σ E A
where 3 == [31 , ...., 8(-1JT
extracts optimal coefficients approximating the influence of new task data on each previous task SVDD-aaed low-rank adapters. 'hen the parametric knowledge injection in the CL. model initialization for task T, is constructed follows:
0 0 W. + - k- V*EV* - a &U'''V' where 1 a = 3
(6)
In summary, while updating with low-rank models in CL. can provide an implicit regularization effect to mitigate the forgetting issue, their generalization capability is limited due to the low-rank perturbation of parameters. This necessitates effective initialization to overcome the generalization limitations of low-rank updates. Specifically, an initialization that can effectively capture the shared knowledge across tasks based on their similarities to augment the knowledge captured by the pre- trained model is required to facilitate knowledge transfer and improve generalization when performing low-rank updates in CL setting.

---

2.1 Learning from Knowledge Extraction and Injection
Sequential Task Low-rank Adapter. In our continual learning framework, for a pre-trained weight matrix Wo. o. it parameterizes the incremental updates for task To by SVD-based low-rank matrices: 0,- 0z- A Wo + E U + U'E'V. Similar [50], define Jh, = (U Af,3 Vi.s. as the i-th triplet (i-th singular value and its vectors) of layer I for task Tr, which is different from the dependent and not orthogonal doublet in LoRA-based low-rank matrices. With the SVD-based flexible regularizer, the training objective for task Tr represented as (U Σ', V*) (U E', V*) +')+>>> R(U,, V.), where is the number of weight matrices and 7 > 0 is the regularization coefficient.
Sensitivity of the Parameters. Parameter sensitivity quantifies the impact on the loss function when a specific parameter is altered, typically set to zero, providing insight into the parameter's s importance [14 19 26 30 [53]. Consider teacher model with parameters O [ 0 N..
where N. denotes the total number of parameters, the i-th parameter is 0, = [0, .0.....e S......07 0.. The sensitivity St, for sample 2y from task T is calculated as S.j Vo
Re [(0)- 0,)1
where the approximation uses the first-order Taylor expansion of (-)) at A, Thus, we formulate the sensitivity of each triplet of low-rank matrices, rather than each parameter of the model. By masking the 1-th layer i-th triplet Gf (U Mr, V of low-rank matrices for previous task Ts. following [50], we can obtain the sensitivity score of the I-th layer i-th triplet of low-rank matrices for previous task Tk. where E {I...., t 1}. on new data sample 2t from new task Te:
da
de Σ s(Vi.)) 1=1
st. = S(
dj Σ S(U
d2
Knowledge Extraction and Injection. For simplicity in analysis and mathematical expression, we assume that each layer within every task shares the same SVD-based low-rank adapter rank r. Our first step is to assess the layers of the previous task Ts low-rank matrices where {l...., t 1}. by Eq. 7 we calculate triplct-specific sensitivity scores in layer and this layer sensitivity score is represented as S = Ei= St. To preserve the inherent structure of the tasks' low-rank adapters, the layers are subsequently mapped to the new task Tr adapter maintaining their original sequential order. Having computed the sensitivity scores of all triplets for layer I in previous tasks (Ti.... TT-1), we proceed to arrange them with weighted sensitivity scores to initialize the layer in new task Te:
Gi. (U
1ess M1. Vi,..)
at
U Loii
ai
V Lio
(8)
km
Amm
St Ei= St
where af,, is the weight for the components of the triplet Gi, and af, sensitivity founded in Eq. By aggregating weighted triplets across layers, we derive the extracted triplets from previous tasks' adapters and inject these triplets into the new task's adapter.
obtained from the
|
2.2 Training in Orthogonal Subspaces
We utilize the SVD-based adapter's low-rank subspace to represent the gradient subspaces of previous tasks, asserting that these parameters capture essential model update directions rather than just numerical adjustments (43. This hypothesis allows us to minimize interference with previously learned tasks by training within a subspace orthogonal to that of the SVD-based subspace. We approximate the layer subspace for task Ta as the subspace consisting of the triplets g To ensure orthogonal to the previously learned tasks layer subspaces, we first project the gradients of layer triplets of new task T, onto the previously learned tasks layer subspaces spanned by {g²) where k {l....,t 1}, and then make the gradients far away from these subspaces. The gradients of layer / triplets of data from new task T, become:
(g = Lg [
proj Lc
(9)
where proj v) (u,0) (0,0) is the projection of in the direction of v
5

---

Experiments
3.1 Experimental Setup
Our experiments employ the encoder-decoder architecture of the T3-large and T5-base models [2), consistent with previous work in CL for NLP. All experiments conducted NVIDIA A6000 GPUs, utilizing the DeepSpeed repository.
Standard CL benchmark. We evaluate approach using CL benchmark specifically designed for language models. This benchmark comprises five text classification datasets: AG News, Amazon Reviews, Yelp Reviews, DBpedia, and Yahoo Answers, as introduced by [5T]. We adhere to the CL setup for the T5 model as outlined in LFPT5 31 and experiment with three different task orders within this benchmark.
Large number of tasks. Our method's ' efficacy is further tested on extended task sequences through comprehensive CL. benchmark encompassing 15 datasets, as detailed in [35]. This benchmark integrates tasks from three sources: five from the standard CL benchmark, four from the GLUE benchmark (MNLI, QQP, RTE, SST-2), five from the SuperGLUE benchmark (WiC, CB, COPA, MultiRC, BoolQ), and the IMDB movie reviews dataset. For each task, we train using 1000 randomly selected samples and validate using 500 samples per class, following the methodology of (35).
Metrics. We define the testing accuracy on the task Ti after training on the task Tj as 0s.j- The main metric for evaluation is Average Accuracy (AA), calculated as the mean accuracy across all
tasks after training on the last task: + Σi= a,,T-
Baselines. We compare our method against various baseline approaches:
sequence of tasks (without adding any regularization SeqFT 3 train all model parameters
replaying samples from the previous tasks).
SeqLoRA: fixed-size LoRA parameters are trained on sequence of tasks (without adding any regularization or replaying samples from the previous tasks).
IncLoRA: incremental learning of new LoRA parameters on a sequence of tasks (without adding any regularization replaying samples from the previous tasks).
SeqSVD: fixed-size SVD parameters are trained on a sequence of tasks (without adding any regularization or replaying samples from the previous tasks).
Replay: fine-tune the whole model with a memory buffer, and replay samples from old tasks when learning new tasks to avoid forgetting.
EWC [33: fine-tune the whole model with a regularization loss that prevents updating parameters that could interfere with previously learned tasks.
LwF [18]: constrains the shared representation layer to be similar to its original state before learning the new task.
L2P [44] uses the input to dynamically select and update prompts from the prompt pool in an instance- wise fashion.
LFPT5 31: continuously train a soft prompt that simultaneously learns to solve the tasks and generate training samples, which subsequently used in experience replay.
L-CL: incremental learning of new SVD parameters on sequence of tasks with initialization and SVD regularization.
B-CL: incremental learning of new SVD parameters on a sequence of tasks with gradient projection
and SVD regularization.
NLNB-CL: incremental learning of new SVD parameters on sequence of tasks (with adding only SVD regularization and without replaying samples from the previous tasks), also called IncSVD. ProgPrompt [35] adopts task-specific soft prompts for each task, training distinct models per task and using task IDs during inference.
O-LoRA 43 incrementally train new tasks in an orthogonal subspace while fixing the LoRA
matrices of previous tasks.
PerTaskFT: train separate model for each task.
MTL: train model on tasks as multi-task learning, serving as the benchmark's upper bound of the performance limit.
3.2 Main Results
presents a performance comparison LB-CL and baseline continual learning methods across two CL benchmarks. Following LFPT5, we report results from three random runs with different task orders on the CL benchmark. For fairness, we use the same rank of the LoRA-based and SVD-based matrix in each corresponding comparison experiment. To reduce computation time,
Tab.

---

Table Testing performance on two standard CL benchmarks with T5-large.
Standard CL Benchmark
Large Number of Tasks
Order-1
Order-2
Order-3
avg
Order-4
Order-5
Order-6
avg
SeqFT
18.9
24.9
41.7
28.5
7.4
7.3
7.4
7.4
S
39.5
31.9
46.6
39.3
4.9
3.5
4.2
4.2
IncLoRA
63.4
62.2
65.1
63.6
63.0
57.9
60.4
60.5
SDqSVD
40.0
63.3
44.9
49.4
13.7
13.8
12.2
13.2
Replay
50.3
52.0
56.6
53.0
54.5
54.3
53.5
54.1
EWC
46.3
45.3
52.1
47.9
44.9
44.0
45.4
44.8
LwF
52.7
52.9
48.4
51.3
49.7
42.8
46.9
46.5
L2P
59.0
60.5
59.9
59.8
57.7
53.6
56.6
56.0
LFPTS
66.6
71.2
76.2
71.3
69.8
67.2
69.2
68.7
L-CL
75.3
73.5
71.9
73.6
66.5
64.0
69.0
66.5
B-CL
76.4
71.5
75.1
74.3
65.7
66.4
69.2
67.1
NLNB-CL
76.0
73.4
74.0
74.5
67.6
65.3
62.6
65.2
O-LoRA
74.9
75.3
75.9
75.4
70.5
65.5
70.5
68.8
LB-CL.
76.9
76.5
76.8
76.7
68.4
67.3
71.8
69.2
ProgPrompt PerTaskFT 70.0
76.1
76.0
76.3
76.1
78.7
78.8
77.8
78.4
70.0
70.0
70.0
78.1
78.1
78.1
78.1
MTL
80.0
80.0
80.0
80.0
76.3
76.3
76.3
76.3
we reasonably focus on some high-level layers to narrow the searching range of exploring critical parametric knowledge, since previous research, c.g. 50), has already demonstrated that high-level layers are important for model performance.
Results on Standard Continual Learning Benchmarks. Across task orders the standard CL benchmark, LB-CL consistently surpasses previous methods by a significant margin. Notably, LB-CL achieves performance improvements all task orders compared to O-LoRA, the prior state-of-the-an Our approach demonstrates performance on par with multi-task learning and significantly exceeds that of Per TT. This indicates that LB-CL. not only effectively prevents catastrophic forgetting but also efficiently utilizes knowledge from prior tasks enhance the learning of new tasks.
Performance with Large Number of Tasks. In a more demanding benchmark featuring large number of tasks, LB-CL surpasses the of O-LoRA, in terms of average performance across three task orders. Although ProgPrompt shows superior performance managing long- notable. ProgPrompt is strictly dependent
sequence tasks, its limitations been trained on and relies heavily on task IDs during inference, which restricts its generalization capabilities and adaptability for use in LLMs. However, LB-CL does not use task ID during testing, which keeps its generalization. It is important to note that nearly all existing continual learning methods fall significantly short of the performance achieved by PerTaskFT and MTT,, underscoring that continual learning with large array of tasks remains a formidable challenge.
the tasks it has
3.3 Discussions
How do initialization and gradient projection effectively influence the performance of LB-CL? We systematically evaluated the influence of two components: initialization and gradient projection. The results across various orders and their average performances, as presented in the last rows in Tab. 2 reveal distinct trends. L-CI., which employs only initialization, suggests that while this component does provide beneficial starting point for task learning. it falls short maintaining knowledge across tasks in Order 3. B-CL, utilizing only gradient projection, demonstrates slightly higher performance than L-CL in Order and Order However, the absence of initialization limits its effectiveness, particularly in establishing a robust foundation in Order 2. NLNB- which neither initializes nor employs gradient projection, surprisingly performs slightly better than both L-CL and B-CL on average, but it does not excel in any specific order. This outcome suggests that the model may possess inherent adaptive capabilities or rely on other compensatory mechanisms. LB-CL, integrating both initialization and gradient projection, exhibits the highest overall performance with consistent scores across all orders. This robust performance indicates that the synergistie contribution of both components significantly enhances the model's ability to effectively handle CL tasks.

---

How do different initialization strategies affect the performance? To elucidate the impact of distinct initialization strategies, we compare two different initialization strategies motivated by 29 to initialize new task T:: (i) with previous tasks low-rank matrix triplets g and (ii) without the singular values Σ but leveraging {U*, V*} where & E fl....,t 1). These strategies are assessed using Ta-large model over three task orders in the standard CL benchmark. Our results, depicted in Fig. 3 reveal that utilizing only {U, ] from prior tasks' triplets surpass the full triplet configuration in average performance across three task orders. While "with E" strategy exhibits peak performance in Order 3, "without Σ" approach demonstrates better consistency and stability. This suggests that excluding Σ may lead to more robust generalization across diverse tasks, but it cannot fully represent previously learnt tasks' important subspaces, thus we use "with "n as the initialization of LB-CL, and "without V" can be used as an improvement strategy in implementation. Furthermore, both strategies outperform the performance of the O-LoRA method over these three orders.
78
Order 1
Order 2
--. Avaraee time erders
Order 3
Accuracy
76
75
tosting
74
73
with
without
Number Samples
36
32
Figure 3: Comparison of different initialization strategies across three orders of standard CL benchmark. The "Avg" value represents the aver- age testing accuracy, illustrating how each strat- egy stabilizes learning performance.
Figure 4: Impact analysis of seed sample quan- tity on the performance in LB-CI,, evaluated across three orders of standard CL. benchmark. This investigation highlights the influence of ini- tial seed samples on model effectiveness.
How does the number of seed samples affect the performance of the model? The number of seed samples significantly influences the reliability and efficiency of the sensitivity score computations derived from the teacher model. We explored the impact of varying the number of seed samples from new tasks on the sensitivity of parameters from previous tasks. As illustrated in Fig. 4 increasing the number of seed samples improves performance incrementally at a slight rate. Notably, the variance in performance metrics is considerably lower with 4 and 8 seed samples. Based on these findings, we have selected 8 seed samples as the optimal number for our hyperparameter setting.
How does the training computation cost perform? We compare the training computation cost between LB-CL and O-LoRA in Tab. B] using Order in the standard CL benchmark with Ta-le model. Tab. 3 shows that the GPU memory consumption of both methods is similar, indicating comparable resource efficiency during training. For the number of training parameters, we examine single layer and denote dimension, and n as the output dimension of the layer. Given that r < min nt,, the number of training parameters for both methods remains close, further highlighting their efficiency in parameter.
the rank of SVD-based and LoRA-based matrix, m as the input
Table 3: Comparison of training computation cost between LB-CL. and O-LoRA.
Method
GPU Memory
Num of training params/task
O-LoRA
24.82 GB
r(m + n) r ((m + n) +r
LB-CL
28.28 GB
What's the distribution of parametric knowledge across layers of the model? We analyze the distribution of parametric knowledge across the model's layers to identify those most critical for retaining task-specific information. In Fig. our analysis of sensitivity and Fisher information on the average of three task orders in standard CL benchmark, reveals that higher-level layers, particularly in decoder layers, exhibit significant sensitivity. Particularly, the top layers of the decoder are notably sensitive, suggesting that focusing sensitivity analyses on these layers could represent the entire decoder effectively, thus reducing computational demands. Validation with Fisher information confirms that these high-level layers are crucial in both the encoder and decoder, especially the top 3 layers of the decoder. This alignment underscores that our sensitivity scores effectively identify the most crucial layers for task-specitt knowledge transfer. Given the fime-intessvv nature of
8

---

Encoder
Decoder
Laye
(a) Sensitivity Scores
(b) Fisher Information
Figure 5: Comparison of sensitivity scores and Fisher information of encoder and decoder Layers, and both results the average results of three task orders in standard CL benchmark.
Fisher information calculations, sensitivity score approach provides more efficient alternative, enhancing training efficiency by focusing on several critical high-level layers.
Table 5: Comparisons of different models' per- formances across three task orders in standard CL benchmark.
Table 4: Comparisons of different rank of low- rank matrix. This experiment is conducted based on Targe in standard CL. benchmark.
(T5-base)
Order
Order
Method
1
3
avg
r-dim
2
avg
O-LoRA
72.9
72.3
72.6
72.6
2
76.7
77.2
75.2
76.3
LB-CL
73.8
74.4
72.4
73.5
4
77.0
76.8
75.9
76.6
(T5-large)
Order
8
76.9
76.5
76.8
76.7
16
77.4
76.0
75.5
76.3
Method
1
3
avg
Std
0.25
0.44
0.60
0.18
O-LoRA
74.9
75.3
75.9
75.4
LB-CL
76.9
76.5
76.8
76.7
What's the optimal rank r for LB-CL? To explore the impact of the rank r on the performance of LB-CL, we conduct experiments using the T5-large model on the standard CL benchmark. The results, presented in Tab, 4 examine how varying affects the accuracy across different task orders. It shows that increasing the rank does not lead to significant improvement in model performance. Furthermore, the small standard deviations across different orders for each rank underscore the model's consistent performance, irrespective of rank variations. This suggests that by leveraging more knowledge from previous tasks, our method allows the gradient space of the new task to diverge more significantly from those of prior tasks, thereby enhancing stability across various ranks.
How do different pre-trained models influence performance? We investigate the impact of model scale performance by comparing T5-base and T5-large models using standard CL benchmark. We evaluate both our method and O-LoRA across three task orders. The results, presented in Tab. 5. clearly demonstrate significant performance differences between the two model sizes and the methods employed. For the T5-base model, LB-CL consistently outperforms O-LoRA. While for the T5-large model, LB-CL significantly surpasses O-LoRA's outcomes. Moreover, LB-CL shows exceptional consistency across all task orders in the T5-large model, highlighting its robustness and effectiveness when scaled up. This analysis confirms the influence of model size the success of different continual learning strategies, with LB-CL, proving particularly effective in larger models.
4 Conclusion
In this paper, we investigate the balance between overcoming forgetting and achieving generalization in the continual learning of L.LMs, decompose the generalization error with the task low-rank matrix initialization, then propose a novel framework, exploring parametric knowledge transfer between tasks and utilizing the inherent forgetting less ability of low-rank matrix. Instead of storing extra task- specific auxiliary parameters, we just utilize the low-rank parameters which would be merged into the pre-trained model. Our experiments across standard CL benchmarks validate the effectiveness of this approach. Furthermore, we analyze the critical factors influencing initialization in CL, providing insights for further enhancements this field.

---

Acknowledgment
This work is partially supported by NSF CAREER Award #2239374 and NSF EFMA Award 2318101.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[2] Dan Biderman, Jose Gonzalez Ortiz, Jacob Portes, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, al. Lora learns less and forgets less. arXiv preprint arXiv:2405.09673, 2024.
[3] Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference on computer vision (ECCV), pages 532-547,
[4] Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny, Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.0044 2018.
[5] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1 113, 2023.
[6] Cyprien de Masson D' Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory in lifelong language learning. Advances in Neural Information Processing Systems, 32, 2019.
[7] Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics, pages 3762-3773. PMLR, 2020.
[8] Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. Continual relation learning via episodic memory activation and reconsolidation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6429 6429-6440, 2020.
[9] Junxian Hc, Chunting Zhou, Xuezhe Ma, Taylor Kong-Kirkattrikk. and Graham Neubig, To- wards unified view of parameter-efficient transfer learning. arXiv preprint arXiv:21 10.00366. 2021.
[10] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly, Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790-2799. PMLR, 2019.
[11] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2
2021.
[12] Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang- Wen Li, Xiaokai Wei, Andrew Amold, and Xiang Ren. Lifelong pretraining: Continually adapting language models to emerging corpora. arXiv preprint arXiv: 10.08533,, 2021.
[13] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 1 114(13):3521-335 2017.
[14] Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. Snip: Single-shot network pruning based on connection sensitivity, arXiv preprint arXiv:1810.02340.22 2018.
10

---

[15] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning, arXiv preprint arXiv:2104.0866 2021.
[16] Haoran Li, Jingfeng Wu, and Vladimir Braverman. Fixed design analysis of regularization-ban continual learning. In Conference on Lifelong Learning Agents, pages 513-533. PMLR, 2023.
[17] Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In International conference on machine learning, pages 3925-3934, PMLR, 2019.
[18] Zhizhong Li and Derek Hoiem. Learning without forgetting, IEEE transactions on pattern analysis and machine intelligence, 40 40(1)):933522947, 2017.
[19] Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, and Tuo Zhao. No parameters left behind: Sensitivity guided adaptive learning rate for training large transformer models. arXiv preprint arXivv2202.02666 02600, 2022.
[20] Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference-free free low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23638-23644 2024.
[21] Sen Lin, Peizhong Ju, Yingbin Liang, and Ness Shroff. Theory forgetting and generalization of continual learning. In International Conference on Machine Learning, pages 21078-20 200000 PMLR, 2023.
[22] Sen Lin, Li Yang. Deliang Fan, and Junshan Zhang. Beyond not-forgetting: Continual learning with backward knowledge transfer. Advances in Neural Information Processing Systems, 35:16165-16177, 2022.
[23] Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Trgp: Trust region gradient projection for continual learning. arXiv preprint arXiv:2202.02931. 2022.
[24] Shih- Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Swang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation, 2024.
[25] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
[26] Ekdeep Singh Lubana and Robert P Dick. A gradient flow framework for analyzing network pruming. arXiv preprint arXiv:2009.11833 2020.
[27] Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, and Zhiguang Wang. Continual learning in task-oriented dialogue systems. arXiv preprint arXiv:2012.1 20ti::002..5504, 2020.
[28] Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109- 165. Elsevier, 1989.
[29] Nikhil Mehta, Kevin Liang, Vinay Kumar Verma, and Lawrence Carin. Continual learning using a bayesian nonparametric dictionary of weight factors. In International Conference on Artificial Intelligence and Statistics, pages 100-108. PMLR, 2021.
[30] Michael C Mozer and Paul Smolensky. Skeletonization: A technique for trimming the fat from a network via relevance assessment. Advances in neural information processing systems, 1,
1 1988.
[31] Chengwei Qin and Shafiq Joty. Lfpt5: A unified framework for lifelong few-shot language learning based on prompt tuming of 15. arXiv preprint arXiv:2110.072988 2021.
[32] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1-67, 2020.
11

---

[33] Dushyant Rao, Francesco Visin, Andrei Rusu, Razvan Pascanu, Yee Whye Teh, and Raia Hadsell. Continual unsupervised representation learning. Advances in neural information processing systems, 32, 2019.
[34] Roger Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
[35] Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. Progressive prompts: Continual learning for language models. In The Eleventh International Conference on Learning Representations, 2023.
[36] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Expe- rience replay for continual learning, Advances in neural information processing systems, 32, 2019.
[37] Andrei Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: 1606. 04671, 2016.
[38] James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual diffusion: Continual customization of text-to-image diffusion with c-lora. arXiv preprint arXiv:2304.0602. 2023.
[39] Chenyang Song, Xu Han, Zheni Zeng, Kuai Li, Chen Chen, Zhiyuan Liu, Maosong Sun, and Tao Yang. Conpet: Continual parameter-efficient tuning for large language models. arXiv preprint M.Ki::2309.17660 2023.
[40] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Ama Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.1399 13971, 2023.
[41] Alex Wang. Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32, 2019.
[42] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman Glue: multi-task benchmark and analysis platform for natural language understanding. arXiv preprint
2018.
[43] Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. Orthogonal subspace learning for language model continual learning, arXiv preprint arXiv:23 /41 2023.
[44] Zifeng Wang. Zizhao Zhang, Chen Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 139-144, 2022.
[45] Tongtong WU, Massimo Caccia, Zhuang Li, Yuan-Fang Li, Guilin Qi, and Gholamreza Haffari. Pretrained language model in continual learning: A comparative study. In International conference on learning representations, 2021.
[46] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv: /708.0154445 2017.
[47] Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Ping Hu, Dong Wang. Huchuan Lu. and You He. Boosting continual learning of vision-language models via mixture-of-experts adapters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pages 23219-23230.
2024.
[48] Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-efficient fine-tuning for transformer-bant masked language-models, arXiv preprint arXiv:2 10199, 2021.
12

---

[49] Han Zhang, Lin Gui, Yuanzhao Zhai, Hui Wang, Yu Lei, and Ruifeng Xu. Copf: Continual learning human preference through optimal policy fitting, arXiv preprint arX1v:2310.15694, 2023.
[50] Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-effiiien fine-tuning. The Eleventh International Conference Learning Representations, 2023.
[51] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.
[52] Zihan Zhang, Meng Fang, Ling Chen, and Mohammad-Redd Namazi-Rad. Citb: A benchmark for continual instruction tuning. arXiv preprint arXiv:2310. 14510, 2023.
[53] Ming Zhong, Chenxin An, Weizhu Chen, Jiawei Han, and Pengcheng He. Seeking neural nuggets: Knowledge transfer in large language models from a parametric perspective. In The Twelfth International Conference on Learning Representations, 2024.
13

---

A Appendix
A.1 Additional Related Works
Continual Learning. Continual learning is a field focused on developing algorithms that can accu- mulate and refine knowledge over time, particularly as they encounter non-stationary data streams. The primary challenge in this domain is overcoming catastrophic forgetting, where the performance of model old tasks significantly diminishes when trained with new data. To address this, existing methods
research has generally been divided into three main categories: (i) The rehearsal-ber utilize memory buffer retain data samples from previous tasks, incorporating techniques such as experience replay (36 or constrained optimization to simultancously train the model on current methods introduce additional terms into the loss function to penalize changes in important model parameters, thereby limiting interference with EWC [3] remembers old tasks by selectively slowing down learning on the weights important for those tasks. OGD 7 constrains the parameters to move within the orthogonal space defined by the stored gradients of previous tasks. (iii) The prchitecture-basse methods the aim is to minimize interference between tasks by dynamically expanding model capacity or creating isolated components for each task 17
and previous tasks 8 25 (ii) The regularization-bate
previously learned tasks 7 1
18 38
33 35
37 46). Progressive Prompts (35) enhances
forward transfer and mitigates catastrophic forgetting by learning distinct prompt for each new task and sequentially appending these new task-specific soft prompts to the ones previously learned.
Parameter-efficien Tuning. Recent works on parameter efficient tuning (PET) 9 have shown that training only a subset of model parameters can yield performance comparable training the full model, reducing computational demands and annotation efforts [10
II
15
48 50). BitFit [48]
finds that updating only the bias-items during fine-tuning is very effective. Prompt tuning [5] uses learnable 'soft prompts' through back proppaaoaaitiion to condition frozen language models for specific tasks. LoRA [ uses low-rank adapters in adapting models to new tasks with minimal additional parameters, and AdaL.oRA [50] improves the performance of LoRA by adaptively allocating the parameter budget based on the weight matrices importance score. While the majority of PET has focused on learning one single task, there has been several efforts to apply PET to continual learning. AdapterCL [27] introduces an individual adapter block for each task. LFPT5 [31 utilizes a large soft prompt that is continuously trained on all tasks. O-LoRA 43 incrementally learns new tasks in orthogonal subspaces while keeping the LoRA parameters learned from previous tasks fixed to minimize catastrophic forgetting. InfLoRA 20 mitigates catastrophic forgetting by reparameterizing pre-trained weights with a small set of parameters, enabling fine-tuning within a subspace to maintain previous knowledge. The proposed MoE-Adapten and DDAS collaborate in 47 mitigates long-term forgetting by dynamically expanding pre-trained CLIPP model with Mixture-of-Fae Experss adapters and preserves zero-shot recognition through Distribution Discriminative Auto-Selector for routing in- and out-of-distrine -iittii-iittittiionn inputs. ConPET 33 adapts existing continual learning strategies, originally developed for relatively smaller models to LLMs by incorporating PET with a dynamic replay approach. While O-LoRA addresses catastrophic forgetting through its incremental learning within orthogonal subspaces. it focuses on LoRA-bases architecture rather than more general low-rank matrices and does not explore the knowledge transfer across different tasks.
A.2 Implementation Details
All our experiments involving T5 models were performed on a server outfitted with four NVIDIA A6000 GPUs, utilizing the DeepSpeed repository for implementation. For every sequence of tasks across different orders, we standardized our experimental setup as follows: A constant rate of Ic-3 was maintained throughout the experiments. We used a total batch size of 32, distributed as 8 per GPU to leverage the computational capabilities of all four A6000 GPUs efficiently. We set the dropout rate at 0. We applied a regularization rate of 0.1 to the orthogonal matrices derived from the Singular Value Decomposition (SVD). A rate of 0.0 was employed, indicating no additional penalty on the model's 'ss weights during training.
A.3 Datasets
Tab. 6 provides detailed information on the 15 datasets utilized in our continual learning (CL.) experi- ments, including the evaluation metries used for assessment. Our selection encompasses datasets
14

---

from established benchmarks:: the standard CL benchmark STI benchmarks [41 and added IMDB movie reviews dataset.
GLUE 42 and SuperGLUE
Table 6: The details of 15 datasets used in our CL experiments. NLI denotes natural language inference, QA denotes questions and answers task. The first five tasks correspond to the standard CL benchmark, all other tasks are used in ung-sequence experiments.
Dataset name
Category
Task
Domain
Metric
Yelp
CL Benchmark
Sentiment Analysis
Yelp Reviews
Accuracy
2. Amazon
CL Benchmark
Sentiment Analysis
Amazon Reviews
Accuracy
DBPedia
CL Benchmark
Topic Classification
Wikipedia
Accuracy
4. Yahoo
CL Benchmark
Topic Classification
Yahoo Q&A
Accuracy
AG News
CL Benchmark
Topic Classification
News
Accuracy
6. MNLI
GLUE
NLI
Various
Accuracy
QQP
GLUE
Paragraph Detection
QUora
Accuracy
8. RTE
GLUE
NLI
News, Wikipedia
Accuracy
SST-2
GLUE
Sentiment Analysis
Movie Reviews
Accuracy
10. WIC
SuperGLUE
Word Sense Disambiguation
Lexical Databases
Accuracy
11. CB
SuperGLUE
NLI
Various
Accuracy
12. COPA
SuperGLUE
QA
Blogs, Blogs_Encyclopedia
Accuracy
13. BooiQA
SuperGLUE
Boolean QA
Wikipedia
Accuracy
14. MultiRC
SuperGLUE
QA
Various
Accuracy
15. IMDB
SuperGLUE
Sentiment Analysis
Movic Reviews
Accuracy
Table 7: Six different task sequence orders utilized in continual learning experiments. Orders 1-3 follow the standard continual learning benchmark established by previous research, focusing on more traditional task sequence. Orders 4-6 customized for long-sequence experimentation, encompass 15 tasks each and structured according to the methodologies outlined in (35
Order
Model
Task Sequence
1
15-laree.T5-ber
dbpedia-> amazon yahoo - ag dbpedia amazon - ag yahoo yahoo amazon ag dbpedia
2
15-large,T5--e T5-base
3
T5rlarg,,T5- T5-base
4
Targe
mnli - cb - wic copa - qqp - boolqa rte - imdb + yelp - amazon + sst-2 - dbpedia - ag multirc + yahoo multire - boolqa - wic - mnli + eb - copa + qqp + rtc + imdb + sst-2 - dbpedia - ag + yelp + amazon + yahoo yelp - amazon + mnli - cb > copa + qqp - rte > imdb sst-2 dbpedia + ag - yahoo - multire - boolqa + wic
5
Ta-le
6
TS-large
A.4
Sensitivity Scores Fisher Information
In our analysis, we aim to elucidate the distribution of parametric knowledge intrinsic to different task orders and compare the sensitivity scores with Fisher Information. The insights from Fig. 6 indicate the distributions of parametric knowledge across layers remains consistent among different task orders. It suggests that regardless of the task sequence, the layer-wise distribution of parameters critical for task performance does not significantly vary. Both sensitivity scores and Fisher Information depict similar patterns, underscoring the robustness of our model's 's learning mechanisms, Our findings also highlight that higher-level decoder layers exhibit increased sensitivity compared to their lower- level counterparts. This heightened sensitivity in the decoder suggests that these layers play a more crucial role in refining the outputs, possibly due to their direct involvement in generating end task results. Notably, the sensitivity scores of the encoder layers in task order 3 are higher than those observed in the first two task orders. This variation could be attributed to the specific nature or complexity of the tasks in order 3, which might demand more nuanced feature extraction capabilities from the encoder layers.
15

---

Table Instructions for different tasks
Task
Prompts
NLI
What is the logical relationship between the "sentence 1" and the "sentence 2"?
Choose one from the option.
QQP
Whether the "first sentence" and the "second sentence" have the same meaning?
Choose one from the option.
SC
What is the sentiment of the following paragraph?
Choose one from the option.
TC
What is the topic of the following paragraph?
Choose one from the option.
BoolQA
According to the following passage, the question or false?
Choose one from the option.
MultiRC
According to the following passage, is the question true or false?
Choose one from the option.
WiC
Given word and two sentences, whether the word is used with the same sense in both sentence? Choose one from the option.
(a) Encoder
(b) Decoder
(a) Encoder
(b) Decoder
(a) Sensitivity Scores
(b) Fisher Information
Figure 6: Order 1: Sensitivity scores and Fisher information of encoder and decoder Layers
A.5 Comparison of ROUGE score
We compare Average ROUGE-L scores (measures the longest common subsequence between the predicted and reference summaries, capturing sentence-level structure similarity) between O-LoRA and LB-CL the standard CL benchmark in Tab. [9] It shows that the ROUGE-L scores of LB- CL achieve performance improvements across all three task orders of the Standard CL benchmark compared to O-LoRA, demonstrating the effectiveness of LB-CL.
Table 9: Comparison of ROUGE score between LB-CL, and O-LoRA
Method
Order
Order 2
Order
O-LoRA
0.7902
0.7868
0.7859
LB-CI. 0.8169
0.8090
0.7894
16

---

(a) Encoder
(b) Decoder
(a) Encoder
(b) Decoder
Layers
Layers
Sensitivity Scores
Fisher Information
Figure 7: Order 2: Sensitivity scores and Fisher information of encoder and decoder Layers
(a)
Encoder
(b) Decoder
Sensitivity Scores
Fisher Information
Figure 8: Order 3: Sensitivity scores and Fisher information of encoder and decoder Layers
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
In the abstract and introduction, we list our contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers,
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [No]
Justification:
The paper has limitations, but those are not discussed in the paper.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximation: only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms
and how they scale with dataset size.
If applicable, authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
18

---

The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems. The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We present datasets, models, and implementation details in the experimental section and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken
to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (c.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification:
We use soun-source datasets and models, but do not attach the code.
19

---

Guidelines:
The authors should provide scripts to reproduce all experimental results for the new reproducible, they The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7/nips. cc/ wwwppblccc/acecccccccccccccceppe http:///guudes/odeuuubiiisis CodeSubmisss colommsssion wwoeiisssooooloi onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. If only subset of experiments
should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We present datasets, models, and implementation details in the experimental section and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them. The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
The experimental results are not presented as error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
20

---

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative
error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the details in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We follow the ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics. If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
We focus on the techniques.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal
impact why the paper does address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the
21

---

technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We cite the datasets and baseline papers in the experiment section.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms of use in the batessttssppprssstttt paperswi theode. com/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
22

---

Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such the institution conducting the review.
23


Reviews:
[{"review_id": "tj5tDffNMG", "reviewer": "Reviewer_dHuW", "summary": "This paper presents a novel approach to continual learning for large language models (LLMs). The proposed method, LB-CL, incorporates parameter-efficient tuning using low-rank subspace learning and orthogonal subspace projection to mitigate catastrophic forgetting. The study leverages incremental SVD-based low-rank matrix parameters for fine-tuning LLMs across a sequence of tasks. Comprehensive evaluations on benchmark datasets demonstrate the superiority of LB-CL over existing state-of-the-art methods in continual learning.", "questions": "see Weaknesses", "limitations": "see Weaknesses", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The use of incremental SVD-based low-rank matrix parameters and orthogonal subspace projection do address key challenges in continual learning such as catastrophic forgetting and efficient knowledge transfer.\n2. The method is rigorously evaluated on multiple benchmark datasets, providing strong empirical evidence of its effectiveness. The baseline compared with this paper includes the latest papers published in 2024.\n3. The paper includes an in-depth analysis of parametric knowledge transfer dynamics, initialization strategies, and the impact of seed samples on model performance, contributing to a deeper understanding of continual learning in LLMs.", "weaknesses": "1. The paper does not provide open access to the code and datasets used, which may hinder reproducibility and wider adoption of the proposed method.\n2. The method in this paper is similar to O-LoRA [1] and has some improvements over O-Lora. This paper claims that \"O-LoRA does not explicitly address knowledge transfer across different tasks.\" However, there is no specific evaluation in the experimental part of this paper to prove that the improvement of this paper on O-LoRA can improve \"knowledge transfer across different tasks\".\n3. In the past, there have been some continual learning methods based on orthogonal subspaces in non-LLM fields, such as [2] and [3]. This paper does not point out the core differences between this method and these previous methods. Is this method just an application of past methods in the fields of LLM and LoRA?\n\n[1] Wang, Xiao, et al. \"Orthogonal subspace learning for language model continual learning.\" arXiv preprint arXiv:2310.14152 (2023).\n[2] Chaudhry, Arslan, et al. \"Continual learning in low-rank orthogonal subspaces.\" Advances in Neural Information Processing Systems 33 (2020): 9900-9911.\n[3] Farajtabar, Mehrdad, et al. \"Orthogonal gradient descent for continual learning.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thorough review as well as constructive feedback. Your comments have been extremely helpful. We have carefully addressed each of your concerns and provided detailed answers to your questions below:\n\n\n> **W1: About open access to the code and datasets**\n\nWe strongly agree with the importance of reproducibility and accessibility of our experiments and appreciate your suggestion. We have shared our code and datasets via an anonymized link with the Area Chair in a separate comment to maintain anonymity during the review process, per NeurIPS author response guidelines, which will be made public ultimately.\n\n> **W2: About specific evaluation on the improvement due to \"knowledge transfer across different tasks\"**\n\nThank you for raising this comment. To specifically demonstrate the importance of knowledge transfer, we have conducted a detailed analysis of the influence of each component of our approach—initialization (knowledge transfer) and orthogonal gradient update—on overall performance in the original version of our paper. Table 3 presents the average testing accuracies under three scenarios:\n\n- **(1) No initialization (no knowledge transfer) and no orthogonal gradient update, called NLNB-CL**: This baseline scenario helps establish the effectiveness of our basic model without any enhancements for knowledge transfer or gradient management.\n- **(2) Only initialization (only knowledge transfer), called L-CL**: In this scenario, we focus on the impact of knowledge transfer. The results show an improvement in average testing accuracy, indicating that initializing new tasks with knowledge from previous tasks helps the model learn more effectively.\n- **(3) Only orthogonal gradient update, called B-CL**: This scenario isolates the effect of the orthogonal gradient update mechanism. The results demonstrate how managing gradient directions reduces interference between tasks, leading to better retention and performance.\n\nThese three different scenarios demonstrate the importance of individual ingredients, which can show our approach improves the knowledge transfer across different tasks, compared to O-LoRA.\n\n> **W3: About core differences between this method and these previous continual learning methods based on orthogonal subspaces in non-LLM fields? Is this method just an application of past methods in the fields of LLM and LoRA?**\n\nThank you for mentioning this point. We are happy to clarify the core differences between our method and previous methods and explain why our approach is not merely an application of past methods but a tailored solution for parameter-efficient continual learning in large language models (LLMs). We list the core differences in the following discussion:\n\n**Replay-based Approach in [2]**:\n\n- **Method**: [2] (Chaudhry et al.) use a memory buffer to store previous task data and replay it during the training of new tasks. It divides a random orthogonal space into several subspaces and allocates these subspaces one-to-one to each task with pre-defined orthogonal projections.\n\n- **Limitations**: Storing and replaying previous data can become impractical for large-scale models due to memory constraints and data privacy concerns.\n\n- **Our Approach**: We do not store any previous data, thereby ensuring data privacy and making our method more scalable for LLMs. Instead, we leverage the inherent low-rank structure of the model to manage orthogonal subspaces without relying on data replay.\n\n**Gradient Storage in [3]**:\n\n- **Method**: [3] (Farajtabar et al.) propose Orthogonal Gradient Descent (OGD), which stores a set of gradient directions in memory for previous tasks and projects new task gradients onto these stored orthogonal directions.\n\n- **Limitations**: Storing gradient directions for large-scale models is memory-intensive and impractical, especially as the number of tasks increases.\n\n- **Our Approach**: We avoid storing previous task gradients. Instead, we utilize low-rank subspaces to project new task gradients. This method is more memory-efficient and suitable for parameter-efficient continual learning in LLMs.\n\nIn summary, both methods mentioned ([2] and [3]) involve storing either data or gradients, while our work addresses the unique challenges of parameter-efficient continual learning in LLMs by avoiding data and gradient storage, leveraging low-rank approximations, and ensuring data privacy. Our work is not just an application of past methods in the field of LLM and LoRA. We believe these core differences highlight the innovation and practicality of our approach. We will highlight these differences in the subsequent version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FffDBp9IwS", "reviewer": "Reviewer_X83x", "summary": "This paper proposes the Learn More but Bother Less Continual Learning (LB-CL) algorithm for Continual Learning (CL) of Large Language Models. Unlike previous research, this paper introduces the idea of using SVD-based low-rank matrices to inject knowledge learned from previous tasks into new tasks, thereby enhancing plasticity during the CL process. Additionally, it suggests a method for learning in an orthogonal subspace to prevent forgetting of previous tasks. Experimental results on various text classification datasets demonstrate that the proposed algorithm outperforms existing algorithms. Furthermore, various ablation studies and analyses experimentally show the role and effectiveness of each component of the proposed algorithm.", "questions": "I find LB-CL proposed in this paper is somewhat novel. However, due to the similarity with the existing algorithm in certain aspects and the lack of significant performance improvement compared to the existing algorithm, I believe additional experiments and analysis are needed to properly validate the merits of the proposed idea. Please check the Weakness Section for more details.", "limitations": "There is no potential negative societal impact of this paper.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The strengths of this paper are as follows:\n\n1. The paper is well-written and easy to read overall. In particular, the explanation of the proposed algorithm in Section 2, \"Generalization and Forgetting Tradeoff of Low-rank Finetuning,\" and Section 2.1 is seamlessly integrated, making it easy to understand the motivation and ideas behind the proposed algorithm in a natural flow.\n\n2. The idea of Knowledge Extraction and Injection using SVD-based low-rank matrices adapters in CL to utilize knowledge from previous tasks for learning new tasks is novel and innovative.\n\n3. The proposed algorithm demonstrated superior performance compared to existing algorithms in experiments conducted on various text classification benchmark datasets.\n\n4. The extensive ablation studies and analyses experimentally validated the role and effectiveness of each component of the proposed algorithm, making this section particularly interesting to read.", "weaknesses": "LB-CL is motivated by O-LoRA and proposes a similar idea (Section 2.2: Training in Orthogonal Subspaces). Although the proposed algorithm achieves state-of-the-art performance in various experiments, the performance improvement over the previous SOTA algorithm, O-LoRA, is not very substantial (e.g., an average improvement of 1.3% on standard CL benchmarks and 0.4% on a large number of tasks). While LB-CL introduces additional ideas for knowledge transfer, this might increase the overall cost of the algorithm. Therefore, to verify the superiority of LB-CL over O-LoRA, it is necessary to compare the computation costs of these algorithms (e.g., training time, number of training parameters, or FLOPS).\n\nAdditionally, I have identified the following corrections that need to be made while reading the paper:\n\n1. Line 192: It seems that Eq.5 should be corrected to Eq.7.\n2. Line 224: It would be beneficial to include a brief explanation of O-LoRA under the 'Baselines' section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your appreciation and excellent summary of our work. We also appreciate the time and effort you dedicated to reviewing our research. We have addressed your questions and concerns below:\n\n> **W1: Line 192: It seems that Eq.5 should be corrected to Eq.7.**\n\nThanks for your correction. Yes, Eq.5 should be corrected to Eq.7.\n\n> **W2: Line 224: It would be beneficial to include a brief explanation of O-LoRA under the ’Baselines’ section.**\n\nThanks for your suggestions. We will include the explanation of O-LoRA should be briefly explained in the 'Baselines' section, such as \"O-LoRA: incrementally train new tasks in an orthogonal subspace while fixing the LoRA matrices of previous tasks.\" in the subsequent revision.\n\n> **Q: Computation cost comparison**\n\nThanks for your insightful suggestions. \n\n**Exp.Details**: We conduct computation cost comparisons on 4 NVIDIA A6000 GPUs and compare training costs between O-LoRA and LB-CL on task order 1 from the standard CL benchmark with T5-large model.\n\n| **Method** | **GPU Memory** | **Num of training params/task** |\n|------------|----------------|---------------------------------|\n| O-LoRA     | 24.82 GB       | $r(m+n)$                        |\n| LB-CL      | 28.28 GB       | $r(m+n)+r$                      |\n\n**Discussion:**\n\n- The GPU memory footprint of the two methods is quite close.\n- For the number of training params, we compare the trainable params within one layer. $r$ is the SVD matrix rank and LoRA rank, $m$ is the input dimension of the layer, and $n$ is the output dimension of the layer. Since $r \\ll \\min(m, n)$, the number of training params of two methods is also close.\n- LB-CL implementation computation cost is slightly more than O-LoRA. First, the orthogonal gradient update itself takes extra cost. Second, our code implementation computation cost is mainly incurred by DeepSpeed, a deep learning optimization library developed by Microsoft designed to train large-scale models efficiently. While we use DeepSpeed's `ZeRO stage 2' for efficient memory management, it currently does **NOT** support extracting gradients during training, as noted on their GitHub page. This limitation requires us to write an additional gradient computation module for orthogonal gradient update, increasing training costs. We plan to optimize our code implementation to reduce these costs.\n\n**Additional merits**:\n\nROUGE (Recall-oriented Understudy for Gisting Evaluation) is a set of metrics designed to evaluate the quality of summaries by comparing them with reference summaries, which are widely used in natural language processing (NLP) tasks. ROUGE scores measure the overlap between the predicted and reference summaries, indicating how well the model-generated summaries capture the essential content of the reference summaries. We compare Average ROUGE-L scores (measures the longest common subsequence between the predicted and reference summaries, capturing sentence-level structure similarity) between O-LoRA and LB-CL on the standard CL benchmark:\n\n| **Method** | **Order 1** | **Order 2** | **Order 3** |\n|------------|-------------|-------------|-------------|\n| O-LoRA     | 0.7868      | 0.7759      | 0.7902      |\n| LB-CL      | 0.8169      | 0.7994      | 0.8090      |\n\n\nIt shows that the ROUGH-L scores of LB-CL achieve performance improvements across all three task orders of the Standard CL benchmark compared to O-LoRA, demonstrating the effectiveness of LB-CL. Furthermore, the following table presents the average accuracy in Order 1 with different numbers of seed samples, which illustrates that LB-CL is more flexible and can achieve higher accuracy. We ultimately chose 8 seed samples in the original version of our paper because the variance across the 3 task orders was the least, as shown in Figure 4 of the original version of our paper (page 8).\n\n| **# Seed Sample** | **4**   | **8**   | **16**   | **64**   |\n|-------------------|---------|---------|----------|----------|\n| LB-CL             | 76.78%  | 76.90%  | 77.16%   | 77.32%   |\n\n\nWe greatly appreciate your insightful feedback. Based on your valuable suggestions, we will include this computation cost analysis and provide a more detailed comparison of computation costs with other different algorithms. Additionally, we will incorporate the additional merits as discussed and include additional metrics such as the forward transfer score to offer a more thorough understanding of the model's behavior during continual learning in the experimental section of the subsequent version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A1tL1dN4Jg", "reviewer": "Reviewer_aBm6", "summary": "This paper introduces LB-CL, a continual learning algorithm designed to tackle the issues of catastrophic forgetting and forward knowledge transfer. The approach integrates orthogonal low-rank SVD decomposition and sensitivity-based parameter initialization. The orthogonal subspace learning component addresses catastrophic forgetting by ensuring the SVD decompositions of different tasks remain orthogonal. The sensitivity-based initialization enhances forward transfer by optimizing parameter initialization weights. Comprehensive experiments across multiple representative datasets demonstrate that LB-CL outperforms many existing methods.", "questions": "Could you provide more details on why SVD decomposition is preferred over LoRA and what specific advantages it offers?\nHave you tested the sensitivity of LB-CL to the order in which tasks are presented? How might task order impact the performance of the method?\nHow does the method perform when tasks are highly similar and distinguishing features overlap? Does the orthogonality constraint still effectively prevent forgetting in such cases?\nDuring inference, does the model require the task-ID to be known? If so, how does this affect the practicality and flexibility of LB-CL?\nThe choice of rank in the low-rank SVD decomposition significantly impacts performance. How is this rank chosen optimally and how sensitive is the method to this hyperparameter?", "limitations": "*", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper introduces a novel approach that combines sensitivity-based knowledge transfer with orthogonal subspace learning, a unique method in the context of continual learning for LLMs.\nThe methodology is rigorously developed and supported by comprehensive experimental evaluations, demonstrating the effectiveness of LB-CL against existing state-of-the-art methods.\nThe experimental results robustly support the claims, showing that LB-CL outperforms state-of-the-art methods on standard continual learning benchmarks.", "weaknesses": "1. The paper does not sufficiently clarify the advantages and necessity of the proposed SVD decomposition over existing methods like LoRA. \n2. The paper does not provide experiments to demonstrate the sensitivity of the method to the order of tasks.\n3. The paper does not address whether the classification head is distinguished by task-ID during inference.\n4. The paper's experimental section lacks a thorough comparison with other common initialization methods, which makes it difficult to assess the superiority of the proposed initialization strategy.\n5. The computational complexity and scalability of maintaining orthogonal subspaces for a growing number of tasks are not addressed. As the number of tasks increases, the computational overhead may become prohibitive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback on our paper. We truly appreciate the time you invested in the review. We have carefully considered your insights and addressed the highlighted concerns. We hope our responses provide clarity on the matters raised.\n\n\n**W1 & Q1: About advantages and necessity of the proposed SVD decomposition preferred over LoRA**\n\nThanks for your interest. SVD decomposition offers key advantages: (1) singular values are important for identifying the relationship between singular vectors in the orthogonal matrices, evaluating the importance of triplets ($\\{\\boldsymbol{U}_i,\\boldsymbol{\\Sigma}_i,\\boldsymbol{V}_i\\}$) efficiently, while $\\boldsymbol{A}$ and $\\boldsymbol{B}$ of LoRA are not orthogonal, making the doublets $(\\{\\boldsymbol{A}_i,\\boldsymbol{B}_i\\})$ dependent with each other and discarding the doublets can result in greater variation from the original matrix. (2) SVD matrix easily evaluates the importance of its components, making it more flexible, for example, SVD only masks singular values for pruning and maintains singular vectors, while LoRA may prune all elements if it is measured as unimportant, hindering reactivation. We will highlight this comparison in the subsequent revision.\n\n**W2 & Q2 & Q3: Sensitivity to task order**\n\nThank you for your question. We have tested LB-CL's sensitivity to different task orders in original version of our paper. Table 2 (page 7) shows average testing accuracy for 3 different task orders in each CL benchmark. Let's revisit results on the standard CL benchmark for clarity:\n\n***Task Order 1:*** dbpedia$\\rightarrow$amazon$\\rightarrow$yahoo$\\rightarrow$agnews\n\n***Task Order 2:*** dbpedia$\\rightarrow$amazon$\\rightarrow$agnews$\\rightarrow$yahoo\n\n***Task Order 3:*** yahoo$\\rightarrow$amazon$\\rightarrow$agnews$\\rightarrow$dbpedia\n\n| **Method** | **Order 1** | **Order 2** | **Order 3** |\n|------------|-------------|-------------|-------------|\n| LB-CL      | 76.9%       | 76.5%       | 76.8%       |\n\n\nResults indicate a moderate sensitivity to task order. Achieving task order-invariance is challenging in continual learning, as the order of tasks can affect model performance. For example, learning Task A before Task B might yield different results compared to learning Task B before Task A.\n\n**Q4 & Q5: Perform when similar tasks and overlapping features, and whether prevent forgetting?**\n\nThank you for raising this comment. (1) Our approach uses weighted triplets to transfer knowledge. When previous tasks are highly similar with distinguishing overlapping features for the new task, these tasks' important triplets for the new task are also similar. We use weighted scores for these triplets, constructing the new SVD matrix accordingly. (2) In such cases, the orthogonal gradient subspace is the intersection of these weighted triplets orthogonal subspaces, making the new task gradient orthogonal update to effectively prevent forgetting.\n\n**W3 & Q6 & Q7: During inference, does the model require task-ID to be known?**\n\nThanks for mentioning this point. No, during the inference, we don't require task-ID. We use explicit instructions or demonstrations during training, injecting knowledge into the new SVD matrix, enabling the model to generalize well and handle unseen tasks efficiently. We are committed to emphasizing this in our subsequent revision.\n\n**Q8: Choice of rank in low-rank SVD decomposition and how sensitive is the method to rank?**\n\nThanks for your interest. In Table 4 of original version of our paper (page 9), we compared average testing accuracy for different ranks of the low-rank SVD matrix. Let's revisit results for clarity:\n\n\n| **r-dim**  | **Order1**      | **Order2**     | **Order3**     | **Std.**  |\n|------------|------------|-----------|-----------|-----------|\n| 2          | 76.7       | 77.2      | 75.2      | 0.85      |\n| 4          | 77.0       | 76.8      | 75.9      | 0.48      |\n| 8          | 76.9       | 76.5      | 76.8      | 0.17      |\n| 16         | 77.4       | 76.0      | 75.5      | 0.80      |\n| **Std**    | 0.25       | 0.44      | 0.60      |           |\n\n\n\nIt shows that in our scenarios, increasing rank does not significantly improve performance, and differences between ranks 2 and 16 are not significant. In our experiments, we used rank 8 for its smallest standard deviation, indicating consistent performance across different orders.\n\n**W4: Comparison with other common initialization methods**\n\nThank you for your valuable feedback. We acknowledge the importance of thoroughly comparing our proposed initialization strategy with other common methods in continual learning to better assess its effectiveness. In our experiments, we compared our initialization strategy with a standard random initialization, using NLNB-CL as a baseline. This approach involves freezing SVD matrices of previous tasks and using a new randomly initialized SVD matrix for new tasks. This comparison highlighted our method's improvements. We are committed to providing a thorough evaluation of different initializations in subsequent revision, such as model-agnostic meta-learning.\n\n**W5: Computational complexity and scalability of maintaining orthogonal subspaces**\n\nThanks for your insightful comment. As the number of low-rank SVD matrices grows, we merge updates into the initial parameters to mitigate GPU memory inflation, maintaining computational feasibility and preventing excessive memory usage. While our approach has shown effectiveness in empirical evaluations, its performance and scalability with a large number of tasks, such as hundreds, need further study. We will further focus on optimizing techniques and improving scalability to ensure practical use in extensive continual learning applications.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HqcTbFAyRa", "reviewer": "Reviewer_qwD9", "summary": "The paper presents a new parameter-efficient method for continual learning in LLMs. The method focuses on two aspects of continual learning: (1) Catastrophic forgetting and (2) Forward transfer.  To address catastrophic forgetting the gradient update of the current task is performed on the orthogonal space of previous tasks. To address forward transfer, they propose to initialize the new SVD parameters of the current task using a combination of previous tasks via learned coefficients.  The method is tested on two benchmarks with three task orders. An ablation is provided for each proposed component. The method is compared to another parameter-efficient method for CL, variations of the proposed method, some basic baselines (MTL, standard finetuning), and few previous CL methods on standard architectures.", "questions": "- Is the training performed in two stages? One for determining sensitivity parameters and one for training the task?\n- Do you assume that the task identity is not available at the test time? If yes, is the input image at the test time forwarded to the sum of the SVD of all tasks?\n- In Table 3, it is not clear to me why NLNB -CL is better than L-CL and B-CL, any thoughts on that?\n- Do the same observations in Table 3 hold for the other benchmark?\n- Figure 3 shows that excluding Sigma leads to better performance, I am wondering what could be the reasons behind this and why you don’t consider doing this in your proposed approach. \n- How is your method compared with the mentioned works above?", "limitations": "Limitations and societal impact are not discussed.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Unlike most methods which focus on forgetting, the method addresses forward transfer as well. \n- The method shows improvement in accuracy over a recent parameter-efficient method.\n- Ablation and analysis of the proposed components and hyperparameters are provided.", "weaknesses": "- Despite that forgetting is one of the main aspects addressed by the paper, it is not evaluated and no forgetting metric is reported. \n- Some very related works are missing [1,2].\n- Some design choices/observations are not clear, see the questions section. \n- Minor: Paper needs proofread. Suggestions for a few improvements are below:\n    - L149: “since the the” \n    - Section 3.1: A description of O-LoLA is missing in the baselines section\n    - L 84 description of triplet could be provided here for clarity. \n    - 3 rows (out of 4) from Table 3 are already presented in Table 2 \n    - Since the parameters of previous tasks are kept frozen, I am not sure whether forgetting is the right terminology or one should use interference (since forgetting is avoided by design somehow) \n\n[1] Yu, Jiazuo, et al. \"Boosting continual learning of vision-language models via mixture-of-experts adapters.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[2] Liang, Yan-Shuo, and Wu-Jun Li. \"InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback on our paper. We have thoughtfully addressed your insights and concerns, and hope our responses offer the necessary clarity on the issues raised.\n\n> **Q1: Training stage**\n\nThanks for your interest. Yes, first we initialize the SVD parameters using sensitivity-determined important parameters for the new task (injecting knowledge), then fine-tuning the new task in orthogonal subspaces.\n\n> **Q2: Task identity not available at test time? If yes, is the input image at the test time forwarded to the sum of the SVD of all tasks?**\n\nThank you for raising the concern. (1) Yes, task identity is not available at test time. (2) Inputs are forwarded through the whole model, including previously learned SVDs. We will make it clear in the subsequent revision.\n\n> **Q3: Performance of NLNB-CL, L-CL and B-CL**\n\nThanks for your interest in this observation. As we discussed this result in section 3.3, NLNB-CL, while neither initializes nor employs gradient projection, performs slightly better on average testing accuracy but important to note that it does **NOT** achieve the best performance in any specific task order. \n\n- **Inherent adaptive capabilities**: pre-trained model with low-rank SVD matrices may have inherent adaptive capabilities or rely on other compensatory mechanisms, making performance more stable on average. This stability might lead to slightly better average testing accuracy compared to L-CL and B-CL.\n- **Orthogonal subspace in B-CL**: it preserves previous tasks' subspaces, retaining knowledge from previous tasks but somehow hindering optimization for new tasks, leading to suboptimal performance.\n- **Knowledge transfer in L-CL**: it emphasizes knowledge transfer, potentially disrupting previously learned subspaces, which may affect results.\n\n> **W4.4 \\& Q4: Table 3**\n\nThank you for raising the concern. No, Table 3 does not hold for the other benchmark. The reason we showed Table 3 separately is to compare each component (stage) clearly. We will remove Table 3 to avoid confusion due to repeated information in Table 2.\n\n> **Q5: About excluding Sigma**\n\nThanks for your interest. Excluding Sigma and Including Sigma are different initialization strategies. Since the initialization of LB-CL idea is to learn previous tasks' important subspaces, constructed by triplets, if without $\\boldsymbol{\\Sigma}_i$, only doublets $\\{\\boldsymbol{U}_i,\\boldsymbol{V}_i\\}$ cannot fully represent these subspaces. We consider excluding Sigma as an initialization strategy, which can be used as an improvement strategy in implementation. We will clarify this in the revision.\n\n> **W2 \\& Q6: Compare with the mentioned works above**\n\nThank you for pointing out these related works. We appreciate the opportunity to compare our method with them.\n\n- **Model Focus**: [1] proposes a CL framework for a specific vision-language model, pre-trained CLIP, with Mixture-of-Experts (MoE) adapters. [2] proposes a LoRA-based continual learning method for a pre-trained Vision Transformer (ViT). However, our approach is for large language models and not a specific large language model.\n\n- **Method**:\n  - [1] uses MoE to dynamically activate the most related experts (adapters) by routers for a task, but the number of experts $N_{E}$ in the vision-language model is predefined and fixed, limiting flexibility and potentially affecting learning new tasks when none of the experts have the required knowledge. However, our approach uses a low-rank SVD matrix per task, learning tasks separately and mitigating forgetting.\n  - [2] algorithm not only stores previous LoRA parameters but also additionally stores gradient subspaces of previous tasks. Our approach initializes new task subspaces using previous SVD matrices without storing extra components, making it more memory-efficient. Also, large pre-trained models mainly fine-tune within a specific low-rank subspace, encapsulating crucial model update directions.\n\n- **Dataset**: [1] and [2] use image datasets, while our approach is applied to text (natural language) datasets.\n\nWhile [1] and [2] provide CL methods to vision-language models, our method offers a scalable and effective approach to CL in large language models, proven in NLP tasks. We are committed to including these related works and comparisons in the subsequent revision.\n\n> **W1: Evaluating forgetting and its metric**\n\nThank you for mentioning this important aspect. We followed existing continual learning methods by evaluating model's performance primarily by average testing accuracy, which provides a good measure of how well the model performs on all tasks. In subsequent revisions, we will include additional metrics such as forward transfer score, backward transfer score, and forgetting metrics to offer a more thorough understanding of the model's behavior in CL.\n\n> **W4.1: Typo**\n\nThanks for pointing out it. We will correct the typos in the subsequent revision.\n\n> **W4.2: Description of O-LoLA in baselines section**\n\nThanks for pointing out this point. We will describe it in the subsequent revision, such as \"O-LoRA: incrementally train new tasks in an orthogonal subspace while fixing the LoRA matrices of previous tasks.\"\n\n> **W4.3: Description of triplet** \n\nThanks for mentioning this point. We will describe triplet in the subsequent revision in L 84, such as \"a singular value and its corresponding vectors\".\n\n> **W4.5: About catastrophic forgetting**\n\nCatastrophic forgetting occurs when a neural network learns new tasks, inadvertently overwriting or conflicting with knowledge of earlier tasks thus reducing performance. This is critical in continual learning, as it undermines the principle of consistently accumulating knowledge without negatively impacting prior learning. We compared NLNB-CL (only keeps SVD matrices of previous tasks frozen) with LB-CL. Results show performance degradation in NLNB-CL when learning new tasks, highlighting the need to address both forgetting and interference.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Reviewers,

We greatly appreciate your insightful feedback and valuable suggestions. We have provided specific responses to each reviewer’s questions separately. We sincerely thank you for your contributions to improving our work. If there are any further concerns or queries, we are fully prepared to address them.

Thank you for your time and effort.

Meta-review: The paper proposes a method for continual learning of LLMs to defy forgetting based on LoRA. The approach is novel and considers the knowledge transfer and parameters relevance among different tasks which is an interesting direction for large overparameterized models.
The method shows SOTA performance on various continual learning benchmarks.
The concerns raised by reviewers were mostly addressed by the authors. We encourage the authors to include the clarification and the additional results and computation cost estimation in the final version.
Decision: Accept (poster)