Continual Learning with Global Alignment
Authors: Xueying Bai
Jinghuan Shang
Yifan Sun
Niranjan Balasubramanian
Venue: NeurIPS 2024 poster (2024)
Keywords: Continual Learning
Global Alignment
Composition Learning
Primary area: online_learning
TL;DR: To address interference when continually learning across tasks, we learn globally aligned data representations by interpolating pre-trained token representations; and apply probing first strategy to reduce interference caused by the classifier.

Abstract: Continual learning aims to sequentially learn new tasks without forgetting previous tasks' knowledge (catastrophic forgetting). One factor that can cause forgetting is the interference between the gradients on losses from different tasks. When the gradients on the current task's loss are in opposing directions to those on previous tasks' losses, updating the model for the current task may cause performance degradation on previous tasks. In this paper, we first identify causes of the above interference, and hypothesize that correlations between data representations are a key factor of interference. We then propose a method for promoting appropriate correlations between arbitrary tasks' data representations (i.e., global alignment) in individual task learning. Specifically, we learn the data representation as a task-specific composition of pre-trained token representations shared across all tasks. Then the correlations between different tasks' data representations are grounded by correlations between pre-trained token representations. We explore different ways to learn such compositions. Without experience replay, our model achieves SOTA performance in continual learning tasks. It also achieves advanced class-incremental performance through task-incremental training.

Continual Learning with Global Alignment
Xueying Bai
Yifan Sun Department of Computer Science Stony Brook University {xubai, jishang, yaun, niranjan)@cs wtw.:ecssssooobooo.ooo
Jinghuan Shang
Niranjan Balasubramanian
Abstract
Continual learning aims to sequentially learn new tasks without forgetting previous tasks' knowledge (catastrophic forgetting). One factor that can cause forgetting is the interference between the gradients on losses from different tasks. When the gradients on the current task's loss are in opposing directions to those on previous tasks' losses, updating the model for the current task may cause performance degradation on previous tasks. In this paper, we first identify causes of the above interference, and hypothesize that correlations between data representations key factor of interference. We then propose method for promoting appropriate correlations between arbitrary tasks' data representations (i.e., global alignment) in individual task learning. Specifically, we learn the data representation as a task- specific composition of pre-trained token representations shared across all tasks. Then the correlations between different tasks' data representations are grounded by correlations between pre-trained token representations. We explore different ways to learn such compositions. Without experience replay, our model achieves SOTA performance in continual learning tasks. also achieves advanced class- incremental performance through task incremmeettlll training. The code is available at: hatpps://et crgithuu.ccom com/SttoyBrroool.PPP wtg.ooo.../////llllllgogppoooaaa glwblllolllaooooo gnment.
Introduction
Continual Learning (CL) aims to develop models that can sequentially learn from streams of data and tasks, which is important need for many real-world applications [35, 18). One main challenge in developing CL models lies in reducing catastrophic forgetting, where models forget knowledge obtained from previous tasks after learning new tasks [38, 44, 15).
Catastrophic forgetting can happen when there interference during task learning, especially in models that use shared parameters for all tasks [46]. Specifically, when learning new task, if the model's gradients on the new task's 's loss are contradictory (e.g. in opposing directions) to those on the previous tasks' loss, the model will be updated towards direction that increases the losses previous tasks, causing forgetting. In this paper, we first identify factors that may lead to interference by analyzing the dot product between models' (flattened) gradients losses from two tasks. Then, we design methods to address interference based on each factor.
Our analysis in Section 3 shows that interference mainly depends on two factors: correlations between hidden representations of the data from different tasks; and correlations between columns of the classifier (which we call class vectors) that map data representations corresponding classes.
To address interference caused by the first factor, we want models learn aligned data representations that do not have destructive correlations (i.e., leading to interference) when switching tasks and during task learning. Specifically, this requires models accommodate future task representations when learning the current task. This motivates learning data representations based on some corle-cor))late) global representations, for which we use pre-trained token representations for language data [1 1]. Specifically, we compose data representations task-specific interpolations of pre-trained token
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Task
Input Data
(g)
Reprooracation
A buy in hooded top stalling The bay & upect
Contradiction
Chuo Vediars
spec
Us Ug-Bi 1)
cry
Petuorn Talme
Some octors have much charisma that you'd 'd be
Positive
2
lissen them reading
4)
1yo7s
positive
phone book.
clecsira
What Denzel Washington' 5 best moxie" Man on Fire was enery! Made me ery.
Entersaimmnt
City 00
Figure 1: Overview of our methods. Task i's data representations are denoted as h, with pre- trained token representations grey dots in the "Representation" block. Correlations between aligned data representations from different tasks depends on correlations between pre-trained token representations. In the 'Class Vectors' block, class vectors for different classes have different focuses on representations after probing, which can reduce interference caused by overlapped representations.
representations. This allows the correlations between tasks' data representations to be grounded by correlations between pre- pre-trained representations. We design three transformer- based [54] models that target such global alignment: (1). learning data representations by interpolating the pre-trained token representations through the attention mechanism; (2). the above model with additional neighborhood information to expand the search space of task information; (3). a controlled LoRA [20] model that adapts the pre-trained token representation with small scaling factor.
To address interference caused by the second factor, we first train the classifier only when switching the task and then tune the whole model. This probing first strategy was first proposed in [27] to reduce representation distortion in single-task learning. Here we use it to reduce interference especially when there are destructive correlations between representations (e.g.. caused by overlapping representations [5]). Probing enables different class vectors to focus on different features in data representations when switching tasks, which is useful when different tasks' representations are overlapped unexpectedly.
An overall view of our methods is in Fig. 1. Evaluations show that both the aligned representations and the probing first strategy improve CL. performance in multiple settings. Specifically, global alignment models perform well in class-incremental evaluation after task-incremental training.
In conclusion, we make the following key contributions in this paper:
1. We identify factors that cause interference in CL, and propose to address the interference issues by learning aligned representations and applying the probing first strategy.
2. We design three models to learn aligned representations, which learn task-specific attention with different levels of adaptations on pre-trained token representations.
3. We conduct extensive experiments on multiple CL settings. Results show that our models can significantly reduce forgetting even without the use of experience replay.
2 Related Work
Continual Learning CL Models can be divided into three main categories: regularization based models which constrain the deviation of new parameters from the older ones [25, 62, 1, 29]; replay- based models which reduce forgetting by rehearsing on real or pseudo samples from previous tasks [35, 7] generative models [49, 24]; and architecture-be based models which learn evolving architectures for sequential tasks, with their capacities for each task carefully assigned [48, 60].
CL in NLP is an emerging area [32, 3]. MBPA++ [10] uses experience replay and local adaptation to mitigate forgetting; LAMOL [53] generates pseudo samples for replay: IDBR [21] disentangles task-agnostic and task-specific information; CTR (23) uses capsule network for knowledge transfer.
All the above models are based on pre-trained LM [11, 4, 43]. Recent works show that pre-training can alleviate catastrophic forgetting [59, 39, 28]. Our work is also based on the pre-trained LM, but we use it for alignment purposes without experience replay.
Alignment in CL Recent CL works have studied the importance of alignment between different tasks' learning. Riemer et al. [46] aligns gradients between tasks to reduce destructive interference; Guo et al. [16] preserves holistic information for future tasks. If the previously learned knowledge does not align with that for future tasks, models may abruptly change previously learned knowledge and cause forgetting [5, 36, 22]. However, previous works focus on the alignment between observed tasks, which lack global view and can cause forgetting in the future [26]. Instead, our work achieves alignment by pre-trained semantic features, which are general even to unseen future tasks.

---

Adaptation Models With limited trainable parameters, our alignment models have connections to adaptation models, which originally aimed at parameter efficiency. Different adaptation models add limited trainable parameters on the frozen transformer layer [19, 40, 17, 20]; or selectively update existing parameters [42, 61]. Recent works do adaptation by prompt tuning [31, 30, 34), which learns prompt embeddings for target tasks.
Adaptation models have also been used for CL [57, 13, 56, 45, 50]. However, most works use the models' parameter efficiency to construct progressive memory. Whether different adaptation structures influence CL, why and how they help remain unexplored. Our model has a similar form to adaptation models after derivation, but our design focuses on representation alignment rather than computational efficiency, with no progressive memory.
Problem Statement
In this paper, we focus on the catastrophic forgetting caused by cross-task interference [46, 47]. We conduct a case study to identify factors of the interference in Section 3, which motivates our global alignment approach in Section
3.1 Continual Learning Settings
Tasks and Data We consider the setting where models continually learn sequence of tasks, with the condition that the previous tasks' data becomes inaccessible when learning new tasks. We denote representative data for each task as (x,, Vr). where is the model input and yı c; is its class logit. C; the set of all class logits task i, and we denote the set of class logits across all tasks as C= (c,
Scenarios We consider two CL scenarios: inkkinncremetall learning and ctass-nncremenal learning. The main difference between them is that at inference time for task the model knows its task-specific classes C; task-incremental learning (lask-aware), while the model has to predict the class from all classes C in class-incremental learning (lask-agnostic)) [37].
Models Our models consist of encoder that encodes the input x; d-dimensional representation dx whose y4-th column Vy, maps the hidden representation h, to the space the class y;- The probability of yı being predicted by the model is calculated by the softmax function: p(y: h,) softmax v)u.- At training time, the softmax is computed over classes in each task, while at inference time the softmax is computed over the range of classes specified in the task- class-incremental scenarios. We sequentially train each task with the cross-entropy loss: L(h,, 31) = p(y: (h,) for task i's data (x,, 81).
h; Rdd and matrix of class vectors (i.e., classifier)
3.2 Cross-Taan Interference
According to Riemer al. [46], catastrophic forgetting can occur when model learns a new task its gradients the new task's loss are contradictory (e.g. in opposing directions) to its gradients for the previous tasks' losses. In other words, the gradient descent for the new task might update model towards a direction that increases its losses on previous tasks, and thus cause forgetting.
In this section, we analyze factors that can lead to interference between gradients. Specifically, we study a case based on representative toy model. The model's encoder contains two linear layers with corresponding weight matrices denoted by W [gd where indexes the layers. The encoder outputs the representation: h; W x; for data in task i. Here the input x; E IR is d-dimensional vector.
Suppose we are learning task j after task using gradient descent. Consider (xi, Wi). (x,, Mi). two arbitrary data instances tasks and respectively. The interference between gradients of the weight matrix Wi with respect to the cross-entropy loss is given by:
I(W") L(h, 31) Vw: v,).
Destructive interference occurs when I(W") < 0, which can cause the model to forget task i's knowledge after updating W for task j. the other hand, when Z(W') > 0, the gradients of two tasks can enhance each other which encourages knowledge transfer across tasks [46].
3

---

Maseting Competty
Sampany
king
ditting
roporiation
getro
Jutre Rar
After first task
After the task
After first
After the task
Model withoat global aligument
(b) Model glohal alignment
Figure 2: T-SNE plots of all tasks' data representations after learning the first (with classes Village, Athlete) and last task. Under the vanilla sequential learning in (a), after the first task, representations of data from unseen tasks are overlapped. This may cause interference when switching tasks, which makes representations indistinguishable after learning the last task. With our global alignment model (Wire-Neigh) in (b), representations remain distinguishable after the first and last tasks.
We expand I( Z(")) below. For simplicity, we calculate gradients related to the y--th and yy-th class vectors in the matrix v: I(W Vy. Vu,) (p(y: hh)) 1) (p(y)) bb,)
(h D"
product of probubility terms
correlation between comelation between class vectors
hidlee representationss
is the hidden representation at the (1-1)-th layer, 52 about the weight matrix at the (1+1)-th layer. The class vector Vy. E R is the y column of the classifier v, mapping the output representation to the space of class Vi-
h 1/-1 w x:, 1=2
{ (w'+hy 1=1
1=2
X.. 1=1
Based Eq. ((W) depends on the correlation between data's hidden representations the (1-1)- th layer; and the correlation between class vectors transformed by weight matrices of the subsequent (e.g. (1+1)-th) layers. To address the interference issue, we discuss each correlation below.
Correlations Between Hidden Representations We consider correlations between different hidden representations two time points (Fig. 2): (1) The model has learned task i and is switching to learn task j. At this time, the model's hidden representations of task j's data should not destructively interfere with those of task i's data, even though the model hasn't 't been trained for task yet. (2) After learning Task j. At this time, the model should still produce good hidden representations for task i's data, even though it no longer has access to task i's training data.
Some previous works address the interference issue by forcing models to learn different tasks in orthogonal subspaces [9, 14, 55]. However, they may constrain models" knowledge transfer ability as Z(W") will always be zero. Other works minimize the destructive interference at time (2), by learning task with the replay of task i's data [46, 47, 2]. However, if hidden representations do not correlate well when switching to task j. they may cause and propagate interference in task j's learning (Fig. 2 (a)). Then even with replay, representations of task i's data may drift towards representations of task; j data, leading to disruptive model updates [5].
Our work tackles correlations of hidden representations at both time points. When learning across tasks, the model expected to produce aligned hidden representations of both the current and previous tasks' data at all times. The aligned representations should have appropriate correlations, which will not lead to destructive interference but retain the model's ability for knowledge transfer. To achieve this, we encourage the model to learn data representations for each task as different (task-specificc) interpolations of pre-trained token representations. This then enables the correlations between data representations to be grounded by correlations between pre-trained token representations (details are in Section 4).
Correlations Between Class Vectors The interference also depends on correlations between class vectors. Assuming no shared classes in task and j. the class vector Va, not involved in learning task and thus remains unchanged after task i. When learning the class vector Vy, for task j. we denote V by at the time step as V Var.t. Then the correlation between class vectors Vp. and Vyy,t is:
VT 91 Vu,.0 av T y.
Σ. Vv.y L(hj,, Wi);
yy u
where hy.e is the output data representation at time step 4, Vuy.o the initialization of the class vector Vags is the learning rate.

---

In Eq. 2, the correlation of class vectors depends on the initialization of the class vector vg, and the learning of data representations hy, (in the gradient Vv,, L(h W5)). The learning of the representation h depends on the correlations between representations, which we have discussed above. addition, we hypothesize that good initialization of class vectors can help mitigate the interference problem. To obtain suitable initialization for class vectors, we apply the probing then fine-tuning (PF) strategy [27] which first learns the class vectors (classifier) only and then fine-tune the whole model. We describe details in Section 4.
4 Methodology
In this section, we introduce our models that align data representations and initialize class vectors. First, we introduce our global alignment models which learn task data representations as interpolations of pre-trained token representations. Then we discuss the probing and then fine-tuning strategy and the effects of initializing the class vectors for CL.
4.1 Data Representation as Interpolation of Pre-Trained Token Representations
Pre-trained Token Representations In this paper, we focus on language models which are typically pre-trained in a self-supervised manner [11, 41J. For example, some models are pre-trained by the masked language modeling objective, which first masks tokens in imput texts and then learns models to predict masked tokens. By pre-training, models learn semantic relationships between tokens.
ceatformer-basee [54] language models. Typically, for an arbitrary task', the input of the model is a sequence of n tokens. At the 1-th transformer layer, denote the input representations of all tokens as G'-l igi gh- 1'' (I 2 1) where each token representation is an Rd vector. Then the output token representation is:
We consider
G' = Attnt (G G'
A where w and W E RRdxd are query and key matrices, Alln( K) = softmax QK" /vd) is function calculating the attention matrix based on the query Q and key K. The initial token representation G is the output of an embedding layer in the model. A feed-forward layer is applied after self-attention for token-wise transformation. We omit it here for simplicity.
After pre-training, we obtain contextual token representations G' at each layer I using the pre-trained g. W and W Such token representations contain semantic information of input tokens, which are general enough to accommodate diverse uses in different downstream tasks. We refer to representations G' as pretrained token representations.
matrices W
Data Representations To address a downstream task, models learn a data representation that summarizes the task-specific information in the entire input. Typically, these data representations are learned by fine-tuning all parameters in the pre-trained model. This type of full fine-tuning has been shown to distort the pre- pre-trained token representations [27] and may not consider correlations between data representations across tasks. This does not fit the alignment goal stated in Section 3.2.
To align data representations across tasks, we propose to wire (interpolate) the pre-trained token representations construct the data representation. Our proposal is based on the following hypothesis:
Task-specific information of data can be composed from the general semantics of tokens in that data.
For example, given the input text 'Some actors have so much charisma that you't 'd be happy to listen to them reading the phone book' from 'positive' class in a sentiment analysis task, composition of tokens (charisma, happy] } can convey the information of ppositive'.
Based on the hypothesis and the fact that pre-trained token representations contain the information of general token semantics, appropriately interpolating pre-trained token representations can represent task- ses-ispecfiic information in the data. Since correlations between pre-trained token representations are general across tasks, this type of composition aligns data representations from different tasks.
Alignment Effect For an arbitrary task, the pre-trained token representations G' can be interpolated to yield the data representation at the 1-th layer as: = (b' where b' E R xx is a learnable
For an arbitrary task, do not specify the task id by subscripts in the notations for simplicity.
5

---

Ressi
b
AW
Pre-trained
AW
Pro-trained
AW:
AW
Pre-truined
AW
(1) Fixed Wiring
Wiring with Neighbor Attention
(c) Controlled LoRA
Figure 3: Comparison between alignment models. Modules in blue are pre-trained and in orange are learnable. Representations in grey are mainly adapted and in blue are close to the pre-trained ones. We specify hidden representations for [CLS] and any other token as h [CLs] and h others
stochastic row vector with weights for the interpolation. To assess the alignment effect of this scheme, obtain the correlation between data representations hf and h from task and j as:
(h/) = b/G (
where by, by are learned interpolations, G. Gjj are pre-trained token representations for data in task i and j.
At any time step, the correlation between data representations is grounded by the correlation between pre- praitreined token representations: Eq. 4 involves the correlation (G)))) between pre-trained token representations; and the task-specific interpolation weights b/, by are also learned with the guidance of G and G', respectively. This grounding to pre-trained token representations thus aligns the data representations across tasks.
4.2 Global Alignment Models
We develop global alignment models to learn data representations as interpolations of pre-trained token representations. Parameters in models are trained by the cross- ennsopy loss for each task. Following common practice in traditional models, we append [CLS] token to the input text and use the representation of [CLS] as the data representation, denoted h [c
Fixed Wiring Applying the interpolation weights b' on pre-trained token representation G' generated by Eq. 3, we have b' = b' Attn(G w g. G W. G W Since the product of stochastic G
G' can be viewed as task-specific attention on the pre-trained token representations G
row vector and now-stochastic matrix is stochastic, b' AttnnG
Based on this, we develop a fixed wiring model that learns task-specific attention for [CLS] only, while using the pre-trained parameters to compute hidden representations of other tokens as the pre-trained representations G. In the model, the task-specific attention is calculated as the attention from [CLS] 's query, using new learnable key matrix A W! and the pre-trained query matrix w Formally, the data representation ht(cus) is:
(h (bfcss) )" Alln(( (CLS) TWi G AW G
(
where hicLs] is the pre-trained embedding of [CLS]. w is low-ranked [20] for efficiency.
By constraining non-cls tokens' hidden representations to be close to the pre-trained token representa- tions, the fixed-wiring model may have limited learning capacity. To avoid this, design two other methods with improved model capacity, which we describe below.
Wiring with Neighbor Attention Sometimes, the task information may not be easy to extract from pre-trained representations of input tokens. For example, in a text entailment task, give a sentence pair 'The boy is crying; He's happy about the view." with the label 'contradiction" The pre-trained representations of task-related tokens 'crying' and 'happy' may not be negatively correlated, which makes the model hard to learn their contradiction. However, 'crying' usually has neighbor token 'sad', and pre- trained representations 'sad' and 'happy' are more likely to have negative correlations. Therefore, using the information of 'sad' may make the model easier to learn the task.
Therefore, to increase the model capacity while preserving alignment ability, we retain the guidance of pre-traine pre-trained token representations while exploring the tokens' neighborhood to better search for the

---

task-specific information. The data representation can be written as:
(h )" Attn(
1-1
W
G
eapand AW expand W
[cLS]
where Gezpond [ G concatenates input tokens' pre-trained representations G and their neighbors' representations G
1
Since each token has its own neighbors, to obtain the data representation (acss) in Eq. 6, we first adapt each pre-trained token representation individually to incopporate task-specific information from their neighbors. Then we calculate task-specific attention on adapted token representations, using the attention mechanism in Eq. Specifically, we adapt the p-th pre-trained token representation Ep by:
( (1 (1-5) - +
Attn((g))")) w G
nei,p G AW
GI-1 W
where G
"eiip E Rexd contains neighbor representations for the p-th token, and Rriipp is the pre-trained embedding of the neighbor tokens. IR scaling factor. The neighbor tokens are selected by comparing cosine similarities between token embeddings. To stay close to the pre-trained token representations but incorporate task-specific information, we update the neighbor representations as: G G = (1 - 8) expand(g)) + 8 Altra G W. G W ) W where expand E kxd is the matrix which duplicates g for k tokens.
nei.p
Gzei provides extra capacity in learning data representations. Meanwhile, the aligmment effect is preserved by controlling the scale s, and making the neighborhood Gnei not deviate far away from pre-trained token representations G.
Controlled-Loke Another way to increase the model capacity is adapt representations of all tokens (including both [CLS] and other tokens in the text) by learning low-rank matrices added to the pre-trained query and value matrices (LoRA [20]). This has the model capacity close to fine-tuning, while keeping reference to the pre-trained parameters.
Denote the input representations of all tokens at layer as H where H G the pre-trained token embeddings. In LoRA, all token representations are updated by the same attention mechanism, with learnable query and value matrices A AW and AW! The data representation h ((css) is:
(h(cs)) T Attn((h [cis] )T (W + AW H W/ + s. AWW.).
When s O, we have H' G When s > O, the added query and value matrices not only learn the task-specific attention, but also adapt the token representations which can deviate away from the pre-trained ones. To keep alignment with pre-trained token representations, we control the scaling factor to make adapted token representations close to the pre-trained ones.
4.3 Initialization of Class Vectors
Although data representations are grounded by pre-trained token representations, at the start time of learning task j after task i, the interpolation for task j's data may not be well learned. In this case, properly initializing class vectors of classes in task j can help reduce interference.
To initialize the new class vectors when switching tasks, we adopt the probing and then fine-tuning (PF) strategy first proposed in Kumar al. [27]: when learning new task, it first freezes the encoder and only trains the classifier for the task (probing); and then tunes the encoder and classifier together funninun This beneficial in the case, for example, when the two tasks have similar input distributions but target different classes (e.g., news sentiment analysis vs. news categorization). In this case, data representations for two tasks may overlap when switching tasks. However, the class vectors can focus on different features in data representations after probing. Therefore, the correlation between class vectors may be small (or 0) and can reduce the interference.
Experiments
5.1 Datasets and Metrics
We evaluate four sequences of CL tasks: (1) Yahoo: a split of Yahoo dataset for news question-anseee -nswwer categorization [63] with 5 disjoint tasks containing 2 classes each; (2) DB: split of DBPedia data for
7

---

Table 1: Results for task-incremental learning using BERT-base encoder. We report the averaged accuracy (ACC) and forgetting (FGT) with their standard deviations (sid) on five random seeds. Bold scores are the best scores and underline scores are the second best. 'OOT" means out of time.
Model
Yahoo
DB
News Series
All
ACC
FG ml
ACC
FGT
aCC
FGT ol
aCCaa
FGT
Classifier-only
Probing
88.43 0.0
99.30
74.81 6.4
89.84
Adaptation Models
FT
73.07
18.67
73.15
24,90
59,98 194
21.13
60.92
30.53
Adapter
79.85 1.3
11.86 1.3
98.70
1.19 2.0
65.43
15.53 4:29
76.31 s31
15.97 8:1
LoRA
86.32 1:75
5.61
11.25
69.59 c.
12.43
77.37 usss
14.89
Prefix
89.75
3.04
99.83
0.07 c04
75.03 o07
6.13
87.53 om
as80
CL
ER
87.42 nm
5.61 ons
91.05 10:35
8,70 10.14
75.47 1:91
7.81 1:27
66.42 17
24.91 mm
Models
A-GEM
89.43 u.s
2.95 064
94.71 4.70
5.98 5.40
75,90 1:34
6.60 1S4
71.60 6..00
19.40 s.
MBPX++
86.50
5.30
97.17
2.65
72.55
7.23
82.60
9.09
IDBR (-R)
89.32 1:17
2.19 1os
96.47 mm
3.39 1:9
72.362.m
6.50
84.09 mz
a.00
IDBR
90.48
1.32
99.84
0.04
76,90 1.08
3,24
88.
2.21
CTR
87.06
1.02 oM
99. 99.04.mm o.t1
0.25 o/
1:
2,55 1:19
OOT
L2P
n.58
0.60 0.56
99.6 0.36
0,29 0.36
73,99 2.36
3,43 2.42
87.30 1.0
3.
CODA
88.33
0.40
99.33 0.23
0,37 o.25
75.13 1/22
0.69
87.95 0.33
2.76
Alignment
Wire-Fixed
91.11 0.24
0,70
99,85 6/2
0.03 o/2
76,28 1:1
3.66
88.63 LM3
3.55
Models
+PF
90.99 0.25
0.73 0.29
99.88 cu
0,02 c.00
77.75 1.19
1.81 Lo4
90.18 0.51
1.79...7
(ours)
Wire Wire-Neigh
90.98
0.89 c.3
99.86 cm
0.02
77.10
2.81
88.71 0:11
0.44
+PF
91.10 0.19
0.63 02)
99.88 col
0.01 cm
77.90 0:45
1.98 LMs
89.87 0.57
2.03
C-LoRA
90.72
1.47
99.79 cus
0.11 omt
74 .83 1:35
5.80
87.95
4.71 1:
+PF
91_38 0.19
0.61 om
99.89 cm
0.01 om
78.59 o.
2.42
89.48 mm
Non-CL
MTL
91.69 m02
99.61 0.1
79.67 1.9
90.75 0.6
Wikipedia article classification [63] with 7 disjoint tasks containing 2 classes each; (3) News Series: a sequence of tasks on news-related data, including AG AG_news (news classification, classes), MRPC (paraphrase detection, 2 classes) [12], RTE (text entailment, 2 classes) [58] and SST (sentiment analysis. 2 classes) [51]: (4). All: All tasks in the above sequences. For each task, we randomly sample 1245 samples per class, which is the least number of class samples in datasets.
We train alignment and adaptation models in musk-incremental (Task-IL -L) settings, where in-task classes are specified during training. Then we evaluate models on both Task-II and Class-II. inferences, where in-task classes are not specified for Class-II. inference [37]. We measure models' average accuracy and forgetting (Appendix B) over five random seeds.
5.2 Models
We compare different models on the pre-trained BERT-base model, including:
Alignment Models (ours): (1) Wire-Fixed: the model freezes pre-trained token representations and learns task-specific attention for data representations (i.e. [CLS] token). (2) Wire-Neigh: the wiring model with neighbor attention. We set 8 = 0.1. For computation efficiency, we fix the number of neighbors as k 5, and randomly select neighbors from top-K (K 20) nearest neighbors to control the range of neighborhood. (3) C-LoRA: the controlled LoRA model with the scaling factor = 0.1. For both Wiring and C-LoRA models, we set the matrix rank 8. We also evaluate above models with the probing then fine-tuing (PF) strategy, denoted as Model+PF.
Adaptation Models: (1) Fine-tuning (FT): fine-tuning all parameters sequentially. (2) Prefix Tuning (Prefix) [33]: freezing the pre-trained parameters and adding learnable embeddings to attention layers. (3) Adapter [19]: freezing the pre-trained parameters and injecting learnable linear projections after self-attention. (4) LoRA [20]: the LoRA model with suggested scaling s for single task learning.
CL Models: (1) ER: the FT model storing all seen examples and performs sparse (1 (%) experience replay. (2) A-GEM [7]: the FT model constraining gradients to prevent degrading performance on previous tasks. (3) +BPA++ ++ [10]: the FT model that stores and retrieves samples to locally adapt the model inference time like [52]. (4) IDBR [21]: the FT model with pntormatioo-dieettaggloo
based regularization and replay. We also compare to IDBR without replay, denoted as IDBR(-R) (5) CTR [23]: an adapter- based model with capsules and task transfer routing. (6) L2P [57]: a prompt- based model that dynamically prompts for different data and tasks. (7) CODA [50]: a prompt- based model that learns attention over extensive prompt components for tasks. (8). ERACE [5]: a model for class-II. that calculates the current task's 'ss loss over in-task classes, while calculating the replay loss over all seen classes in the replay buffer. We also show the performance of MTI. which is an FT model jointly trained on all tasks (not CL). Detailed settings for models are shown in Appendix A.
8

---

Class INDEENATT Leaning
Learihg
D5 Dass incremettal Learring
Yahoo Cass Incoenetii Leaming
Avaraged Accuracy
Ausr Atsunacy
15 8
ANNO Accurscy
18
Time
Tims Siep
(a) Class-I After the Last Task
(b) Class-I--ccaracyy across Time
Figure 4: (a). Class-IL IL accuracy after the last task. Dashed lines show accuracies of ERACE, which replays previous tasks' data with Class-IL loss. (b). Average Class-IL accuracies after each task.
Table 2: Tokens decoded from data representations by the pre-trained MLM decoder. Red tokens are tokens related to target classes in inputs; blue tokens are those related to red tokens in decodings.
Data (Tisk
Task
Yahoo (news categorization, targel class: Science) meaning blog?
DB (urticle classification target class: Film)
Sentence Sentence 2
No One
is a little space of puper of webpuge space where can
No One Man is a 1932 American dramu film starrin Car- ole Lombard Ricardo Cortez directed Lloyd
little scutence expressing general idea that repre-
sents what feel or think.
Corrigan. It based on novel by Rupert Hughes.
Top-10
FT
1 One a An an 2 and E.G.
the :: all its his their as from pro card into B stick long
Decoded
LoRA
Every Hint End
Tokens
Wire-Fixed
blog ittly user fttln Facebook hlogs #tting: Twitter weh
and released - ddinected made/ / Western
of [CLS]
Wire Neigh
blog post site posting web page content crawl
directed 2 film- produced released is, film of and are hut - follows
C-LoRA
content blog Wikipedia posting if post page
5.3 Results
Alignment models are effective in Task-IL, and Class-II. without replay. Results for Task-Il. are shown in Table Our alignment models forget less than CL models that use experience replay during fine-tuning. They also outperform adaptation models including Adapter, uncontrolled LoRA A and prefix tuning. This suggests the reduced forgetting in our alignment models is not just due to tuning fewer parameters, a feature in common with the adaptation models. In particular, C-LoRA performs better than the original LoRA model, which does not control the scaling factor for alignment. Results for Class- IL are shown in Fig. 4. On DB, alignment models achieve advanced Class- IL performance close to ERACE, a strong Class-IL model with replay, even under the Task-IL training where models are not trained on out-task classes. On Yahoo, alignment models also forget less than FT and A.
Wiring models forget less. Without the probing and fine-tuning (PF) strategy, wiring models achieve best performances in all Task-IL experiments and Class-IL on DB (Fig. 4). We hypothesize that the good performance of wiring models is due the strong alignment effect achieved by referencing the pre-trained token representations. With improved capacity brought by neighbors, Wire-Neigh outperforms Wire-Fixed on hard tasks like those in News Series. However, wiring models may have less capacity compared to models with more adaptations, which can limit their classification (separation) capability in Class-IL (e.g. Class-IL IL on Yahoo).
PF reduces forgetting. Fig. 4 shows that applying the PF strategy improves CL. performance for all models. Although C-LoRA itself performs worse than wiring models in Task-II. with PF it outperforms wiring models on 3/4 sequences. It also achieves best performance on Yahoo Class-IL. We hypothesize that is because PF can reduce interference when data representations have destructive correlations, as discussed in Section 3.2. Generally, PF and alignment on representations compensate each other on reducing interference. When the alignment effect is strong (e.g., wiring models), the improvement caused by PF is not very significant. However, when the alignment effect is not as strong (e.g.. C-LoRA, LoRA, FT), PF can significantly improve models' performance in CL.
5.4 Additional Analysis and Ablations
Table 3: Alignment quantification on SNLI and News Series. We report Re- call @ 20 on three random seeds.
Interpretability of Alignment Models Since alignment models learn data representations as the interpolation of pre-trained token representations, we interpret learned data representations using the pre-trained ML.M decoder. Specifically, we decode data representations to the token space by the pre-trained decoder and then get the tokens with top- 10 probabilities. This helps us understand how
Model
SNLI
News Series
FT
6,80 1 1.72
7.74 1 3,3
C-LORA
0.63
23.13
2.75
Wire-Fixed
37.01 + 1.54
27.80
+ 1.48
Wire-Neigh
36.24 + 188
32.32-t 2.58

---

data representations are closed to pre-trained token representations the model aligns to. Results are shown in Table 2. From the table, decoded tokens of alignment models are close to tokens that are related to target classes, while those of non-alignment models (FT, LoRA) are hard to interpret.
We also quantify the model's alignment ability using E-SNLI data [6], where each data's task-related tokens are annotated by human. We calculate the Recall @ 20 of annotated task-related tokens being retrieved from data representations, on SNLI and News Series data. The results are shown in Table 3. Results suggest that wiring models have more alignment ability than C-LoRA, in both in-task (SNLI) and CL evaluations on similar NLI tasks (News Series).
This interpretability may explain the effectiveness of alignment models in Class-ILa even when trained with local classes, data representations in each task correlated to pre-trained token representations that relate to all tasks' classes. This may help to separate representations from different tasks.
Infuence of Scaling Factor We show Task-II. accuracies of different scaling factors s for Wire- Neigh and C-LoRA in Table 4, C-LoRA's CL performance tends to decrease when & increases. This may be due to the decrease of the global alignment effect, which increases the interfer- ence. After applying PF, C-LoRA's accuracy first increases and then slightly decreases. This may be because PF reduces the interference caused by the class vectors, and the model can fully utilize its global alignment ability when increasing plasticity. However, when the scaling factor goes too large, the loss of alignment will lead to more forgetting even with PF. For Wire-Neigh, the observation is similar C-LoRA: when s goes up, the model's ' accuracy first increases and then decreases because of the trade-ofTs between global alignment and plasticity. And since Wire-Neigh interpolate pre-trained token representations with their neighbor representations, the increase of s leads to the decrease of pre-trained information. And therefore we observe a more rapid performance drop when increases.
Table 4: Average ACC with different s in C-LoRA, Wire-Neigh on News Series.
Model
==0
0.1
0.4
0.7
1.0
Wire-Neigh C-LoRA
76.28
77.10
72.59
68.18
66.59
74.81
74.83
72.99
71.02
69.59
+PF
74.81
78.59
77.41
76.83
76.81
Influence of Neighborhood In Wire-Neigh, we randomly select five neighbor tokens from top- K nearest neighbors. Here we study the effect of the range of neighborhood with different K value in Table 5. For relatively simple sequences DB and Yahoo, Wire-Neigh under different K has stable performance. However, for hard sequence News Series, when K increases, the model has more neighbor information (more capacity) to solve the task, which first improves its CL performance. However, when K is too large (K = 100), the neighbor information may become noisy, which makes the CL performance drop.
Table 5: Average ACC with different neighborhood range K in Wire-Neigh.
K
Yahoo
DB
News Series
5
99.86
91.16
76.90
20
99.86
90.98
77.10
50
99,86
91.16
77.20
100
99.87
91.13
76.58
6 Limitation
We discuss our limitations in model and assumption perspectives: For models, without replaying previous data, there can be a problem of shifting attention; e.g. the model shifts attention on previous task-related tokens after learning new tasks. This may lead to forgetting, which we leave as our future study. For assumptions, our model assumes we have pre-trained model. For domains that do not have well-established pre-trained semantic features, our model may not be immediately applicable. However, since fundamental models are consistently established and shown to be beneficial across different domains, we can expect our models to apply more domains in the future.
7 Conclusion
In this paper, we investigate methods to address correlations between data representations and class vectors to reduce interference when training across tasks. Specifically, for alignment, we propose to learn data representations as task-specific compositions of pre-trained token representations. To learn the composed representations, we propose wiring models with or without neighbor attention and a controlled LoRA model. To address correlations between class vectors, we adopt the probing and then fine-tuning strategy, which can effectively reduce interference even when the representations do not correlate well. Experiments show that our models can successfully learn the composed representations for alignment and achieve SOTA performance in CL.
10

---

Acknowledgments
We thank the anonymous reviewers for their insightful feedback to improve the paper. This material based on research that is supported in part by the Air Force Research Laboratory (AFRL), DARPA, for the KAIROS program under agreement number FA8750-19-2-1003 and in part by the National Science Foundation under the award IIS #2007290.
References
[1] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny. Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 139-154, 2018.
[2] Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. Advances neural information processing systems, 32, 2019.
[3] Magdalena Biesialska, Katarzyna Biesialska, and Marta R. Costa-jussà. Continual lifelong learn- ing in natural language processing: survey. In Proceedings of the 28th International Confer- ence Computational Linguistics, pp.
Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics. doi: 10. 10.18653/v1/20222 1/t03/001122222onn/.gg col.:00-ooiinmmiii555. main.574. URL https: / //ww:://wwwww aclweb. org/ att/aaathhoooooooeee wwwp:llloyyyyo...oooo coling main.574. 574.
[4] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.. /4165, 2020.
[5] Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights reducing abrupt representation change in online continual learning. In International Conference on Learning Representations, 2022. URL https: //penr/eiee/eeeeeeee aett/forum?idd/BMaBByylala
[6] Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. e-snli: Natural language inference with natural language explanations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018
[7] Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny, Efficient lifelong learning with a-GEM. In International Conference on Learning Representations, 2019. URL https::t////pptt httpp:://penreevixr.. http:p::///pppeeeeeeeeeeee/rrr// wtw.eeeeeneeeettomiddeedeeee sC5FX.
[8] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, P Dokania, Torr, and M Ranzato. Continual learning with tiny episodic memories. Workshop on Multi-Task and Lifelong Reinforcement Learning, 2019.
[9] Arslan Chaudhry, Naeemullah Khan, Puneet Dokania, and Philip Torr. Continual learning in low-rank orthogonal subspaces. Advances in Neural Information Processing Systems, 33: 9900-9911, 2020.
[10] Cyprien de Masson D' Autume, Sebastian Ruder, Lingpeng Kong, and Dani Yogatama. Episodic memory in lifelong language learning. Advances in Neural Information Processing Systems, 32, 2019.
[11] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), 2019.
[12] Bill Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases.
In Third International Workshop Paraphrasing (IWP2005), 2005.

---

[13] Beyza Ermis, Giovanni Zappella, Martin Wistuba, Aditya Rawal, and Cedric Archambeau. Memory efficient continual learning with transformers. Advances in Neural Information Processing Systems, 35:10629- 2022.
[14] Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International Conference on Artificial Intelligence and Statistics, pp. 3762-3773. PMLR, 2020.
[15] Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135, 1999.
[16] Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In International Conference on Machine Learning, pp. 8109-8126. PMLR, 2022.
[17] Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Kerg-Kirkpatricc, and Graham Neubig. To- wards unified view of parameter-efficient transfer learning. In International Conference Learning Representations, 2022. URL. pettt:::/opeereeeee net.eeewweeeeeeee..me.eee.ee.eee www.eeeeeoooodooeeeeeeeeeee. dd=oRDed5xxxoo
[18] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang. and Dahua Lin. Learning unified classifier incrementally via rehalancing. In Proceedings of the IEEF/CVF conference on Computer Vision and Pattern Recognition, pp. 831-839 2019.
[19] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruma Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly, Parameter-effiieet transfer learning for nip. In International Conference on Machine Learning, pp. 2790-2799. PMLR, 2019.
[20] Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https::/ htte/forrmmidddzeeeeeeeee
[21] Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, and Diyi Yang. Continual learning for text classification with information disentanglement based regularization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021.
[22] Khurram Javed and Martha White. Meta-learning representations for continual learning, Ad- vances in Neural Information Processing Systems, 32, 2019.
[23] Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. Achieving forgetting prevention and knowledge transfer in continual learning. Advances in Neural Information Processing Systems, 34:22443-22456, 2021.
[24] Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning.
In International Conference on Learning Representations, 2018. URL net/forum?idse id=$J1Xmf Rbb.
[25] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):551-3556, 2017
et al.
[26] Jeremias Knoblauch, Hisham Husain, and Tom Diethe. Optimal continual learning has perfect memory and is NP-hard. Hal Daumé and Aarti Singh (eds.), Proceedings of the 37th International Conference Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 5327-5337. PMLR, 13-18 Jul 2020. URL https: 1/ //ppoceedings... mlr,
htw.eedddiin.......eeees/eee wtt........ees11///eoooooaeeeeee Tieess/v11///oooaauch000 html.
[27] Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, and Percy Liang. Fine- tuning can distort pretrained features and underperform out-of-distriiution.. In International Conference on Learning Representations, 2022. URL htttps://operreviee wtwpe/rriiiieeeee/oruuuuee net/forum? id-UYneFZXSJWh.
12

---

[28] Kuan- Ying Lee, Yuanyi Zhong, and Yu-Xiong Wang. Do pre-trained models benefit equally in continual learning? In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 6485-6493, January 2023.
[29] Sang-Woo Lee, Hwa Kim, Jachyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017.
[30] Brian Lester, Rami AI-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021.
[31] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021.
[32] Tianlin Liu, Lyle Ungar, and Joao Sedoc. Continual learning for sentence representations using conceptors. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume I (Long and Short Papers), 2019.
[33] Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2021.
[34] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. arXiv preprint arXiv:2108 103.1085, 2021.
[35] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
[36] Divyam Madaan, Jachong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang. Representa- tional continuity for unsupervised continual learning. In International Conference on Learning Representations, 2021.
[37] Mare Masana. Xialei Liu, Bartlomiej Twardowski. Mikel Menta, Andrew D Bagdanov, and Joost van de Weijer. Class-incremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
[38] Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
[39] Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role of pre-training in lifelong learning. Journal of Machine Learning Research, 24(214): 2023.
[40] Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé Kyunghyun Cho, and Iryna Gurevych. AdapterFusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. pp. 487-503, Online, April 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021 main.39......
[41] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. http:://s3-us-weste amazonaws. comfopenai- pnees//errtoorr///ggggggegnneeee
understanding paper. pdf. 2018.
[42] Evani Radiya-Dixit and Xin Wang. How fine can fine-tuning be? learning efficient language models. In Silvia Chiappa and Roberto Calandra (eds.), Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 2435-2443. PMLR, 26-28 Aug 2020.
13

---

[43] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(14))1-67, 2020.
[44] Roger Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
[45] Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. Progressive prompts: Continual learning for language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreviie net/foorum?dddUJTg@BBB.
[46] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Representations, 2019. URL https: / //openreview weet/forumidddBIITSAAACC B1gTShAct7.
[47] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experi- ence replay for continual learning. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' d'Alché-uue, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: http://ppoeeeeddnneeeeeeee neurips. cctccc/ppppa aec/paper_filest pdf.
paper/2019/file// http:///eeeaaaaaaaaaaaaaaaaaaa/a ccffcdleaaafs77eeeeeeaaaaa1l.aa.
[48] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: 1606 04671, 2016.
[49] Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017.
[50] James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Mescante--onilla Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11999-11919, 2023.
[51] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D) Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference empirical methods in natural language processing, pp. 1631-1642, 2013.
[52] Pablo Sprechmann, Siddhant Jayakumar, Jack Rac, Alexander Pritzel, Adria Puigdomenech Badia, Benigno Uria, Oriol Vinyals, Demis Hassabis, Razvan Pascanu, and Charles Blundell. Memory-based parameter adaptation. In International Conference on Learning Representations, 2018. URL http:: http:::://///////eeeeeeeee www.eeewwwwwwwweeeeeeeeeeeeeeeee www.wewwwwwwwwweeeeeeeeeeeeeeeee seet/foorumid-rr2f00aaa
[53] Fan-Keng Sun, Cheng-Hao Ho, and Hung- Yi Lee. Lamol: Language modeling for lifelong language learning. In International Conference on Learning Representations, 2020. URL
[54] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
[55] Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. Orthogonal subspace learning for language model continual learning. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL https: / httpp:://ppenneeve http///pneevei.eeee iaaprrrrieee.ettooouuiiiiiieeee 2ttif/foorrmmm1117222222
[56] Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Ye Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, pp. 631-648. Springer, 2022.
14

---

[57] Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 139-149, 2022.
[58] Adina Williams, Nikita Nangia, and Samuel Bowman. A croer-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers). pp. 11 1112-1122, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10. 10.18653/1//118-11000 URL https://// https/// //www. aclweb. org/ (achhooogyy888-111 -1011
[59] Tongtong Wu, Massimo Caccia, Zhuang Li, Yuan-Fang Li, Guilin Qi, and Gholamreza Haffari. Pretrained language model in continual learning: A comparative study. In International Conference on Learning Representations, 2022. URL
id=figapGMraD.
[60] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In International Conference on Learning Representations, 2018. URL https::t//// htttts:://ppereviiee wtepiiiwweeeeeeeeeeee fttt/forummdd-Skksft///
[61] Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple carameter-efficiee fine-tuning for transformer- -aasenedrr--rdddddd masked language- models. In Proceedings of the 60th Anmual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2021.
[62] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3995. PMLR, 2017.
[63] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances neural information processing systems, 28, 2015.
15

---

A Detailed Experimental Settings
We provide detailed experimental settings in addition to the main paper Section 5. We train all models (except ERACE) via Task-IL training, while evaluating them in both Task-IL and Class-IL settings. We perform all experiments on one Nvidia RTX A6000 machine.
Probing: We fix the encoder and only train the classifier. We train 5 epochs for each taskwith the learning rate 5e-4.
FT: We fine-tune the whole model, including the encoder and classifier. We train 3 epochs for each task BERT, with the learning rate 2c-5.
we select learning rates from {5e-5, le-4, 1e-3) and train (5, 20) epochs for each task. For all continual learning tasks, we train with the learning rate 5e-5 and 20 epochs. The rank of Adapter projection is 32 suggested in the original paper.
Adapter:
select learning rates from (5e-4, 1e-3) and train LoRA for (5, 8) epochs for each task.
LoRA (C-LoRA)::
Wire-Fixed and Wire-Neigh: we select learning rates from [2c-4, 5e-4, 1e-3) and train [5, 8) epochs for each task. The rank of the learnable key matrix is 8. For Wire-Neigh, the number of neighbors is In practice, they are randomly sampled from larger neighborhood ranging from (10, 20,50, 100]. We set the mixing ratio as 0.1.
IDBR: We train IDBR with the learning rate 3e-5 for 3 epoches per task. We follow the k-means memory selection rule, and the replay batch size is 16 (training batch size) x number of tasks in the memory.
CTR: We follow the settings in the original paper, training epochs for each task.
L2P: We have the prompt pool with 100 prompt tokens and select 50 of them prepend to
the input. We train the model with the learning rate le-3 for 20 epochs for each task.
CODA: We have the prompt component size 20 for each task, and set the prompt length as
20. We train the model with the learning rate 1e-3 for 10 epochs for each task.
ER: We apply sparse experience replay with 1% replay ratio. At each replay time, we sample 32 samples from the memory and perform one-step gradient descent based them.
A-GEM: We store all previous data in the memory. At each gradient step, we randomly extract 32 samples from the memory and apply the A-GEM gradient projection.
MBPA++ We fine-tune the model with ER and then adapt the model at the inference time. At the inference time, we retrieve 32 nearest samples in the memory for local adaptation.
B Evaluation Metrics
Recall@k Denote the set of task-related tokens of the i-th sample as rel;, the set of top-k tokens predicted from the learned data representation as pred, @k. The metric Recall @ calculates the proportion of task-related tokens rel; that are predicted in pred, @k, which is defined as:
pred, @ reel
Recall @ k = [
[rel,]
Because each data instance in E-SNLI has 5-10 task-related tokens, we use Recall@20 for evaluation.
Average Accuracy and Forgetting We use the average accuracy and average forgetting similar in [8] to evaluate the performance in CL scenarios. The specific definitions described below.
Average Accuracy (ACC E [0,1]): Let di,j be the performance of the model on the test set of task j after the model is trained on task The average accuracy after training on all task T is:
1
ACCT
aT
In this paper, select 7 as the end of CL task sequence.
16

---

Average Forgetting (FGT E [-1,1]): Denote fig as the forgetting on task after the model is trained on task fiz is calculated by:
fiJ le(1.....-11 01
Oz.j-
And the forgetting after training the task T is:
T-1 FGTT Σ JTJ-
Our forgetting score divides the number of all tasks instead of 1. We do this to make the above metrics also indicate models' capacities single tasks, i.c. single-task capacity s ACCT + FGTr.
C Computation Costs
In this section, discuss the computation costs of alignment models. All our models use shared parameters across tasks, which do not progressively increase parameters. Since the learnable new matrices in our models are all low-ranked, they require limited usage of additional memory. So focus on the models' time consumption. For all models, we have the number of input tokens as n.
C-LoRA Our controlled LoRA model has the same time complexity as LoRA.
Wire-Fixed In the Wire-Fixed model, besides forwarding the pre-trained model to computing pre-trained token representations, we need extra computation for Eq. 5. Since only query for the [CLS] token, the extra time complexity is (n).
Wire-Neigh In Wire-Neigh, the time consumption comes from searching for neighbors and updating the neighbor representations. We find neighbors for each token based on their embeddings the embedding layer. The time complexity is O(Vn + VlogV) for computing the cosine similarity and sorting, where V is the size of the token vocabulary. Then the k neighbor representations are updated for each layer, with a complexity of O(nk?).
In practice, since the embedding layer fixed in our model, for each data instance we only need to find their neighbors once and then store the neighbor indices for iterative training (i.e., for several training epochs). The neighbor selection can also be accelerated by reducing the search space of neighbor tokens, for example, only searching neighbors from frequently used tokens instead of the whole vocabulary.
Wire-Neigh also needs extra memory to store neighbor representations and update them. To control the extra consumption of memory, we keep the number of neighbors as 5 and randomly sample neighbors from top- similar tokens to control the range of the neighborhood. We leave the improvement of Wire-Neigh's efficiency for future works.
Computation induced by PF Another computation costs come in the probing stage, in which we fix the encoder and only train the classifier. This takes less than 40% training time (including language model forwarding) and 30% GPU memory compared full fine-tuning.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: This paper studies some key factors of forgetting in continual learning and proposes an approach to mitigate that problem.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We included a limitation section (Section 6).
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
This paper is not for theoretically proofing something.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have included our detailed experimental settings in Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
We are cleaning our code and will release the code soon.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/ Stapcc/guiees//oodeeee
wwwpammssssooooooioooooooooooooc for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ //nips. wttpccccpppcccceeeececceccccccct ppppui/gguiudes...ppppppppppaaee CCodeSubmieeeeee
setoommsssiioo wwwssssssisiiiieeeeeeeoee oniioniyl) /ttia//uuiieeeeeuuuuuiuiiiiii for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We included details in Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We report mean and standard deviation from 5 runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
20

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We included this information in our Appendix A.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https //terr///euu../.eeeeee ct/ccc//p/// httppppllc//tiiccuieeeeeee
Answer: [Yes]
Justification:
We followed it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work does not have negative societal impacts. For positive impacts, since we study the CL problem training tasks sequentially without revisiting previous tasks or data, we have a positive impact on security. However, this is not our key point so we do not discuss it in the paper.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We used open-source datasets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets introduced.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: No these types of studies.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "kEPPUNJajC", "reviewer": "Reviewer_LhK3", "summary": "This paper studies the cause of cross-task interference in class-incremental learning of transformer-based language models. The authors disentangle the cause into the correlation (i) between data representations and (ii) between class vectors in the linear classifier. To tackle (i), the authors propose three ways to construct data representations at each layer by learning an attention over the pretrained token representations. To tackle (ii), the authors propose to only train the classifier for a new task (to obtain a good initialization) before jointly training both the classifier and the encoder.\n\nThe authors perform experiments with the pretrained BERT-base model and various text datasets. Training is in the task-incremental setup (where task labels are provided and the model predicts over in-task classes); the model is evaluated on both task-incremental and class-incremental (where task labels are not provided and the model predicts among all classes) setups. The authors found that their methods, \"alignment models\", outperform existing adaptation and continual models.", "questions": "1. Could the authors explain the choice of training in TIL while evaluating in CIL, as opposed to training and evaluating in the same setup?\n\n2. (Eq. 2) Since weights are usually initialized from a distribution centered around 0, shouldn't the first term be quite small? \n\n3. (Sec 4.3) Does probing perform softmax over only the new classes? If so, how does it help two class vectors for different tasks to focus on different features, since the loss does not require differentiating between the two classes?\n\n4. (Sec. 4.2) In the wiring models, what is the intuition behind only replacing the key matrix but not the query and value matrices? \n\n\nMinor:\n\n5. (Eq. 1) It seems that interference is eliminated once any of the three components is 0. Is it possible to fix the class vectors to be the canonical basis which are guaranteed to be perpendicular, and only learn the data representations?\n\n6. (Sec. 4.2) In Wire-Neigh, do you need to find the $K$ nearest neighbors for each token? If so, how expensive it is in practice?\n\n7. (L#219) Could you elaborate on what \"grounded\" means here?\n\n8. Missing references: [1, 2] are two prompting-based CL methods that are shown to perform better than L2P. \n\n9. (L#41-44) I recommend changing the indices to, e.g., \"(i), (ii), (iii)\", as a different \"(2)\" is referred to in the next paragraph.\n\n10. (L#53) \"by\" -> \"after\"\n\n11. (L#182) \"grounded192\" -> \"grounded\"\n\n\n[1] DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning. Wang et al. ECCV 2022.\n\n[2] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning. Smith et al. CVPR 2023.", "limitations": "The authors have addressed the limitations clearly in Sec. 6.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written and I did not find major technical flaws.\n\n2. I enjoyed reading the motivation of the paper in Sec. 3 where the authors examine the causes of cross-task interference.\n\n3. I find that the initialization of class vectors may influence cross-task interference interesting, although I have some related questions.", "weaknesses": "1. While the main goal of the paper is reducing cross-task interference, the main results (Table 1) is using the task-incremental setup, where task labels are known during prediction. The class-incremental learning results are only in Fig. 4, comparing the proposed method only with LoRA and ERACE.\n\n2. I find it a bit hard to infer where the performance improvement comes from from the results. I wonder if it is possible to do some fine-grained ablation analysis that verifies that the proposed method indeed helps by reducing overlap in data representations and in class vectors' features for different tasks. For example, one could measure the accuracy on the task level or look at the confusion matrix, which may reveal some information about cross-task confusions.\n\n3. Is $\\Delta W$ shared across tasks? If so, I don't understand why the proposed method does not forget. The authors hypothesize that this is potentially due to referencing pretrained representations. However, since the [CLS] representations are constructed involving $\\Delta W$, I'd expect the model to lose some ability to generate good representations for past tasks. Could the authors elaborate on this point?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and feedback. We address your concerns as follows.\n___\n**W1. While the main goal of the paper is reducing cross-task interference, the main results are using the Task-IL setup.**\n\nWe focus on Task-IL because: \n\n* Cross-task interference is a fundamental problem in Task-IL, which focuses on the gradients of shared parameters (i.e., the encoder) across all tasks. If destructive interference happens, the encoder may be updated to generate drifted representations for previous tasks, and the previous classifier may not be able to distinguish them (i.e., forget previous knowledge). Previous works study interference over the Task-IL setting as well, as cited in the paper. \n\n* Task-IL focuses on the interference in the shared parameters, while Class-IL further requires distinguishing between classes from different tasks. Although our paper shows they are inherently related, the distinction between cross-task classes is not our original goal. \n\nWe do not compare with more Class-IL methods because they are explicitly designed to distinguish cross-task classes during training like ERACE, while we only use Task-IL training without information of previous tasks’ classes. However, our method can be combined with those methods to achieve stronger performance in Class-IL. \n___\n**W2. Ablation analysis of proposed models in reducing overlap in data representations and class vectors for different tasks.**\n\n* In paper Fig. 2, we show a T-SNE plot of data representations on DB sequence for our model and the fine-tuning model. Our model generates non-overlapping representations after the first and last task. However, fine-tuning representations overlap after the first task, and further mix up after the last task.\n\n* By evaluating our Task-IL models in the Class-IL evaluation, the Class-IL results show the separation of learned representations and class vectors. If data representations and class vectors are not separable enough across tasks, then models may fail to assign data representations to corresponding classes (across all tasks). Our models have improvements in Class-IL as well, which suggests they reduce overlapping in representations and class vectors.     \n___\n**W3. Is $\\Delta W$ shared across tasks?**\n\nYes,  $\\Delta W$  is shared across all tasks. The proposed model still has forgetting as shown in experiments. However, the forgetting is significantly reduced by using our global alignment methods, which results in less interference during training and thus less forgetting.\n___\n**Q1. The choice of training in TIL while evaluating in CIL.**\n\n* For Class_IL, without replay or pre-separating class vectors, training with the loss on in-task classes can reduce forgetting compared to the loss over all classes [1]. The reason is that, if we use the loss on all seen tasks' classes but do not have data from all those classes (since no replay), it will cause data imbalance on classes during training. This may distort previously learned representations and cause forgetting. \n\n* As mentioned in **W2**, this setting can also evaluate the separation of data representations and class vectors learned in Task-IL.\n___\n**Q2. Influence of the first term in Eq. 2 when the weights are usually initialized to be centered around 0。**\n\nWe write Eq. 2 with the learning rate $\\alpha$ as:\n${\\bf{v}}\\_{y_i}^T{\\bf{v}}\\_{y_j} = {\\bf{v}}\\_{y_i}^T{\\bf{v}}\\_{{y_j},0} - \\alpha{\\bf{v}}\\_{y_i}^T\\sum\\nolimits_{t}\\nabla_{{\\bf{v}}\\_{y_j}} \\mathcal{L}({\\bf{h}}\\_{j, t}, y_{j})$.\n\n* Whether the first term is small depends on the class vector learned from the previous task, i.e., ${\\bf v}\\_{y_i}$. \n\n* The influence of the first term also depends on the scale of the second term. Since models are fine-tuned with relatively small learning rates like $\\alpha$ = \\{1e-3, 1e-4, 1e-5\\}, the second term is also 'small'. So the first term can still have a large impact even if the class vectors are initialized centered around 0.\n___\n**Q3. How does probing help two class vectors for different tasks to focus on different features?**\n\nIn the probing stage, we fix the encoder learned from previous tasks and only train the classifier for the new task. Therefore, the encoder generates the new task’s representations based on the previous task’s knowledge. And the class vectors of new and previous tasks can have different focuses on such knowledge, based on different objectives. \n___\n**Q4. Replacing the key but not the query and value matrices.**\n\nPlease see the general response 1.\n___\n**Q5. Interference is eliminated once any of the three components is 0. Is it possible to fix the class vectors to be the canonical basis?**\n\n* Our goal is to avoid destructive interference when learning across tasks, which encourages both zero and positive interference at adequate times. If we fix the class vectors to be perpendicular to each other, there will be no positive interference that can help positive knowledge transfer. \n\n* Also, the perpendicular class vectors may not be good class vectors that prevent feature distortions when tuning a pre-trained model. This will cause the loss of pre-trained knowledge during learning and make models lose generalization on OOD data, which can also make models perform inferior in CL.\n___\n\n**Q6. Computation cost for Wire-Neigh.**\n\nPlease see the general response 2.\n___\n\n**Q7. 'Grounded' in L219.**\n\n“Grounded” means that the correlations between data representations are decided by the correlations between corresponding pre-trained representations $G_i$, $G_j$, and the attention on these representations (learned in tasks). For example, if the two tasks pay attention to tokens that have orthogonal pre-trained token representations ($G_iG^T_j = 0$), e.g. when the two tasks’ information is irrelevant, then the alignment models will generate orthogonal data representations as well. \n___\n[1] Masana et.al. Class-incremental learning: survey and performance evaluation on image classification. TPAMI, 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kRQIjFdabL", "reviewer": "Reviewer_Ep6T", "summary": "This paper addresses the problem of Task-Incremental-Learning (TIL) with pre-trained transformer in the context of NLP. The author extended their experiments in the Class-Incremental Learning (CIL) scenario. The authors identify potential forgetting causes as (1) negative correlation between data representations and (2) negative correlation between class prototypes. After theoretically justifying their claim, the authors propose to align the model representations with either a) learning new key matrices for pre-trained queries and values b) learning new key matrices for with additional token chosen as the nearest neighbors of the considered token c) learning new queries and values from the original ones with a low-rank adaptation strategy. The author additionally align prototypes (class vectors) leveraging a probing and fine-tuning strategy. This paper shows superior performances on various dataset in CIL and briefly discuss the effect of various components.", "questions": "See weaknesses.", "limitations": "Limitations have been partially addressed in the main draft. I believe a discussion regarding the poor performances on CIL scenarios should be included, as well as a discussion on the potential computation overhead.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- clear and understandable paper\n- the equations are well derived and clear\n- the proposed approach has interesting theoretical justifications\n- The obtained performances on the TIL settings are compelling", "weaknesses": "1. While I believe the evaluation to be sufficient, it would improve the paper to include more recent prompt-learning techniques such as CODA [1] which showed stronger performances than L2P. I believe it would equally be interesting to see this approach applied to Computer Vision problems, even though I understand this paper focuses on NLP.\n\n2. Although the proposed approach is theoretically justified, it would be interesting to quantify such alignment through experiments, by computing data representations covariance/correlation between transformed class vectors; for each alignment strategy.\n\n3. I appreciate the effort to include results in CIL scenarios. However, I think more discussions as to how the proposed approach performs poorly compared to ER-ACE could be introduced.\n\n4. What is the link between the findings of this paper and previous work on orthogonal subspaces in continual learning [2, 3]? In these works, amongst others, it seems that the correlation between hidden representation should be zero, and not positive. I think such work should be included in related work section.\n\n5. A discussion regarding the extra computation induced by the alignment strategies and the PF would be welcomed as well, as it seems to increase it considerably.\n\n6. The code is unfortunately not accessible to the reviewers.\n\nIf the authors can address most of the above points I would happily increase my score.\n\n**Typos and presentation**\n\n- LoRA l44 and PF l.46 are not defined. Please either define the acronyms or cite the corresponding paper.\n- l.114 $h_{\\tau}$ should be h_{i}\n- I do not like the use of RHS l.163 and 164.\n- l. 182: \"grounded192\"?\n\n[1] Smith, James Seale, et al. \"Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\n[2] Wang, Xiao, et al. \"Orthogonal subspace learning for language model continual learning.\" Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10658–10671\n\n[3] Chaudhry, Arslan, et al. \"Continual learning in low-rank orthogonal subspaces.\" Advances in Neural Information Processing Systems 33 (2020): 9900-9911.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and feedback. We address your concerns as follows.\n___\n**W1. Comparison to more recent prompt-learning techniques and the application to CV tasks.**\n\n* Comparison to CODA: we show CODA's average accuracy on Task-IL below. Since CODA's hyperparameters are set for CV tasks, the recommended settings may not be optimal for our NLP evaluation. This may be the reason that CODA does not outperform L2P in our case. However, after finding the optimal NLP hyperparameters CODA may perform better than L2P as shown on CV tasks. We leave this into our future work. And we will add more recent prompt-learning techniques to our related works.\n| Model | DB | Yahoo |News Series\n|-|-|-|-|\nCODA|98.67|88.05|75.07\nL2P|99.63|90.82|73.99\n\n* Application to CV tasks: please see the general response 5.\n___\n**W2. Quantify alignment through experiments.** \n* **Representations' correlation between transformed class vectors**: By evaluating our Task-IL models in the Class-IL evaluation, the Class-IL results show the correlation between learned representations and class vectors. If data representations and class vectors are not correlated well across tasks, then models may fail to assign data representations to corresponding classes (across all tasks). Our models have improvements in Class-IL as well, which suggests they have learned representations correlated to all class vectors.\n* **Representations' correlation in the pre-trained space**: To further evaluate the correlation between data representations, we decode representations to the token space using the pre-trained decoder. If data representations are well correlated and guided by the pre-trained representations, they should be decoded to tokens that are related to tasks, as shown in paper Table 2. We quantify the model's alignment ability using E-SNLI [1] data, where the data's task-related tokens are highlighted by human annotators. We calculate the Recall@20 of task-related tokens decoded after training on News Series and SNLI (single task). The results are shown below. \n| Model | SNLI |News Series\n|-|-|-|\nFT|6.80|7.74\nC-LoRA|14.53|23.13\nWire-Fixed|37.01|27.80\nWire-Neigh|36.24|32.32\n\nResults suggest that wiring models have more alignment ability than C-LoRA, in both in-task (SNLI) and CL evaluations on similar NLI tasks (News Series). And Wire-Neigh has a better alignment ability in CL evaluations.\n___\n**W3. How the proposed approach performs poorly compared to ER-ACE.**\n* ER-ACE replays previous tasks’ data at each training step. This means it is learned to explicitly distinguish classes from different tasks, which is effective for class-IL. However, our methods are only trained in a task-IL manner without experience replay, which means (1). They have no information about classes in previous tasks; (2). They do not have access to previous data so they are not explicitly trained to distinguish classes from different tasks. \n* We believe it is unfair to directly compare our models to ER-ACE since ER-ACE uses more information for class-IL during training. However, even under this setting, our models perform close to ER-ACE on DB, and achieve good class-IL performance on Yahoo compared to fine-tuning. Their ability to separate representations from different tasks/classes is a result of our global alignment design.\n___\n**W4. What is the link between the findings of this paper and previous work on orthogonal subspaces in continual learning?**\n* Thanks for pointing out these related works. As you mentioned, they focus on getting zero interference between gradients, which minimizes the interference but also reduces the chances of the gradients being in the same direction and enhancing different tasks’ learning. \n* Our work does not restrict the gradients to be orthogonal. On the other hand, we guide the gradients by the pre-trained representations. If there are two tasks that have positive knowledge transfer between them, models may focus on tokens that have positively correlated pre-trained representations, which make the inner product between their gradients positive and enhance the two tasks’ learning (if the class vectors are not negatively correlated). \n___\n**W5. Extra computation induced by the alignment strategies and the PF.**\n\nPlease see the general response 2. \n___\n**W6. The code is unfortunately not accessible to the reviewers.**\n\nWe are cleaning the code and will send the link to AC soon. \n\n**Typos and presentations.**\n\nThanks for pointing out the typos and the presentation suggestions. We will correct and modify the paper accordingly.\n___\n[1] Camburu et al. e-SNLI: Natural Language Inference with Natural Language Explanations. NIPS 2018.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m9zH5vbrZt", "reviewer": "Reviewer_NuNi", "summary": "In Continual Learning (CL), the interference caused by the constant modification of the representation is the leading cause of catastrophic forgetting. Motivated by this and the idea that gradients in opposite directions are one cause of this interference, the authors proposed new ways of adding knowledge to a model. The authors motivate the proposal with a toy model that exemplifies where and why the interference occurs, concluding that it comes from the discrepancy between the representations of the different tasks and the relationship between the class vectors. These conclusions lead the authors to propose three proposals that slightly modify the transformer architecture, specifically adding learnable weights in the self-attention layers to generate a task-specific attention representation. The authors also propose using the probing and then fine-tuning approach presented in the past to help initialize class vectors. The results are presented in task and class incremental learning benchmarks in sequences of text datasets.", "questions": "- My main concern is the causes of interference that motivate this work. The author hypothesised that a gradient in the same direction can alleviate forgetting between tasks, a concept also adopted by GEM, despite its susceptibility to forgetting. Even with a gradient in the same direction, if it induces significant weight changes, the representations could be compromised, leading to catastrophic forgetting.\n    - How much does having a model pre-trained in tasks similar to those used in CL affect the proposed methods? The gradient will likely be low for these tasks.\n    - Text benchmarks need minor modifications to the model weights due to the similar distribution between the pre-trained model and the new tasks. In cases where higher modifications are needed, can the scaling factor (s) help reduce this effect? Did you explore different values of the scaling factors?\n    - Can this method work on image benchmarks? Images can have a more complex distribution than text, making avoiding interference between tasks difficult.  \n- Is FP applied to CL or adaptation methods in Table 1?\n    - Another technique used in CL is freezing class vectors in the classifier so that only the classes present in the batch during training are modified. This is especially useful when there is no memory. Can this help further reduce interference in the proposed methods?\n- Fig 2.b, what method of global alignment was used?\n- Three methods to mitigate interference are presented. Can these methods be complemented by each other?\n- How many neighbours are used in Wire-Neigh? Did you perform experiments to find the optimal number of neighbours?\n- Did you train only the classifier for the Task Incremental Learning problem? This method should not forget, and the pre-trained model can have good prior knowledge.\n- For the CIL experiments, the proposed methods have very similar results to previous methods when applying FT in Yahoo. This somewhat contradicts the claim that the proposed methods mitigate forgetting. Do you have any intuition about why FT+PF works so well?\n    - In Fig 4.b., is the performance with or without FP?", "limitations": "As the authors mention, this method heavily relies on a pre-trained model with a similar distribution to the incoming tasks. This assumption is only sometimes true and can be more complex in scenarios with a more complex input data distribution.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- The authors present a motivation from which they exemplify the problem in a simple way and where they want to aim their solution. This motivation may help the reader understand the context and problem to be solved.\n- The paper introduces a method that alters the architecture of a transformer model. While the paper does not explicitly state this, the provided motivation helps to understand why certain parts of the self-attention layer (the k matrix) are modified.\n    - The authors must describe why only the K matrix is modified and not the others.", "weaknesses": "- What is the actual contribution of applying wiring or C-LoRA? The results show that applying FP can benefit the methods much more than adding the proposals presented.\n    - What are the implications of applying FP to the Adaptation or CL methods? The performance gain from the proposed methods is relatively low (compared to L2P), and this could further decrease with the application of FP. This requires a deeper exploration of the proposed methods and the alternatives.\n- The writing is unclear and often unnecessarily complex, making reading difficult for someone unfamiliar with the subject. The notation used is only sometimes in line with the literature, which can confuse readers. In addition, there are problems explaining some terms and easily avoidable problems. For example:\n    - Line 46, FP, is mentioned but needs to be correctly defined. However, the document defines the abbreviation FP multiple times after that when only one should be enough.\n    - Line 182, there is an extra “192”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and feedback. We address your concerns as follows.\n___\n**W1. What is the actual contribution of applying wiring or C-LoRA?**\n\n* As shown in Eq. 1, the interference depends on two factors: (1). **Correlation between representations.** The wiring and C-LoRA models are designed to address this,  by learning aligned data representations with the reference of pre-trained token representations. (2). **Correlation between class vectors.** Probing and then fine-tuning (PF) are used to address this. Both alignment models and PF are our proposals to address interference in CL. \n\n* We respectfully disagree with the point that ‘*applying PF can benefit the methods much more than adding the proposals presented*’.  Without PF, wiring models can already achieve superior performance (Table 1). PF is especially helpful when the alignment between representations is not strong, as discussed in paper l327 -l330. \n\n* About the comparison to L2P, almost all of our models, with and without PF, achieve better performance than L2P in Table 1. If we misunderstood your point, please let us know and we will be happy to discuss it.\n\n___\n**Q1.a. Gradient direction and significant weight changes.**\n\n* The interference problem which focuses on the direction of gradients is fundamental in CL, as studied by many previous works like GEM you mentioned. The weight change is another important problem. We believe these two do not contradict each other. \n\n* The weight changes depend on both the learning rate and the gradients. In our experiments, models with similar structures (e.g. models with pre-trained encoder frozen) are tested with similar learning rates for a fair comparison. \n___\n\n**Q1.b. Effect of having a model pre-trained in tasks similar to those used in CL.**\n\n* **Pre-trained and downstream CL tasks**: In pre-training, models are pre-trained in a self-supervised manner (masked language modeling), which does not have implications for any supervised task used in CL. \n\n* **Gradient will likely be low**: In downstream tasks for CL, the task data usually have different distributions from that in pre-training (e.g., detecting text entailment of sentence pairs), and the classifier for the task has to be learned from random initializations. Both of these will make the gradient not low (i.e., the loss is not small) at the beginning of tuning.  \n\n* **Use of pre-trained model in our work**: Our work does not hypothesize the similarity between pre-training and fine-tuning tasks. On the other hand, we use pre-trained token representations as a basis/reference, and the models have to learn task-specific attention to them (with the randomly initialized key matrix). A better pre-trained model can help if it learns true correlations between pre-trained token representations, which will provide better pre-trained token representations for better alignment effects in our models.\n___\n**Q1.c. In cases where higher modifications are needed, can the scaling factor(s) help reduce this effect?**\n\nWe assume the ‘higher modifications are needed’ means we need more plasticity to learn every single task. In this case, we can increase the scaling factor in Controlled-LoRA or expand the range of neighbor tokens in Wire-Neigh. \n\nThe scaling factor controls the balance between the alignment effect and the models’ tuning capacity. If the datasets are hard and we need more plasticity, then increasing the scaling factor will help. However, when the scaling values become too large, the alignment effect will be reduced and the model may have more risk of forgetting as well. For comparison, please refer to the results of C-LoRA (with the scaling factor 0.1) and LoRA (with the scaling factor 1) in Table 1. Although LoRA achieves better single-task performance than C-LoRA, it also forgets more in our CL experiments.\n___\n**Q2. Is FP applied to CL or adaptation methods in Table 1?**\n\n* No. Since PF is proposed based on our analysis in Eq. 1, applying it to CL is also a part of our proposed models. So we did not apply it to other baselines. \n\n* **Modify only the classes in the batch**: Our standard training process has already used the mentioned strategy. In our training, for each task, only the class vectors of classes presented in that task are trained. For each batch, data are randomly sampled over all classes in the task. Besides this, our proposed model can achieve additional improvements in CL by the global alignment design and PF.   \n___\n**Q3. Fig 2.b, what method of global alignment was used?**\n\nSorry for the confusion. Fig 2.b is plotted under the wiring model with neighbor attention.\n___\n**Q6. Did you train only the classifier for the Task-IL?**\n\nYes. We have provided this result in the paper, row for ‘Probing’ (above adaptation models) in Table 1. The pre-trained model has good prior knowledge in DB, but the prior knowledge is not sufficient for Yahoo and News Series as there are large performance gaps between MTL and Probing. And all of our methods outperform the probing performance in Table 1. \n___\n**Q7. Why FT+PF works so well?**\n\n* As shown in Eq. 1, both correlations between data representations and class vectors will affect the interference. And PF can reduce the interference caused by class vectors. This may be the reason that using FT + PF works well, as discussed in l323 - l330. \n* **Why Alignment models + PF have similar results to FT + PF**: for CIL on DB, alignment models outperform FT + PF. For CIL on Yahoo, models may need more separation ability in each task's learning. Since FT has a strong separation ability for each task, it may have potential in CIL with PF. Similarly, C-LoRA has more single-task capacity than wiring models and also has alignment effects, so it achieves the best results in CIL on Yahoo.\n* **Fig 4.b.**: The performance is without PF, since wiring and C-LoRA themselves perform well in Fig 4.a. \n___\nFor questions Q4 and Q5, please refer to the general response 4, 3.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fcKauhIHZq", "reviewer": "Reviewer_R2vv", "summary": "This paper tackles continual learning by addressing interference between tasks. For the interference between different tasks, the authors propose a method called ‘global alignment’ to align the data representations using task-specific compositions of pre-trained token representations. Then the authors conduct extensive experiments to verify the effectiveness of the proposed method.", "questions": "Please refer to Weaknesses.", "limitations": "NA", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written and easy to understand.\n\n2. The motivation is clear. From the perspective of cross-task interference, this paper provides some simple but effective methods to avoid the interference of the data representation and classifier.\n\n3. The proposed methods are easy to follow and experiments verify their effectiveness.", "weaknesses": "1. The analysis of interference in Section 3.2 does not consider the activation function between the two layers of the network. Multiplying the two linear weight matrices is equivalent to using only one weight matrix, so the analysis of this linear case is different from that of the non-linear case and I’m curious about the inference in the non-linear case.\n\n2. About wiring with neighbor attention:\n\n    a) The author needs to explain the rationality of neighborhood tokens and the impact of the size of K on the model.\n\n    b) This method requires matching the most similar K tokens for each token given a sample. This process may require a large amount of computation.\n\n    c) The author needs to further explain the difference between this method and controlled-LoRA. Both these two methods add a learnable term to the original token representations but adopt different generation strategies. Specifically, this method adds a term composed of neighborhood tokens, while controlled-LoRA uses low-rank matrices. So what are the advantages of this method?\n\n3. About Controlled-LoRA:\n\n    a) Computational burden and parameter size: As the number of tasks increases, linearly increasing model parameters is not in line with the spirit of CL, and the increased parameter size should be limited even if LoRA only adds small parameters compared to the large model.\n\n    b) Why does Controlled-LoRA perform worse than other methods? In the case of task-IL, given the task ID for each test sample, we can directly use the best LoRA model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and feedback. We address your concerns as follows.\n___\n\n**W1. The analysis of interference in Section 3.2 does not consider the activation function between the two layers of the network.**\n\n* We consider the ReLU activation which is widely used in neural networks. After ReLU, for each layer we have the output ${\\bf{h}}^{(l)} = {\\bf{D}}^{(l)}{\\bf{W}}^{(l)}{\\bf{h}}^{(l-1)}$. ${\\bf{D}}^{(l)}$ is a diagonal matrix with the $i$-th diagonal value as 1 if the $i$-th element in ${\\bf{W}}^{(l)}{\\bf{h}}^{(l-1)}$ is positive, and otherwise 0. Then we have $\\frac{\\partial {\\bf{h}}^{(l)}}{\\partial {\\bf{W}}^{(l)}} = {\\bf{D}}^{(l)} \\otimes ({\\bf{h}}^{(l-1)})^T$ and $\\frac{\\partial {\\bf{h}}^{(l)}}{\\partial {\\bf{h}}^{(l-1)}} = {\\bf{D}}^{(l)}{\\bf{W}}^{(l)}$ where $\\otimes$ is the Kronecker product. We can calculate the gradient of each ${\\bf{W}}^{(l)}$ based on the chain rule. \n* However, adding the non-linearity will complicate our analysis and also make the results hard to understand. Since it's typical to remove activation analysis for a first-pass analysis, and we do not believe the nonlinearity significantly changes the observations, we did not include it in the paper.\n___\n**W2.a. The rationality of neighborhood tokens and the impact of neighbor size k.**\n* **Why using neighborhood tokens**: Based on Eq. 6, our model wires the pre-trained representations of input tokens to learn a task, and thus its task-learning ability depends on the range of input tokens. However, the task information may not be limited to the information of input tokens, but also their neighbor tokens. **For example**, in a text entailment task, give a sentence pair: {*s1: the boy is crying; s2: he’s happy about the view.*} with the label ‘*contradiction*’. The pre-trained representations of ‘*crying*’ and '*happy*’ may not be negatively correlated, which makes the model hard to learn their contradiction. However, ‘*crying*’ has a neighbor token '*sad*', and pre-trained representations of '*sad*' and '*happy*' are likely to have negative correlations. Therefore, introducing the information of '*sad*' may make the model easier to learn the task, and thus enhance its task learning capacity. \n* **The impact of the size of K**: Please see the general response 3.\n___\n**W2.b. The computation cost of Wire-Neigh.** \n\nPlease see the general response 2.\n___\n**W2.c. Comparison between Wire-Neigh and Controlled-LoRA.** \n\nPlease see the general response 4.\n___\n**W3. Computational burden and parameter size of Controlled-LoRA. Why does Controlled-LoRA perform worse than other methods?**\n* a. **Controlled-LoRA does not linearly increase model parameters with tasks.** Controlled-LoRA adds low-rank query and value matrices as LoRA does. These low-rank matrices are *shared* across all tasks. We agree with your point that it is not desirable to increase model parameters when tasks grow. And that’s why we use shared parameters across tasks, and focus on alignment methods to reduce destructive interference and forgetting.\n* b. Controlled-LoRA still has forgetting because we do not progressively store parameters for each task. And it performs worse than wiring models because by learning both query and value matrices, it has weaker alignment effects. This can be compensated by applying PF (probing and then fine-tuning). After adding PF it outperforms wiring models in Table 1.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all reviewers for their thoughtful reviews and feedback. We address common questions as follows.
___
**1. The intuition behind only replacing the key matrix but not the query and value matrices in wiring models.**
* **Why not replace value matrices**: when replacing the value matrices with $\Delta {\bf W}_v$, the data representation ${\bf{h}}^{(l)}$ turns out to be:  $({\bf{h}}^{(l)})^T = ({\bf{b}}^{(l)})^T{\bf{A}}^{(l)}{\bf{G}}^{(l-1)}({\bf W}\_v^{(l)} + \Delta {\bf W}^{(l)}_v)$. Compared to our goal in Eq. 6, this will reduce the guidance of pre-trained token representations, which does not fit our wiring goal. 
* **Why not replace query matrices**: since the wiring models only relearn the attention for [CLS] token, for all tasks they only query from [CLS] (other tokens' queries are pre-trained). However, the keys are from all tokens in the input and are different when tokens from different tasks have different distributions. Intuitively, we learn the key matrices applied over all input tokens instead of the query matrices applied on the same [CLS] across different tasks.
___

**2. Extra computation induced by the alignment strategies and the PF.**
* **Computation induced by searching for neighbors in Wire-Neigh**: We find $k$ neighbors for each token based on their embeddings at the embedding layer. The time complexity is $O(vn)$ where $v$ is the size of vocabulary and $n$ is the size of input tokens. Then the neighbor representations are updated for each layer, with a complexity of $O(nk^2)$. 
In practice, since the embedding layer is fixed in our model, for each data instance we only need to find their neighbors once and then store the neighbor indices for iterative training (i.e., for several training epochs). 
The neighbor selection can also be accelerated by reducing the search space of neighbor tokens, for example, only searching neighbors from frequently used tokens instead of the whole vocabulary.

* **Computation induced by PF**: in the probing stage, we fix the encoder and only train the classifier. This takes less than 40% training time (including LM forward) and 30% GPU memory compared to full fine-tuning. 
___

**3. Impact of hyperparameter K in Wire-Neigh.**

For computation efficiency, we fix the number of neighbors as $k=5$, and randomly select the neighbors from top-$K$ nearest neighbors to control the range of tokens' neighborhood. We show the Task-IL accuracies with different $K$ below:
| Wire-Neigh | DB | Yahoo |News Series
|-|-|-|-|
|  $K=5$  |   99.86  |   91.16  |76.90|
|  $K=20$  | 99.86|   90.98  |77.10|
|  $K=50$  | 99.86|  91.16 |77.20|
|  $K=100$  | 99.87 | 91.13|76.58|

For relatively simple sequences DB and Yahoo, Wire-Neigh under different $K$ has stable performance. However, for hard sequence News Series, when $K$ increases the model has more neighbor information (more capacity) to solve the task, which first improves its CL performance. However, when $K$ is too large ($K=100$), the neighbor information may become noisy, which makes the CL performance drop. 
___

**4. Difference between Wire-Fixed, Wire-Neigh and C-LoRA.**

We illustrate the difference between alignment models based on Eq. 6, where aligned data representations ${\bf{h}}^{(l)}$ are expected to have the form $({\bf{h}}^{(l)})^T = ({\bf{b}}^{(l)})^T{\bf{A}}^{(l)}{\bf{G}}^{(l-1)}{\bf W}\_v^{(l)}$.
* **Wire-Fixed**: keep the pre-trained representations ${\bf{G}}^{(l-1)}{\bf W}\_v^{(l)}$ fixed, only learn the attention $ ({\bf{b}}^{(l)})^T{\bf{A}}^{(l)}$ by the self-attention mechanism with new key matrices, denoted as $SA(\Delta {\bf{W}}^{(l)}_k)$. This is only applied to [CLS] tokens. Then $({\bf{h}}^{(l)})^T = SA(\Delta {\bf{W}}^{(l)}_k){\bf{G}}^{(l-1)}{\bf W}\_v^{(l)}$. Wire-Fixed is strongly guided by pre-trained token representations but may have limited capacity in solving hard tasks.
* **Wire-Neigh**: apply the same wiring strategy as Wire-Fixed but add neighbor representations ${\bf{G}}\_{neigh}^{(l-1)}$ for better capacity. Then $({\bf{h}}^{(l)})^T = SA(\Delta {\bf{W}}^{(l)}_k)[{\bf{G}}^{(l-1)}; {\bf{G}}\_{neigh}^{(l-1)}]{\bf W}\_v^{(l)}$. The advantage of Wire-Neigh is it keeps the guidance of pre-trained token representations but increases the model capacity by expanding the neighborhood.
* **C-LoRA**: adapt both query and value matrices with $\Delta {\bf{W}}^{(l)}_q, \Delta {\bf{W}}^{(l)}_v$ but with a small scaling factor $s = 0.1$. This adaptation is applied to all tokens. Then $({\bf{h}}^{(l)})^T = SA({\bf{W}}^{(l)}_q + s\Delta {\bf{W}}^{(l)}_q){\bf{H}}^{(l-1)}({\bf W}\_v^{(l)}+s\Delta{\bf W}^{(l)}\_v)$, where ${\bf{H}}^{(l-1)}$ is the adapted token representation of the previous layer. Compared to wiring models, C-LoRA modifies both the attention and the value matrices applied to all tokens. It can still enjoy the guidance of pre-trained representations with the small scaling factor $s$, but the guidance may be weaker than wiring models. Meanwhile, it has more task-learning capacity. C-LoRA does not use neighbor information. 
___

**5. The application of the methods to computer vision (CV) problems.**

* We believe our main contributions on alignment models and utilizing PF to reduce interference are general to CV tasks. First, our analysis of interference in Section 3 is general to both NLP and CV. Second, there are effective pre-trained models in CV as well, which can be used for alignment purposes. 
* We think the keys to applying our model to CV tasks are: (1) how effective the pre-trained model is in providing self-supervised token representations for alignment; (2) how to properly set scaling ratios or select neighborhoods to balance the alignment effect and the task capacity on CV tasks.  

We will study our models' application to CV in our future works.
___

**6. Typos and definitions in the writing.**

Thanks for pointing out the typos and the presentation suggestions. We will correct and modify the paper accordingly.

Meta-review: This paper presents a global alignment method to alleviate catastrophic forgetting in continual learning. It first identifies the correlation between data representations is the main cause of gradient interference from different tasks. Then proposes a global alignment method to learn data representation as a composition of pre-trained token representations shared by all tasks, and explore different ways to learn such composition.  Extensive experiments demonstrate the effectiveness of the proposed method for both class-incremental and task-incremental learning.

The proposed approach has interesting theoretical justifications, and the code is provided for reproducing the experiments in the paper.

Reviewers raised some questions on why replacing the key matrix but not the query and value matrices in wiring models, and the authors explained the intuition behind the approach. Some reviewers also have concern on the extra computation induced by the alignment strategies and the PF. The authors' rebuttal addresses the above concern.

Most reviewers have raised their scores after reviewing the authors' rebuttal.
The reviewers find the technical novelty and contributions are significant enough for acceptance. The authors' rebuttal helps address some issues. The area chair agrees with the reviewers and recommend it be accepted.
Decision: Accept (poster)