Continual Learning with Global Alignment

Authors: xueying  bai
jinghuan  shang
yifan  sun
niranjan  balasubramanian
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 17:09:45

Abstract: Continual learning aims to sequentially learn new tasks without forgetting previous tasks' knowledge (catastrophic forgetting). One factor that can cause forgetting is the interference between the gradients on losses from different tasks. When the gradients on the current task's loss are in opposing directions to those on previous tasks' losses, updating the model for the current task may cause performance degradation on previous tasks. In this paper, we first identify causes of the above interference, and hypothesize that correlations between data representations are a key factor of interference. We then propose a method for promoting appropriate correlations between arbitrary tasks' data representations (i.e., global alignment) in individual task learning. Specifically, we learn the data representation as a task-specific composition of pre-trained token representations shared across all tasks. Then the correlations between different tasks' data representations are grounded by correlations between pre-trained token representations. We explore different ways to learn such compositions. Without experience replay, our model achieves SOTA performance in continual learning tasks. It also achieves advanced class-incremental performance through task-incremental training.

Main paper: 
 introduction: 
 Continual Learning (CL) aims to develop models that can sequentially learn from streams of data and tasks, an important need for many real-world applications [35, 18] . One main challenge in developing CL models lies in reducing catastrophic forgetting, where models forget knowledge obtained from previous tasks after learning new tasks [38, 44, 15] . Catastrophic forgetting can happen when there is interference during task learning, especially in models that use shared parameters for all tasks [46] . Specifically, when learning a new task, if the model's gradients on the new task's loss are contradictory (e.g. in opposing directions) to those on the previous tasks' loss, the model will be updated towards a direction that increases the losses on previous tasks, causing forgetting. In this paper, we first identify factors that may lead to interference by analyzing the dot product between models' (flattened) gradients on losses from two tasks. Then, we design methods to address interference based on each factor. Our analysis in Section 3 shows that interference mainly depends on two factors: correlations between hidden representations of the data from different tasks; and correlations between columns of the classifier (which we call class vectors) that map data representations to corresponding classes.To address interference caused by the first factor, we want models to learn aligned data representations that do not have destructive correlations (i.e., leading to interference) when switching tasks and during task learning. Specifically, this requires models to accommodate future task representations when learning the current task. This motivates learning data representations based on some (well-correlated) global representations, for which we use pre-trained token representations for language data [11] . Specifically, we compose data representations as task-specific interpolations of pre-trained token Figure 1 : Overview of our methods. Task i's data representations are denoted as h i with pretrained token representations as grey dots in the 'Representation' block. Correlations between aligned data representations from different tasks depends on correlations between pre-trained token representations. In the 'Class Vectors' block, class vectors for different classes have different focuses on representations after probing, which can reduce interference caused by overlapped representations.representations. This allows the correlations between tasks' data representations to be grounded by correlations between pre-trained representations. We design three transformer-based [54] models that target such global alignment: (1) . learning data representations by interpolating the pre-trained token representations through the attention mechanism; (2) . the above model with additional neighborhood information to expand the search space of task information; (3) . a controlled LoRA [20] model that adapts the pre-trained token representation with a small scaling factor.To address interference caused by the second factor, we first train the classifier only when switching the task and then tune the whole model. This probing first strategy was first proposed in [27] to reduce representation distortion in single-task learning. Here we use it to reduce interference especially when there are destructive correlations between representations (e.g., caused by overlapping representations [5] ). Probing enables different class vectors to focus on different features in data representations when switching tasks, which is useful when different tasks' representations are overlapped unexpectedly.An overall view of our methods is in Fig. 1 . Evaluations show that both the aligned representations and the probing first strategy improve CL performance in multiple settings. Specifically, global alignment models perform well in class-incremental evaluation after task-incremental training.In conclusion, we make the following key contributions in this paper:1. We identify factors that cause interference in CL, and propose to address the interference issues by learning aligned representations and applying the probing first strategy. 2. We design three models to learn aligned representations, which learn task-specific attention with different levels of adaptations on pre-trained token representations. 3. We conduct extensive experiments on multiple CL settings. Results show that our models can significantly reduce forgetting even without the use of experience replay. 
related work: 
 Continual Learning CL Models can be divided into three main categories: regularization-based models which constrain the deviation of new parameters from the older ones [25, 62, 1, 29] ; replaybased models which reduce forgetting by rehearsing on real or pseudo samples from previous tasks [35, 7] or generative models [49, 24] ; and architecture-based models which learn evolving architectures for sequential tasks, with their capacities for each task carefully assigned [48, 60] .CL in NLP is an emerging area [32, 3] . MBPA++ [10] uses experience replay and local adaptation to mitigate forgetting; LAMOL [53] generates pseudo samples for replay; IDBR [21] disentangles task-agnostic and task-specific information; CTR [23] uses a capsule network for knowledge transfer.All the above models are based on pre-trained LM [11, 4, 43] . Recent works show that pre-training can alleviate catastrophic forgetting [59, 39, 28] . Our work is also based on the pre-trained LM, but we use it for alignment purposes without experience replay.Alignment in CL Recent CL works have studied the importance of alignment between different tasks' learning. Riemer et al. [46] aligns gradients between tasks to reduce destructive interference; Guo et al. [16] preserves holistic information for future tasks. If the previously learned knowledge does not align with that for future tasks, models may abruptly change previously learned knowledge and cause forgetting [5, 36, 22] . However, previous works focus on the alignment between observed tasks, which lack a global view and can cause forgetting in the future [26] . Instead, our work achieves alignment by pre-trained semantic features, which are general even to unseen future tasks.Adaptation Models With limited trainable parameters, our alignment models have connections to adaptation models, which originally aimed at parameter efficiency. Different adaptation models add limited trainable parameters on the frozen transformer layer [19, 40, 17, 20] ; or selectively update existing parameters [42, 61] . Recent works do adaptation by prompt tuning [31, 30, 34] , which learns prompt embeddings for target tasks.Adaptation models have also been used for CL [57, 13, 56, 45, 50] . However, most works use the models' parameter efficiency to construct progressive memory. Whether different adaptation structures influence CL, why and how they help remain unexplored. Our model has a similar form to adaptation models after derivation, but our design focuses on representation alignment rather than computational efficiency, with no progressive memory. 
problem statement: 
 In this paper, we focus on the catastrophic forgetting caused by cross-task interference [46, 47] . We conduct a case study to identify factors of the interference in Section 3, which motivates our global alignment approach in Section 4.Tasks and Data We consider the setting where models continually learn a sequence of tasks, with the condition that the previous tasks' data becomes inaccessible when learning new tasks. We denote a representative data for each task i as (x i , y i ), where x i is the model input and y i ∈ c i is its class logit. c i is the set of all class logits in task i, and we denote the set of class logits across all T tasks as C = {c i } T i=1 . Scenarios We consider two CL scenarios: task-incremental learning and class-incremental learning. The main difference between them is that at inference time for task i the model knows its task-specific classes c i in task-incremental learning (task-aware), while the model has to predict the class from all classes C in class-incremental learning (task-agnostic) [37] .Models Our models consist of an encoder that encodes the input x i to a d-dimensional representation h i ∈ R d , and a matrix of class vectors (i.e., a classifier) v ∈ R d×|C| whose y i -th column v yi maps the hidden representation h i to the space of the class y i . The probability of y i being predicted by the model is calculated by the softmax function: p(y i |h i ) = softmax(h T i v) yi . At training time, the softmax is computed over classes in each task, while at inference time the softmax is computed over the range of classes specified in the task-or class-incremental scenarios. We sequentially train each task with the cross-entropy loss: L(h i , y i ) = -log p(y i |h i ) for task i's data (x i , y i ).According to Riemer et al. [46] , catastrophic forgetting can occur when a model learns a new task if its gradients on the new task's loss are contradictory (e.g. in opposing directions) to its gradients for the previous tasks' losses. In other words, the gradient descent for the new task might update the model towards a direction that increases its losses on previous tasks, and thus cause forgetting.In this section, we analyze factors that can lead to interference between gradients. Specifically, we study a case based on a representative toy model. The model's encoder contains two linear layers with corresponding weight matrices denoted by W l ∈ R d×d , where l indexes the layers. The encoder outputs the representation:h i = W 2 W 1 x i for data in task i. Here the input x i ∈ R d is a d-dimensional vector.Suppose we are learning task j after task i using gradient descent. Consider (x i , y i ), (x j , y j ), two arbitrary data instances in tasks i and j respectively. The interference between gradients of the weight matrix W l with respect to the cross-entropy loss is given by:I(W l ) = ∇ W l L(h i , y i ) • ∇ W l L(h j , y j ).Destructive interference occurs when I(W l ) < 0, which can cause the model to forget task i's knowledge after updating W l for task j. On the other hand, when I(W l ) > 0, the gradients of two tasks can enhance each other which encourages knowledge transfer across tasks [46] . We expand I(W l ) below. For simplicity, we calculate gradients related to the y i -th and y j -th class vectors in the matrix v:EQUATION)h l-1 i = W l-1 xi, l=2xi, l=1 is the hidden representation at the (l-1)-th layer,Ω l+1 = I, l=2 (W l+1 ) T W l+1 , l=1is about the weight matrix at the (l+1)-th layer. The class vector v yi ∈ R d is the y i -th column of the classifier v, mapping the output representation to the space of class y i .Based on Eq. 1, I(W l ) depends on the correlation between data's hidden representations at the (l-1)th layer; and the correlation between class vectors transformed by weight matrices of the subsequent (e.g. (l+1)-th) layers. To address the interference issue, we discuss each correlation below.We consider correlations between different hidden representations at two time points (Fig. 2 ): (1) The model has learned task i and is switching to learn task j. At this time, the model's hidden representations of task j's data should not destructively interfere with those of task i's data, even though the model hasn't been trained for task j yet. (2) After learning Task j. At this time, the model should still produce good hidden representations for task i's data, even though it no longer has access to task i's training data. Some previous works address the interference issue by forcing models to learn different tasks in orthogonal subspaces [9, 14, 55] . However, they may constrain models' knowledge transfer ability as I(W l ) will always be 0. Other works minimize the destructive interference at time (2) , by learning task j with the replay of task i's data [46, 47, 2] . However, if hidden representations do not correlate well when switching to task j, they may cause and propagate interference in task j's learning (Fig. 2 (a)). Then even with replay, representations of task i's data may drift towards representations of task j data, leading to disruptive model updates [5] .Our work tackles correlations of hidden representations at both time points. When learning across tasks, the model is expected to produce aligned hidden representations of both the current and previous tasks' data at all times. The aligned representations should have appropriate correlations, which will not lead to destructive interference but retain the model's ability for knowledge transfer. To achieve this, we encourage the model to learn data representations for each task as different (task-specific) interpolations of pre-trained token representations. This then enables the correlations between data representations to be grounded by correlations between pre-trained token representations (details are in Section 4).The interference also depends on correlations between class vectors. Assuming no shared classes in task i and j, the class vector v yi is not involved in learning task j and thus remains unchanged after task i. When learning the class vector v yj for task j, we denote v yj at the time step t as v yj ,t . Then the correlation between class vectors v yi and v yj ,t is:EQUATIONwhere h j,t is the output data representation at time step t, v yj ,0 is the initialization of the class vector v yj , α is the learning rate.In Eq. 2, the correlation of class vectors depends on the initialization of the class vector v yj ,0 and the learning of data representations h j,t (in the gradient ∇ vy j L(h j,t , y j )). The learning of the representation h j,t depends on the correlations between representations, which we have discussed above. In addition, we hypothesize that a good initialization of class vectors can help mitigate the interference problem. To obtain suitable initialization for class vectors, we apply the probing then fine-tuning (PF) strategy [27] which first learns the class vectors (classifier) only and then fine-tune the whole model. We describe details in Section 4.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
methodology: 
 In this section, we introduce our models that align data representations and initialize class vectors. First, we introduce our global alignment models which learn task data representations as interpolations of pre-trained token representations. Then we discuss the probing and then fine-tuning strategy and the effects of initializing the class vectors for CL.Pre-trained Token Representations In this paper, we focus on language models which are typically pre-trained in a self-supervised manner [11, 41] . For example, some models are pre-trained by the masked language modeling objective, which first masks tokens in input texts and then learns models to predict masked tokens. By pre-training, models learn semantic relationships between tokens.We consider transformer-based [54] language models. Typically, for an arbitrary task1 , the input of the model is a sequence of n tokens. At the l-th transformer layer, denote the input representations of all tokens asG l-1 = [g l-1 1 , ..., g l-1 n ] T (l ≥ 1)where each token representation is an R d vector. Then the output token representation is:EQUATIONwhere W l q and W l k ∈ R d×d are query and key matrices, Attn(Q, K) = softmax QK T / √ d is a function calculating the attention matrix based on the query Q and key K. The initial token representation G 0 is the output of an embedding layer in the model. A feed-forward layer is applied after self-attention for token-wise transformation. We omit it here for simplicity.After pre-training, we obtain contextual token representations G l at each layer l using the pre-trained matrices W l q , W l k and W l v . Such token representations contain semantic information of input tokens, which are general enough to accommodate diverse uses in different downstream tasks. We refer to representations G l as pretrained token representations.Data Representations To address a downstream task, models learn a data representation that summarizes the task-specific information in the entire input. Typically, these data representations are learned by fine-tuning all parameters in the pre-trained model. This type of full fine-tuning has been shown to distort the pre-trained token representations [27] and may not consider correlations between data representations across tasks. This does not fit the alignment goal stated in Section 3.2.To align data representations across tasks, we propose to wire (interpolate) the pre-trained token representations to construct the data representation. Our proposal is based on the following hypothesis:Task-specific information of data can be composed from the general semantics of tokens in that data.For example, given the input text 'Some actors have so much charisma that you'd be happy to listen to them reading the phone book' from 'positive' class in a sentiment analysis task, a composition of tokens {charisma, happy} can convey the information of 'positive'.Based on the hypothesis and the fact that pre-trained token representations contain the information of general token semantics, appropriately interpolating pre-trained token representations can represent task-specific information in the data. Since correlations between pre-trained token representations are general across tasks, this type of composition aligns data representations from different tasks.Alignment Effect For an arbitrary task, the pre-trained token representations G l can be interpolated to yield the data representation at the l-th layer as: We specify hidden representations for [CLS] and any other token as h lh l = b l G l T where b l ∈ R 1×n is a learnable[CLS] and h l others .stochastic row vector with weights for the interpolation. To assess the alignment effect of this scheme, we obtain the correlation between data representations h l i and h l j from task i and j as:EQUATIONwhere b l i , b l j are learned interpolations, G l i , G l j are pre-trained token representations for data in task i and j.At any time step, the correlation between data representations is grounded by the correlation between pre-trained token representations: Eq. 4 involves the correlation G l i (G l j ) T between pre-trained token representations; and the task-specific interpolation weights b l i , b l j are also learned with the guidance of G l i and G l j respectively. This grounding to pre-trained token representations thus aligns the data representations across tasks.We develop global alignment models to learn data representations as interpolations of pre-trained token representations. Parameters in models are trained by the cross-entropy loss for each task. Following common practice in traditional models, we append a l G l = b l Attn(G l-1 W l q , G l-1 W l k )G l-1 W l v .Since the product of a stochastic row vector and a row-stochastic matrix is stochastic, b l Attn(G l-1 W l q , G l-1 W l k ) can be viewed as task-specific attention on the pre-trained token representations G l-1 .Based on this, we develop a fixed wiring model that learns task-specific attention for [CLS] only, while using the pre-trained parameters to compute hidden representations of other tokens as the pre-trained representations G. In the model, the task-specific attention is calculated as the attention from [CLS]'s query, using a new learnable key matrix ∆W l k and the pre-trained query matrix W l q . Formally, the data representation h l[CLS] is:EQUATIONwhere [20] for efficiency. By constraining non-cls tokens' hidden representations to be close to the pre-trained token representations, the fixed-wiring model may have limited learning capacity. To avoid this, we design two other methods with improved model capacity, which we describe below.h 0 [CLS] is the pre-trained embedding of [CLS]. ∆W l k is low-rankedWiring with Neighbor Attention Sometimes, the task information may not be easy to extract from pre-trained representations of input tokens. For example, in a text entailment task, give a sentence pair 'The boy is crying; He's happy about the view.' with the label 'contradiction'. The pre-trained representations of task-related tokens 'crying' and 'happy' may not be negatively correlated, which makes the model hard to learn their contradiction. However, 'crying' usually has a neighbor token 'sad', and pre-trained representations of 'sad' and 'happy' are more likely to have negative correlations. Therefore, using the information of 'sad' may make the model easier to learn the task.Therefore, to increase the model capacity while preserving its alignment ability, we retain the guidance of pre-trained token representations while exploring the tokens' neighborhood to better search for the task-specific information. The data representation can be written as:EQUATIONwhereG expand = [G l-1 ; G l-1 nei ]concatenates input tokens' pre-trained representations G l-1 and their neighbors' representations G l-1 nei . Since each token has its own neighbors, to obtain the data representation h l[CLS] in Eq. 6, we first adapt each pre-trained token representation individually to incorporate task-specific information from their neighbors. Then we calculate task-specific attention on adapted token representations, using the attention mechanism in Eq. 5. Specifically, we adapt the p-th pre-trained token representation g l p by:(g l p ) T ← (1 -s) • (g l p ) T + s • Attn((g l-1 p ) T W l q , G l-1 nei,p ∆W l k )G l-1 nei,p W l v ,where G l-1 nei,p ∈ R k×d contains k neighbor representations for the p-th token, and G 0 nei,p is the pre-trained embedding of the neighbor tokens. s ∈ R is a scaling factor. The neighbor tokens are selected by comparing cosine similarities between token embeddings. To stay close to the pre-trained token representations but incorporate task-specific information, we update the neighbor representations as:G l nei,p = (1 -s) • expand(g l p ) + s • Attn(G l-1 nei,p W l q , G l-1 nei,p ∆W l k )G l-1 nei,p W l v, where expand(g l p ) ∈ R k×d is the matrix which duplicates g l p for k tokens. G nei provides extra capacity in learning data representations. Meanwhile, the alignment effect is preserved by controlling the scale s, and making the neighborhood G nei not deviate far away from pre-trained token representations G.Controlled-LoRA Another way to increase the model capacity is to adapt representations of all tokens (including both [CLS] and other tokens in the text) by learning low-rank matrices added to the pre-trained query and value matrices (LoRA [20] ). This has the model capacity close to fine-tuning, while keeping reference to the pre-trained parameters.Denote the input representations of all tokens at layer l as H l-1 , where H 0 = G 0 is the pre-trained token embeddings. In LoRA, all token representations are updated by the same attention mechanism, with a learnable query and value matrices ∆W l q and ∆W l v . The data representation h l [CLS] is:EQUATIONWhen s = 0, we have H l = G l . When s > 0, the added query and value matrices not only learn the task-specific attention, but also adapt the token representations which can deviate away from the pre-trained ones. To keep alignment with pre-trained token representations, we control the scaling factor s to make adapted token representations close to the pre-trained ones.Although data representations are grounded by pre-trained token representations, at the start time of learning task j after task i, the interpolation for task j's data may not be well learned. In this case, properly initializing class vectors of classes in task j can help reduce interference.To initialize the new class vectors when switching tasks, we adopt the probing and then fine-tuning (PF) strategy first proposed in Kumar et al. [27] : when learning a new task, it first freezes the encoder and only trains the classifier for the task (probing); and then tunes the encoder and classifier together (fine-tuning). This is beneficial in the case, for example, when the two tasks have similar input distributions but target different classes (e.g., news sentiment analysis vs. news categorization). In this case, data representations for two tasks may overlap when switching tasks. However, the class vectors can focus on different features in data representations after probing. Therefore, the correlation between class vectors may be small (or 0) and can reduce the interference. 
datasets and metrics: 
 We evaluate four sequences of CL tasks: (1) Yahoo: a split of Yahoo dataset for news question-answer categorization [63] with 5 disjoint tasks containing 2 classes each; (2) DB: a split of DBPedia data for [12] , RTE (text entailment, 2 classes) [58] and SST (sentiment analysis, 2 classes) [51] ; (4). All: All tasks in the above sequences. For each task, we randomly sample 1245 samples per class, which is the least number of class samples in our datasets.We train alignment and adaptation models in task-incremental (Task-IL) settings, where in-task classes are specified during training. Then we evaluate models on both Task-IL and Class-IL inferences, where in-task classes are not specified for Class-IL inference [37] . We measure models' average accuracy and forgetting (Appendix B) over five random seeds.We compare different models on the pre-trained BERT-base model, including:Alignment Models (ours): (1) Wire-Fixed: the model freezes pre-trained token representations and learns task-specific attention for data representations (i.e.[CLS] token). (2) Wire-Neigh: the wiring model with neighbor attention. We set s = 0.1. For computation efficiency, we fix the number of neighbors as k = 5, and randomly select neighbors from top-K (K = 20) nearest neighbors to control the range of neighborhood. (3) C-LoRA: the controlled LoRA model with the scaling factor s = 0.1. For both Wiring and C-LoRA models, we set the matrix rank r = 8. We also evaluate above models with the probing then fine-tuning (PF) strategy, denoted as Model+PF.Adaptation Models: (1) Fine-tuning (FT): fine-tuning all parameters sequentially. (2) Prefix Tuning (Prefix) [33] : freezing the pre-trained parameters and adding learnable embeddings to attention layers.(3) Adapter [19] : freezing the pre-trained parameters and injecting learnable linear projections after self-attention. ( 4) LoRA [20] : the LoRA model with suggested scaling s = 1 for single task learning.CL Models: (1) ER: the FT model storing all seen examples and performs sparse (1%) experience replay. (2) A-GEM [7] : the FT model constraining gradients to prevent degrading performance on previous tasks. (3) MBPA++ [10] : the FT model that stores and retrieves samples to locally adapt the model at inference time like [52] . (4) IDBR [21] : the FT model with information-disentanglementbased regularization and replay. We also compare to IDBR without replay, denoted as IDBR(-R). (5) CTR [23] : an adapter-based model with capsules and task transfer routing. (6) L2P [57]: a promptbased model that dynamically prompts for different data and tasks. ( 7) CODA [50] : a prompt-based model that learns attention over extensive prompt components for tasks. (8) . ERACE [5] : a model for class-IL that calculates the current task's loss over in-task classes, while calculating the replay loss over all seen classes in the replay buffer. We also show the performance of MTL, which is an FT model jointly trained on all tasks (not CL). Detailed settings for models are shown in Appendix A. Alignment models are effective in Task-IL and Class-IL without replay. Results for Task-IL are shown in Table 1 . Our alignment models forget less than CL models that use experience replay during fine-tuning. They also outperform adaptation models including Adapter, uncontrolled LoRA and prefix tuning. This suggests the reduced forgetting in our alignment models is not just due to tuning fewer parameters, a feature in common with the adaptation models. In particular, C-LoRA performs better than the original LoRA model, which does not control the scaling factor for alignment. Results for Class-IL are shown in Fig. 4 . On DB, alignment models achieve advanced Class-IL performance close to ERACE, a strong Class-IL model with replay, even under the Task-IL training where models are not trained on out-task classes. On Yahoo, alignment models also forget less than FT and LoRA.Wiring models forget less. Without the probing and fine-tuning (PF) strategy, wiring models achieve best performances in all Task-IL experiments and Class-IL on DB (Fig. 4 ). We hypothesize that the good performance of wiring models is due to the strong alignment effect achieved by referencing the pre-trained token representations. With improved capacity brought by neighbors, Wire-Neigh outperforms Wire-Fixed on hard tasks like those in News Series. However, wiring models may have less capacity compared to models with more adaptations, which can limit their classification (separation) capability in Class-IL (e.g. Class-IL on Yahoo).PF reduces forgetting. Fig. 4 shows that applying the PF strategy improves CL performance for all models. Although C-LoRA itself performs worse than wiring models in Task-IL, with PF it outperforms wiring models on 3/4 sequences. It also achieves best performance on Yahoo Class-IL. We hypothesize that is because PF can reduce interference when data representations have destructive correlations, as discussed in Section 3.2. Generally, PF and alignment on representations compensate each other on reducing interference. When the alignment effect is strong (e.g., wiring models), the improvement caused by PF is not very significant. However, when the alignment effect is not as strong (e.g., C-LoRA, LoRA, FT), PF can significantly improve models' performance in CL.Table 3 : Alignment quantification on SNLI and News Series. We report Recall@20 on three random seeds. Interpretability of Alignment Models Since alignment models learn data representations as the interpolation of pre-trained token representations, we interpret learned data representations using the pre-trained MLM decoder. Specifically, we decode data representations to the token space by the pre-trained decoder and then get the tokens with top-10 probabilities. This helps us understand how data representations are closed to pre-trained token representations the model aligns to. Results are shown in Table 2 . From the table, decoded tokens of alignment models are close to tokens that are related to target classes, while those of non-alignment models (FT, LoRA) are hard to interpret.We also quantify the model's alignment ability using E-SNLI data [6] , where each data's task-related tokens are annotated by human. We calculate the Recall@20 of annotated task-related tokens being retrieved from data representations, on SNLI and News Series data. The results are shown in Table 3 .Results suggest that wiring models have more alignment ability than C-LoRA, in both in-task (SNLI) and CL evaluations on similar NLI tasks (News Series).This interpretability may explain the effectiveness of alignment models in Class-IL: even when trained with local classes, data representations in each task are correlated to pre-trained token representations that relate to all tasks' classes. This may help to separate representations from different tasks. This may be due to the decrease of the global alignment effect, which increases the interference. After applying PF, C-LoRA's accuracy first increases and then slightly decreases. This may be because PF reduces the interference caused by the class vectors, and the model can fully utilize its global alignment ability when increasing plasticity. However, when the scaling factor goes too large, the loss of alignment will lead to more forgetting even with PF. For Wire-Neigh, the observation is similar to C-LoRA: when s goes up, the model's accuracy first increases and then decreases because of the trade-offs between global alignment and plasticity. And since Wire-Neigh interpolate pre-trained token representations with their neighbor representations, the increase of s leads to the decrease of pre-trained information. And therefore we observe a more rapid performance drop when s increases. Influence of Neighborhood In Wire-Neigh, we randomly select five neighbor tokens from top-K nearest neighbors.Here we study the effect of the range of neighborhood with different K value in Table 5 . For relatively simple sequences DB and Yahoo, Wire-Neigh under different K has stable performance. However, for hard sequence News Series, when K increases, the model has more neighbor information (more capacity) to solve the task, which first improves its CL performance. However, when K is too large (K = 100), the neighbor information may become noisy, which makes the CL performance drop. 
limitation: 
 We discuss our limitations in model and assumption perspectives: For models, without replaying previous data, there can be a problem of shifting attention; e.g. the model shifts attention on previous task-related tokens after learning new tasks. This may lead to forgetting, which we leave as our future study. For assumptions, our model assumes we have a pre-trained model. For domains that do not have well-established pre-trained semantic features, our model may not be immediately applicable. However, since fundamental models are consistently established and shown to be beneficial across different domains, we can expect our models to apply to more domains in the future. 
conclusion: 
 In this paper, we investigate methods to address correlations between data representations and class vectors to reduce interference when training across tasks. Specifically, for alignment, we propose to learn data representations as task-specific compositions of pre-trained token representations. To learn the composed representations, we propose wiring models with or without neighbor attention and a controlled LoRA model. To address correlations between class vectors, we adopt the probing and then fine-tuning strategy, which can effectively reduce interference even when the representations do not correlate well. Experiments show that our models can successfully learn the composed representations for alignment and achieve SOTA performance in CL. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We included this information in our Appendix A.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes]Justification: We followed it.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: Our work does not have negative societal impacts. For positive impacts, since we study the CL problem training tasks sequentially without revisiting previous tasks or data, we have a positive impact on security. However, this is not our key point so we do not discuss it in the paper.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: No risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We used open-source datasets.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: We provide detailed experimental settings in addition to the main paper Section 5. We train all models (except ERACE) via Task-IL training, while evaluating them in both Task-IL and Class-IL settings. We perform all experiments on one Nvidia RTX A6000 machine.• Probing: We fix the encoder and only train the classifier. We train 5 epochs for each taskwith the learning rate 5e-4. • FT: We fine-tune the whole model, including the encoder and classifier. We train 3 epochs for each task in BERT, with the learning rate 2e-5. • Adapter: we select learning rates from {5e-5, 1e-4, 1e-3} and train {5, 20} epochs for each task. For all continual learning tasks, we train with the learning rate 5e-5 and 20 epochs. The rank of Adapter projection r is 32 as suggested in the original paper. • LoRA (C-LoRA): we select learning rates from {5e-4, 1e-3} and train LoRA for {5, 8} epochs for each task. • Wire-Fixed and Wire-Neigh: we select learning rates from {2e-4, 5e-4, 1e-3} and train {5, 8} epochs for each task. The rank of the learnable key matrix is 8. For Wire-Neigh, the number of neighbors is 5. In practice, they are randomly sampled from a larger neighborhood ranging from {10, 20,50,100}. We set the mixing ratio as s = 0.1. • IDBR: We train IDBR with the learning rate 3e-5 for 3 epoches per task. We follow the k-means memory selection rule, and the replay batch size is 16 (training batch size) × number of tasks in the memory. • CTR: We follow the settings in the original paper, training 5 epochs for each task.• L2P: We have the prompt pool with 100 prompt tokens and select 50 of them to prepend to the input. We train the model with the learning rate 1e-3 for 20 epochs for each task. • CODA: We have the prompt component size 20 for each task, and set the prompt length as 20. We train the model with the learning rate 1e-3 for 10 epochs for each task. • ER: We apply sparse experience replay with 1% replay ratio. At each replay time, we sample 32 samples from the memory and perform one-step gradient descent based on them. • A-GEM: We store all previous data in the memory. At each gradient step, we randomly extract 32 samples from the memory and apply the A-GEM gradient projection. • MBPA++: We fine-tune the model with ER and then adapt the model at the inference time.At the inference time, we retrieve 32 nearest samples in the memory for local adaptation.Recall@k Denote the set of task-related tokens of the i-th sample as rel i , the set of top-k tokens predicted from the learned data representation as pred i @k. The metric Recall@k calculates the proportion of task-related tokens rel i that are predicted in pred i @k, which is defined as:Recall@k = E i |pred i @k ∩ rel i | |rel i | .Because each data instance in E-SNLI has 5-10 task-related tokens, we use Recall@20 for evaluation.Average Accuracy and Forgetting We use the average accuracy and average forgetting similar in [8] to evaluate the performance in CL scenarios. The specific definitions are described below.• Average Accuracy (ACC ∈ [0,1]): Let a i,j be the performance of the model on the test set of task j after the model is trained on task i. The average accuracy after training on all task T is:ACC T = 1 T T j=1a T,j .In this paper, we select T as the end of CL task sequence.• Average Forgetting (FGT ∈ [-1,1]): Denote f i,j as the forgetting on task j after the model is trained on task i. f i,j is calculated by:f i,j = max l∈{1,...,i-1}a l,j -a i,j .And the forgetting after training on the task T is:F GT T = 1 T T -1 j=1 f T,j .Our forgetting score divides the number T of all tasks instead of T -1. We do this to make the above metrics also indicate models' capacities on single tasks, i.e. single-task capacity ≈ ACC T + F GT T .In this section, we discuss the computation costs of alignment models. All our models use shared parameters across tasks, which do not progressively increase parameters. Since the learnable new matrices in our models are all low-ranked, they require limited usage of additional memory. So we focus on the models' time consumption. For all models, we have the number of input tokens as n.C-LoRA Our controlled LoRA model has the same time complexity as LoRA.Wire-Fixed In the Wire-Fixed model, besides forwarding the pre-trained model to computing pre-trained token representations, we need extra computation for Eq. 5. Since we only query for the [CLS] token, the extra time complexity is O(n).Wire-Neigh In Wire-Neigh, the extra time consumption comes from searching for neighbors and updating the neighbor representations. We find neighbors for each token based on their embeddings at the embedding layer. The time complexity is O(V n + V logV ) for computing the cosine similarity and sorting, where V is the size of the token vocabulary. Then the k neighbor representations are updated for each layer, with a complexity of O(nk 2 ).In practice, since the embedding layer is fixed in our model, for each data instance we only need to find their neighbors once and then store the neighbor indices for iterative training (i.e., for several training epochs). The neighbor selection can also be accelerated by reducing the search space of neighbor tokens, for example, only searching neighbors from frequently used tokens instead of the whole vocabulary.Wire-Neigh also needs extra memory to store neighbor representations and update them. To control the extra consumption of memory, we keep the number of neighbors as k = 5 and randomly sample neighbors from top-K similar tokens to control the range of the neighborhood. We leave the improvement of Wire-Neigh's efficiency for future works.Computation induced by PF Another computation costs come in the probing stage, in which we fix the encoder and only train the classifier. This takes less than 40% training time (including language model forwarding) and 30% GPU memory compared to full fine-tuning.NeurIPS Paper Checklist Justification: We included a limitation section (Section 6). Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.For an arbitrary task, we do not specify the task id i by subscripts in the notations for simplicity. 


Decision: This paper presents a global alignment method to alleviate catastrophic forgetting in continual learning. It first identifies the correlation between data representations is the main cause of gradient interference from different tasks. Then proposes a global alignment method to learn data representation as a composition of pre-trained token representations shared by all tasks, and explore different ways to learn such composition.  Extensive experiments demonstrate the effectiveness of the proposed method for both class-incremental and task-incremental learning.

The proposed approach has interesting theoretical justifications, and the code is provided for reproducing the experiments in the paper.

Reviewers raised some questions on why replacing the key matrix but not the query and value matrices in wiring models, and the authors explained the intuition behind the approach. Some reviewers also have concern on the extra computation induced by the alignment strategies and the PF. The authors' rebuttal addresses the above concern.

Most reviewers have raised their scores after reviewing the authors' rebuttal.
The reviewers find the technical novelty and contributions are significant enough for acceptance. The authors' rebuttal helps address some issues. The area chair agrees with the reviewers and recommend it be accepted.

Reviews: review: paper_summary: paper_summary: This paper studies the cause of cross-task interference in class-incremental learning of transformer-based language models. The authors disentangle the cause into the correlation (i) between data representations and (ii) between class vectors in the linear classifier. To tackle (i), the authors propose three ways to construct data representations at each layer by learning an attention over the pretrained token representations. To tackle (ii), the authors propose to only train the classifier for a new task (to obtain a good initialization) before jointly training both the classifier and the encoder.

The authors perform experiments with the pretrained BERT-base model and various text datasets. Training is in the task-incremental setup (where task labels are provided and the model predicts over in-task classes); the model is evaluated on both task-incremental and class-incremental (where task labels are not provided and the model predicts among all classes) setups. The authors found that their methods, "alignment models", outperform existing adaptation and continual models.
strength_weakness: strength_weakness: 1. While the main goal of the paper is reducing cross-task interference, the main results (Table 1) is using the task-incremental setup, where task labels are known during prediction. The class-incremental learning results are only in Fig. 4, comparing the proposed method only with LoRA and ERACE.

2. I find it a bit hard to infer where the performance improvement comes from from the results. I wonder if it is possible to do some fine-grained ablation analysis that verifies that the proposed method indeed helps by reducing overlap in data representations and in class vectors' features for different tasks. For example, one could measure the accuracy on the task level or look at the confusion matrix, which may reveal some information about cross-task confusions.

3. Is $\Delta W$ shared across tasks? If so, I don't understand why the proposed method does not forget. The authors hypothesize that this is potentially due to referencing pretrained representations. However, since the [CLS] representations are constructed involving $\Delta W$, I'd expect the model to lose some ability to generate good representations for past tasks. Could the authors elaborate on this point?
questions: questions: 1. Could the authors explain the choice of training in TIL while evaluating in CIL, as opposed to training and evaluating in the same setup?

2. (Eq. 2) Since weights are usually initialized from a distribution centered around 0, shouldn't the first term be quite small? 

3. (Sec 4.3) Does probing perform softmax over only the new classes? If so, how does it help two class vectors for different tasks to focus on different features, since the loss does not require differentiating between the two classes?

4. (Sec. 4.2) In the wiring models, what is the intuition behind only replacing the key matrix but not the query and value matrices? 


Minor:

5. (Eq. 1) It seems that interference is eliminated once any of the three components is 0. Is it possible to fix the class vectors to be the canonical basis which are guaranteed to be perpendicular, and only learn the data representations?

6. (Sec. 4.2) In Wire-Neigh, do you need to find the $K$ nearest neighbors for each token? If so, how expensive it is in practice?

7. (L#219) Could you elaborate on what "grounded" means here?

8. Missing references: [1, 2] are two prompting-based CL methods that are shown to perform better than L2P. 

9. (L#41-44) I recommend changing the indices to, e.g., "(i), (ii), (iii)", as a different "(2)" is referred to in the next paragraph.

10. (L#53) "by" -> "after"

11. (L#182) "grounded192" -> "grounded"


[1] DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning. Wang et al. ECCV 2022.

[2] CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learning. Smith et al. CVPR 2023.
limitations: limitations: The authors have addressed the limitations clearly in Sec. 6.
paper_summary: paper_summary: This paper addresses the problem of Task-Incremental-Learning (TIL) with pre-trained transformer in the context of NLP. The author extended their experiments in the Class-Incremental Learning (CIL) scenario. The authors identify potential forgetting causes as (1) negative correlation between data representations and (2) negative correlation between class prototypes. After theoretically justifying their claim, the authors propose to align the model representations with either a) learning new key matrices for pre-trained queries and values b) learning new key matrices for with additional token chosen as the nearest neighbors of the considered token c) learning new queries and values from the original ones with a low-rank adaptation strategy. The author additionally align prototypes (class vectors) leveraging a probing and fine-tuning strategy. This paper shows superior performances on various dataset in CIL and briefly discuss the effect of various components.
strength_weakness: strength_weakness: 1. While I believe the evaluation to be sufficient, it would improve the paper to include more recent prompt-learning techniques such as CODA [1] which showed stronger performances than L2P. I believe it would equally be interesting to see this approach applied to Computer Vision problems, even though I understand this paper focuses on NLP.

2. Although the proposed approach is theoretically justified, it would be interesting to quantify such alignment through experiments, by computing data representations covariance/correlation between transformed class vectors; for each alignment strategy.

3. I appreciate the effort to include results in CIL scenarios. However, I think more discussions as to how the proposed approach performs poorly compared to ER-ACE could be introduced.

4. What is the link between the findings of this paper and previous work on orthogonal subspaces in continual learning [2, 3]? In these works, amongst others, it seems that the correlation between hidden representation should be zero, and not positive. I think such work should be included in related work section.

5. A discussion regarding the extra computation induced by the alignment strategies and the PF would be welcomed as well, as it seems to increase it considerably.

6. The code is unfortunately not accessible to the reviewers.

If the authors can address most of the above points I would happily increase my score.

**Typos and presentation**

- LoRA l44 and PF l.46 are not defined. Please either define the acronyms or cite the corresponding paper.
- l.114 $h_{\tau}$ should be h_{i}
- I do not like the use of RHS l.163 and 164.
- l. 182: "grounded192"?

[1] Smith, James Seale, et al. "Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

[2] Wang, Xiao, et al. "Orthogonal subspace learning for language model continual learning." Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10658–10671

[3] Chaudhry, Arslan, et al. "Continual learning in low-rank orthogonal subspaces." Advances in Neural Information Processing Systems 33 (2020): 9900-9911.
questions: questions: See weaknesses.
limitations: limitations: Limitations have been partially addressed in the main draft. I believe a discussion regarding the poor performances on CIL scenarios should be included, as well as a discussion on the potential computation overhead.
paper_summary: paper_summary: In Continual Learning (CL), the interference caused by the constant modification of the representation is the leading cause of catastrophic forgetting. Motivated by this and the idea that gradients in opposite directions are one cause of this interference, the authors proposed new ways of adding knowledge to a model. The authors motivate the proposal with a toy model that exemplifies where and why the interference occurs, concluding that it comes from the discrepancy between the representations of the different tasks and the relationship between the class vectors. These conclusions lead the authors to propose three proposals that slightly modify the transformer architecture, specifically adding learnable weights in the self-attention layers to generate a task-specific attention representation. The authors also propose using the probing and then fine-tuning approach presented in the past to help initialize class vectors. The results are presented in task and class incremental learning benchmarks in sequences of text datasets.
strength_weakness: strength_weakness: - What is the actual contribution of applying wiring or C-LoRA? The results show that applying FP can benefit the methods much more than adding the proposals presented.
    - What are the implications of applying FP to the Adaptation or CL methods? The performance gain from the proposed methods is relatively low (compared to L2P), and this could further decrease with the application of FP. This requires a deeper exploration of the proposed methods and the alternatives.
- The writing is unclear and often unnecessarily complex, making reading difficult for someone unfamiliar with the subject. The notation used is only sometimes in line with the literature, which can confuse readers. In addition, there are problems explaining some terms and easily avoidable problems. For example:
    - Line 46, FP, is mentioned but needs to be correctly defined. However, the document defines the abbreviation FP multiple times after that when only one should be enough.
    - Line 182, there is an extra “192”.
questions: questions: - My main concern is the causes of interference that motivate this work. The author hypothesised that a gradient in the same direction can alleviate forgetting between tasks, a concept also adopted by GEM, despite its susceptibility to forgetting. Even with a gradient in the same direction, if it induces significant weight changes, the representations could be compromised, leading to catastrophic forgetting.
    - How much does having a model pre-trained in tasks similar to those used in CL affect the proposed methods? The gradient will likely be low for these tasks.
    - Text benchmarks need minor modifications to the model weights due to the similar distribution between the pre-trained model and the new tasks. In cases where higher modifications are needed, can the scaling factor (s) help reduce this effect? Did you explore different values of the scaling factors?
    - Can this method work on image benchmarks? Images can have a more complex distribution than text, making avoiding interference between tasks difficult.  
- Is FP applied to CL or adaptation methods in Table 1?
    - Another technique used in CL is freezing class vectors in the classifier so that only the classes present in the batch during training are modified. This is especially useful when there is no memory. Can this help further reduce interference in the proposed methods?
- Fig 2.b, what method of global alignment was used?
- Three methods to mitigate interference are presented. Can these methods be complemented by each other?
- How many neighbours are used in Wire-Neigh? Did you perform experiments to find the optimal number of neighbours?
- Did you train only the classifier for the Task Incremental Learning problem? This method should not forget, and the pre-trained model can have good prior knowledge.
- For the CIL experiments, the proposed methods have very similar results to previous methods when applying FT in Yahoo. This somewhat contradicts the claim that the proposed methods mitigate forgetting. Do you have any intuition about why FT+PF works so well?
    - In Fig 4.b., is the performance with or without FP?
limitations: limitations: As the authors mention, this method heavily relies on a pre-trained model with a similar distribution to the incoming tasks. This assumption is only sometimes true and can be more complex in scenarios with a more complex input data distribution.
paper_summary: paper_summary: This paper tackles continual learning by addressing interference between tasks. For the interference between different tasks, the authors propose a method called ‘global alignment’ to align the data representations using task-specific compositions of pre-trained token representations. Then the authors conduct extensive experiments to verify the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. The analysis of interference in Section 3.2 does not consider the activation function between the two layers of the network. Multiplying the two linear weight matrices is equivalent to using only one weight matrix, so the analysis of this linear case is different from that of the non-linear case and I’m curious about the inference in the non-linear case.

2. About wiring with neighbor attention:

    a) The author needs to explain the rationality of neighborhood tokens and the impact of the size of K on the model.

    b) This method requires matching the most similar K tokens for each token given a sample. This process may require a large amount of computation.

    c) The author needs to further explain the difference between this method and controlled-LoRA. Both these two methods add a learnable term to the original token representations but adopt different generation strategies. Specifically, this method adds a term composed of neighborhood tokens, while controlled-LoRA uses low-rank matrices. So what are the advantages of this method?

3. About Controlled-LoRA:

    a) Computational burden and parameter size: As the number of tasks increases, linearly increasing model parameters is not in line with the spirit of CL, and the increased parameter size should be limited even if LoRA only adds small parameters compared to the large model.

    b) Why does Controlled-LoRA perform worse than other methods? In the case of task-IL, given the task ID for each test sample, we can directly use the best LoRA model.
questions: questions: Please refer to Weaknesses.
limitations: limitations: NA

Comments: title: Thank you for the response
Thank you for the rebuttal
Discussion
Thank you for the response
Thank you for the rebuttal
Thanks for rebuttal
comment: We thank the reviewer for the thoughtful response.
___

**W1 and W3.** Thank you for your understanding. 

**W2.** Thank you for the suggestion. We will add this part to the main paper.
Thanks for answering my questions. After going through all the other reviews and the given rebuttal, I've increased my score to 6.

W1. I understand the difficulty of the non-linearity analysis and thanks for the authors' explanation.

W2. The comparison between Wire-Fixed, Wire-Neigh, and C-LoRA is clear, and I suggest adding this part to the main text for better understanding.

W3. I'm sorry for the misunderstanding about C-LoRA and my concerns have been addressed.
Dear Reviewers,

This paper has received diverging reviews ranging from 4 to 7. The authors have submitted a rebuttal and the anonymous link to their code https://drive.google.com/file/d/1PNjqBgr-ZUBmarNK-yPZ7GWUdFjNf7t0/view?usp=sharing. Most reviewers have raised their scores after reviewing the authors' rebuttal except one.

@Reviewer R2vv: Please read the authors' rebuttal/code and other reviewers' comments. Have the authors' responses addressed your concerns? If so, please reconsider the borderline decision.

AC
We thank the reviewer for the thoughtful response and helpful suggestions.
___
**W1** 

We agree with that. We are working on the hyperparameter searching on CODA and will include it for comparison in the later version of the paper.

**W2** 

Thank you for the suggestions. We will include them in the main draft.

**W3** 

Thank you for your understanding.

**W4 and Time consumption** 

Thank you for the suggestions. We will include the discussions in the paper.
I thank the authors for taking the time to answer my question and improve my understanding of the manuscript.

**W1**

I believe it would make it a fairer comparison to include CODA as a SoTA prompt-learning based method after hyper-parameter, since it outperforms L2P on vision tasks, at least for CIL scenarios. I acknowledge that results might differ for the TIL scenario. I still appreciate the effort and I understand that the limited time for rebuttal is not necessarily sufficient to conduct an extensive hyper-parameter search.

**W2**

Thank you for the extra experiments, which are convincing. Including them in the main draft could also clarify the impact of the proposed loss on the model alignment.

**W3**

Thank you for the clarification. I agree that direct comparison to replay-based methods might be unfair.

**W4**

Thank you for the clarification. I would advise the authors to include such discussion in the related work.

**Time consumption**

Thank you for the additional information. Such details could be included in the paper or the appendix, as computation is a major focus focus in Continual Learning.

Overall, I do not have major concern and I will **increase my score to 7**.
We thank the reviewer for the thoughtful response and helpful reference. 
___
Yes, we agree that tuning fewer parameters may help to mitigate forgetting. In the paper Table 1, we observe popular PEFT methods (Adapter, LoRA, Prefix-Tuning) have overall less forgetting than pure fine-tuning (FT). 

In this paper, we provide an angle of interference that connects PEFT models’ superior performance to a global alignment effect. According to [1], token representations in different PEFT models can be viewed as combinations of the pre-trained token representations and some modification vectors. When the models have limited parameters, the modifications may also be limited (e.g., prefix tuning with a limited number of prompts). This makes the token representations strongly guided by the pre-trained token representations, which has the global alignment effect to reduce interference and mitigate forgetting.

However, if models have too few parameters, they may lose the plasticity to learn hard tasks and perform inferiorly in CL. Therefore, we also study ways to reduce forgetting when increasing model parameters/adaptations for plasticity. 
___
[1] He et al, Towards a Unified View of Parameter-Efficient Transfer Learning, ICLR 22
We thank the reviewer for the thoughtful response and detailed explanation. 
___
**W1.** Thank you for your understanding.
___
**Q1.a.** We agree that fixing the columns of classes not present in the batch can mitigate interference *in the classifier*. Under this setting, updating the columns of class vectors from new tasks will not influence the class vectors learned in the previous tasks (i.e., zero gradient for class vectors of previous tasks). As stated in the rebuttal Q2,  we have already used this technique in training all our models, including baselines. 

However, this technique can not fully mitigate interference *in the encoder*. Since the encoder is shared across all tasks, the value of new class vectors will influence the gradients on the encoder (Eq. 1), in both the magnitude and direction. Since the loss is usually non-zero at the beginning of task learning, the gradients of the encoder are unlikely to be zero*. This will cause interference based on the encoder’s gradients, on which we focus in this paper.

*If we have a good pre-trained model that can solve the target task by only tuning the classifier, then the gradients on the encoder are zero (i.e., the loss is zero) after probing. However, that is not always the case as shown in the probing results of Table 1.  
___
**Q1.c.**  Thanks for your suggestions. We show the Task-IL accuracies of different scaling factors $s$ for controlled-LoRA (C-LoRA) and Wire-Neigh in the tables below.  We test on the News-Series sequence, where the tasks are from nature language inference (NLI) datasets that have different data distributions from pre-training. The probing performance on News Series also has a large gap to MTL, which indicates that models need more plasticity to solve the tasks. 

| Model | s = 0 (Probing) | s = 0.1  |s = 0.4|s = 0.7|s = 1.0|
|-|-|-|-|-|-|
|  C-LoRA  |   74.81  |   74.83  |72.99|71.02|69.59|
|  C-LoRA + PF  |  74.81 |   78.59|77.41|76.83|76.81| 

* For C-Lora, when $s$ increases, the model's global alignment effect decreases while the plasticity increases. The results suggest: (1) C-LoRA’s CL performances decreases when $s > 0.1$. This may be because the encoder loses the global alignment effect, which also misleads the learning of the classifier and increases the interference. (2) After applying PF, C-LoRA’s accuracy first increases and then slightly decreases, overall consistently outperforms probing. This may be because PF reduces the interference caused by the class vectors, and the model can fully utilize its global alignment ability when increasing plasticity. However, when the scaling factor goes too large, the loss of alignment ability will lead to more forgetting even with PF.  

| Model | s = 0 (Wire-Fixed) | s = 0.1  |s = 0.3|s = 0.5|
|-|-|-|-|-|
|  Wire-Neigh  |   76.28|77.20 |75.19|72.46|

* For Wire-Neigh, we can improve its plasticity by expanding the neighborhood (general response 3), or increasing the scaling factor $s$ that represents the interpolation between the pre-trained token representations and their neighbor representations. The observation is similar to C-LoRA: when $s$ goes up, the model’s accuracy first increases and then decreases because of the trade-offs between global alignment and plasticity.  
___
**Q6.** Yes, we will mark the probing as the classifier-only learning baseline in the first column.
I thank the authors for answers my questions. My concerns are resolved and I've increased my score to a 6.

Regarding W3, I wonder if the reason that a shared $\Delta W$ shows low forgetting could also be due to that it does not contain a lot of parameters. This recent paper [1] shows that PEFT methods enjoy low forgetting when you don't have too many tunable parameters.

[1] Thede et al. Reflecting on the State of Rehearsal-free Continual Learning with Pretrained Models. CoLLAs 2024.
I thank the authors for their detailed answers.

W1. Yes, I am sorry about my error. I associate IDBR's results with L2P, and because IDBR uses memory, I understand that their comparison is not completely fair. 

For Q1.a, I was referring to the gradient in the model. I understand that the gradient in the classifier can be high, and it should be high to learn the downstream task. However, some techniques have been proposed to mitigate the classifier's forgetting. For example, fix the columns of those classes not present in the batch (to mitigate the interference at the gradient level).

Q1.c. Yes, this is exactly what I was referring to. If the task distribution is very different, the strategy must modify the model weights, increasing the scaling factor. I understand that this can increase forgetting, but it is interesting to understand how much the scaling factor can move to learn a very out-of-distribution downstream task but mitigate forgetting. This is the main challenge in CL when using pre-trained models.

Q.6. Thanks. I didn't see it. Would it be a good idea to add something in the first column?

Due to this, I will increase my score to a 5.