TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion
Authors: Preetha Vijayan
Prashant Shivaram Bhat
Bahram Zonooz
Elahe Arani
Venue: NeurIPS 2023 poster (2023)
Keywords: Continual Learning
Catastrophic Forgetting
Experience Replay
Lifelong Learning
Bio-Inspired
Active Forgetting
Scalable Neurogenesis
TL;DR: A novel approach to continual learning that leverages task-based neuron saliency to selectively retain and revise important information while actively promoting the learning of less active neurons through rewinding and relearning.

Abstract: Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks. Several techniques such as weight regularization, experience rehearsal, and parameter isolation have been proposed to alleviate CF. Despite their relative success, these research directions have predominantly remained orthogonal and suffer from several shortcomings, while missing out on the advantages of competing strategies. On the contrary, the brain continually learns, accommodates, and transfers knowledge across tasks by simultaneously leveraging several neurophysiological processes, including neurogenesis, active forgetting, neuromodulation, metaplasticity, experience rehearsal, and context-dependent gating, rarely resulting in CF. Inspired by how the brain exploits multiple mechanisms concurrently, we propose TriRE, a novel CL paradigm that encompasses retaining the most prominent neurons for each task, revising and solidifying the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Across CL settings, TriRE significantly reduces task interference and surpasses different CL approaches considered in isolation.

TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion
Prashant Bhat Bahram Zonooz Elahe Arani-t NNaInnfo Europe "Eindhoven University of Technology (TU/e) TTomTon preetha. vijayan@naainfo.euu (p.s. bhat, e. ararni, b. zon
Preetha Vijahan',Prashant
Abstract
Continual learning (CL) has remained persistent challenge for deep neural ner- works due to catastrophic forgetting (CF) of previously learned tasks. Several techniques such as weight regularization, experience rehearsal, and parameter isolation have been proposed alleviate CF. Despite their relative success, these re- search directions have predominantly remained orthogonal and suffer from several shortcomings, while missing out on the advantages of competing strategies. On the contrary, the brain continually learns, accommodates, and transfers knowledge across tasks by simultaneously leveraging several neurophysiological processes, including neurogenesis, active forgetting, neuromodulation, metaplasticity, experi- ence rehearsal, and context-dependent gating, rarely resulting CF. Inspired by how the brain exploits multiple mechanisms concurrently, propose TriRE, novel CL paradigm that encompasses retaining the most prominent neurons for each task, revising and solidifying the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Across CL settings, TriRE significantly reduces task interference and surpasses different CL approaches considered isolation
Introduction
Continual learning (CL) over a sequence of tasks remains an uphill task for deep neural networks (DNNs) due to catastrophic forgetting of older tasks, often resulting in rapid decline in performance and, the worst-case scenario, complete loss of previously learned information (38). Several
approaches, such parameter isolation [3 4. weight regularization [56] 45). and experience
rehearsal [41] 42 have been proposed in the literature address the problem of catastrophic forgetting DNNs. Despite their relative success, these research directions have predominantly
remained orthogonal and suffer from several shortcomings. Parameter isolation approaches suffer from capacity saturation and scalability issues in longer task sequences, while weight regularization approaches cannot discriminate classes from different tasks, thus failing miserably in scenarios such class-incremental learning (Class-IL) (88) In scenarios where buffer size is limited due to memory constraints (e.g., edge devices), rehearsal-based approaches are prone to overfitting on the buffered data [ As these research directions have rarely crossed paths, there is need for an integrated approach to leverage the advantages of competing methods to effectively mitigate catastrophic forgetting CL.
Catastrophic forgetting is a direct consequence of more general problem, namely the stability- plasticity dilemma 36]: the extent to which the CL model must be plastic to accommodate newly acquired knowledge and stable to not interfere with previously learned information (38] In stark
"quual contribution. 'Equal advisory advisoryy role.
"ooee is available at
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Merge S into 8
Beginning of task
Retain Stage
Revise Stage
Rewind Stage
End of task
Neuron in $ Neuron St
Neuron not in UUUSS S
Rewinded Neurons
Figure 1: TriRE consists of three-phase learning paradigm that reduces task interference and drastic weight changes by using task modularity. In the Retain stage, the method selects and preserves the most active neurons and weights in mask S,, which is used in the subsequent Revise stage finetune the joint distribution of current and past tasks along with a cumulative subnetwork mask S. The Rewind stage is responsible for reintroducing less active neurons to the learning process for future tasks by actively forgetting and relearning the non-sparse subnetwork.
contrast to DNNs, biological systems manage this dilemma better and are able to learn continually throughout their lifetime with minimal interference. CL the brain is administered by a rich set of neurophysiologccal processes that encompass different kinds of knowledge, and conscious processing that integrates them coherently [17]. Empirical studies suggest that metaplasticity [27] and experience replay play a prominent role in memory consolidation in the brain 440 15 In addition, neurogenesis in the brain is crucial for the growth and restructuring necessary to accommodate new skills (3). Neuromodulatory systems facilitate swift learning and adaptability in response contextual changes induced by new stimuli or shifts in motivation [33]. Whereas context-dependent gating (34) and active forgetting [19] improve the separation between the representations of patterns belonging to different tasks. By simultaneously leveraging these processes, the brain exploits task similarity and exhibits positive forward transfer, rarely resulting in catastrophic forgetting.
Inspired by the biological underpinnings of the CL mechanisms the brain, we propose 'REtain, REvise & REwind' (TriRE), novel CL paradigm mitigate catastrophic forgetting. Specifically, TriRE involves experience rehearsal, scalable neurogenesis, selective forgetting, and relearning to effectively mitigate catastrophic forgetting. Within each task, the proposed method consists of three stages: (i) Retain, where the most active neurons and their corresponding most important weights of the task extracted and retained avoid task interference and drastic weight changes, (ii) Revise, where the extracted network is finetuned to revise and solidify the current task as well as the joint distribution of the past tasks, (iii) Rewind, where the free neurons undergo active forgetting and relearning to promote the less active neurons back into the learning circuit for the next task as illustrated in Figure]] T TriRE effectively combines multiple mechanisms and leverages the advantages offered by different CL approaches.
We find that TriRE significantly reduces task interference and surpasses the aforementioned CL approaches across various CL. scenarios. Specifically, TriRE outperforms rehearsal-basee approaches in ToqgTTnlllmaeNee for Class-IL scenario by almost 14%, even under low-buffer regimes, by promoting generalization through weight and function space regularization. Experience rehearsal enables discrimination of classes belonging to different tasks in TriRE, resulting in least twice as good performance over weight regularization methods for the same dataset. Unlike parameter isolation approaches, TriRE scalable and produces at least 7% relative improvement compared parameter isolation approaches in CIFARI0R110 Task-IL setting without requiring access to task identity at inference time.
Related works
Rehearsal-based Approaches: Prior works attempted to address the problem of catastrophic forget- ting by explicitly storing and replaying previous task samples, akin to experience rehearsal in the brain. Experience rehearsal (ER) approaches [4] 42 maintain fixed capacity memory buffer to

---

store data sampled from previous task distributions. Several approaches are built on top of ER to better preserve the previous task information: GCR [49] proposed a coreset selection mechanism that approximates the gradients of the data seen so far to select and update the buffer. DER++ (9 and CLS-ER [6) enforce consistency in predictions using soft targets in addition to ground-truth labels. DRI 52 uses generative model to further support experience rehearsal in low buffer regimes. More recent works like TARC (8). ER-ACE (1I) and Co²L (12) focus on reducing representation drift right after task switch to mitigate forgetting through asymmetric update rules. Under low-buffer regimes and longer task sequences, however, these approaches suffer from overfitting on the buffered samples.
Weight Regularization Approaches:
Catastrophic forgetting mainly emanates from large weight changes in DNNs when learning a new task. Therefore, penalize the sudden changes to model parameters that are crucial for the previous tasks. Depending on the type of regularization, these approaches can he broadly categorized into prior- and data-focused
methods seek to
approaches. Prior-focused approaches, such as elastic weight consolidation (EWC) [25] online EWC
(oEWC) [45], memory-aware synapses (MAS) (5). and synaptic intelligence (SI) (56) employ prior information model parameters and estimate the importance of parameters associated with previous
tasks based either on the gradients of the learned function output or through Fisher's 's information matrix. On the other hand, data-focused methods, such as Learning without Forgetting (LwF) (30] instead perform knowledge distillation from models trained on previous tasks when learning on new data. Although weight regularization approaches do not require memory buffer and are scalable, they only impose soft penalty, thus failing to entirely prevent forgetting of previous task information.
Parameter Isolation Approaches: Parameter isolation has been predominantly done in two ways: either within fixed capacity or by growing in size. In the former, dynamic sparse methods such as PackNet (32) CLNP (16), PAE (23), and NISPA (18) make use of DNN's over peee-paramtterizatoon to learn multiple tasks within a fixed model capacity. Similar to the brain, these models simultaneously learn both connection strengths and a sparse architecture for each task, thereby isolating the task- specific parameters. However, these methods suffer from capacity saturation longer task sequences, limiting their ability to accommodate new tasks. In contrast, the latter methods, such as PNNs (39), Expert-gate (4) and DEN (55) expand in size, either naively or intelligently, to accommodate new tasks while minimizing forgetting. Although these approaches are extremely efficient in mitigating catastrophic forgetting, they do not scale well with longer task sequences, rendering them inapplicable in real-world scenarios.
Contrary to DNNs, the brain simultaneously exploits multiple neurophysiological processes, including neurogenesis (3] active forgetting (19), metaplasticity (27). experience rehearsal [40), and context- dependent gating 34 to continually acquire, assimilate, and transfer knowledge across tasks without catastrophic forgetting (26). Inspired by how the brain exploits multiple mechanisms concurrently, we propose a novel CL paradigm, TriRE, that leverages the advantages of multiple aforementioned
mechanisms to effectively mitigate catastrophic forgetting in CL.
3 Method
CL problems typically comprise t {1, 2, .., T) sequential tasks, with classes per task, and data that appear gradually over time. Each task has a task-specific data distribution associated with it (I,, 3.) E D,. We take into account two well-known CL scenarios, class- dlas-inccemmenal learning (Class-IL) and task- insk-incremmetal learning (Task-IL). Our working model consists a feature extractor network fo and single head classifier go that represents all classes of all tasks.
Sequential learning through DNNs has remained a challenging endeavor, since learning new in- formation tends to dramatically degrade performance previously learned tasks. As a result, to better retain information from past tasks, we maintain a memory buffer D.,. that contains data from tasks previously viewed. Considering the desiderata of CL., we assume that the model does not have infinite storage for previous experience and thus Dml < Dz]. To this end, we use loss-aware balanced reservoir sampling (10) to maintain the memory buffer. We update the working model, 20 = go (.), using experience rehearsal at each iteration by sampling a mini-batch from both D, and Dm as follows:
L
(C vill + A
E (.
IC (o
(D.D))DD,
3

---

where Lee is the cross-entrope loss, c(.) is the softmax function, L is the task-wise loss and Cer is the rehearsal-buit based loss. The objective in Eq. encourages plasticity through the supervisory signal from D, and increases stability through Dm However, as CL. training advances, model predictions earry more information per training sample than ground truths [7]. Hence, soft targets can be utilized in addition to ground-truth labels to better preserve the knowledge of the earlier tasks. Traditional methods to enforce consistency in predictions include using an exponential moving average (EMA) of the weights of the working model (6) or holding previous predictions in buffer (9]. As the former result in better knowledge consolidation and decision boundaries, we use EMA of the working model weights to ensure consistency in predictions:
Per
A
E
[ Do (1))))
(1..3)))
where DogMa is the EMA of the working model ty and || ||r is the Frobenius norm. We update the EMA model as follows:
a [pOEMA + (1- 0, TCc2((011) OEMA:
BEMA
(3)
otherwise
where μ is the decay parameter and S is the update rate. Finally, the EMA model acts as self- ensemble of models with distinct task specializations for inference rather than the working model.
Notations: Let St be the extracted subnetwork mask corresponding exclusively to the current task and S be the cumulative dynamic network mask corresponding to the tasks learned so far. At the end of the training, S would contain the most active neurons and the best corresponding weights across all tasks. In the following, we describe in detail various components of TriRE learning paradigm.
3.1 Retain
In CL. typically, models from previous tasks are seen as initialization and are "washed out" by new updates from the current task, which causes CF (35]. However, the brain uses context-dependent gating (26) to selectively filter neural information based on the context in which it is presented, allowing the development of specialized modules that can be added or removed from the network without disrupting previously Icarned skills (37, [50]. Inspired by this, the Retain phase induces modularity in the model by training hyper-network first and then extracting a subnetwork that is equivalently representational of the current task knowledge. This extracted subnetwork not only helps in creating task-wit wise specialized modules, but also helps the model preserve capacity for future tasks. Retention of this subnetwork is done using heterogeneous dropout of activations and weight pruning.
The Retain stage appears at the beginning of each task. At this stage, initially (/o 0 é S) is trained using a mini-batch of D. and (Jo 0 E S) trained using a mini-batch of Dm- This to ensure that the weights not in the cumulative network learn the new task to maintain plasticity, while the weights in the cumulative network learn a combination of old and new tasks to maintain stability. At the convergence of this training, we perform activation pruning followed by weight pruning to extract S, as shown in Figure? 2
This involves identifying and extracting neurons that contribute the most to the overall activation or output of the network. We monitor the frequency of neuron activations when a network is trained on a task. In essence, each neuron is given an activation counter that increases when a neuron's 's activation is among the top-k activations in its layer. We use these activation counts to extract the k-winner activations and retain them as the knowledge base for the current task. Heterogeneous dropout [ is used to map the activation counts of each neuron to a Bernoulli variable, indicating whether the said neuron is extracted or dropped. This, essentially, leaves the less activated neurons free to learn the next tasks.
Activation Pruning:
Weight Pruning: After retaining the most activated neurons for the task, we prune the less important connections corresponding to these neurons. In contrast to conventional methods, which only leverage weight magnitude or Fisher information for pruning, our method also takes into account the significance of weights with respect to data saved the rehearsal buffer. Continual Weight Importance (CWT) (53) criteria ensure that we maintain: (1) weights of greater magnitude for output stability, (2) weights significant for the current task for learning capacity, and (3) weights significant for past data to prevent catastrophic forgetting.
4

---

Dense
After Heterogeneous Dropout
After CWI based Weight Pruning
Extracted Subnetwork
Figure 2: Schematic representation of extraction of subnetwork at the end of Retain stage. The dense network first pruned using k- T- criteria, resulting a subnetwork the most activated neurons. This subnetwork is then pruned using CWI criteria, resulting in final extracted subnetwork, S..
For the working model, the CWI of weight 0 is defined follows,
CWI(Ø) |00| + all &Le
111 + 8Le (Dmi 0)
(4)
80
58
where and 3, coefficients that regulate the weight of current and buffered data, respectively. In addition, for denotes the single-head form of the cross-entropy loss, which only takes into account the classes relevant to the current task by masking out the logits of other classes.
At the end of this stage, we end up with mask of the most activated neurons and their most important weights for the current task, S..
3.2 Revise
Although context- context-dependent gating is a helpful phenomenon for CL in tackling CF, there are other biological phenomena such as neuromodulation, metaplasticity, and neurogenesis among others which also contribute to the brain's 's ability to combat CF [26]. Neuromodulation, for instance, is used decide whether a new feature is novel and unfamiliar (that is, creates new memory) or common and familiar (that is, consolidates into an existing memory). Taking the cue from that, Revise stage mainly focuses on revising and solidifying the knowledge that the extracted subnetwork of the current task, S, and the cumulative subnetwork of past tasks, S, currently possess. It is clear that S; is specialized on the current task and specialized on the past tasks. However, finetuning these two networks jointly can improve the knowledge overlap, and the features thus generated would be better approximation all the seen classes.
Firstly. {fe 0 & (S))))} is trained with a mini-batch of D, so that a part of the extracted network, S,, can be utilized to regain the performance for the current task. Subsequently, {.fe (8 8.)3 is trained with a mini-batch of D,, to preserve forward transfer and knowledge overlap between the past and the current tasks. The optimization learning rate is also considerably reduced this stage compared to the Retain stage to prevent drastic weight changes in subnetworks, which in turn decreases the forgetting. This could be seen as an adaptation of the metaplasticity in the brain (26) 20 which refers to the ability to adjust the amount of plasticity in the network based on the current organism. At end of this finetuning, the currently extracted subnetwork S, merged with the cumulative extracted mask from past tasks, S. The merging of the subnetworks can be seen as a process of integrating new neurons (S,) into the existing neural network (S), similar to the way neurogenesis allows for the integration of new neurons into existing neural circuits to accommodate new memories.
and future needs
3.3
Rewind
Evidence implies that the brain uses active forgetting as tool for learning due to its limited capacity Studies also suggest that although learning and memory become more difficult to access during the forgetting process, they still exist (54 48 There are still memory remnants in the brain that can be reactivated [3T]. In this work, we define the rewinding of weights an earlier state as
47
29
active forgetting. We also make sure that rather than reinitializing the model back to random very
5

---

Algorithm Proposed Approach TriRE
input: Data streams D,, working model 2 go go(fe())) EMA model PoEMA
g0gna (forma (.)), sparsity factor y, learning rates > nf, Retain epochs E,, Revise epochs Eg. Rewind epochs Eg.
1: S +-,.M + 0
2: for all tasks [1. do
3: for epochs E (1.2....Pm do
Di Retain
4:
for minibatch {(r; 9/ )=1= E Dr and ((rm Um Um) m= E M do
5:
Update {.fa $ S}.9e with n on { 9:) 1 using Eq. (C,)
6:
Update { $),90 with 7 on [Tim ym ) using Eqs. (Crr) and
7:
if <=== then
8:
Save the weights 0z
9:
Update θEMA using Eq.
10:
Extract new subnetwork S, with 7 sparsity based on CWI from Eq. for epochs e2
11:
F2 21. do for minibatch { (xx 31) i== E DD and ((rm Vm) } Jm= E . do
Revise
12:
13:
Finetune {fo $ (S S S)), 9o with r/ on f( 1
14:
Finetune (Jo E (.) go with n/' on {Trm Bm }) m=
15:
Update 0EMA
16:
Update cumulative set S SUS, Reinitialize non-cumulative weights {Se 0 $ 8} with 0s. for epochs ea 2.. do for minibatch {(T,, 344 31)) E D2 do
17:
18:
D. Rewind
19:
20:
Update { $ S), go with on {(Ti v:)
21:
Update AEMA
Update buffer M
return model 4s, model & ΦEMA
early weights, it is rewound to a point where the model has learned some features and has a generic perception of the objective closer convergence (but not absolute convergence). To aid this, the weights from a later epoch from the Retain phase are saved to be used in the Rewind stage.
Specifically, after the Retain and Revise steps, we rewind the weights belonging to non-cumulative subnetwork {fe 0 & S} back to the epoch & weights. Then the rewinded weights are finetuned for few epochs using mini-batch of D.. This is helpful because studies [46 13) show that in the human brain, less active neurons follow a 'use-it-or-lose-if philosophy. Therefore, forgetting and relearning act warm-up for these less active neurons making them relevant again for the learning circuit and making them more receptive to learning the next task.
In summary, TriRE above each task within a lifelong learning setting. Our method effectively combines multiple biological phenomena and harnesses the advantageous characteristics provided by popular CL. approaches. The step-by-step procedure is given in Algorithm
involves iteratively applying the three phases mentioned
Results
We expand the Mammoth CL repository in PyTorch [9] On the basis of Class-IL and Task-IL scenarios, assess the existing CL techniques against the proposed one. Although the training procedure for Class-IL and Task-IL is the same, during inference, Task-IL has access to the task-id. We consider number of rehearsal-basee based, weight regularization, and parameter-isolation approaches as useful baselines because TriRE necessitates experience rehearsal and model modularity. We use ResNet- 18 (21) the feature extractor for all of our investigations. In order to reduce catastrophic forgetting, we additionally offer a lower bound SGD without any support and upper bound Joint where the CL model is trained using the full dataset.
Experimental Setup:
Experimental Results: We compare TriRE with contemporary rehearsal- and weight regulariza- tion methods in Class IL and Task-IL settings, As shown in Table I TriRE consistently outperforms

---

Table I: Comparison of prior methods across various CL scenarios. We provide the average top-1 (%) accuracy all tasks after training. i Results of the single EMA model.
Buffer
Seq-CIFARIO
Seq-CIFARI00
Seq-TinylmageNer
Methods
size
Class-IL
Task-IL
Class-IL
Task-IL
Class-IL
Task-IL.
SGD Joint
19.6240005
61.02=3.33
17.49+0.28
40.46+0.99
07.92+0.26
18.31
12.20+0.15
98.31
70.56+0.28
19.044443
59.99
82.04
LwF
+9.61
63.29=2.35
18.47+0.14
26.45+0.22
8.46=0022
15.85=0.58
oEWC
19.49=0.12
68.29
7.58=0.10
19.20 19.20=0031
19.48.10.17
68.05 15.91
6.58.10.11
36.32 1..133
ER
1..79+1.86
91.19=0.94
20,40
61.36+0035
8.57=0.04
38. 17.20000000
DER+
64.88
91.92.10.60
29.60.1.14
62.49
10.96.1.17
40.8711.16
CLS-ER
61,88:2.43
93.59 0.887
43.38
72.01-1097
17.68
12.60
ER-ACE
6..08
92.20
35.17
63.09+11.3
11.25
44.17=1.02
200
Co²L
65.57+1377
93.43
31.90
55.02
13.88=00.0
42.37=0.74
GCR
6..84+1.63
90.8+1.05
33.69+1.40
64.24
13.05
42. 42.11
DRI
65.16.11.1
92.87
17.58:11.24
14.28
TriRE
68.17.1033
92.45.0.18
43.91 10.18
10.11
71.66:1044
20.14:10.19
55.95 10.78
Seq-CIFAR100 (20 Tasks)
100
Task-IL Accuracy
70
50
ThiRE (82.84%)
CLNP (66.68%)
PackNet (67.48%)
PAE (77.6%)
NISPA (75.87%)
PNN (77.33%)
4
5
7
8
11 Tasks
12
13
14
15
16
17
18
19
20
Figure 3: Comparison of TriRE against evolving architectures in terms of Task-IL accuracy Seq-CIFAR 100 dataset divided into 20 tasks. The graph reports the accuracy of individual tasks at the end of CL training.
rehearsal-basee methods across most datasets, highlighting the significance of dynamic masking in CL. Although methods like Co and ER-ACE excel simpler datasets, they struggle with more challenging ones. The same applies to methods like DRI and GCR, which augment memory buffers through core-set and generative replay. Their performance, for instance, lags behind TriRE in Seq-TinyImageNe where the buffer-to-class ratio low. Retaining task-wise dynamic subnetworks and revising extracted subnetworks to preserve past knowledge significantly reduces task interference in TriRE. In essence, TriRE boosts generalization through weight and function space regularization, selective forgetting, and releaming, yielding superior performance across tasks.
As evident from Table [ weight regularization methods such as LWF, oEWC, and SI perform miserably in both Class-IL and Task-IL settings across datasets. The reason being, these approaches encounter classes solely from the current task at any point CL training. Therefore, they fail to discriminate between classes from different classes resulting in subpar performance. On the other hand, TriRE leverages samples from previous classes through experience rehearsal to learn discriminatory features across tasks. In addition, TriRE entails forming subnetworks through Retain and Revise resulting in modularity and reduced task interference between tasks.
Parameter isolation approaches minimize task interference in CL by creating distinct sub-networkd either within given model capacity or by dynamically growing the network. Figure illuttrates comparison between methods trained on Seq-CIFAR CIFAR100 100 with 20 tasks i.c., it depicts final accuracies

---

C
SPwNE
Number of tasks
11
21
31
41
Subset of neurons in the last conv layer
F
11
21 neurons shortcut layer
Figure 4: (Top) depicts the distribution of neuron activations across tasks in the last convolutional layer of the feature extractor and (Bottom) depicts the same for the last shorteut layer. The black cubes represent the extracted ones after Retain stage.
70
50
30
Class-IL Accuracy
20
Seq CIFAR1O
Seq Seq-CIFAR100
Sin
10
0.7
1.0
0.4
0.7
Rewind Epoch Percentile
Figure 5: The effect of rewinding on Class-IL accuracy for all three datasets. The region from 70% to 90% of all epochs gives the best results consistently across datasets.
on 1s 2nd 20th task after training. Upon completing all tasks, TriRE achieves average accuracy of 80.85% surpassing the performance the baselines considered. TriRE leverages the benefits of experience rehearsal, weight regularization, and function space regularization to learn compact and overlapping subnetworks, resulting reduced task interference while maintaining scalability. In line with biological principles, TriRE incorporates selective forgetting and relearning mechanisms to activate less active neurons and enhance their receptiveness to learning subsequent tasks, thereby mitigating the risk of capacity saturation.
5
Model Analysis
Task Interference: Figure 4 shows the changes in neuronal activity for the subset of neurons in the last convolutional layer (top) and the last shortcut layer (bottom) of ResNet- trained on 5 tasks in Seq-CIFAR 100 dataset. The neurons that are blacked out are the most active neurons for each particular task after the Retain phase. For each task, observed that there are exclusive subnetworks forming on their own (horizontally) that capture task-specific information. However, with CL training, several neurons become generalizable by capturing information that can be shared across tasks. Therefore, there is neuron overlap between the extracted neurons across tasks (vertically). More in the shortcut layer, since the number of parameters these layers low. TriRE optimally manages the model capacity vs. task modularity trade-off better by re-using neurons that can be shared across tasks while maintaining the modularity of knowledge in each task intact.
How Much to Rewind? In Figure 5] we examine Class- IL accuracy when the model is rewound to different points in the Retain stage in order to comprehend how rewinding affects accuracy the model. We observe that the accuracy of the inference model decreases if the model forgets too much and is rewound to an early stage in the training. This alignment with the observations made by 14 that rewinding to extremely early stages is not recommended for DNNs because the network has

---

Table 2: Comparison of the contribution of each phase in TriRE. Note that the combination of Revise alone or Revise & Rewind has not been considered, as is not feasible without the Retain phase.
Seq CIa-CIIARTTO
Seq Tiny Imggeee
Retain
Revise
Rewind
Class-IL
Task-IL
Class-IL
Task-IL
X
38.01
66.23
11.54
40.22
J
x
33.08
60.03
8.44
31.90
V
43.03
72.09
16.25
48.89
43.91
71.66
20.14
55.95
Table 3: Relative number of learnable parameters and corresponding memory footprint in Seq- CIFAR100 with varying number task sequence.
Learnable Parameters (Million)
Memory Consumption (Million)
Methods
Tasks
10 Tasks
20 Tasks
Tasks
10 Tasks
20 Tasks
DER ++
1x
1x
1x
1x
1x
1x
EWC
1x
1x
1x
3x
3x
3x
TriRE
1x
1x
1x
6x
6x
6x
PNNs
27x
79x
240x
27x
79x
240x
not learned enough meaningful features by then to regain the lost accuracy. Additionally, we notice that accuracy also suffers when the model is rewound to point extremely close to the end of training time. Rewinding to a very late point in the training phase close to convergence is not idcal because there is not enough time for relearning. Our experiments indicate that rewinding to between 70% and 90% the training time results in the best accuracy.
As explained previously, TriRE employs three-stage learning paradigm to reduce task interference and improve weight reuse in CL. We seek to uncover how each Retain, Revise, and Rewind in TriRE influence Class-IL and Task-IL accuracies in Seq-CIFAR 100 and Seq-Tiny ImageNet datasets through Table 2 can be seen that although Retain alone can extract the subnetworks containing the most active neurons and decrease task interference, it falls short in aspects of forward transfer and weight reuse. Similarly, Retain and Revise together can solidify the knowledge extracted from current and past tasks, but such model suffers from capacity issues without the reactivation of less active neurons for future tasks. Likewise, Retain and Rewind together can encourage task-wise delimitation of knowledge and promote efficient usage of available networks, but lose out the forward transfer introduced by the learning of joint distributions. Finally, analogous to the brain, is evident that the harmony of all components is what achieves the best results in both datasets.
Ablation Study:
Memory and Computational Cost: We conduct comparative analysis the computational and memory overhead of TriRE in contrast to related works. Table provides provides analysis of the learnable parameters and memory required by TriRE in contrast to those of DER++, EWC, and PNNs, (Opting for method from each individual family of CL methods). Firstly, similar to DER+ and EWC, TRiRE does not add any learnable parameters to the model. However, is evident that PNNs have an infeasible amount of learnable parameters which gets progressively worse with longer task sequences. Secondly, the observed increase memory consumption in TriRE can be attributed to several factors: (1) the application of multiple masking mechanisms for parameter isolation; (2) the incorporation of the Rewind phase necessitating weight retention from a previous epoch; and (3) the utilization of the Exponential Moving Average (EMA) model to enhance knowledge consolidation. All of these factors hold memory but do not add any learnable parameter to the training.
Conclusion
We introduced TriRE, novel biologically inspired CL paradigm that entails experience rehearsal, scalable neurogenesis, and selective forgetting and relearning to effectively mitigate catastrophic forgetting in CL. Within each task, TriRE entails retaining the most prominent neurons for each task, revising the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Analogous to multiple neurophysiologice mechanisms in the brain, TriRE leverages the advantages of different CL approaches, thus significantly lowering task interference and surpassing different CL. approaches when considered isolation. For Seq-Tinyl TinyImageNe TriRE outperforms the closest rival in rehearsal-bas based basclines by 14%,

---

surpasses the best parameter isolation baseline by 7%, and nearly doubles the performance of the best weight regularization method. Extending our method to CL scenarios oblivious to task boundaries and to few- and zero-shot learning settings are some of the future research directions for this work.
7 Limitations and Future Work
We proposed TriRE, a novel paradigm that leverages multiple orthogonal CL approaches to effectively reduce catastrophic forgetting in CL. As orthogonal CL approaches may not always be complementary, the selection of such approaches needs careful consideration in TriRE. In addition, having multiple objective functions naturally expands the number of hyperparameters, thereby requiring more tuning to achieve optimal performance. Therefore, additional computational complexity and memory overhead due to the staged approach and extensive hyperparameter tuning some of the major limitations of the proposed method. For the same reason, we highlight that TriRE is not directed toward compute-intensive architectures such as vision transformers.
TriRE involves different stages of training within each task, requiring knowledge of task boundaries. In line with state-of-the- art methods in CL, each task entails non-overlapping set of classes, and data within each task shuffled to guarantee i.i.d. data. However, in the case of online learning where data streams and the distribution gradually shift, TriRE cannot be applied in its current form. Therefore, additional measures such as task book-boundry approximation and modification to learning objectives are necessary enable TriRE work in such scenarios. Furthermore, traditional CL datasets considered in this work entail independent tasks and data points without intrinsic cumulative structure. As TriRE does not leverage structures learned in previously encountered tasks, structure learning forms another limitation of this proposed method. Reducing computational and memory overhead, extending to task-free CL scenarios with recurring classes, and leveraging intrinsic structures within underlying data are some of the future research directions for this work.
8 Broader Impacts
Inspired by the multifaceted learning mechanisms of the brain, we propose TriRE, which replicates the brain's 's ability to leverage multiple mechanisms for CI., enhancing the generalization capabiliitess of CL. models across tasks. Its success not only encourages the exploration of neuro-inspired methods for deep neural networks, but also opens up opportunities to augment existing CL approaches by leveraging the advantages of competing strategies. By enabling models to learn continuously and adapt new tasks, TriRE contributes to the responsible and ethical deployment of AI technologies, as models can improve and update their knowledge without requiring extensive retraining. This advancement has significant implications for various real-world applications and promotes the development of Al systems that can continually improve and adapt their performance.
The work conducted while all the authors were affiliated with InvInfo Europe, Eindhoven, The Netherlands.
Acknowledgement:
References
[1] Ali Abbasi et al. "Sparsity and heterogeneous dropout for continual learning in the null space of neural activations". In: Conference Lifelong Learning Agents. PMLR. 2022, pp. 617-628.
[2] Wickliffe C Abraham and Anthony Robins. "Memory retention-the synaptic stability versus plasticity dilemma". In: Trends in neurosciences 28.2 (2005), pp. 73-78.
[3] James B Aimone, Janet Wiles, and Fred H Gage. "Potential role for adult neurogenesis in the encoding of time in new memories". In: Nature neuroscience 9.6 (2006), pp. 723-727.
[4] Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. "Expert gate: Lifelong learning with network of experts". In: Proceedings of the IEEE Conference Computer Vision and Pattern Recognition. 2017, 7, pp. 3366-3375.
[5] Rahaf Aljundi et al. "Memory aware synapses: Learning what (not) to forget". In: Proceedings of the European conference on computer vision (ECCV). 2018, pp. 139-154.
10

---

[6] Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. "Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System". In: International Conference on Learning Representations. 2022. URT.: http::ttts:://pppreeeeee http://///ppeeeeeeeeeee//tr
id=uxxFrDwrE7Y
[7] Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. "Consistency is the key to further mitigating catastrophic forgetting in continual learning". In: Conference on Lifelong Learning
Agents. PMLR. 2022, pp. 1192-1212.
[8] Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. "Task agnostic representation consolidation: a supervised based continual learning approach". In: Conference on Lifelong Learning Agents. PMLR. 2022, pp. 590-405.
[9] Pietro Buzzega et al. "Dark Experience for General Continual Learning: Strong, Simple Baseline". In: Advances in Neural Information Processing Systems. Vol. 33, Curran Associates, Inc., 2020. pp. 15920-15930.
[10] Pietro Buzzega et al. "Rethinking experience replay: bag of tricks for continual learning". In: 2020 25th International Conference on Pattern Recognition (ICPR). IEEE. 2021, pp. 2180- 2187.
[11] Lucas Caccia et al. "New Insights on Reducing Abrupt Representation Change in Online Continual Learning". In: International Conference Learning Representations, 2022.
[12] Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. "Co21: Contrastive continual learning". In: Pro- ceedings of the JEEF/CVF International conference on computer vision. 2021, pp. 9516- 9525.
[13] Diane Feldman and William C Gordon. "The alleviation of short-term retention decrements with reactivation". In: Learning and Motivation 10.2 (1979), pp. 198-210.
[14] Jonathan Frankle and Michael Carbin. "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks". In: International Conference on Learning Representations. 2018.
[15] Gabrielle Girardeau et al. "Selective suppression of hippocampal ripples impairs spatial memory". In: Nature neuroscience 12.10 (2009), pp. 1222-1223.
[16] Siavash Golkar, Micheal Kagan, and Kyunghyun Cho. "Continual Learning via Neural Prun- ing". In: Real Neurons & Hidden Units: Future directions at the intersection of neuroscience and artificial intelligence @ NeurlPS 2019. 2019.
[17] Anirudh Goyal and Yoshua Bengio. "Inductive biases for deep learning of higher-level cogni- tion". In: Proceedings of the Royal Society A 478. 476.2266 (2022), p. 20210068.
[18] Mustafa B Gurbuz and Constantine Dovrolis. "NISPA: Neuro-Inspired Stability-Plastichi Adaptation for Continual Learning in Sparse Networks". In: International Conference on Machine Learning. PMLR. 2022, pp. 8157-8174.
[19] Oliver Hardt, Karim Nader, and Lynn Nadel. "Decay happens: the role of active forgetting in memory". In: Trends in cognitive sciences 17.3 (2013), pp. 111-120.
[20] Demis Hassabis al. "Meta-learning and the neuroscience of learning and memory". In: Nature Reviews Neuroscience 18.5 (2017), Meta-Learning and the Neuroscience of Learning and Memory.
[21] Kaiming He al. "Deep residual learning for image recognition". In: Proceedings of the IEEE conference computer vision and pattern recognition. 2016, pp. 770-778.
[22] Saihui Hou et al. "Learning unified classifier incrementally via rebalancing". In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. 2019, pp. 831-839,
[23] Steven CY Hung et al. "Increasingly packing multiple facial-informatics modules in a unified deep-learning model via lifelong learning". In: Proceedings of the 2019 on International Conference on Multimedia Retrieval. 2019, pp. 339 -39-343..
[24] Diederik P Kingma and Jimmy Ba. "Adam: A method for stochastic optimization", In: arXiv preprint arXiv:1412.00 Nee::411269990 42.669800 (2014).
[25] James Kirkpatrick et al. "Overcoming catastrophic forgetting in neural networks". In: Proceed- ings of the national academy of sciences 114.13 (2017), pp. 3521-3526.
[26] Dhireesha Kudithipudi et "Biological underpinnings for lifelong learning machines".. In: Nature Machine Intelligence 4.3 (2022), pp. 196-210.
[27] Jesse J Langille and Richard E Brown. "TTe synaptic theory of memory: historical survey and reconciliation of recent opposition".. In: Frontiers in systems neuroscience 12 (2018), p. 52.
11

---

[28] Timothée Lesort, Andrei Stoian, and David Filliat. "Regularization shortcomings for continual learning". In: arXiv preprint SEKi::1922.00000 (2019).
[29] Penelope A Lewis and Simon J Durrant. "The role of active forgetting in learning and memory". In: Frontiers in psychology 9 (2018), p. 1314.
[30] Zhizhong Li and Derek Hoiem. "Learning without forgetting". In: IEEE transactions on pattern analysis and machine intelligence 40.12 (2017), pp. 2935-2947.
[31] He Liu et al. "Forgetting generates a novel state that is reactivatable". In: Science advances 8.6 (2022), eabi9071.
[32] Arun Mallya and Svetlana Lazebnik. "Packnet: Adding multiple tasks to a single network by iterative pruning". In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2018, pp. 7765-7773.
[33] Eve Marder and Vatsala Thirumalai. "Cellular, synaptic and network effects of neuromodula- tion". In: Neural Nenworks 15.4-6 (2002). pp. 479-493.
[34] Nicolas Masse, Gregory D Grant, and David Freedman. "Alleviating catastrophic forgetting using context-dependee gating and synaptic stabilization". In: Proceedings of the National Academy of Sciences 115.44 (2018), E10467-E1044
[35] Michael McCloskey and Neal J Cohen. "Catastrophic interference in connectionist networks: The sequential learning problem". In: Psychology of learning and motivation. Vol. 24. Elsevier, 1989, pp. 109-165.
[36] Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The stability-paastiiity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects. 2013.
[37] Vernon B Mountcastle. "The columnar organization of the neocortex". In: Brain 80.2 (1957),
pp. 214-241.
[38] German Parisi et al. "Continual lifelong learning with neural networks: A review". In: Neural Networks 113 (2019), pp. 54-71.
[39] Gyeong- Park, Sahng- Min Yoo, and Jong- Hwan Kim. "Convolutional neural network with developmental memory for continual learning". In: IEEE Transactions on Neural Net- works and Learning Systems 32.6 (2020), pp. 2691-2705.
[40] Björn Rasch and Jan Born. "Maintaining memories by reactivation". In: Current Opinion in Neurobiology 17.6 (2007). Motor systems / Neurobiology of behaviour, pp. 698-703, ISSN: 0959-4388.
[41] Roger Ratcliff. "Connectionist models of recognition memory: constraints imposed by learning and forgetting functions." In: Psychological review 97.2 (1990). p. 285.
[42] Anthony Robins. "Catastrophic forgetting, rehearsal and pseudorehearsal" In: Connection Science 7.2 (1995), pp. 123-146.
[43] Andrei A. Rusu et al. Progressive Neural Networks. 2016.
[44] Fahad Sarfraz, Elahe Arani, and Bahram Zonooz. "Synergy between synaptic consolidation and experience replay for general continual lcarning". In: Conference on Lifelong Learning Agents. PMLR. 2022, pp. 920-936.
[45] Jonathan Schwarz et al. "Progress & compress: A scalable framework for continual learning". In: International Conference on Machine Learning. PMLR. 2018, pp. 4528-4537.
[46] Tracey J Shors et al. "Use it or lose it: how neurogenesis keeps the brain fit for learning". In: Behavioural brain research 227.2 2 (2012), pp. 450-458.
[47] Norman E Spear. "Retrieval of memory in animals." " In: Psychological Review 80.3 (1973),
p. 163.
[48] Benjamin C Storm and Elizabeth L Bjork. "Active forgetting of episodic memories: Is it time to rethink the 'retrieval failure' theory of forgetting?" In: British Journal of Psychology 1 110.3 (2019), pp. 549-571.
[49] Rishabh Tiwari et al. "GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning" In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, pp. 99-108.
[50] Jane X Wang et al. "Neural modularity helps organisms evolve to learn new skills without forgetting old skills". In: PLoS computational biology 13.2 (2017), e1005354.
[51] Fu- Yu-Yun Wang et al. "Foster: Feature boosting and compression for class-incremental learning". In: European conference on computer vision. Springer. 2022, pp. 398-414.
12

---

[52] Zhen Wang et al. "Continual learning through retrieval and imagination". In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. 8. 2022, pp. 8594-8602.
[53] Zifeng Wang al. "SparCL: Sparse Continual Learning on the Edge". In: Advances in Neural Information Processing Systems.
[54] John T Wixted. "The psychology and neuroscience of forgetting". In: Annu. Rev. Psychol. 55 (2004), pp. 235-269.
[55] Jaehong Yoon et al. "Lifelong Learning with Dynamically Expandable Networks". In: Interna- tional Conference on Learning Representations.
[56] Friedemann Zenke, Ben Poole, and Surya Ganguli. "Continual learning through synaptic intelligence". In: International conference machine learning. PMLR. 2017, pp. 3987-3995. [57] Da- Wei Zhou et al. "A Model or 603 Exemplars: Towards Memory- Efficient Class- Jaass-nncremettll Learning... In: The Eleventh International Conference on Learning Representations. 2022.
13

---

A Effect of Pruning Criteria
We seek to illustrate the effectiveness of different pruning criteria in TriRE. As explained in Section 3.1 the dense network is first pruned using k- k-WTA criteria, resulting in a subnetwork of the most activated neurons, and then this subnetwork is pruned using CWI criteria, resulting in a final extracted subnetwork at the end of Retain stage. Table demonsstetsss the comparison of Class-IL accuracy between various pruning criteria, namely, magnitude-based, Fisher information-based and CWI- based, across all three datasets. The idea behind magnitude pruning is that small valued weights impact the network's 's output less and can be safely pruned without significantly affecting performance. Fisher information based pruning evaluates the importance of connections based on their contributions to the Fisher information matrix. Connections with low contributions, indicating less relevance or importance, are pruned set to zero. However, both these criteria calculate the importance of weights within the current task, but do not consider the possibility of it being crucial for other tasks. On the other hand, CWI considers the significance of weights with respect data saved in the rehearsal buffer as well, resulting in superior performance across all datasets.
Table 4: Comparison of the effect of various pruning criteria TriRE on different datasets.
Dataset
Magnitude
Fisher Information
CWI
SCQ-CIFARIO
65.09+0.83
64.40+004
68.17=0.33
Seq-CIFARI00
41.89+0074
40.26:+0.1
43.91=0.18
Seq-TinylmageNet
19.07+0.97
18. 16.075
20.14
B
Model Analysis
B.1
Task Recency Bias
In any CL. setting, the model entails learning on few samples from previous tasks while aplenty of the most recent task [22]. This tilts learning toward the most recent task, resulting in decisions biased toward new classes and confusion among the old classes. However, the CL model should ideally have predictions distributed evenly across all tasks with the least possible recency bias. Figure 6 provides the confusion matrix for various CL models to evaluate the task recency bias. After training on Seq-CIFARIOS 100 for tasks with buffer size of 200, the model is deemed to have correctly predicted the task label if it predicts any of the classes that make up the sample's 's true task label. As can be seen, ER and DER++ have propensity to frequently classify the majority of samples as classes in the most recent task. However, TriRE's predictions across the diagonal. TriRE essentially decreases interference between tasks, captures task-specific information through extracted sub-networks, and produces the least recency bias.
uniformly distributed
ER
DER+
TriRE
2000
Task Label
1500
3
1000
2
3
2
2
3
Predicted Task Label
Figure 6: Confusion matrix of different rehearsal-basee CL models. Unlike ER and DER++, TriRE predictions are evenly distributed across the tasks with the least recency bias.
B.2 Stability-Plasticity Trade-off
CL model said to stable it can retain previously learned information, and plastic it can effectively acquire new information. The mubility-plasticin dilemma refers to inherent trade-off
14

---

100
80
ER
ER (65.64)
Top-1 Accuracy (%)
DER++
DER++ (41.29)
60
TriRE
TriRE (27.86)
Accuracy
60
40
40
20
Stability
Plasticity
Tradeoff
0.0
0.2
0.4
0.6
0.8
1.0
Confidence
Figure 7: (Left) Stability-Plasticiyy Trade-off for CL models trained on Seq-CIFAR100 with 5 tasks. ER and DER++ are more plastic than stable leading to recency bias. TriRE maintains a better balance between stability and plasticity and achieves the highest trade-off amongst the baselines. (Right) Reliability diagram depicting model calibration: The red dashed line represents the ideal scenario. Compared the other methods, TriRE is better calibrated with the lowest ECE value. All models were trained on Seq-CIFARI00 with 5 tasks.
in which the CL model masters one of these aspects at the expense of the other. Sarfraz er al. [44 introduced trade-off measure that serves as an approximation of how the model balances its stability and plasticity. Once the model completes the final task T, its stability (S) is assessed by calculating the average performance across all preceding T tasks follows:
S
ATi
(5)
The plasticity of the model (P) is evaluated by computing the average performance of each task after its initial learning i.e.,
P
A
(6)
Thus, the trade-off measure determines the optimal balance between the stability (S) and the plasticity (P) of the model. This measure is calculated as the harmonic mean of S and P.
2SP
Trade-oft off S P
Figure 7 (Left) provides the stability-llastictty plasticity trade-off measure for different CL methods across different datasets for buffer size 200. ER and DER++ exhibit high plasticity, enabling them to rapidly adapt to information. However, they lack the ability to effectively retain previously acquired knowledge. On the other hand, TriRE exhibits substantially high stability with low plasticity, resulting in higher sability-pllasiiile trade-off.
B.3 Model Calibration
Ensuring the reliability of safety-critical CL systems necessitates the presence well-calibrated model. Calibration refers to the task of accurately predicting probability estimates that reflect the true likelihood of correctness. Miscalibration, on the other hand, refers to the disparity between confidence and accuracy expectations. To assess the degree of miscalibration in classification, the Expected Calibration Error (ECE) involves partitioning the predictions into bins of equal size and calculating the difference between the weighted average of accuracy and confidence within each bin. lower ECE value indicates better calibration in the underlying models.
Figure 7 (Right) shows comparison of different CL approaches using calibration framework trained on Seq-CIFAR 100 with a buffer size of 200. Well-calibrated CL systems accurately represent true likelihood of accuracy (indicated by the red dashed line). Among the baselines, TriRE achieves the lowest ECE value and exhibits high calibration, demonstrating effectiveness in minimizing task interference and reducing overconfidence in CL, thus enabling more informed decision making.
15

---

C Hyperparameter Selection
The hyperparameters required to replicate the results of TriRE can be found in Table 5 These hyperparameters were determined through tuning process involving different random initializations and small portion of the training set reserved for validation. All experiments were conducted using a batch size of 32 and trained for 50 epochs. TriRE was optimized using the Adam optimizer (24) implemented in PyTorch. Furthermore, the number of epochs allocated to each phase specified in Algorithm was consistently set a ratio of Ej E2 Eg 1:
Table 5: Best hyperparameters of TriRE chosen for optimal performance on different datasets.
EMA Parameters
Rewind Percentile
Dataset
"
r/
7
À
Seq- CIO-CIFARTO
0.0006
0.0001
0.4
0.06
0.999
0.18
0.9
Seq-CIFAR 100
0.002
0.0001
0.2
0.04
0.999
0.12
0.9
Seq-TinylmageNet
0.002
0.0001
0.3
0.05
0.999
0.01
0.8
C.1 Hyperparameters Sensitivity Analysis
In order showcase the robustness of TriRE to choice of values for each hyperparameter, conducted additional experiments by finetuning different hyperparameters. Specifically, we evaluated the performance of TriRE in the Seq-CIFAR 100 dataset with buffer size of 200 and 5 tasks. The results are visualized in the Figure [ Our experimentation focused on adjusting hyperparameters such sparsity, k-winner sparsity, and EMA model update frequency. Upon examining the graphs, is evident that performance remains relatively stable across range of values for these hyperpa- rameters. This observation suggests that satisfactory results can be achieved without exhaustive hyperparameter search.
Accuracy (%)
35
0.3
0.5 Sparsity
0.7
0.9
Accuracy (96)
45
40
0.1
0.3
0.5 K-winner Sparsity
0.7
0.9
Accuracy (%)
45
35
0.1
0.3
0.5
0.7
0.9
Stable model update frequency
Figure 8: Evaluation of performance across different values of Sparsity, K-Winner Sparsity, and Stable model update frequency in Seq-CIFAR 100 with buffer size 200. As can be seen, the performance does not drop significantly for different choices of values for hyperparameters.
16

---

D TriRE vs. CLS-ER: Commonalities and Differences
CLS-ER emulates the interplay between fast and slow learning systems by incorporating two supplementary semantic memories that aggregate the weights the working model in a stochastic manner via exponential moving average. That is, CLS-ER operates with three models: the working model, the stable model, and the plastic model. Conversely, in the TriRE framework, the architecture consists of two models: the working model and a stable model (referred Therefore, the similarity is that both methods use the concept of progressively aggregating the weights of the working memory as sequentially learns tasks allowing us consolidate the information efficiently.
the EMA model).
However, there are two main differences. Firstly, CLS-ER uses two supplementary semantic memo- ries, whereas TriRE only utilizes one supplementary memory. Nevertheless, our results outperform CLS-ER (with one EMA model, referred to Mean-ER in the mentioned paper) in all three datasets, as shown in Table T Secondly, CLS-ER only uses experience replay to tackle catastrophic forgetting whereas method harmoniously integrates all the families of methods in CL., i.e. weight regular- ization, parameter isolation, and experience replay. That is, CLSSER only focuses on one aspect of biological learning whereas effectively consider multiple aspects.
E
Comparison with Recent Baselines
Table 6 provides a comparison between Foster 5T]. Memo [57]. and TriRE in Class-IL. The experiments are conducted on Seq- Seq-CIFAR100 with buffer size 200 with 5 tasks and random seeds. We caution that the implementation details are slightly different: TriRE uses reservoir sampling with ResNet-18 as backbone while both Foster and Memo employ modified ResNet- with custom sampling methods. Foster entails dynamic expansion and compression to accommodate new information. Similarly, Memo entails expansion in the later layers while preserving generic information in the earlier layers. On the other hand, TriRE promotes generalization through combination weight and function space regularization, selective forgetting, and relearning thereby producing superior performance across tasks. As can be seen, TriRE outperforms both Foster and Memo.
Table 6: Comparison with Foster and Memo on Seq-CIFARE 100 with 5 tasks and buffer size 200.
Method
Foster
Memo
TriEE
Top-I Accuracy (%)
40.48 +0.53
43.57
10.44
43.91
As the CL. training progresses, both Foster and Memo are constrained to accommodate new informa- tion in limited number of parameters resulting in lower performance in later tasks. However, TriRE suffers from no such limitation resulting in superior performance (see Figure
Foster
Memo
TriRE
After Task
74.0
70.0
34.5
After Task
74.6
57.9
70.0
46.6
60.1
46.1
After Task 3
74.8
57.9
31.5
70.0
46.6
37.7
44.3
49.5
56.1
After Task
74.8
57.9
31.5
23.3
70.0
46.6
37.7
33.1
33.6
42.1
50.0
57.5
After Task 5-
74.8
57.9
31.5
23.3
18.5
70.0
46.6
37.7
33.1
28.8
33.1
39.6
38.8
54.5
53.6
T1
T2
T3
T4
T5
T3
T4
T5
T1
T2
T4
T5
Figure 9: Task-wise performance Seq-CIFARI00 with 5 tasks and buffer size 200. The models are assessed after completing each task (y-axis) to gauge how the progress of training impacts task performance (x-axis).
17

---

F
Training Cost
We compare the training times of various CL models that were taken into account for this work. We consider one method from each family of CL models. Table 7 specifically lists the training times required to learn a single task (the first) for 3 epochs on an NVIDIA RTX 2080 Ti for Seq- 100 dataset with buffer size of 200. We conducted the experiment for 3 epochs across all CL methods considering the three-phased approach in TriRE and to maintain the fairness of comparison.
Table 7:
Training time comparison across various CL methods for three epochs on an NVIDIA RTX 2080 TÝ for Seq-CIFAR10N dataset.
Phases of TriE
Method Training Time (sec)
SI
DER++
PNNs
TriRE
Retain
Revise
Rewind
32
72
25
75
~33
30
12
Table 7 shows that PNNs have the least training cost for our experimental setup. However, it is to be noted that in the case of PNNs, the number of tasks increases, the model size also increases. This indicates that the training time also eventually increases towards the later tasks. Furthermore, we see that SI also costs less training time-wise. However, according to Table the Class-IL accuracies of weight regularization methods mentioned are significantly lower for larger datasets. Interestingly, DER+ and TriRE exhibit similar training costs, yet our approach proves to be more effective in mitigating catastrophic forgetting and task interference. This can be attributed to our method's unique amalgamation of various CL method families, making it more robust choice for CL tasks.
We also compared the training costs for each stage within TriRE's learning paradigm. We found that 'Retain' incurs the highest cost among the three, as it involves activation and weight pruning. The 'Revise' stage, where the joint distribution of past tasks and the current task is learned, follows. Finally, 'Rewinding' to a saved weight and learning for few epochs to activate less active neurons requires significantly less training time compared to the other two stages.
G Datasets and Settings
We assess the effectiveness of our approach in two different types of CL scenarios: Class Incremental Learning (Class-IL) and Task Incremental Learning (Task-IL). In Task-IL and Class-IL, each task consists of predetermined number of new classes that the model needs to learn. A CL model learns multiple tasks in sequence while being able to differentiate between all classes it has encountered so far. -I. similar to Class-Il., but it has the advantage of having access to task labels during the inference process, making it one of the easiest scenarios.
To evaluate the performance of our method in Task-IL and Class-IL scenarios, we employ three different datasets: Seq-CIFAR10, Seq-CIFARION 100, and Seq-TinylmageNet. These datasets are derived from CIFARIO, CIFAR 100, and TinylmageNet, respectively. In CIFARO,, CIFARIO is divided into five tasks, each task containing two classes. Similarly, in Seq-CIFARIO CIFAR 100 is divided into five tasks, each consisting of 20 classes. Lastly, in Seq- Tinylmagee,,,, we partition Tiny Tit into ten tasks, each of which comprises 20 classes. These datasets are designed to introduce more challenging scenarios for comprehensive analysis of various CL methods. By increasing the number of tasks or the number of classes per task, we can thoroughly examine the effectiveness of different CL approaches in handling different levels of complexity. Following (6). we used ResNet- 18 as the backbone in all our experiments. The training process remains consistent for both CI and TLs To compare various state-of-the-arr approaches, we present the average accuracy across all tasks encountered in CIass-IL. According to Task-IL conventions, we take advantage of task identity and selectively deactivate neurons in the linear classifier that are not related to the current task.
Contrary to the common practice of using dense CL models, dynamic sparse methods take a different approach by starting with a sparse network and maintaining the same level of connection density throughout the learning procedure to incorporate sparsity into a CL model: it is necessary to disentan- gle interfering units to prevent forgetting and establish new pathways to encode new knowledge. This presents challenges when implementing batch normalization and residual connections for both the NISPA and CLNP methods. Consequently, these methods do not employ the ResNet-18 18 architecture. Instead, they opt for simpler CNN architecture without 'skip connections' and batch normalization.
18


Reviews:
[{"review_id": "4K8vHkwx6L", "reviewer": "Reviewer_tURo", "summary": "This article introduces a multi-faceted approach to continual learning. combining features of many other approaches and relying on a three-phase training process, such that, as each new task in a sequence of tasks is encountered, subsets of weights in the network are 'retained', 'revised' or 'rewound' to values that maintain old knowledge while beginning to learn the new task, facilitate integration of weights important for the new task with those important for previous tasks, and maintain plasticity for future learning, respectively.  The authors find advantages for this approach relative to a large range of other approaches, especially with more complex task settings, and perform several ablations helping to clarify the roles of the three stages, along with a few other explorations.", "questions": "I had difficulty understanding the CLS-ER results, which are almost as good as TriRE.  A clearer understanding of what the CLS-ER model shares with TriRE and how it differs from it would be useful in evaluating the contributions of TriRE over and about the use of an EMA model smoothing weights across tasks.  The note in Table 1 is insufficient for me to understand the relationship between CLS-ER and TriRE.", "limitations": "I initially wrote: \"The paper mentions limitations in the appendix.  As stated they hint at some of the same limitations I see in the work as described above.  Per the importance placed on limitations in instructions to authors and reviewers, these ought to be placed in the main text.\"\n\nThe authors have addressed this concern through the discussion during the rebuttal, leading me to increase my rating to 'Accept'.  While I think the weaknesses described above still apply, they are, as I have said, largely shared by a mini-paradigm in which these issues have been addressed.  I hope the authors will be encouraged by acceptance of this work to work on a new paradigm.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper cites a wide range of relevant related work and situates its approach within the context established by other approaches.  The other approaches considered seem fairly extensively sampled.  I appreciated the consideration of the effects of the different ablations which show the importance of the 'rewind' phase as well as the stability plasticity analysis shown in the appendix.", "weaknesses": "The work seems thoughtful in relation to the relevant existing literature but it may be that the CL paradigm as explored in this project is in need of re-framing, if new breakthroughs are to be achieved.  While the advantages relative to many of the baselines seem clear, I felt that the additional complexity of the TriRE scheme made the gaps between it and some of the other approaches relatively difficult to get excited about.  Some of the features that seem relatively important, such as CWI, are direct importations from previous work, making it difficult to assess whether TriRE really advances our understanding, given its additional complexity compared to most other work.\n\nTo me the greatest weakness of the approach is one that appears to be widely shared across the CL literature:  This is the fact that this and all of the cited work rely on triggering complex meta-processes at the boundaries between tasks in ways that seem very distant from what might occur in biological networks on in naturalistic continual learning settings where task boundaries are not announced.  \n\nWe need approaches that can address the continual learning problem when tasks shift more gradually and task boundaries are not available.  The approach also exploits features limited to the setting in which tasks are defined by the fact that they use completely non-overlapping sets of output neurons (i.e. distinct class labels).  For example the Loss on the new task items in Eq 4 only considers the class labels relevant to the current task.  Finally, continual learning as explored in this literature is restricted to tasks with no intrinsically cumulative structure.  Catastrophic forgetting also occurs in settings where new learning could productively build on structure learned in previously learned tasks (in the way that addition builds on counting, multiplication builds on addition, etc).  The whole body of work thus seems narrowly focused on paradigms of limited general interest.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer for providing thoughtful feedback and sharing a constructive evaluation of our work. Your valuable input has greatly contributed to the enhancement of our paper.\n\n`complexity of the TriRE scheme made the gaps between it and some of the other approaches relatively difficult to get excited about. `\n\nWe agree with the implication that TriRE has additional complexity when compared to the other mentioned baseline methods. But the exciting element in what we are proposing is that there is a very plausible research direction which was under explored and has shown promise. Although the work before ours has pushed the boundary in CL domain, they have missed out on the benefits of effectively integrating several neuro-physiological processes emulating how biological learning works. We manage to coherently combine the concepts of meta-plasticity, active forgetting, relearning, neurogenesis and context gating to form a new CL paradigm and empirically prove that it is a viable direction with results.\n\n`the greatest weakness of the approach is one that appears to be widely shared across the CL literature:`\n\nWe appreciate the reviewer for highlighting a common gap observed in both our proposed work and the related literature. Your discerning analysis has emphasized an area of significance that merits careful consideration and refinement. That being said, we would like to indulge in the argument that CL as a topic is in its early phase and what you suggested : task boundary less training, rehearsal free training; are all part of the desiderata of CL [1]. Therefore, as the domain is progressing, we believe that there is a lot of scope for improvement. For instance, neuro-symbolic continual learning [2] is a subtopic within CL that broadly deals with the reviewer’s suggestion; the concept of new learning productively building on structure learned in previously learned tasks. \n\nHowever, in this work, we would reiterate that our focus was on demonstrating the efficacy of harmoniously combining the existing CL methods to tackle catastrophic forgetting better and it achieves the same. Ours is one of the earliest works in this direction and we hope that it spurs the CL research community to explore more methods which combine neuro-physiological processes as in the biological brain.\n\n[1]\tFarquhar, Sebastian, and Yarin Gal. \"Towards robust evaluations of continual learning.\" arXiv preprint arXiv:1805.09733 (2018).\n\n[2]\tMarconato, Emanuele, Gianpaolo Bontempo, Elisa Ficarra, Simone Calderara, Andrea Passerini, and Stefano Teso. \"Neuro symbolic continual learning: Knowledge, reasoning shortcuts and concept rehearsal.\" arXiv preprint arXiv:2302.01242 (2023).\n\n`A clearer understanding of what the CLS-ER model shares with TriRE and how it differs from it would be useful`\n\nCLS-ER emulates the interplay between fast and slow learning systems by incorporating two supplementary semantic memories that aggregate the weights of the working model in a stochastic manner via an exponential moving average. That is, CLS-ER operates with three models: the working model, the stable model, and the plastic model. Conversely, in the TriRE framework, the architecture consists of two models: the working model and a stable model (referred to as the EMA model). Therefore, the similarity is that both the methods use the concept of progressively aggregating the weights of the working memory as it sequentially learns tasks allowing us to consolidate the information efﬁciently. \n\nHowever, there are two main differences:\n\n1. CLS-ER uses two supplementary semantic memories whereas TriRE only uses one supplementary memory and still our results are better than CLS-ER in all three datasets as shown in Table 1 of the main paper.\n\n2. CLS-ER only uses experience replay to tackle catastrophic forgetting whereas our method harmoniously integrates all the families of methods in CL, i.e weight regularization, parameter isolation and experience replay. That is, CLS-ER only focuses on one aspect of biological learning whereas we effectively consider multiple aspects. \n\nWe once again thank the reviewer for detailed and insightful feedback. Please let us know in case we have missed any open points.\n"}, {"review_id": "WqjYtAaEQD", "reviewer": "Reviewer_SoXi", "summary": "The paper proposes a method for avoiding catastrophic forgetting in continual supervised learning that operates in three stages. In the first stage (retain), a subnetwork for the current task is identified by detecting most/least activated neurons of the main network. In the second stage (revise), both the main network and the extracted subnetwork are re-trained / fine-tuned with examples from the current task and examples from a replay memory. The subnetwork is later integrated into the main network. In the final stage (rewind), the weights belonging to non-cumulative subnetworks are finetuned for a few epochs. The paper presents experimental results on three benchmark continual learning datasets, in both the task-incremental and class-incremental settings, by also comparing with counterpart methods in the area.\n\n--Rebuttal--\nI read the rebuttal, along with other reviewers' comments, and increased my score accordingly (to borderline accept) during the rebuttal phase.", "questions": "Please refer to questions listed in the \"weaknesses\" section.", "limitations": "Limitations in terms of the network architecture used are clearly stated in the paper.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The originality of the paper relies on a new multi-stage method to tackle catastrophic forgetting which relies on the important concepts of modularity and example replay. \n\n- The paper is in general easy to follow. \n\n- The paper is properly aligned with literature in the area.", "weaknesses": "- Although the proposed approach touches on a lot of important points in catastrophic forgetting and continual learning, it seems to be a mere combination of existing ideas into a so-called three-stage approach, therefore undermining the novelty of the proposed mechanism. Furthermore, what seem to be novel aspects of the proposed approach are not explained in full detail, and therefore it is difficult to estimate their impact. For instance, in the retain phase, activation pruning is performed by using the existing heterogeneous dropout, while weight pruning used the existing CWI approach. For activation pruning, in lines 158-159, it is stated that a counter is used to determine the top k-winner activations. However, not technical details are provided regarding how this counter works. In the revise stage, lines 191-193 mention that the learning rate is \"considerably\" reduced at this phase; however, no insights onto how to decide what the reduction rate to use. For this same stage, in lines 195-199 it is mentioned that the S and S_{t} subnetworks are eventually merged; however, no technical details of how this merge occurs are provided. Finally for the rewind stage, in lines 209-214, it is unclear how to decide for how many epochs (k?) to rewind the network, what are the criteria, the requirements to decide this. \n\n- The experiments lack analysis of computational cost (e.g. memory consumption, training time). As can be inferred from Algorithm 1 and section 3, the multi-stage procedure proposed in this paper involves several calculations and passes through the network/subnetwork. What is the cost of this multi-stage procedure? How does this compare to counterpart methods? \n\n- In the experiments, some results are left unexplained while some others have an ambiguous explanation. For example, why does the proposed method seem to underperform on Task-IL for two of the three datasets evaluated? The explanation for the \"miserably\" (please change this word) performance of methods such as LwF and SI provided in lines 240-246 looks ambiguous and incorrect, since these methods have been used previously in Task-IL. \n\n- From the results in Figure 3, it can be implied that the proposed method actually improves performance as the number of tasks increases. Is that the case? Did you run multiple task orders? Are tasks 10 to 20 naturally easier?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Firstly, we would like to express our gratitude for the time and attention you dedicated to reviewing our paper. Below, we have carefully addressed each of your concerns to the best of our knowledge to enhance the paper's overall contribution.\n\n`no technical details are provided regarding how the activation counter works.`\n\nFor a given task, each neuron in a layer is monitored for its frequency of activations during training. In essence, each neuron is given an activation counter that increases when a neuron’s activation is among the top-k activations in its layer. In the Retain phase, the likelihood that a neuron would be dropped is inversely proportional to its activation counts. As a result, the model is not only encouraged to preserve the knowledge of the current task but also learn the new task utilizing neurons that were less active during earlier tasks. At the end of each task, we reset this counter to make sure that we track the neuron activity of every task separately.\n\n`the learning rate is \"considerably\" reduced at Revise phase; however, no insights into how to decide what reduction rate to use.`\n\nWe understand that the indication to decrease the learning rate “considerably” in the Revise stage could be misleading for the reader and appreciate you pointing it out. Our notion from the suggestion was that depending on the optimizer used, model architecture and the complexity of the dataset involved, slowing down the learning in the Revision stage can help us by not drastically overwriting the existing weights while simultaneously reaping the benefits of joint distribution based training. We acknowledge that this is a hyperparameter that needs tuning but for the datasets mentioned in the paper, we have provided the learning rates that gave us the best results in the Appendix which could be a good starting point for anyone in the community who wants to develop on this idea. Also as a thumb of rule, from our experiments, we found that $\\eta'$ should be approximately 1/20th of the initial learning rate $\\eta$ for smaller datasets and 1/10th for larger datasets. \n\n`mentioned that the S and S_t subnetworks are eventually merged; however, no technical details of how this merge occurs are provided.`\n\nBy merging we mean updating the cumulative set S = S $\\cup$ S_t at the end of the revise step. Specifically, the new cumulative mask includes those weights that were already part of the cumulative mask and the new set of weights that are identified to be most active by S_t. This information is part of Algorithm 1, line 16. In any case, we will update Figure 1 and provide more clarity in the next revision. \n\n`it is unclear how to decide for how many epochs (k?) to rewind the network`\n\nThis has been addressed in Section 5 under “How much to Rewind?”. As explained in the paper, rewinding to very early and very late stages in the training tends to decrease the accuracy. The former does not work because the network has not learned enough meaningful features by then to regain the lost accuracy and the latter does not work because there is not enough time for relearning. The empirical analysis as shown in Figure 5 proves that rewinding to between 70% and 90% of the training time in the Retain phase results in the best accuracy. \n\n`explanation for performance of methods such as LwF and SI provided in lines 240-246 looks ambiguous and incorrect`\n\nWe regret that the explanation for the unsatisfactory performance of LwF and SI when compared to our work seems ambiguous. However, we do want to point out that our method does considerably better (Refer Table 1) than weight regularization methods like LwF and SI and we were simply trying to bring out the difference in the accuracy in both scenarios. There is a relative increase of ~40% between LwF and TriRE in terms of Task-IL accuracy in Seq-CIFAR10. Furthermore, TriRE’s Task-IL accuracy shows a relative increase of ~200% in Seq-TinyImageNet which is a harder dataset considering its low buffer-to-class ratio. This proves that our method which uses a combination of weight regularization, experience replay and parameter isolation is better than using weight regularization alone. However, under the reviewer’s suggestion, we would be removing the word “miserably” as it’s an unfairly harsh description and would be adding more explanation.\n\n`implied that the proposed method actually improves performance as the number of tasks increases. Is that the case? Did you run multiple task orders? Are tasks 10 to 20 naturally easier?`\n\nWe would like to clarify that Figure 3 provides results on all 20 tasks after training on Seq-CIFAR100 with 20 task training i.e. Figure 3 depicts final accuracies on 1st, 2nd.. 20th task after training. Therefore, it cannot be inferred that moving from 10 tasks to 20 tasks is naturally easier. On the contrary,  catastrophic forgetting worsens as the number of tasks in a sequence increases, referred to as long-term catastrophic forgetting. The number of samples in the buffer representing each previous task drastically reduces in longer task sequences resulting in poor performance. With the Revise phase forcing joint distribution based training thus preserving forward transfer and with the Rewind phase forcing less active neurons to participate in the task learning, TriRE combats task interference better. We employ the same task order as the compared baselines to maintain fairness in comparison. Therefore, Figure 3 contains a single task order averaged over multiple random seeds. \n\nWe once again thank the reviewer for detailed feedback. We have made an utmost effort to address all the concerns raised. Please let us know in case we have missed something."}, {"review_id": "KzWFFeKZuY", "reviewer": "Reviewer_6DDf", "summary": "The paper proposed a new continual learning (CL) method that updates partial neurons while rewinding other neurons to the previously stored weights. To do this, the authors utilize sparsity constraints to the network weights and select highly activated neurons. Consequently, the proposed method outperforms the baselines.\n", "questions": "See Weakness", "limitations": "See Weakness", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The proposed method is generally technically sound.\n2. The ablation study in Table 2 helps us understand the contribution of each component.\n", "weaknesses": "1. The proposed method has three components, retain, revise, and rewind. The key contribution part is rewind while other parts are periphery or have no novelty.\n\n2. What is the overhead cost for rewinding, such as training time,  memory cost for holding previous weights, and the performance drop for the current task?\n\n3. There are 7 different parameters and it seems that it is arbitrarily chosen to make the best performance for each task (Appendix D) as the authors mentioned in Appendix C, which is appreciated.\n\n4. The statement in L211-212, “This is helpful because studies show that in the human brain, less active neurons follow a ‘use-it-or-lose-it’ philosophy” is difficult to understand why it is “helpful.” It seems that the authors blindly think that “human-like” is good.\n\n5. It will be better if the authors include recent baselines [1-2] and check whether the proposed method is state-of-the-art in the current setting.\n\n[1] Fu-Yun, et al. \"Foster: Feature boosting and compression for class-incremental learning.\" ECCV. 2022.\n\n[2] Zhou, Da-Wei, et al. \"A model or 603 exemplars: Towards memory-efficient class-incremental learning.\" ICLR. 2023\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere gratitude for your thorough review of our paper. Below, we have addressed each of your concerns to the best of our understanding, aiming to enhance the paper's overall contribution.\n\n`What is the overhead cost for rewinding, such as training time, memory cost for holding previous weights, and the performance drop for the current task? `\n\nWe appreciate the reviewer's insightful observation of the absence of computation cost analysis. Furthermore, we concur with the interpretation regarding the potential drawback of the Rewind phase in terms of memory consumption and training time. Yet, we wish to emphasize that the novelty in our paper lies in effectively integrating existing neuro-inspired concepts, aiming to guide the CL research community towards a less-explored direction. We are actively considering the efficient interpretation and implementation of TriRE as a natural progression of this work. However, we acknowledge the present version's drawbacks, including memory and training costs, which will be duly addressed in the limitation section.\n\nAdditionally, we would like to address the concern regarding the possible performance drop in the current task because of the Rewind phase. The analysis for this can be found in Section B.2 in the Appendix. We analyze this from the perspective of the stability-plasticity dilemma. As shown in Figure 7, while other baselines like ER and DER++ suffer from recency bias and are more plastic, TriRE clearly balances the stability - plasticity tradeoff managing to preserve existing knowledge from past tasks while acquiring new knowledge from current tasks. This is because while rewinding the weights, we only rewind the weights that are not in the cumulative subnetwork S. And to reiterate, by the time we reach the Rewind stage, S contains the most important weights and activations for the past and current task. So by keeping them intact and only rewinding the rest of the weights, we preserve what we have already learned and manage to bring the less active neurons back into the learning circuit.\n\n`It seems that the authors blindly think that “human-like” is good.`\n\nThere is existing literature [1,2] which shows that active forgetting is a part of biological learning and that neuron decay due to unuse is one of the reasons for catastrophic forgetting. Therefore, we aimed the rewind and subsequent relearning to act as a warm-up for the less active neurons, that are not there in cumulative subnetwork S, making them relevant again for the learning circuit and engaging them to be more receptive to learning the next task. This is to also make sure that the model is scalable and is able to utilize the available parameters efficiently. Moreover, we prove this hypothesis with empirical analysis in Section 5 (under Ablation Study) which separately focuses on the impact of the Rewind phase on the overall algorithm.\n\n[1]\tShors, Tracey J., Megan L. Anderson, D. M. Curlik Ii, and M. S. Nokia. \"Use it or lose it: how neurogenesis keeps the brain fit for learning.\" Behavioural brain research 227, no. 2 (2012): 450-458.\n\n[2]\tAlmond, N. M. \"The Use-It-Or-Lose-It Theory; The Cognitive Reserve Hypothesis and the Use-Dependency Theory: Methodological Issues, Previous Research, Current Research and Future Perspectives.\" K, Edison.(Ed). Episodic Memory: Formation, Clinical Disorders and Role of Aging. Nova Science Publishers, Inc: New York (2014).\n\n`It will be better if the authors include recent baselines [1-2]`\n\nWe extend our gratitude to the reviewer for bringing to our attention the possibility of establishing new baselines. That being said, the experimental setup of the suggested papers and TriRE are different so it will be difficult to incorporate that in the current version. However, your suggestion has enriched our perspective and we will extend our Related Works section to include these newer methods in the final version. Currently, we have made an earnest effort to consider baselines that are representative of each category of the existing CL methods that combat task interference and catastrophic forgetting.\n\nOnce again, we thank you for your thoughtful evaluation and consideration. We have diligently tried to address each concern you raised, and we are committed to ensuring that our paper contributes meaningfully to the conference proceedings. "}, {"review_id": "3IhwfJdj1r", "reviewer": "Reviewer_3Tuq", "summary": "The paper concerns catastrophic forgetting in neural networks and draws from neural mechanisms contributing to the absence of such phenomena in the brain to propose a method to overcome CF through targeted neuron retraining, task knowledge revision, and enhanced learning for less active neurons. The authors present an impressive suite of experimental results, benchmarking their proposed method as superior or comparable to notable existing approaches in class- and task-IL scenarios and with further analysis of the different stages of the method.", "questions": "* I am curious as to why rewinding improves accuracy, perhaps this could be another direction for exploration.\n\n* Small clarifications regarding the method itself:\n\t* Learn: finite replay buffer with loss-aware experience rehearsal?\n\t* Retain: parameter isolation where important connections and weights are learned?\n\t* Revise: How are the subnetworks combined? I wonder if there is a way to specifically focus on free neurons when learning new tasks, so during the combination stage they are orthogonal to the combined network?\n\t* Rewind: Does relearning target specific unactivated neurons/parameters? This step is perhaps redundant if simply taking weights from a few epochs before and training more on new mini-batches to relearn.\n\nMinor:\n* Lines 159-160: Put “k” in math mode (top-k, k-winner)\n* Line 215: “Retatin” -> “Retain”\n", "limitations": "The authors touch upon the limitations of their work in the main text and more so in the Appendix.\n\nSocial impact — not directly applicable but is mentioned in broader impacts discussion.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "+ The work creatively and effectively links to biological motivations toward continual learning (CL) capabilities.\n+ Fairly inclusive summary of existing CL approaches with contributive discussion as to their strengths and weaknesses toward integration in the proposed approach.\n+ Figures are well-crafted and easily understandable.\n+ Performance is comparable to SOTA methods.\n+ It's nice to see in the experimental data the significance of dynamic masking, retaining, rewinding.", "weaknesses": "- (As mentioned by the authors:) The work is primarily applicable only to CNN-based architectures in its present state, and additional tuning is required for the increased number of hyperparameters. While the paper is nice, one can see how TriRE won't work on transformers due to computational expense.\n\n- Regarding Related Work, the authors could elaborate on whether there are any other notable CL approaches that draw upon strengths of previous works as TriRE does and clarify how TriRE is superior/comparable (if applicable).\n\n- The authors could perhaps move discussion of limitations more heavily to the main text as opposed to the Appendix. The limitations discussed in Appendix C pertain to the practical applicability of the work and thus have merit for inclusion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer taking time to review our work in detail. We appreciate your encouraging words on our manuscript. Our response to weaknesses and questions are as follows:\n\n`While the paper is nice, one can see how TriRE won't work on transformers due to computational expense.`\n\nWe agree with the reviewer that added computational complexity is one of the major limitations of our approach. Therefore, a naive extrapolation of TriRE to transformer-based architectures might face some hurdles such as computational complexity, hyperparameter tuning etc. We extend our Limitations section to include these in the next revision. However, from a distinctive standpoint, TriRE is a novel training paradigm that enables effective assimilation of multiple orthogonal CL approaches and is one of the earliest works in this direction. Although TriRE suffers from some limitations, it has been successful in showcasing the combined efficacy of multiple orthogonal CL approaches. Also, it is to be noted that, other baseline methods mentioned in Table 1 and Figure 3 could also not work on transformers considering the architectures’ innate complexity and need for more resources and training time.\n\n`whether there are any other notable CL approaches that draw upon strengths of previous works as TriRE does`\n\nTo the best of our knowledge, we are one of the earliest works in this direction. However, we might have missed some of the recent publications. As suggested by you and other reviewers, we intend to build upon our related works to provide more information on any other notable CL approaches that draw upon strengths of previous works as TriRE does. \n\n`move discussion of limitations more heavily to the main text as opposed to the Appendix`\n\nDue to the space constraints, we moved the Limitations section to Appendix. We intend to expand the limitation section further by incorporating suggestions from you and other reviewers for the revised version. Considering that, we would appreciate your guidance on identifying specific sections from the main paper that could potentially be moved to the appendix to make space for the Limitations section. \n\n`why rewinding improves accuracy, perhaps this could be another direction for exploration`\n\nThere is existing literature which shows that active forgetting is a part of biological learning and that neuron decay due to unuse is one of the reasons for catastrophic forgetting. Inspired by that, we aimed the rewind and subsequent relearning to act as a warm-up for the less active neurons, that are not there in cumulative subnetwork S, making them relevant again for the learning circuit and engaging them to be more receptive to learning the next task. By keeping the cumulative subnetwork S intact, we also make sure that the rewinding of weights does not cause any performance drop. Moreover, we concur with the perspective that exploring the concept of \"rewind\" as a potential research direction holds promise and could open up new avenues for further investigation.\n\n`Learn: finite replay buffer with loss-aware experience rehearsal? `\n\n(Assuming the clarification is to understand loss-aware experience rehearsal better) In loss aware balanced reservoir sampling, we compute a score vector proportional to the number of items of each class and estimate an importance score given by the opposite of the loss value for each example. Then, we normalize these two terms to ensure an equal contribution and sum them to form a single score vector. Finally, we assign replacement probability to each item that is proportional to the combined score. This ensures that there is balance within the buffer in terms of the number of examples per class as well as help us identify and replace the elements displaying low loss values to make space for harder examples.\n\n`Retain: parameter isolation where important connections and weights are learned?`\n\nIn the Retain phase, neurons and weights are learned for the current task. At the end of that stage, we employ k-WTA and heterogenous dropout based activation pruning and subsequent weight pruning to retain the best weights and activations for that task.\n\n`Revise: How are the subnetworks combined?`\n\nIn the Revise stage, initially the cumulative subnetwork containing knowledge from past tasks and the subnetwork containing knowledge from the current task are trained / fine tuned together on the rehearsal buffer to learn their joint distribution. After this revising process, we update the cumulative set $S = S \\cup S_t$. This makes sure that at the end of the current task, the cumulative set contains the best weights and activations of both past tasks and the current task which helps in preserving knowledge.\n\n`if there is a way to specifically focus on free neurons when learning new tasks, so during the combination stage they are orthogonal to the combined network?`\n\nWe don’t encourage completely mutually exclusive subnetworks for each task as empirical analysis in Figure 4 shows that there is neuron overlap while learning different tasks. This knowledge sharing is particularly beneficial when it comes to scaling the model for longer task sequences. Furthermore, when all subnetworks are orthogonal to each other, it could lead to capacity saturation and task discovery problems during inference.\n\n`Rewind: Does relearning target specific unactivated neurons/parameters?`\n\nIn the rewind phase, as correctly inferred by the reviewer, we are rewinding and relearning the less active neurons. The cumulative subnetwork S which contains the most important weights and activations remains unchanged. The rest of the less active neurons go through the Rewind phase.\n\nWe hope that the provided clarification has addressed your concerns and inquiries to your satisfaction. However, if further assistance or elaboration is required, we would be more than happy to provide additional information to ensure a complete comprehension of our work.\n"}]

Author rebuttal:
In this section, we aim to address the overarching concerns raised by reviewers, ensuring a comprehensive response to the broader themes highlighted in their feedback.

`The key contribution part is rewind while other parts are peripheral or have no novelty / It seems to be a mere combination of existing ideas into a so-called three-stage approach.`

We respectfully disagree with this characterization. We intend to showcase that the novelty of the proposed method lies in the fact that it is able to effectively combine the existing families of CL methods to form a sophisticated new CL paradigm. The key contribution of the work is not the stages or phases by themselves but our effective interpretation of how these stages can harmoniously work together to create a novel CL method. As mentioned in the ‘Related Works’ section, each of these categories of methods has a plethora of research available individually, but they also have their drawbacks. For instance, weight regularization (inspired by meta plasticity) alone only imposes a soft penalty, which doesn’t entirely prevent catastrophic forgetting. Rehearsal methods (inspired by experience replay) do better but suffer overfitting in low buffer regimes, whereas parameter isolation (inspired by context gating and/or neurogenesis) alone is not a good candidate for longer task sequences due to capacity saturation. Notably, mammalian brain doesn’t depend on a singular concept but on an amalgamation of the above mentioned neuro-physiological phenomena working coherently together. As obvious as it may seem, the current CL research is not focused in that direction, and with this work, we are trying to bring focus to this idea by showing its effectiveness and plausibility. To the best of our knowledge, ours is one of the earliest works in this direction and we have proved its potential with empirical results.

`Weight pruning uses the existing CWI criterion`

While we recognise that CWI is an existing weight pruning criterion, it is essential to note that the proposed work focuses on a novel CL paradigm and considers weight pruning as a means to an end. The purpose here is not to propose a new pruning method. Nonetheless, in the process, other pruning criteria were considered like magnitude pruning and fisher information based pruning (Refer file attached). As shown in the Table, it is evident that CWI only contributes marginally to the overall accuracy improvement. The method is conceptually compatible with other criteria as well. The rationale behind selecting CWI is primarily rooted in its capacity to assess the weight significance relative to data stored in the rehearsal buffer. As shown in the Table, even other pruning criteria yields better results than CLS-ER even without hyperparameter tuning due to time constraints. Also, as weight pruning is an active research area, we believe newer pruning criteria could also work in conjunction with our method.

`There are 7 different parameters and it seems that it is arbitrarily chosen`

As this work is one of the earliest ones in the direction of effectively combining existing CL methods, it is not the most optimal method and the number of hyperprarameters adds a perception of complexity. However, we can assure you that it is not arbitrarily chosen to make the best performance for each task. Although the combination of hyperparameters that gave the best accuracy have been tabulated in Table 4 as a means for reproducibility, our method is not highly sensitive to the particular choice of hyperparameters and is robust enough to handle different settings to attain similar performance. Regarding the intuition behind the hyperparameters, for instance, Rewind percentile, which decides how much to rewind, has been analyzed and the instinct for choosing values in the range of 70-90% percentile has been discussed in Section 5. Furthermore, the parameter is fixed at 0.999 for all experiments to mimic the slow acquisition of structured knowledge by the EMA model. Also, the learning rate in the Revise phase ($\eta$') is dependent on the initial learning rate ($\eta$). From our experiments, we found that for smaller datasets, $\eta$' is approximately 1/20th  of $\eta$ and for larger datasets, its 1/10th of $\eta$. It is a safe bet to use this criterion as a starting point to find the most optimal learning rate for the Revise stage.

`The experiments lack analysis of computational cost and what is the cost of multi-stage procedure?`

We appreciate the reviewers for identifying the lack of computation cost analysis, and we have made a note to include a comprehensive analysis in the final version of the paper. Also, we would agree with the extrapolation that the multi-stage procedure proposed in this paper is not the most optimal solution memory and computation efficiency-wise and has a lot of room for improvement. However, we would also like to reiterate that efficiency of the method was not a goal for this work and the novelty of this paper lies in the effective combining of existing  neuro-inspired concepts. We are trying to spur the CL research community towards this direction which is less explored. Efficient interpretation and implementation of TriRE is a natural progression to this work that we are also considering. 

`Why does the proposed method seem to underperform on Task-IL for two of the three datasets evaluated?`

Considering Task-IL is considered to be an easier CL scenario than the Class-IL, TriRE’s Task-IL performance is not significantly worse or better statistically than the competing methods. However, as the CL scenario gets harder, the difference between the competing methods is clearer due to their ability to mitigate catastrophic forgetting. For instance, their performance is much worse compared to TriRE in Seq-TinyImageNet where the buffer-to-class ratio is low. We regret the lack of clarity in the current version and will update the manuscript as per your suggestion in the next revision. 


Meta-review: This paper tackles class incremental learning from a new perspective of utilizing a multi-mechanism approach which includes rehearsal, network growing, forgetting, and re-learning. Results are shown across datasets such as sequential TinyImageNet including in low-buffer settings. Reviewers appreciated a number of aspects of the paper, including rethinking of the current paradigm where methods are studied in isolation, thoroughness of other approaches considered, and ablations showing the importance of the different phases. Weaknesses included complexity of the method (including computation and memory requirements), addition of significant hyper-parameters, triggering of various processes based on task boundaries, and comparison to the state-of-art. The authors provided a rigorous rebuttal with explanations and additional experiments: comparisons to Foster/Memo, computation/memory overhead, and hyper-parameter sensitivity. 

  After considering these rebuttals, most reviewers expressed that their concerns have been largely satisfied and retained or increased to positive scores. One reviewer has retained a weak reject and expressed a number of concerns, but in my view many of them have been addressed satisfactorily. Overall, considering the paper, reviews, rebuttal, and discussions, I recommend acceptance of this paper and believe it adds an interesting multi-method perspective to the continual learning literature. I highly recommend that the authors incorporate the issues raised in the reviews and during discussions, for example limitations both of the method itself as well as the general continual learning settings used in the paper.
Decision: Accept (poster)