Title: TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion
Venue: NeurIPS 2023
Accepted: yes
Keywords: Continual Learning

Catastrophic Forgetting

Experience Replay

Lifelong Learning

Bio-Inspired

Active Forgetting

Scalable Neurogenesis
Abstract: Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks. Several techniques such as weight regularization, experience rehearsal, and parameter isolation have been proposed to alleviate CF. Despite their relative success, these research directions have predominantly remained orthogonal and suffer from several shortcomings, while missing out on the advantages of competing strategies. On the contrary, the brain continually learns, accommodates, and transfers knowledge across tasks by simultaneously leveraging several neurophysiological processes, including neurogenesis, active forgetting, neuromodulation, metaplasticity, experience rehearsal, and context-dependent gating, rarely resulting in CF. Inspired by how the brain exploits multiple mechanisms concurrently, we propose TriRE, a novel CL paradigm that encompasses retaining the most prominent neurons for each task, revising and solidifying the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Across CL settings, TriRE significantly reduces task interference and surpasses different CL approaches considered in isolation.

TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion

Abstract

Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks. Several techniques such as weight regularization, experience rehearsal, and parameter isolation have been proposed to alleviate CF. Despite their relative success, these research directions have predominantly remained orthogonal and suffer from several shortcomings, while missing out on the advantages of competing strategies. On the contrary, the brain continually learns, accommodates, and transfers knowledge across tasks by simultaneously leveraging several neurophysiological processes, including neurogenesis, active forgetting, neuromodulation, metaplasticity, experience rehearsal, and context-dependent gating, rarely resulting in CF. Inspired by how the brain exploits multiple mechanisms concurrently, we propose TriRE, a novel CL paradigm that encompasses retaining the most prominent neurons for each task, revising and solidifying the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Across CL settings, TriRE significantly reduces task interference and surpasses different CL approaches considered in isolation. 1

1 Introduction

Continual learning (CL) over a sequence of tasks remains an uphill task for deep neural networks (DNNs) due to catastrophic forgetting of older tasks, often resulting in a rapid decline in performance and, in the worst-case scenario, complete loss of previously learned information [38]. Several approaches, such as parameter isolation [43, 4], weight regularization [56, 45], and experience rehearsal [41, 42, 9] have been proposed in the literature to address the problem of catastrophic forgetting in DNNs. Despite their relative success, these research directions have predominantly remained orthogonal and suffer from several shortcomings. Parameter isolation approaches suffer from capacity saturation and scalability issues in longer task sequences, while weight regularization approaches cannot discriminate classes from different tasks, thus failing miserably in scenarios such as class-incremental learning (Class-IL) [28]. In scenarios where buffer size is limited due to memory constraints (e.g., edge devices), rehearsal-based approaches are prone to overfitting on the buffered data [7]. As these research directions have rarely crossed paths, there is a need for an integrated approach to leverage the advantages of competing methods to effectively mitigate catastrophic forgetting in CL.

Catastrophic forgetting is a direct consequence of a more general problem, namely the stabilityplasticity dilemma [2, 36]: the extent to which the CL model must be plastic to accommodate newly acquired knowledge and stable to not interfere with previously learned information [38]. In stark

Beginning of task

End of task

Neuron not in

Neuron in

Neuron in

Merge

contrast to DNNs, biological systems manage this dilemma better and are able to learn continually throughout their lifetime with minimal interference. CL in the brain is administered by a rich set of neurophysiological processes that encompass different kinds of knowledge, and conscious processing that integrates them coherently [17]. Empirical studies suggest that metaplasticity [27] and experience replay play a prominent role in memory consolidation in the brain [40, 15]. In addition, neurogenesis in the brain is crucial for the growth and restructuring necessary to accommodate new skills [3]. Neuromodulatory systems facilitate swift learning and adaptability in response to contextual changes induced by new stimuli or shifts in motivation [33]. Whereas context-dependent gating [34] and active forgetting [19] improve the separation between the representations of patterns belonging to different tasks. By simultaneously leveraging these processes, the brain exploits task similarity and exhibits positive forward transfer, rarely resulting in catastrophic forgetting.

Inspired by the biological underpinnings of the CL mechanisms in the brain, we propose 'REtain, REvise & REwind' (TriRE) , a novel CL paradigm to mitigate catastrophic forgetting. Specifically, TriRE involves experience rehearsal, scalable neurogenesis, selective forgetting, and relearning to effectively mitigate catastrophic forgetting. Within each task, the proposed method consists of three stages: (i) Retain , where the most active neurons and their corresponding most important weights of the task are extracted and retained to avoid task interference and drastic weight changes, (ii) Revise , where the extracted network is finetuned to revise and solidify the current task as well as the joint distribution of the past tasks, (iii) Rewind , where the free neurons undergo active forgetting and relearning to promote the less active neurons back into the learning circuit for the next task as illustrated in Figure 1. TriRE effectively combines multiple mechanisms and leverages the advantages offered by different CL approaches.

We find that TriRE significantly reduces task interference and surpasses the aforementioned CL approaches across various CL scenarios. Specifically, TriRE outperforms rehearsal-based approaches in Seq-TinyImageNet for Class-IL scenario by almost 14%, even under low-buffer regimes, by promoting generalization through weight and function space regularization. Experience rehearsal enables discrimination of classes belonging to different tasks in TriRE, resulting in at least twice as good performance over weight regularization methods for the same dataset. Unlike parameter isolation approaches, TriRE is scalable and produces at least a 7% relative improvement compared to parameter isolation approaches in Seq-CIFAR100 Task-IL setting without requiring access to task identity at inference time.

2 Related works

Rehearsal-based Approaches: Prior works attempted to address the problem of catastrophic forgetting by explicitly storing and replaying previous task samples, akin to experience rehearsal in the brain. Experience rehearsal (ER) approaches [41, 42] maintain a fixed capacity memory buffer to

store data sampled from previous task distributions. Several approaches are built on top of ER to better preserve the previous task information: GCR [49] proposed a coreset selection mechanism that approximates the gradients of the data seen so far to select and update the buffer. DER++ [9] and CLS-ER [6] enforce consistency in predictions using soft targets in addition to ground-truth labels. DRI [52] uses a generative model to further support experience rehearsal in low buffer regimes. More recent works like TARC [8], ER-ACE [11] and Co 2 L [12] focus on reducing representation drift right after task switch to mitigate forgetting through asymmetric update rules. Under low-buffer regimes and longer task sequences, however, these approaches suffer from overfitting on the buffered samples.

Weight Regularization Approaches: Catastrophic forgetting mainly emanates from large weight changes in DNNs when learning a new task. Therefore, weight-regularization methods seek to penalize the sudden changes to model parameters that are crucial for the previous tasks. Depending on the type of regularization, these approaches can be broadly categorized into prior- and data-focused approaches. Prior-focused approaches, such as elastic weight consolidation (EWC) [25], online EWC (oEWC) [45], memory-aware synapses (MAS) [5], and synaptic intelligence (SI) [56] employ prior information on model parameters and estimate the importance of parameters associated with previous tasks based either on the gradients of the learned function output or through Fisher's information matrix. On the other hand, data-focused methods, such as Learning without Forgetting (LwF) [30] instead perform knowledge distillation from models trained on previous tasks when learning on new data. Although weight regularization approaches do not require a memory buffer and are scalable, they only impose a soft penalty, thus failing to entirely prevent forgetting of previous task information.

Parameter Isolation Approaches: Parameter isolation has been predominantly done in two ways: either within a fixed capacity or by growing in size. In the former, dynamic sparse methods such as PackNet [32], CLNP [16], PAE [23], and NISPA [18] make use of DNN's over-parameterization to learn multiple tasks within a fixed model capacity. Similar to the brain, these models simultaneously learn both connection strengths and a sparse architecture for each task, thereby isolating the taskspecific parameters. However, these methods suffer from capacity saturation in longer task sequences, limiting their ability to accommodate new tasks. In contrast, the latter methods, such as PNNs [39], Expert-gate [4] and DEN [55] expand in size, either naively or intelligently, to accommodate new tasks while minimizing forgetting. Although these approaches are extremely efficient in mitigating catastrophic forgetting, they do not scale well with longer task sequences, rendering them inapplicable in real-world scenarios.

Contrary to DNNs, the brain simultaneously exploits multiple neurophysiological processes, including neurogenesis [3], active forgetting [19], metaplasticity [27], experience rehearsal [40], and contextdependent gating [34] to continually acquire, assimilate, and transfer knowledge across tasks without catastrophic forgetting [26]. Inspired by how the brain exploits multiple mechanisms concurrently, we propose a novel CL paradigm, TriRE, that leverages the advantages of multiple aforementioned mechanisms to effectively mitigate catastrophic forgetting in CL.

3 Method

CL problems typically comprise t ∈ { 1 , 2 , .., T } sequential tasks, with c classes per task, and data that appear gradually over time. Each task has a task-specific data distribution associated with it ( x t , y t ) ∈ D t . We take into account two well-known CL scenarios, class-incremental learning (Class-IL) and task-incremental learning (Task-IL). Our working model consists of a feature extractor network f θ and a single head classifier g θ that represents all classes of all tasks.

Sequential learning through DNNs has remained a challenging endeavor, since learning new information tends to dramatically degrade performance on previously learned tasks. As a result, to better retain information from past tasks, we maintain a memory buffer D m that contains data from tasks previously viewed. Considering the desiderata of CL, we assume that the model does not have infinite storage for previous experience and thus | D m | ≪ | D t | . To this end, we use loss-aware balanced reservoir sampling [10] to maintain the memory buffer. We update the working model, Φ θ = g θ ( f θ ( . )) , using experience rehearsal at each iteration by sampling a mini-batch from both D t and D m as follows:

where L ce is the cross-entropy loss, σ ( . ) is the softmax function, L t is the task-wise loss and L er is the rehearsal-based loss. The objective in Eq. 1 encourages plasticity through the supervisory signal from D t and increases stability through D m . However, as CL training advances, model predictions carry more information per training sample than ground truths [7]. Hence, soft targets can be utilized in addition to ground-truth labels to better preserve the knowledge of the earlier tasks. Traditional methods to enforce consistency in predictions include using an exponential moving average (EMA) of the weights of the working model [6] or holding previous predictions in a buffer [9]. As the former result in better knowledge consolidation and decision boundaries, we use EMA of the working model weights to ensure consistency in predictions:

where Φ θ EMA is the EMA of the working model Φ θ and ∥ . ∥ F is the Frobenius norm. We update the EMA model as follows:

where µ is the decay parameter and ζ is the update rate. Finally, the EMA model acts as a selfensemble of models with distinct task specializations for inference rather than the working model.

Notations: Let S t be the extracted subnetwork mask corresponding exclusively to the current task and S be the cumulative dynamic network mask corresponding to the tasks learned so far. At the end of the training, S would contain the most active neurons and the best corresponding weights across all tasks. In the following, we describe in detail various components of TriRE learning paradigm.

3.1 Retain

In CL, typically, models from previous tasks are seen as initialization and are 'washed out" by new updates from the current task, which causes CF [35]. However, the brain uses context-dependent gating [26] to selectively filter neural information based on the context in which it is presented, allowing the development of specialized modules that can be added or removed from the network without disrupting previously learned skills [37, 50]. Inspired by this, the Retain phase induces modularity in the model by training a hyper-network first and then extracting a subnetwork that is equivalently representational of the current task knowledge. This extracted subnetwork not only helps in creating task-wise specialized modules, but also helps the model preserve capacity for future tasks. Retention of this subnetwork is done using heterogeneous dropout of activations and weight pruning.

The Retain stage appears at the beginning of each task. At this stage, initially { f θ | θ / ∈ S} is trained using a mini-batch of D t and { f θ | θ ∈ S} is trained using a mini-batch of D m . This is to ensure that the weights not in the cumulative network learn the new task to maintain plasticity, while the weights in the cumulative network learn a combination of old and new tasks to maintain stability. At the convergence of this training, we perform activation pruning followed by weight pruning to extract S t as shown in Figure 2.

Activation Pruning: This involves identifying and extracting neurons that contribute the most to the overall activation or output of the network. We monitor the frequency of neuron activations when a network is trained on a task. In essence, each neuron is given an activation counter that increases when a neuron's activation is among the top-k activations in its layer. We use these activation counts to extract the k-winner activations and retain them as the knowledge base for the current task. Heterogeneous dropout [1] is used to map the activation counts of each neuron to a Bernoulli variable, indicating whether the said neuron is extracted or dropped. This, essentially, leaves the less activated neurons free to learn the next tasks.

Weight Pruning: After retaining the most activated neurons for the task, we prune the less important connections corresponding to these neurons. In contrast to conventional methods, which only leverage weight magnitude or Fisher information for pruning, our method also takes into account the significance of weights with respect to data saved in the rehearsal buffer. Continual Weight Importance (CWI) [53] criteria ensure that we maintain: (1) weights of greater magnitude for output stability, (2) weights significant for the current task for learning capacity, and (3) weights significant for past data to prevent catastrophic forgetting.

Dense

Network

After

Heterogeneous

Dropout

After CWI based

Weight Pruning

Extracted

Subnetwork

For the working model, the CWI of weight θ is defined as follows,

where α and β , are coefficients that regulate the weight of current and buffered data, respectively. In addition, ˜ L ce denotes the single-head form of the cross-entropy loss, which only takes into account the classes relevant to the current task by masking out the logits of other classes.

At the end of this stage, we end up with a mask of the most activated neurons and their most important weights for the current task, S t .

3.2 Revise

Although context-dependent gating is a helpful phenomenon for CL in tackling CF, there are other biological phenomena such as neuromodulation, metaplasticity, and neurogenesis among others which also contribute to the brain's ability to combat CF [26]. Neuromodulation, for instance, is used to decide whether a new feature is novel and unfamiliar (that is, creates a new memory) or common and familiar (that is, consolidates into an existing memory). Taking the cue from that, Revise stage mainly focuses on revising and solidifying the knowledge that the extracted subnetwork of the current task, S t , and the cumulative subnetwork of past tasks, S , currently possess. It is clear that S t is specialized on the current task and S is specialized on the past tasks. However, finetuning these two networks jointly can improve the knowledge overlap, and the features thus generated would be a better approximation of all the seen classes.

Firstly, { f θ | θ / ∈ ( S ∩ S t ) } is trained with a mini-batch of D t so that a part of the extracted network, S t , can be utilized to regain the performance for the current task. Subsequently, { f θ | θ ∈ ( S ∩ S t ) } is trained with a mini-batch of D m to preserve forward transfer and knowledge overlap between the past and the current tasks. The optimization learning rate is also considerably reduced at this stage compared to the Retain stage to prevent drastic weight changes in subnetworks, which in turn decreases the forgetting. This could be seen as an adaptation of the metaplasticity in the brain [26, 20] which refers to the ability to adjust the amount of plasticity in the network based on the current and future needs of the organism. At the end of this finetuning, the currently extracted subnetwork S t is merged with the cumulative extracted mask from past tasks, S . The merging of the subnetworks can be seen as a process of integrating new neurons ( S t ) into the existing neural network ( S ), similar to the way neurogenesis allows for the integration of new neurons into existing neural circuits to accommodate new memories.

3.3 Rewind

Evidence implies that the brain uses active forgetting as a tool for learning due to its limited capacity [47, 29]. Studies also suggest that although learning and memory become more difficult to access during the forgetting process, they still exist [54, 48]. There are still memory remnants in the brain that can be reactivated [31]. In this work, we define the rewinding of weights to an earlier state as active forgetting. We also make sure that rather than reinitializing the model back to random or very

Algorithm 1 Proposed Approach - TriRE

early weights, it is rewound to a point where the model has learned some features and has a generic perception of the objective closer to convergence (but not absolute convergence). To aid this, the weights from a later epoch k from the Retain phase are saved to be used in the Rewind stage.

Specifically, after the Retain and Revise steps, we rewind the weights belonging to non-cumulative subnetwork { f θ | θ / ∈ S} back to the epoch k weights. Then the rewinded weights are finetuned for a few epochs using a mini-batch of D t . This is helpful because studies [46, 13] show that in the human brain, less active neurons follow a 'use-it-or-lose-it' philosophy. Therefore, forgetting and relearning act as a warm-up for these less active neurons making them relevant again for the learning circuit and making them more receptive to learning the next task.

In summary, TriRE (Retatin-Revise-Rewind) involves iteratively applying the three phases mentioned above to each task within a lifelong learning setting. Our method effectively combines multiple biological phenomena and harnesses the advantageous characteristics provided by popular CL approaches. The step-by-step procedure is given in Algorithm 1.

4 Results

Experimental Setup: We expand the Mammoth CL repository in PyTorch [9]. On the basis of Class-IL and Task-IL scenarios, we assess the existing CL techniques against the proposed one. Although the training procedure for Class-IL and Task-IL is the same, during inference, Task-IL has access to the task-id. We consider a number of rehearsal-based, weight regularization, and parameter-isolation approaches as useful baselines because TriRE necessitates experience rehearsal and model modularity. We use ResNet-18 [21] as the feature extractor for all of our investigations. In order to reduce catastrophic forgetting, we additionally offer a lower bound SGD without any support and an upper bound Joint where the CL model is trained using the full dataset.

Experimental Results: Wecompare TriRE with contemporary rehearsal-based and weight regularization methods in Class-IL and Task-IL settings. As shown in Table 1, TriRE consistently outperforms

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Tasks

50

60

70

80

90

100

Task-IL Accuracy

Seq-CIFAR100 (20 Tasks)

NISPA (75.87%)

CLNP (66.68%)

PNN (77.33%)

PackNet (67.48%)

PAE (77.6%)

rehearsal-based methods across most datasets, highlighting the significance of dynamic masking in CL. Although methods like Co 2 L and ER-ACE excel on simpler datasets, they struggle with more challenging ones. The same applies to methods like DRI and GCR, which augment memory buffers through core-set and generative replay. Their performance, for instance, lags behind TriRE in Seq-TinyImageNet, where the buffer-to-class ratio is low. Retaining task-wise dynamic subnetworks and revising extracted subnetworks to preserve past knowledge significantly reduces task interference in TriRE. In essence, TriRE boosts generalization through weight and function space regularization, selective forgetting, and relearning, yielding superior performance across tasks.

As evident from Table 1, weight regularization methods such as LWF, oEWC, and SI perform miserably in both Class-IL and Task-IL settings across datasets. The reason being, these approaches encounter classes solely from the current task at any point in CL training. Therefore, they fail to discriminate between classes from different classes resulting in subpar performance. On the other hand, TriRE leverages samples from previous classes through experience rehearsal to learn discriminatory features across tasks. In addition, TriRE entails forming subnetworks through Retain and Revise resulting in modularity and reduced task interference between tasks.

Parameter isolation approaches minimize task interference in CL by creating distinct sub-networks either within a given model capacity or by dynamically growing the network. Figure 3 illustrates a comparison between methods trained on Seq-CIFAR100 with 20 tasks i.e., it depicts final accuracies

1

11

21

31

41

Subset of neurons in the last conv layer

1

2

3

4

5

1

11

21

31

41

Subset of neurons in the last shortcut layer

1

2

3

4

5

Number of tasks

■

1

■

■

■

0.1

0.4

0.7

1.0

Rewind Epoch Percentile

30

40

50

Seq-CIFAR100

0.1

0.4

0.7

1.0

50

60

70

Class-IL Accuracy

Seq-CIFAR10

0.1

0.4

0.7

1.0

10

20

30

Seq-TinyImageNet

on 1 st , 2 nd .. 20 th task after training. Upon completing all tasks, TriRE achieves an average accuracy of 80.85%, surpassing the performance of the baselines considered. TriRE leverages the benefits of experience rehearsal, weight regularization, and function space regularization to learn compact and overlapping subnetworks, resulting in reduced task interference while maintaining scalability. In line with biological principles, TriRE incorporates selective forgetting and relearning mechanisms to activate less active neurons and enhance their receptiveness to learning subsequent tasks, thereby mitigating the risk of capacity saturation.

5 Model Analysis

Task Interference: Figure 4 shows the changes in neuronal activity for the subset of neurons in the last convolutional layer (top) and the last shortcut layer (bottom) of ResNet-18 trained on 5 tasks in Seq-CIFAR100 dataset. The neurons that are blacked out are the most active neurons for each particular task after the Retain phase. For each task, it is observed that there are exclusive subnetworks forming on their own (horizontally) that capture task-specific information. However, with CL training, several neurons become generalizable by capturing information that can be shared across tasks. Therefore, there is neuron overlap between the extracted neurons across tasks (vertically). More so in the shortcut layer, since the number of parameters in these layers is low. TriRE optimally manages the model capacity vs. task modularity trade-off better by re-using neurons that can be shared across tasks while maintaining the modularity of knowledge in each task intact.

How Much to Rewind? In Figure 5, we examine Class-IL accuracy when the model is rewound to different points in the Retain stage in order to comprehend how rewinding affects the accuracy of the model. We observe that the accuracy of the inference model decreases if the model forgets too much and is rewound to an early stage in the training. This is in alignment with the observations made by [14] that rewinding to extremely early stages is not recommended for DNNs because the network has

not learned enough meaningful features by then to regain the lost accuracy. Additionally, we notice that accuracy also suffers when the model is rewound to a point extremely close to the end of training time. Rewinding to a very late point in the training phase close to convergence is not ideal because there is not enough time for relearning. Our experiments indicate that rewinding to between 70% and 90% of the training time results in the best accuracy.

Ablation Study: As explained previously, TriRE employs a three-stage learning paradigm to reduce task interference and improve weight reuse in CL. We seek to uncover how each of Retain , Revise , and Rewind in TriRE influence Class-IL and Task-IL accuracies in Seq-CIFAR100 and Seq-TinyImageNet datasets through Table 2. It can be seen that although Retain alone can extract the subnetworks containing the most active neurons and decrease task interference, it falls short in aspects of forward transfer and weight reuse. Similarly, Retain and Revise together can solidify the knowledge extracted from current and past tasks, but such a model suffers from capacity issues without the reactivation of less active neurons for future tasks. Likewise, Retain and Rewind together can encourage task-wise delimitation of knowledge and promote efficient usage of available networks, but lose out on the forward transfer introduced by the learning of joint distributions. Finally, analogous to the brain, it is evident that the harmony of all components is what achieves the best results in both datasets.

Memory and Computational Cost: We conduct a comparative analysis of the computational and memory overhead of TriRE in contrast to related works. Table 3 provides an analysis of the learnable parameters and memory required by TriRE in contrast to those of DER++, EWC, and PNNs, (Opting for one method from each individual family of CL methods). Firstly, similar to DER++ and EWC, TRiRE does not add any learnable parameters to the model. However, it is evident that PNNs have an infeasible amount of learnable parameters which gets progressively worse with longer task sequences. Secondly, the observed increase in memory consumption in TriRE can be attributed to several factors: (1) the application of multiple masking mechanisms for parameter isolation; (2) the incorporation of the Rewind phase necessitating weight retention from a previous epoch; and (3) the utilization of the Exponential Moving Average (EMA) model to enhance knowledge consolidation. All of these factors hold memory but do not add any learnable parameter to the training.

6 Conclusion

We introduced TriRE, a novel biologically inspired CL paradigm that entails experience rehearsal, scalable neurogenesis, and selective forgetting and relearning to effectively mitigate catastrophic forgetting in CL. Within each task, TriRE entails retaining the most prominent neurons for each task, revising the extracted knowledge of current and past tasks, and actively promoting less active neurons for subsequent tasks through rewinding and relearning. Analogous to multiple neurophysiological mechanisms in the brain, TriRE leverages the advantages of different CL approaches, thus significantly lowering task interference and surpassing different CL approaches when considered in isolation. For Seq-TinyImageNet, TriRE outperforms the closest rival in rehearsal-based baselines by 14%,

surpasses the best parameter isolation baseline by 7%, and nearly doubles the performance of the best weight regularization method. Extending our method to CL scenarios oblivious to task boundaries and to few- and zero-shot learning settings are some of the future research directions for this work.

7 Limitations and Future Work

We proposed TriRE, a novel paradigm that leverages multiple orthogonal CL approaches to effectively reduce catastrophic forgetting in CL. As orthogonal CL approaches may not always be complementary, the selection of such approaches needs careful consideration in TriRE. In addition, having multiple objective functions naturally expands the number of hyperparameters, thereby requiring more tuning to achieve optimal performance. Therefore, additional computational complexity and memory overhead due to the staged approach and extensive hyperparameter tuning are some of the major limitations of the proposed method. For the same reason, we highlight that TriRE is not directed toward compute-intensive architectures such as vision transformers.

TriRE involves different stages of training within each task, requiring knowledge of task boundaries. In line with state-of-the-art methods in CL, each task entails a non-overlapping set of classes, and data within each task is shuffled to guarantee i.i.d. data. However, in the case of online learning where data streams and the distribution gradually shift, TriRE cannot be applied in its current form. Therefore, additional measures such as task-boundary approximation and modification to learning objectives are necessary to enable TriRE to work in such scenarios. Furthermore, traditional CL datasets considered in this work entail independent tasks and data points without intrinsic cumulative structure. As TriRE does not leverage structures learned in previously encountered tasks, structure learning forms another limitation of this proposed method. Reducing computational and memory overhead, extending to task-free CL scenarios with recurring classes, and leveraging intrinsic structures within underlying data are some of the future research directions for this work.

8 Broader Impacts

Inspired by the multifaceted learning mechanisms of the brain, we propose TriRE, which replicates the brain's ability to leverage multiple mechanisms for CL, enhancing the generalization capabilities of CL models across tasks. Its success not only encourages the exploration of neuro-inspired methods for deep neural networks, but also opens up opportunities to augment existing CL approaches by leveraging the advantages of competing strategies. By enabling models to learn continuously and adapt to new tasks, TriRE contributes to the responsible and ethical deployment of AI technologies, as models can improve and update their knowledge without requiring extensive retraining. This advancement has significant implications for various real-world applications and promotes the development of AI systems that can continually improve and adapt their performance.

Acknowledgement: The work was conducted while all the authors were affiliated with NavInfo Europe, Eindhoven, The Netherlands.

References

Ali Abbasi et al. 'Sparsity and heterogeneous dropout for continual learning in the null space of neural activations'. In: Conference on Lifelong Learning Agents . PMLR. 2022, pp. 617-628.

Wickliffe C Abraham and Anthony Robins. 'Memory retention-the synaptic stability versus plasticity dilemma'. In: Trends in neurosciences 28.2 (2005), pp. 73-78.

James B Aimone, Janet Wiles, and Fred H Gage. 'Potential role for adult neurogenesis in the encoding of time in new memories'. In: Nature neuroscience 9.6 (2006), pp. 723-727.

Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. 'Expert gate: Lifelong learning with a network of experts'. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 2017, pp. 3366-3375.

Rahaf Aljundi et al. 'Memory aware synapses: Learning what (not) to forget'. In: Proceedings of the European conference on computer vision (ECCV) . 2018, pp. 139-154.

Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. 'Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System'. In: International Conference on Learning Representations . 2022. URL: https://openreview.net/forum? id=uxxFrDwrE7Y .

Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. 'Consistency is the key to further mitigating catastrophic forgetting in continual learning'. In: Conference on Lifelong Learning Agents . PMLR. 2022, pp. 1195-1212.

Prashant Shivaram Bhat, Bahram Zonooz, and Elahe Arani. 'Task agnostic representation consolidation: a self-supervised based continual learning approach'. In: Conference on Lifelong Learning Agents . PMLR. 2022, pp. 390-405.

Pietro Buzzega et al. 'Dark Experience for General Continual Learning: a Strong, Simple Baseline'. In: Advances in Neural Information Processing Systems . Vol. 33. Curran Associates, Inc., 2020, pp. 15920-15930.

Pietro Buzzega et al. 'Rethinking experience replay: a bag of tricks for continual learning'. In: 2020 25th International Conference on Pattern Recognition (ICPR) . IEEE. 2021, pp. 21802187.

Lucas Caccia et al. 'New Insights on Reducing Abrupt Representation Change in Online Continual Learning'. In: International Conference on Learning Representations . 2022.

Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. 'Co2l: Contrastive continual learning'. In: Proceedings of the IEEE/CVF International conference on computer vision . 2021, pp. 95169525.

Diane T Feldman and William C Gordon. 'The alleviation of short-term retention decrements with reactivation'. In: Learning and Motivation 10.2 (1979), pp. 198-210.

Jonathan Frankle and Michael Carbin. 'The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks'. In: International Conference on Learning Representations . 2018.

Gabrielle Girardeau et al. 'Selective suppression of hippocampal ripples impairs spatial memory'. In: Nature neuroscience 12.10 (2009), pp. 1222-1223.

Siavash Golkar, Micheal Kagan, and Kyunghyun Cho. 'Continual Learning via Neural Pruning'. In: Real Neurons & Hidden Units: Future directions at the intersection of neuroscience and artificial intelligence @ NeurIPS 2019 . 2019.

Anirudh Goyal and Yoshua Bengio. 'Inductive biases for deep learning of higher-level cognition'. In: Proceedings of the Royal Society A 478.2266 (2022), p. 20210068.

Mustafa B Gurbuz and Constantine Dovrolis. 'NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks'. In: International Conference on Machine Learning . PMLR. 2022, pp. 8157-8174.

Oliver Hardt, Karim Nader, and Lynn Nadel. 'Decay happens: the role of active forgetting in memory'. In: Trends in cognitive sciences 17.3 (2013), pp. 111-120.

Demis Hassabis et al. 'Meta-learning and the neuroscience of learning and memory'. In: Nature Reviews Neuroscience 18.5 (2017), Meta-Learning and the Neuroscience of Learning and Memory.

Kaiming He et al. 'Deep residual learning for image recognition'. In: Proceedings of the IEEE conference on computer vision and pattern recognition . 2016, pp. 770-778.

Saihui Hou et al. 'Learning a unified classifier incrementally via rebalancing'. In: Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition . 2019, pp. 831-839.

Steven CY Hung et al. 'Increasingly packing multiple facial-informatics modules in a unified deep-learning model via lifelong learning'. In: Proceedings of the 2019 on International Conference on Multimedia Retrieval . 2019, pp. 339-343.

Diederik P Kingma and Jimmy Ba. 'Adam: A method for stochastic optimization'. In: arXiv preprint arXiv:1412.6980 (2014).

James Kirkpatrick et al. 'Overcoming catastrophic forgetting in neural networks'. In: Proceedings of the national academy of sciences 114.13 (2017), pp. 3521-3526.

Dhireesha Kudithipudi et al. 'Biological underpinnings for lifelong learning machines'. In: Nature Machine Intelligence 4.3 (2022), pp. 196-210.

Jesse J Langille and Richard E Brown. 'The synaptic theory of memory: a historical survey and reconciliation of recent opposition'. In: Frontiers in systems neuroscience 12 (2018), p. 52.

Timothée Lesort, Andrei Stoian, and David Filliat. 'Regularization shortcomings for continual learning'. In: arXiv preprint arXiv:1912.03049 (2019).

Penelope A Lewis and Simon J Durrant. 'The role of active forgetting in learning and memory'. In: Frontiers in psychology 9 (2018), p. 1314.

Zhizhong Li and Derek Hoiem. 'Learning without forgetting'. In: IEEE transactions on pattern analysis and machine intelligence 40.12 (2017), pp. 2935-2947.

He Liu et al. 'Forgetting generates a novel state that is reactivatable'. In: Science advances 8.6 (2022), eabi9071.

Arun Mallya and Svetlana Lazebnik. 'Packnet: Adding multiple tasks to a single network by iterative pruning'. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition . 2018, pp. 7765-7773.

Eve Marder and Vatsala Thirumalai. 'Cellular, synaptic and network effects of neuromodulation'. In: Neural Networks 15.4-6 (2002), pp. 479-493.

Nicolas Y Masse, Gregory D Grant, and David J Freedman. 'Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization'. In: Proceedings of the National Academy of Sciences 115.44 (2018), E10467-E10475.

Michael McCloskey and Neal J Cohen. 'Catastrophic interference in connectionist networks: The sequential learning problem'. In: Psychology of learning and motivation . Vol. 24. Elsevier, 1989, pp. 109-165.

Martial Mermillod, Aurélia Bugaiska, and Patrick Bonin. The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects . 2013.

Vernon B Mountcastle. 'The columnar organization of the neocortex'. In: Brain 80.2 (1957), pp. 214-241.

German I Parisi et al. 'Continual lifelong learning with neural networks: A review'. In: Neural Networks 113 (2019), pp. 54-71.

Gyeong-Moon Park, Sahng-Min Yoo, and Jong-Hwan Kim. 'Convolutional neural network with developmental memory for continual learning'. In: IEEE Transactions on Neural Networks and Learning Systems 32.6 (2020), pp. 2691-2705.

Björn Rasch and Jan Born. 'Maintaining memories by reactivation'. In: Current Opinion in Neurobiology 17.6 (2007). Motor systems / Neurobiology of behaviour, pp. 698-703. ISSN: 0959-4388.

Roger Ratcliff. 'Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.' In: Psychological review 97.2 (1990), p. 285.

Anthony Robins. 'Catastrophic forgetting, rehearsal and pseudorehearsal'. In: Connection Science 7.2 (1995), pp. 123-146.

Andrei A. Rusu et al. Progressive Neural Networks . 2016.

Fahad Sarfraz, Elahe Arani, and Bahram Zonooz. 'Synergy between synaptic consolidation and experience replay for general continual learning'. In: Conference on Lifelong Learning Agents . PMLR. 2022, pp. 920-936.

Jonathan Schwarz et al. 'Progress & compress: A scalable framework for continual learning'. In: International Conference on Machine Learning . PMLR. 2018, pp. 4528-4537.

Tracey J Shors et al. 'Use it or lose it: how neurogenesis keeps the brain fit for learning'. In: Behavioural brain research 227.2 (2012), pp. 450-458.

Norman E Spear. 'Retrieval of memory in animals.' In: Psychological Review 80.3 (1973), p. 163.

Benjamin C Storm and Elizabeth L Bjork. 'Active forgetting of episodic memories: Is it time to rethink the 'retrieval failure' theory of forgetting?' In: British Journal of Psychology 110.3 (2019), pp. 549-571.

Rishabh Tiwari et al. 'GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning'. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 2022, pp. 99-108.

Jane X Wang et al. 'Neural modularity helps organisms evolve to learn new skills without forgetting old skills'. In: PLoS computational biology 13.2 (2017), e1005354.

Fu-Yun Wang et al. 'Foster: Feature boosting and compression for class-incremental learning'. In: European conference on computer vision . Springer. 2022, pp. 398-414.

Zhen Wang et al. 'Continual learning through retrieval and imagination'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 36. 8. 2022, pp. 8594-8602.

Zifeng Wang et al. 'SparCL: Sparse Continual Learning on the Edge'. In: Advances in Neural Information Processing Systems .

John T Wixted. 'The psychology and neuroscience of forgetting'. In: Annu. Rev. Psychol. 55 (2004), pp. 235-269.

Jaehong Yoon et al. 'Lifelong Learning with Dynamically Expandable Networks'. In: International Conference on Learning Representations .

Friedemann Zenke, Ben Poole, and Surya Ganguli. 'Continual learning through synaptic intelligence'. In: International conference on machine learning . PMLR. 2017, pp. 3987-3995.

Da-Wei Zhou et al. 'A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning'. In: The Eleventh International Conference on Learning Representations . 2022.

A Effect of Pruning Criteria

We seek to illustrate the effectiveness of different pruning criteria in TriRE. As explained in Section 3.1, the dense network is first pruned using k-WTA criteria, resulting in a subnetwork of the most activated neurons, and then this subnetwork is pruned using CWI criteria, resulting in a final extracted subnetwork at the end of Retain stage. Table 4 demonstrates the comparison of Class-IL accuracy between various pruning criteria, namely, magnitude-based, Fisher information-based, and CWIbased, across all three datasets. The idea behind magnitude pruning is that small valued weights impact the network's output less and can be safely pruned without significantly affecting performance. Fisher information-based pruning evaluates the importance of connections based on their contributions to the Fisher information matrix. Connections with low contributions, indicating less relevance or importance, are pruned or set to zero. However, both these criteria calculate the importance of weights within the current task, but do not consider the possibility of it being crucial for other tasks. On the other hand, CWI considers the significance of weights with respect to data saved in the rehearsal buffer as well, resulting in superior performance across all datasets.

B Model Analysis

B.1 Task Recency Bias

In any CL setting, the model entails learning on a few or no samples from previous tasks while aplenty of the most recent task [22]. This tilts learning toward the most recent task, resulting in decisions biased toward new classes and confusion among the old classes. However, the CL model should ideally have predictions distributed evenly across all tasks with the least possible recency bias. Figure 6 provides the confusion matrix for various CL models to evaluate the task recency bias. After training on Seq-CIFAR100 for 5 tasks with a buffer size of 200, the model is deemed to have correctly predicted the task label if it predicts any of the classes that make up the sample's true task label. As can be seen, ER and DER++ have a propensity to frequently classify the majority of samples as classes in the most recent task. However, TriRE's predictions are uniformly distributed across the diagonal. TriRE essentially decreases interference between tasks, captures task-specific information through extracted sub-networks, and produces the least recency bias.

1

2

3

4

5

1

2

3

4

5

Task Label

ER

1

2

3

4

5

Predicted Task Label

1

2

3

4

5

DER++

1

2

3

4

5

1

2

3

4

5

TriRE

0

500

1000

1500

2000

B.2 Stability-Plasticity Trade-off

A CL model is said to be stable if it can retain previously learned information, and plastic if it can effectively acquire new information. The stability-plasticity dilemma refers to an inherent trade-off

Accuracy (%)

80

60

40

20

0

0.0

0.2

0.4

0.6

0.8

1.0

Confidence

0

20

40

60

80

100

Top-1 Accuracy(%)

ER (65.64)

DER++ (41.29)

TriRE (27.86)

Stability

Plasticity

ER

DER++

TriRE

Tradeoff

in which the CL model masters one of these aspects at the expense of the other. Sarfraz et al. [44] introduced a trade-off measure that serves as an approximation of how the model balances its stability and plasticity. Once the model completes the final task T , its stability ( S ) is assessed by calculating the average performance across all preceding T -1 tasks as follows:

The plasticity of the model (P) is evaluated by computing the average performance of each task after its initial learning i.e.,

Thus, the trade-off measure determines the optimal balance between the stability ( S ) and the plasticity ( P ) of the model. This measure is calculated as the harmonic mean of S and P .

Figure 7 (Left) provides the stability-plasticity trade-off measure for different CL methods across different datasets for a buffer size of 200. ER and DER++ exhibit high plasticity, enabling them to rapidly adapt to new information. However, they lack the ability to effectively retain previously acquired knowledge. On the other hand, TriRE exhibits substantially high stability with low plasticity, resulting in a higher stability-plasticity trade-off.

B.3 Model Calibration

Ensuring the reliability of safety-critical CL systems necessitates the presence of a well-calibrated model. Calibration refers to the task of accurately predicting probability estimates that reflect the true likelihood of correctness. Miscalibration, on the other hand, refers to the disparity between confidence and accuracy expectations. To assess the degree of miscalibration in classification, the Expected Calibration Error (ECE) involves partitioning the predictions into bins of equal size and calculating the difference between the weighted average of accuracy and confidence within each bin. A lower ECE value indicates better calibration in the underlying models.

Figure 7 (Right) shows a comparison of different CL approaches using a calibration framework trained on Seq-CIFAR100 with a buffer size of 200. Well-calibrated CL systems accurately represent the true likelihood of accuracy (indicated by the red dashed line). Among the baselines, TriRE achieves the lowest ECE value and exhibits high calibration, demonstrating its effectiveness in minimizing task interference and reducing overconfidence in CL, thus enabling more informed decision making.

C Hyperparameter Selection

The hyperparameters required to replicate the results of TriRE can be found in Table 5. These hyperparameters were determined through a tuning process involving different random initializations and a small portion of the training set reserved for validation. All experiments were conducted using a batch size of 32 and trained for 50 epochs. TriRE was optimized using the Adam optimizer [24] implemented in PyTorch. Furthermore, the number of epochs allocated to each phase specified in Algorithm 1 was consistently set at a ratio of E 1 : E 2 : E 3 = 3 : 1 : 1 .

C.1 Hyperparameters Sensitivity Analysis

In order to showcase the robustness of TriRE to a choice of values for each hyperparameter, we conducted additional experiments by finetuning different hyperparameters. Specifically, we evaluated the performance of TriRE in the Seq-CIFAR100 dataset with a buffer size of 200 and 5 tasks. The results are visualized in the Figure 8. Our experimentation focused on adjusting hyperparameters such as sparsity, k-winner sparsity, and EMA model update frequency. Upon examining the graphs, it is evident that performance remains relatively stable across a range of values for these hyperparameters. This observation suggests that satisfactory results can be achieved without an exhaustive hyperparameter search.

0.1

0.3

0.5

0.7

0.9

Sparsity

35

40

45

Accuracy (%)

0.1

0.3

0.5

0.7

0.9

K-winner Sparsity

35

40

45

Accuracy (%)

0.1

0.3

0.5

0.7

0.9

Stable model update frequency

35

40

45

Accuracy (%)

D TriRE vs. CLS-ER: Commonalities and Differences

CLS-ER [6] emulates the interplay between fast and slow learning systems by incorporating two supplementary semantic memories that aggregate the weights of the working model in a stochastic manner via an exponential moving average. That is, CLS-ER operates with three models: the working model, the stable model, and the plastic model. Conversely, in the TriRE framework, the architecture consists of two models: the working model and a stable model (referred to as the EMA model). Therefore, the similarity is that both methods use the concept of progressively aggregating the weights of the working memory as it sequentially learns tasks allowing us to consolidate the information efficiently.

However, there are two main differences. Firstly, CLS-ER uses two supplementary semantic memories, whereas TriRE only utilizes one supplementary memory. Nevertheless, our results outperform CLS-ER (with one EMA model, referred to as Mean-ER in the mentioned paper) in all three datasets, as shown in Table 1. Secondly, CLS-ER only uses experience replay to tackle catastrophic forgetting whereas our method harmoniously integrates all the families of methods in CL, i.e. weight regularization, parameter isolation, and experience replay. That is, CLS-ER only focuses on one aspect of biological learning whereas we effectively consider multiple aspects.

E Comparison with Recent Baselines

Table 6 provides a comparison between Foster [51], Memo [57], and TriRE in Class-IL. The experiments are conducted on Seq-CIFAR100 with buffer size 200 with 5 tasks and 3 random seeds. We caution that the implementation details are slightly different: TriRE uses reservoir sampling with ResNet-18 as a backbone while both Foster and Memo employ modified ResNet-18 with custom sampling methods. Foster entails a dynamic expansion and compression to accommodate new information. Similarly, Memo entails expansion in the later layers while preserving generic information in the earlier layers. On the other hand, TriRE promotes generalization through a combination of weight and function space regularization, selective forgetting, and relearning thereby producing superior performance across tasks. As can be seen, TriRE outperforms both Foster and Memo.

Memo

TriRE

After Task 1

74.8

70.0

34.5

74.8

57.9

70.0

46.6

60.1

46.1

AfterTask3

74.8

57.9

31.5

70.0

46.6

37.7

44.3

49.5

56.1

AfterTask4-

74.8

57.9

31.5

23.3

70.0

46.6

37.7

33.1

33.6

42.1

50.0

57.5

After Task5

74.8

57.9

31.5

23.3

18.5

70.0

46.6

37.7

33.1

28.8

33.1

39.6

38.8

54.5

53.6

T1

T2

T3

T4

T5

T1

T2

T3

T4

T5

T1

T2

T3

T4

T5

F Training Cost

We compare the training times of various CL models that were taken into account for this work. We consider one method from each family of CL models. Table 7 specifically lists the training times required to learn a single task (the first) for 3 epochs on an NVIDIA RTX 2080 Ti for Seq-CIFAR100 dataset with a buffer size of 200. We conducted the experiment for 3 epochs across all CL methods considering the three-phased approach in TriRE and to maintain the fairness of comparison.

Table 7 shows that PNNs have the least training cost for our experimental setup. However, it is to be noted that in the case of PNNs, as the number of tasks increases, the model size also increases. This indicates that the training time also eventually increases towards the later tasks. Furthermore, we see that SI also costs less training time-wise. However, according to Table 1, the Class-IL accuracies of weight regularization methods mentioned are significantly lower for larger datasets. Interestingly, DER++ and TriRE exhibit similar training costs, yet our approach proves to be more effective in mitigating catastrophic forgetting and task interference. This can be attributed to our method's unique amalgamation of various CL method families, making it a more robust choice for CL tasks.

We also compared the training costs for each stage within TriRE's learning paradigm. We found that 'Retain' incurs the highest cost among the three, as it involves activation and weight pruning. The 'Revise' stage, where the joint distribution of past tasks and the current task is learned, follows. Finally, 'Rewinding' to a saved weight and learning for a few epochs to activate less active neurons requires significantly less training time compared to the other two stages.

G Datasets and Settings

We assess the effectiveness of our approach in two different types of CL scenarios: Class Incremental Learning (Class-IL) and Task Incremental Learning (Task-IL). In Task-IL and Class-IL, each task consists of a predetermined number of new classes that the model needs to learn. A CL model learns multiple tasks in sequence while being able to differentiate between all classes it has encountered so far. Task-IL is similar to Class-IL, but it has the advantage of having access to task labels during the inference process, making it one of the easiest scenarios.

To evaluate the performance of our method in Task-IL and Class-IL scenarios, we employ three different datasets: Seq-CIFAR10, Seq-CIFAR100, and Seq-TinyImageNet. These datasets are derived from CIFAR10, CIFAR100, and TinyImageNet, respectively. In Seq-CIFAR10, CIFAR10 is divided into five tasks, each task containing two classes. Similarly, in Seq-CIFAR100, CIFAR100 is divided into five tasks, each consisting of 20 classes. Lastly, in Seq-TinyImageNet, we partition TinyImageNet into ten tasks, each of which comprises 20 classes. These datasets are designed to introduce more challenging scenarios for a comprehensive analysis of various CL methods. By increasing the number of tasks or the number of classes per task, we can thoroughly examine the effectiveness of different CL approaches in handling different levels of complexity. Following [6], we used ResNet-18 as the backbone in all our experiments. The training process remains consistent for both Class-IL and Task-IL. To compare various state-of-the-art approaches, we present the average accuracy across all tasks encountered in Class-IL. According to Task-IL conventions, we take advantage of task identity and selectively deactivate neurons in the linear classifier that are not related to the current task.

Contrary to the common practice of using dense CL models, dynamic sparse methods take a different approach by starting with a sparse network and maintaining the same level of connection density throughout the learning procedure to incorporate sparsity into a CL model; it is necessary to disentangle interfering units to prevent forgetting and establish new pathways to encode new knowledge. This presents challenges when implementing batch normalization and residual connections for both the NISPA and CLNP methods. Consequently, these methods do not employ the ResNet-18 architecture. Instead, they opt for a simpler CNN architecture without 'skip connections' and batch normalization.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper concerns catastrophic forgetting in neural networks and draws from neural mechanisms contributing to the absence of such phenomena in the brain to propose a method to overcome CF through targeted neuron retraining, task knowledge revision, and enhanced learning for less active neurons. The authors present an impressive suite of experimental results, benchmarking their proposed method as superior or comparable to notable existing approaches in class- and task-IL scenarios and with further analysis of the different stages of the method.\n\nSTRENGTHS:\n+ The work creatively and effectively links to biological motivations toward continual learning (CL) capabilities.\n+ Fairly inclusive summary of existing CL approaches with contributive discussion as to their strengths and weaknesses toward integration in the proposed approach.\n+ Figures are well-crafted and easily understandable.\n+ Performance is comparable to SOTA methods.\n+ It's nice to see in the experimental data the significance of dynamic masking, retaining, rewinding.\n\nWEAKNESSES:\n- (As mentioned by the authors:) The work is primarily applicable only to CNN-based architectures in its present state, and additional tuning is required for the increased number of hyperparameters. While the paper is nice, one can see how TriRE won't work on transformers due to computational expense.\n\n- Regarding Related Work, the authors could elaborate on whether there are any other notable CL approaches that draw upon strengths of previous works as TriRE does and clarify how TriRE is superior/comparable (if applicable).\n\n- The authors could perhaps move discussion of limitations more heavily to the main text as opposed to the Appendix. The limitations discussed in Appendix C pertain to the practical applicability of the work and thus have merit for inclusion.\n\nQUESTIONS:\n* I am curious as to why rewinding improves accuracy, perhaps this could be another direction for exploration.\n\n* Small clarifications regarding the method itself:\n\t* Learn: finite replay buffer with loss-aware experience rehearsal?\n\t* Retain: parameter isolation where important connections and weights are learned?\n\t* Revise: How are the subnetworks combined? I wonder if there is a way to specifically focus on free neurons when learning new tasks, so during the combination stage they are orthogonal to the combined network?\n\t* Rewind: Does relearning target specific unactivated neurons/parameters? This step is perhaps redundant if simply taking weights from a few epochs before and training more on new mini-batches to relearn.\n\nMinor:\n* Lines 159-160: Put “k” in math mode (top-k, k-winner)\n* Line 215: “Retatin” -> “Retain”\n\nLIMITATIONS:\nThe authors touch upon the limitations of their work in the main text and more so in the Appendix.\n\nSocial impact — not directly applicable but is mentioned in broader impacts discussion.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-28"}

comments: SUMMARY:
The paper proposed a new continual learning (CL) method that updates partial neurons while rewinding other neurons to the previously stored weights. To do this, the authors utilize sparsity constraints to the network weights and select highly activated neurons. Consequently, the proposed method outperforms the baselines.

STRENGTHS:
1. The proposed method is generally technically sound.
2. The ablation study in Table 2 helps us understand the contribution of each component.

WEAKNESSES:
1. The proposed method has three components, retain, revise, and rewind. The key contribution part is rewind while other parts are periphery or have no novelty.

2. What is the overhead cost for rewinding, such as training time,  memory cost for holding previous weights, and the performance drop for the current task?

3. There are 7 different parameters and it seems that it is arbitrarily chosen to make the best performance for each task (Appendix D) as the authors mentioned in Appendix C, which is appreciated.

4. The statement in L211-212, “This is helpful because studies show that in the human brain, less active neurons follow a ‘use-it-or-lose-it’ philosophy” is difficult to understand why it is “helpful.” It seems that the authors blindly think that “human-like” is good.

5. It will be better if the authors include recent baselines [1-2] and check whether the proposed method is state-of-the-art in the current setting.

[1] Fu-Yun, et al. "Foster: Feature boosting and compression for class-incremental learning." ECCV. 2022.

[2] Zhou, Da-Wei, et al. "A model or 603 exemplars: Towards memory-efficient class-incremental learning." ICLR. 2023

QUESTIONS:
See Weakness

LIMITATIONS:
See Weakness
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method for avoiding catastrophic forgetting in continual supervised learning that operates in three stages. In the first stage (retain), a subnetwork for the current task is identified by detecting most/least activated neurons of the main network. In the second stage (revise), both the main network and the extracted subnetwork are re-trained / fine-tuned with examples from the current task and examples from a replay memory. The subnetwork is later integrated into the main network. In the final stage (rewind), the weights belonging to non-cumulative subnetworks are finetuned for a few epochs. The paper presents experimental results on three benchmark continual learning datasets, in both the task-incremental and class-incremental settings, by also comparing with counterpart methods in the area.\n\n--Rebuttal--\nI read the rebuttal, along with other reviewers' comments, and increased my score accordingly (to borderline accept) during the rebuttal phase.\n\nSTRENGTHS:\n- The originality of the paper relies on a new multi-stage method to tackle catastrophic forgetting which relies on the important concepts of modularity and example replay. \n\n- The paper is in general easy to follow. \n\n- The paper is properly aligned with literature in the area.\n\nWEAKNESSES:\n- Although the proposed approach touches on a lot of important points in catastrophic forgetting and continual learning, it seems to be a mere combination of existing ideas into a so-called three-stage approach, therefore undermining the novelty of the proposed mechanism. Furthermore, what seem to be novel aspects of the proposed approach are not explained in full detail, and therefore it is difficult to estimate their impact. For instance, in the retain phase, activation pruning is performed by using the existing heterogeneous dropout, while weight pruning used the existing CWI approach. For activation pruning, in lines 158-159, it is stated that a counter is used to determine the top k-winner activations. However, not technical details are provided regarding how this counter works. In the revise stage, lines 191-193 mention that the learning rate is \"considerably\" reduced at this phase; however, no insights onto how to decide what the reduction rate to use. For this same stage, in lines 195-199 it is mentioned that the S and S_{t} subnetworks are eventually merged; however, no technical details of how this merge occurs are provided. Finally for the rewind stage, in lines 209-214, it is unclear how to decide for how many epochs (k?) to rewind the network, what are the criteria, the requirements to decide this. \n\n- The experiments lack analysis of computational cost (e.g. memory consumption, training time). As can be inferred from Algorithm 1 and section 3, the multi-stage procedure proposed in this paper involves several calculations and passes through the network/subnetwork. What is the cost of this multi-stage procedure? How does this compare to counterpart methods? \n\n- In the experiments, some results are left unexplained while some others have an ambiguous explanation. For example, why does the proposed method seem to underperform on Task-IL for two of the three datasets evaluated? The explanation for the \"miserably\" (please change this word) performance of methods such as LwF and SI provided in lines 240-246 looks ambiguous and incorrect, since these methods have been used previously in Task-IL. \n\n- From the results in Figure 3, it can be implied that the proposed method actually improves performance as the number of tasks increases. Is that the case? Did you run multiple task orders? Are tasks 10 to 20 naturally easier?\n\nQUESTIONS:\nPlease refer to questions listed in the \"weaknesses\" section.\n\nLIMITATIONS:\nLimitations in terms of the network architecture used are clearly stated in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article introduces a multi-faceted approach to continual learning. combining features of many other approaches and relying on a three-phase training process, such that, as each new task in a sequence of tasks is encountered, subsets of weights in the network are 'retained', 'revised' or 'rewound' to values that maintain old knowledge while beginning to learn the new task, facilitate integration of weights important for the new task with those important for previous tasks, and maintain plasticity for future learning, respectively.  The authors find advantages for this approach relative to a large range of other approaches, especially with more complex task settings, and perform several ablations helping to clarify the roles of the three stages, along with a few other explorations.\n\nSTRENGTHS:\nThe paper cites a wide range of relevant related work and situates its approach within the context established by other approaches.  The other approaches considered seem fairly extensively sampled.  I appreciated the consideration of the effects of the different ablations which show the importance of the 'rewind' phase as well as the stability plasticity analysis shown in the appendix.\n\nWEAKNESSES:\nThe work seems thoughtful in relation to the relevant existing literature but it may be that the CL paradigm as explored in this project is in need of re-framing, if new breakthroughs are to be achieved.  While the advantages relative to many of the baselines seem clear, I felt that the additional complexity of the TriRE scheme made the gaps between it and some of the other approaches relatively difficult to get excited about.  Some of the features that seem relatively important, such as CWI, are direct importations from previous work, making it difficult to assess whether TriRE really advances our understanding, given its additional complexity compared to most other work.\n\nTo me the greatest weakness of the approach is one that appears to be widely shared across the CL literature:  This is the fact that this and all of the cited work rely on triggering complex meta-processes at the boundaries between tasks in ways that seem very distant from what might occur in biological networks on in naturalistic continual learning settings where task boundaries are not announced.  \n\nWe need approaches that can address the continual learning problem when tasks shift more gradually and task boundaries are not available.  The approach also exploits features limited to the setting in which tasks are defined by the fact that they use completely non-overlapping sets of output neurons (i.e. distinct class labels).  For example the Loss on the new task items in Eq 4 only considers the class labels relevant to the current task.  Finally, continual learning as explored in this literature is restricted to tasks with no intrinsically cumulative structure.  Catastrophic forgetting also occurs in settings where new learning could productively build on structure learned in previously learned tasks (in the way that addition builds on counting, multiplication builds on addition, etc).  The whole body of work thus seems narrowly focused on paradigms of limited general interest.\n\nQUESTIONS:\nI had difficulty understanding the CLS-ER results, which are almost as good as TriRE.  A clearer understanding of what the CLS-ER model shares with TriRE and how it differs from it would be useful in evaluating the contributions of TriRE over and about the use of an EMA model smoothing weights across tasks.  The note in Table 1 is insufficient for me to understand the relationship between CLS-ER and TriRE.\n\nLIMITATIONS:\nI initially wrote: \"The paper mentions limitations in the appendix.  As stated they hint at some of the same limitations I see in the work as described above.  Per the importance placed on limitations in instructions to authors and reviewers, these ought to be placed in the main text.\"\n\nThe authors have addressed this concern through the discussion during the rebuttal, leading me to increase my rating to 'Accept'.  While I think the weaknesses described above still apply, they are, as I have said, largely shared by a mini-paradigm in which these issues have been addressed.  I hope the authors will be encouraged by acceptance of this work to work on a new paradigm.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}