DualNet: Continual Learning, Fast and Slow
Authors: Quang Pham
Chenghao Liu
Steven HOI
Venue: NeurIPS 2021 Poster (2021)
Keywords: Continual learning
fast and slow learning

Abstract: According to Complementary Learning Systems (CLS) theory~\cite{mcclelland1995there} in neuroscience, humans do effective \emph{continual learning} through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics and individual experiences, and a slow learning system located in the neocortex for the gradual acquisition of structured knowledge about the environment. Motivated by this theory, we propose a novel continual learning framework named ``DualNet", which comprises a fast learning system for supervised learning of pattern-separated representation from specific tasks and a slow learning system for unsupervised representation learning of task-agnostic general representation via a Self-Supervised Learning (SSL) technique. The two fast and slow learning systems are complementary and work seamlessly in a holistic continual learning framework. Our extensive experiments on two challenging continual learning benchmarks of CORE50 and miniImageNet show that DualNet outperforms state-of-the-art continual learning methods by a large margin. We further conduct ablation studies of different SSL objectives to validate DualNet's efficacy, robustness, and scalability. Code is publicly available at \url{https://github.com/phquang/DualNet}.

DualNet: Continual Learning, Fast and Slow
Quang Pham', Chenghao Liu", Steven C.H. Hoi 1.2 Singapore Management University hqpham 20170smu. edu. sg
Salesforce Research Asia
{chenghao.liv) shoi @sallsseeeeoooccoom
Abstract
According to Complementary Learning Systems (CLS) theory [37] in neuroscience, humans do effective continual learning through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics and individual experiences, and slow learning system located in the neocortex for the gradual acquisition of structured knowledge about the environment. Motivated by this theory, we propose novel continual learning framework named "DualNet", which comprises fast learning system for supervised learning of pattern-separaed representation from specific tasks and slow learning system for unsupervised rep- resentation learning of task-agnostic general representation via Self-Supervised Learning (SSL) technique. The two fast and slow learning systems are complemen- tary and work seamlessly in a holistic continual learning framework. Our extensive experiments two challenging continual learning benchmarks of CORE50 and minilmageNeet show that DualNet outperforms state-of- sitte-of-the the-art continual learning methods by a large margin. We further conduct ablation studies of different SSL ob- jectives to validate DualNet's efficacy, robustness, and scalability. Code is publicly available https: // //i thub. com/ haamm//uuag/////// phquang/DualNes DualNet.
Introduction
Humans have the remarkable ability to learn and accumulate knowledge over their lifetime to perform different cognitive tasks. Interestingly, such capability is attributed to the complex interactions among different interconnected brain regions [14]. One prominent model is the Complementary Learning Systems (CLS) theory [37, 30] which suggests the brain achieve such behaviors via two learning systems of the "hippocampus" and the "neocortex." " Particularly, the hippocampus focuses fast learning of pattern-separated representation of specific experiences. Via the memory consolidation process, the hippocampus's memories are transferred to the neocortex over time to form more general representation that supports long-term retention and generalization experiences. The two fast and slow learning systems always interact to facilitate fast learning and long-term remembering. Although deep neural networks have achieved impressive results [31], they often require having access to large amount of i.i.d data while performing poorly on the continual learning scenarios over streams of task [19, 29, 36]. Therefore, the main focus of this study is exploring how the CLS theory can motivate a general continual learning framework with better trade-off between alleviating catastrophic forgetting and facilitating knowledge transfer.
In literature, several continual learning strategies are inspired from the CLS theory principles, from using the episodic memory [36] to improving the representation [26, 44]. However, such techniques mostly use a single backbone to model both the the hippocampus and neocortex, which binds two representation types into the same network. Moreover, such networks trained to minimize the supervised loss, they lack separate and specific slow learning component that supports general representation learning. During continual learning, the representation obtained by repeatedly
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Neal
Data stream
represemiation
Fast net
Classifier
Supervised Loss
M
Supervised learning
The Fast learner's 's learning process
Memory consolidation
Feature adaptation
The Slow learner's leamning process
M
Memory
Slow net
SSL Loss
Self-supervised
Store data learning
experience replay
Figure Overview of the DualNet' architecture. DualNet consists of (i) slow learner (in blue) that learns representation by optimizing an SSL loss using samples from the memory, and (ii) fast learner (in orange) that adapts the slow net's representation for quick knowledge acquisition of labeled data. Both learners can be trained synchronously.
performing supervised learning on a small amount of memory data can be prone overfitting and may not generalize well across tasks. Consider that in continual learning, unsupervised representation [20, 40] often more resisting to forgetting compared supervised representation, which yields little improvements [25]; we propose to decouple representation learning from the supervised learning into two separate systems. To achieve this goal, in analogy to the slow learning system in neocortex, we propose to implement the slow general representation learning system using Self-Supervised Learning (SSL.) [39]. Note that recent SSL works focus on the pre-training phase, which is not trivial to apply for continual learning as it is extensive in both storage and computational cost [28]. We argue that SSL should be incorporated into the continual learning process while decoupling from the supervised learning phase into two separate systems. Consequently, the SSL's slow representation is more general and can capture the intrinsic characteristics from data, which facilitates better generalization to both old and new tasks.
Inspired by the CLS theory [37]. we propose DualNet (for Dual Networks, depicted in Figure 1), novel framework for continual learning comprising two complementary learning systems: a slow learner that learns generic features via self-supervised representation learning, and a fast learner that adapts the slow learner's features to quickly attain knowledge from labeled samples via novel per-sample based adaptation mechanism. During the supervised learning phase, an incoming labeled sample triggers the fast learner to make predictions by querying and adapting the slow learner's representation. Then, the incurred loss will backpropagated through both learners to consolidate the current supervised learning pattern for long-term retention. Concurrently, the slow learner is always trained in the background by minimizing an SSL objective using only the memory data. Therefore, the slow and fast networks learning completely synchronous, allowing DualNet to continue to improve its representation power even in practical scenarios where labeled data delayed [13] even limited, which will demonstrate in Section 4.6. Lastly, focus on developing DualNet for the online continual learning settings [36, 2] since it is more challenging to optimize deep networks in such scenarios [51, 3]. In the batch continual learning setting [46], the model allowed to revisit data within the current task and can achieve good representations when learning the current task.
In summary, our work makes the following contributions:
1. We propose DualNet, novel continual learning framework comprising two key components of fast and slow learning systems, which closely models the CLS theory.
2. We propose a novel learning paradigm for DualNet to efficiently decouple the representation learning from supervised learning. Specifically, the slow learner is trained in the background with SSL to maintain general representation. Concurrently, the fast learner is equipped with novel adaptation mechanism to quickly capture knowledge. Notably, unlike existing adaptation techniques, our proposed mechanism does not require the task identifiers.
3. We conduct extensive experiments to demonstrate DualNet's s efficacy, robustness to the slow learner's 's objectives, and scalability to the computational resources.

---

2 Method
2.1 Setting and Notations
We consider the online continual learning setting [36, 8] over a continuum of data D = far t,, y: Ji where each instance is labeled sample {æ,,84 with an optional task identifier t,. Each labeled data sample is drawn from an underlying distribution Pt(X, X.Y, Y) that represents a task and can suddenly change pt+1, indicating task switch. When the task identifier is given an input, the setting follows the task-aware setting where only the corresponding classifier is selected to make a prediction [36]. When the task identifier is not provided, the model has a shared classifier for all classes observed so far, which follows the task-free setting [7. 2]. We consider both scenarios in our experiments. A common continual learning strategy is employing an episodic memory M to store a subset of observed data and interleave them when learning the current samples [36, 9]. From M, we use M denote randomly sampled mini-batch. and MA, M obtained by applying two different data transformations. Lastly, we denote Φ as the parameter of the slow network that learns general representation from the input data and the parameter of the fast network that learns the transformation coefficients.
denote two views of M
2.2 DualNet Architecture
DualNet learns the data representation independent of the task's label, which allows for better generalization capabilities across tasks in the continual learning scenario. The model consists two main learning modules (Figure 1): (i) the slow learner is responsible for learning general, task- agnostic representation; and (ii) the fast learner learns with labeled data from the continuum to quickly capture the new information and then consolidate the knowledge to the slow learner.
DualNet learning can be broken down into two synchronous phases. First, the self-supervised learning phase in which the slow learner optimizes a Self-Supervised Learning (SSL.) objective using unlabeled data from the episodic memory M.. Second, the supervised learning phase happens whenever a labeled sample arrives, which triggers the fast learner to first query the representation from the slow learner and adapt it to learn this sample. The incurred loss will be backpropagated into both learners for supervised knowledge consolidation. Additionally, the fast learner's adaptation is per-sample- based and does not require additional information such as the task identifiers. Note that DualNet uses the same episodic memory's 's budget as other methods to store the samples and their labels, but the slow learner only requires the samples while the fast learner uses both samples and their labels.
2.3 The Slow Learner
The slow learner is standard backbone network trained to optimize an SSL loss, denoted by LSSL- As a result, any SSL objectives can be applied in this step. However, to minimize the additional computational resources while ensuring a general representation. we only consider the SSL loss that (i) does not require additional memory unit (such as the negative queue in MoCo [23]). (ii)
does not always maintain an additional copy of the network (such as BYOL [21]), and (iii) does not use handcrafted pretext losses (such as RotNet [16] or JiGEN [6]). Therefore, we consider Barlow Twins [59], a recent -tate-of-theeart SSL method that achieved promising results with minimal computational overheads. Formally, Barlow Twins requires two views A and M by applying two different data transformations to a batch of images M sampled from the memory. The augmented data are then passed to the slow net & to obtain two representations Z and B The Barlow Twins loss is defined as:
LST A Ea C2) ABT ΣΣ Ch iti
where ÄgT is trade-off factor, and C is the cross-correlation matrix between 2d and ZB:
E, E V VEDCAAY v EB
Cipy A
with denotes the mini-batch index and i are the vector dimension indices. Intuitively, by optimiz- ing the cross-correlation matrix to be identity, Barlow Twins enforces the network to learn essential information that invariant to the distortions (unit elements on the diagonal) while eliminating the redundancy information in the data (zero element elsewhere). In our implementation, we follow the
3

---

DualNet
Gating mask
Element-vee multiplicattin
Fast learner
Cony, Layer
Conv. Layer
FC
Layer
Adapted features
Slow learner
Residual
Residual Block 4
Block
Slow feature
Final Prediction
Image
A demonstration of the slow and fast learners' interaction during the supervised learning inference on a standard ResNet [22] backbone.
Figure 2:
standard practice in SSL employ projector top of the slow network's ' last layer to obtain the representations Z _ For supervised learning with the fast network, which will be described in Section 2.4,
the slow network's last layer as the representation Z.
In most SSL methods, the LARS optimizer [58] is employed for distributed training across many devices, which takes advantage of large amount of unlabeled data. However, in continual learning, the episodic memory only stores a small number of samples, which are always changing because of the memory updating mechanism. As result, the data distribution in the episodic memory always drifts throughout learning, and the SSL loss DualNet presents different challenges compared to the traditional SSL optimization. Particularly, although the SSL objective in continual learning can be casily optimized using one device, we need to quickly capture the knowledge of the currently stored samples before the newer ones replace them. In this work, propose optimize the slow learner using the Look-ahead optimizer [61], which performs the following updates:
Φ. t - eV Φx LBT,, with 6o + & and k = I....,
(3)
Φ 3(xx - Φ),
(4)
where 3 is the Look-ahead's learning rate and is the Look-ahead's SGD learning rate. As a special case of K 1, the optimization reduces the traditional optimization of LET using SGD. By performing
updates using standard SGD optimizer, the look-ahead weight Φx is used to perform a momentum update for the original slow learner Φ. As a result, the slow learner optimization can explore regions that are undiscovered by the traditional optimizer and enjoys faster training convergence [61]. Note that SSL focuses minimizing the training loss rather than generalizing this loss unseen samples, and the learned representation requires be adapted to perform well on downstream task. Therefore, such properties make the Look-ahead optimizer more suitable choice over the standard SGD to train the slow learner.
Lastly, we emphasize that although we choose to use Barlow Twins as the SSL objective in this work, DualNet is compatible with any existing methods in the literature, which will explore empirically in Section 4.3. Moreover, we can always train the slow learner in the background by optimizing Equation synchronously with the continual learning of the fast learner, which we will detail in the following section.
2.4
The Fast Learner
Given labeled sample {x, y}. the fast learner's goal is utilizing the slow leamner's representation to quickly learn this sample via an adaptation mechanism. this work, we propose a novel context-free adaptation mechanism by extending and improving the thannel-wise transformation [42, 43] to the general continual learning setting. Particularly, instead of generating the transformation coefficients based on the task idennieeeiier, we propose to train the fast learner to learn such coefficients from the raw pixels in the image æ. Importantly, the transformation pixel-wise instead of channel-win wise to compensate for the missing input of task identifiers. Formally, let (haHt. be the feature maps from the slow learner's layers on the image æ, e.g. hy, h22 ha, his are outputs from four residual blocks in ResNets [22], our goal is to obtain the adapted feature h1, conditioned on the image Therefore,

---

design the fast learner as simple CNN with L layers, and the adapted feature h2g is obtained as my -9.,(K(11)) with 11 = a and I....,L A -/y @ mi, VI - -I.....,,4
(5)
(6)
where @ denotes the element-wise multiplication, 90,1 denotes the 1-th layer's output from the fast network 9 and has the same dimension as the corresponding slow feature /T. The final layer's transformed feature hi will be fed into classifier for prediction. Thanks the simplicity of the transformation, the fast learner is weight but still can take advantage of the slow learner's 's rich representation. As result, the fast network can quickly capture knowledge in the data stream, which is suitable for online continual learning. Figure 2 illustrates the fast and slow learners' interaction during the supervised learning or inference phase.
The Fast Learner's Objective To further facilitate the fast learner's knowledge acquisition during supervised learning, we also mix the current sample with previous data in the episodic memory, which is a form of experience replay (ER). Particularly, given the incoming labeled sample (æ: y) and mini-batch of memory data M belonging to a past task k, we consider the ER with soft label loss [53] for the supervised learning phase as:
MM Σ
yk
L. = CD((((ulNNee))))) v)
CE(T(Ú)) + AE. DRL. JM
π
(7)
where CÉ is the cross-entropy loss DKL is the KL-divergence, is the DualNet's prediction, dN is snapshot of the model's logit (the fast learner's 's prediction) of the corresponding sample at the end of task k, sT-)) is the softmax function with temperature and Aer is the trade-off factor between the soft and hard labels in the training loss. Similar to [43, 5], Equation requires minimal additional memory to store the soft label ý in conjunction with the image and the hard label y.
3 Related Work
3.1 Continual learning
The CLS theory has inspired many existing continual learning methods in different settings [41, 12, 27], which can be broadly categorized into groups. First, dynamic architecture methods aims at having a separate subnetwork for each task, thus eliminating catastrophic forgetting to a great extend. The task-specific network can be identified simply allocating new parameters [50, 57, 32]. finding a configuration of existing blocks or activations in the backbone [17, 52], or generating the whole network conditioning on the task identifier [56]. While achieving strong performance, they are often expensive to train and do not work well on the online setting [8] because of the lack of knowledge transfer mechanism across tasks. In the second category of fixed architecture methods, learning is regularized by employing a memory to store information of previous tasks. In regularization-bue methods, the memory stores the previous parameters and their importance estimations{29 [29, 60, 1, 49), which regulates training of newer tasks to avoid changing crucial parameters of older tasks. Recent works have demonstrated that the experience replay (ER) principle [33] is an effective approach and its variants [36, 8, 48, 34, 53, 5] have achieved promising results. Notetably, MER [48] extends the Reptile algorithm [38] for continual learning. Although achieving promising results on simple datasets such as MNIST, MER was later shown to be outperformed by the standard ER strategy on more challenging benchmarks based on CIFAR and minilmageNee [9]. Recently, CTN [43] was proposed to bridge the gap between the two approaches by having fixed backbone network that can model task specific features via the controller that models higher task-level information.
We argue that most existing methods fail to capture the CLS theory's 's fast and slow learning principle by coupling both representation types into one backbone network. Existing CLS-inspired method of FearNet [27] assumes having a powerful pre-trained representation and focuses on the memory management strategy. In contrast, DualNet explicitly maintains two separate systems, which facilitates slow representation learning to support generalization across tasks while allowing efficient and fast knowledge acquisition during continual learning.
3.2 Representation Learning for Continual Learning
Representation learning has been an important research field in Machine Learning and Deep Learn- ing [16, 4]. Recent works demonstrated that general representation could transfer well to finetune
5

---

on many downstream tasks [39], or generalize well under limited training samples [18]. For con- tinual learning, extensive efforts have been devoted to learning a generic representation that can alleviate forgetting while facilitating knowledge transfer. The representation can be learned either by supervised learning [46], unsupervised learning [20, 40, 44], or meta (pre-) Jtraining [26, 24]. While unsupervised and meta training have shown promising results on simple datasets such as MNIST and Omniglot, they lack the scalability to real-world benchmarks. In contrast, our DualNet decouples the representation learning into the slow learner, which is scalable in practice by training synchronously with the supervised learning phase. Moreover, our work incorporates self-supervised representation learning into the continual learning process and does not require any pre-training steps.
3.3 Feature Adaptation
Feature adaptation allows the feature to quickly change and adapt [42]. Existing continual learning methods have explored the use of task identifiers [43, 56, 52] or the memory data [24] to support the fast adaptation. While the task identifier context is powerful since it provides additional information regarding the task of interest, such approaches are limited the task-aware setting or require inferring the underlying task, which can be challenging in practice. On the other hand, data-based context conditioning is useful in incorporating information of similar samples to the current query and has found success beyond continual learning [18, 47, 15, 45]. However, we argue that naively adopting this approach is not practical for real-world continual learning because the model always performs the full forward/backwart computation for each query instance, which reduces the inference speed and defeats the purpose of fast adaptation. Moreover, the predictions are not deterministic because of the dependency on the data chosen for finetuning. For DualNet, feature adaptation plays an important role in the interaction between the fast and slow learners. We address the limitations of existing techniques by developing a novel mechanism that allows the fast learner to efficiently utilize the slow representation without additional information about the task identifiers.
4 Experiments
Throughout this section, we compare DualNet against competitive continual learning approaches with focus on the online scenario [36]. Our goal of the experiments is to investigate the following hypotheses: (i) DualNet's representation learning is helpful for continual learning: (ii) DualNet can continuously improve its performance via self-training in the background without any incoming data samples; and (iii) DualNet presents general framework to unify representation learning and continual learning seamlessly and is robust to the choice of the self-supervised learning objective.
4.1 Experimental Setups
Our experiments follow the online continual learning under both the task-aware and task-free settings.
Benchmarks We consider the "Split" continual learning benchmarks constructed from the mini- ImageNet [54] and CORE50 dataset [35] with three validation tasks and 17, 10 continual learning tasks, respectively. Each task is created by randomly sampling without replacement five classes from the original dataset. For the task-aware (TA) protocol, the task identifier is available, and only the corresponding classifier is selected for evaluation. In contrast, the task-identifiers are not given in the task-free freee (TF) protocol, and the models have to predict all classes observed far. We run the experiments five times and report the averaged accuracy of all tasks/classes at the end of training [36] (ACC), the forgetting measure [7] (FM), and the learning accuracy (LA) [48].
Baselines We compare our DualNet with suite of state-of-the-an continual learning methods. First, we consider ER [9], simple experience replay method that works consistently well across benchmarks. Then we include DER++ [5], an ER variant that augments ER with a & loss on the soft labels. We also compare with CTN [43] a recent state-of-the- method on the online task- aware setting. For all methods, the ppper-parameters are selected by performing grid-search on the cross-validation tasks.
Architecture We use full ResNet18 [22] as the backbone in all experiments. addition, we construct the DualNet's s fast learner as follows: the fast learner has the same number of convolutional layers as the number of residual blocks in the slow learners. A residual block and its corresponding fast learner's 's layer will have the same output dimensions. With this configuration, the fast learner's architecture is uniquely determined by the slow learner's network. Lastly, all networks our experiments are trained from scratch.

---

Table Evaluation metries on the Split minilmageNet and CORESO benchmarks. All methods use an episodic memory of 50 samples per task in the TA setting, and 100 samples per class in the TF setting. The "Aug" suffix denotes using data augmentation during training
Split minilmageNet--T
Split minilmageNNt- TF
Method
ACC(1)
FM(()
LA(1)
ACC(1)
FM(()
LA(t)
ER
08.24
9.22++0.8
65.36-10.71
2..1
49.04=11.50 28.56+11100
ER-Aug
59.80=1.51
4.68=1211
58.9410.69
27.94=2.44
29.3613.23
54.02=1.02
DER+
62.32=0.78
7.00=0.81
67.30+0.57
1..16=1.99
34.56:22.48
19.54=1.53
DER++AAu
63,48=0.98
4.01
62.17:0.52
28.
3..70
62.70=0041
CTN
65.82-1059
3.02
6..43:21.37
N/A
N/A
N/A
CTN-Aug
1..04+1.23
3.94+0.98
69.84+0.78
N/A
N/A
N/A
DualNet
7..20=0.68
3.86=1001
74.12:+0.12
36.86=1.36
28.63+22226 63.46-1..77
CORESO-TA
CORE5O-TF
Method
ACC(t)
FM(4)
LA(1)
ACC())
FM(L)
LA(1)
ER
1.37
9.10+0.80
48.18:00.8
21.80-0.70
14.42++1.10
33.94
ER-Aug
2..16+22.05
5.72+0.02
47.83++1.61
2..34+0.74
37.94+0091 15.28+00.33
DER
46.62+40.6
4.66+00.6
48.32++0.69
22.84+0.84
13.10+00.0
34.50+0.81
DER++-Aug
45.12=0.668
5.02
47.67+0.08
28. 10-0.80
10.43+2.10
36.16110119
CTN
54.17=00.5
5,50
55.3240.34
N/A
N/A
N/A
CTN-Aug
53.40=11.37
6,1811.61
55.40:1.47
N/A
N/A
N/A
DualNet
1..64
4.43 58.8640.66
38.76=1.52
8.06
1.67
Training In the supervised learning phase, all methods are optimized by the (SGD) optimizer over one epoch with mini-batch size 10 and 32 on the Split minilmageNee and CORE50 benchmarks respectively [43]. In the representation learning phase, we use the Look-ahead optimizer [61] train the DualNet's s slow learner as described in Section 2.3. We employ an episodic memory with 50 samples per task and the Ring-buffer management strategy [36] in the task-aware setting. In the task- free setting, the memory is implemented as reservoir buffer [55] with 100 samples per class. We simulate the synchronous training property DualNet by training the slow learner with n iterations using the episodic memory data before observing mini-batch of labeled data.
Data pre-processing DualNet's slow learner follows the data transformations used in Bar- lwwTwins [59]. For the supervised learning phase, we consider two settings. First, the standard data pre-process pre-processing of no data augmentation during both training and evaluation. Second, we also train the baselines with data augmentation for fair comparison. However, we observe the data transformation in [59] is too aggressive; therefore, only implement the random cropping and flipping for the supervised training phase. In all settings, no data augmentation is applied during inference.
4.2 Results on Online Continual Learning Benchmarks
Table 1 reports the evaluation metrics on the CORE50 and Split minilmageNet benchmarks, where omit CTN's performance the task-free setting since it is strictly task-aware method. Our DualNet's slow learner optimizes the Barlow Twins objective for n 3 iterations between every incoming mini-batch of labeled data. Data augmentation creates more samples train the models and provides improvements to all baselines on all benchmarks. Consistent with previous studies, we observe that DER++ performs slightly better than ER thanks to its soft-label loss. Similarly, CTN can perform better than both ER and DER++ because of its ability to model task-specific features. Overall, our DualNet consistently outperforms other baselines by large margin, even with the data augmentation propagated to their training. Specifically, DualNet is more resistant to catastrophic forgetting (lower FM) while greatly facilitating knowledge transfer (higher LA), which results in better overall performance indicated by higher ACC. Since our DualNet has a similar supervised objective as DER++, this result shows that the DualNet's decoupled representations and its fast adaptation mechanism are beneficial to continual learning.
4.3 Ablation Study of Slow Learner Objectives and Optimizers
We now study the effects of the slow learner's objective and optimizer on the final performance of DualNet. We consider several objectives train the slow learner. First, we consider the classification
7

---

Table DualNet's performance under different slow learner objective and optimizers on the Split minilmageNet- TA benchmark
SGD
Look-ahead
Objective
ACC(t)
FM(()
LA(t)
ACC(T)
FM())
LA(1)
Barlow Twins
64.20:1.2.77
4.79:11.19
64.83+1.67
73.20
3.86111.01
74.12:10.12
SimCLR
11.49+1.001
4.23+:0.66
7..64
72.13+00.44
4.13++0.2
73.09=0016
SimSiam
70.55=10.98
1.93
71.90=0.65
71.9410.44
4.21
72.93
BYOL
69.76-12.12
4.23::1411
70.33=0.87
71.73:2.447
3.9621.62
72.06-0.28
Classification
68.50+1.67
5.53+1.67
72.93=1.10
70.9611.08
6.33+0.28
13.92=1.14
Table 3: Performance of DualNet with different self-supervised learning iterations n on the Split manelM benchmarks
Split minilmageNet-TAA
Split minilmageNet-TF
ACC(1)
FM(J)
LA(1)
ACC(t)
FM(L)
LA(1)
72.26=0.71
3.80=10.69
73.16#1.51
33.40:13.28
32.86::3.06
63.96::0.53
3
03.20=#0.68
3.86+1.01
04.12:00.1
36.86=1..36
63.46=1197 28.63:2.26
10
1.00
3.67
0..6
36.43:1177
30.92+2.166
65.33::052
20
1..5
3.48:0045
7..60=0065
38.56:1.91
27.96=1.71
64.06:10.67
loss to train the slow net, which reduces DualNet's representation learning to supervised learning. Second, we consider various contrastive SSL. losses, including SimCLR [10], SimSiam [11], and BYO [21]. We consider the Split minilmageNet-TIA and TF benchmark with 50 memory slots per task and optimize each objective using the SGD and Look-ahead optimizers. Table 2 reports the result of this experiment. In general, observe that SSL objectives achieve better performance than the classification loss. Moreover, the Look-ahead optimizer consistently improves the performances on all objectives compared to the SGD optimizer. This result shows that our DualNet's design is general and can work well on different slow learner's objectives. Interestingly, we also observe that when using the Look-ahead optimizer, the Barlow Twins loss achieves better performance than the remaining objective, which is also the case for the supervised training [59]. Therefore, we expect DualNet improve its performance with a more powerful and suitable slow learning objective.
4.4 Ablation Study of Self-Supervised Learning Iterations
We now investigate DualNet's performances with different SSL optimization iterations n. Small values of indicate there is little no delay of labeled data from the continuum, and the fast learner has to query the slow learner's representation continuously. On the other hand, larger n simulate the situations where labeled data are delayed, which allows the slow learner to train its SSL objective for more iterations between each query from the fast learner. In this experiment, we gradually increase the SSL training iterations between each supervised update by varying from n to n 20. We run the experiments on both the Split minilmageNet benchmarks under the TA and TF settings. Table 3 reports the result of this experiment. Interestingly, even with only one SSL training iteration (n = 1), DualNet still obtains competitive performance and outperforms existing baselines. As more iterations are allowed, DualNet consistently reduces forgetting and facilitates knowledge transfer, which results in better overall performance.
4.5 Ablation Study of DualNet's Fast Learner
DualNet's introduces an additional fast learner on top of the standard backbone used as the slow learner. In this experiment, we investigate the contribution of the fast learner to DualNet's overall performance on the Split minilmageNet on both the TA and TF settings. We compare the full DualNet against a variant that only employs slow learner and report the results in Table 4. We can see that the slow learner variant binds both types of representation into the same backbone and performs significantly worse than the original DualNet on both scenarios. This result corroborates with our motivation in Section that it is more beneficial to separate the two self-supervised and supervised representations into two distinct systems.
8

---

Table 4: Evaluation of DualNet's slow learner on the Split minilmageNet TA and TF benchmarks
Split minilmageNet-III TA
Split minilmageNet-TF
DualNet
ACC(t)
FM(L)
LA(T)
ACC())
FM(L)
LA(1)
Slow Learner
73.20-10.68 3.86+1
7..1 36.86 1.36 28.63::2.26
63.46::1.97
Flow+Fast Leaners 0.57
5.124:0.78
69.20+10.32
27.20 34.60+1.12 59.70
Table 5: Evaluation metrics on the Split minilmageNet-TAA benchmarks under the semi-supervised setting, where p denotes the fraction of data are labeled
p 10%
25%
Method
ACC(t)
FM(()
LA(t)
ACC(t)
FM(()
LA(t)
ER
41.66++2.72
6.80++2.07
42.33++1.51
50.13+2.11
6.76=1.51
51.90:+2.16
DER+
1..56+1141
4.55+0.66
43.03++0.71
51.63+1.11
6.03=1.46
52.36+0.55
CTN
3.96+1.116 49.80++2.66
47.76=10.99
55.90+1.86
3.84=0.32
55.69
DualNet
54.03.12.88
3.46 49.96-10.17 62.80+2.40 0.99
59.60
4.6 Results of the Semi-Supervised Continual Learning Setting
In real-world continual learning scenarios, there exist abundant unlabeled data, which are costly and even unnecessary to label entirely. Therefore, practical continual learning system should be able to improve its representation using unlabeled samples while waiting for the labeled data. To test the performance of existing methods on such scenarios, we create a semi-supervised continual learning benchmark, where the data stream contains both labeled and unlabeled data. For this, we consider the Split minilmageNet-Tie benchmark but provide labels randomly to a fraction (p) of the total samples, which we set to be 10% and 25%. The remaining samples are unlabeled and cannot be processed by the baselines we considered so far. In contrast, such samples can go directly to the DualNet's slow learner to improve its representation while the fast learner stays inactive. Other configurations remain the same as the experiment in Section 4.2.
Table 5 shows the results of this experiment. Under the limited labeled data regimes, the results of ER and DER++ drop significantll. Meanwhile, CTN can still maintain competitive performances thanks to additional information from the task identifiers, which remains untouched. On the other hand, DualNet can efficiently leverage the unlabeled data to improve its performance and outperform other baselines, even CTN. This result demonstrates DualNet's 's potential to work in a real-world environment, where labeled data can be delayed or even unlabeled.
Due to space constraints, we refer to the supplementary materials for DualNet's pseudo-code, additional results, experiments settings such as dataset summary, evaluation metrics, hyper-parameter configurations, and further discussions.
5 Conclusion
In this paper, we proposed DualNet, a novel paradigm for continual learning inspired by the fast and slow learning principle of the Complementary Learning System theory from neuroscience. DualNet comprises two key learning components: (i) a slow learner that focuses on learning general and task-agnostic representation using the memory data; and (ii) a fast learner focuses on capturing new supervised learning knowledge via a novel adaptation mechanism. Moreover, the fast and slow learners complement each other while working synchronously. resulting in holistic continual learning method. Our experiments on two challenging benchmarks demonstrate the efficacy of DualNet. Lastly, extensive and carefully designed ablation studies show that DualNet is robust to the slow learner's objectives, scalable with more resources, and applicable to the semi-supervised continual learning setting.
Our DualNet presents a general continual learning framework that can enjoy great scalability to real-world continual learning scenarios. However, additional computational cost incurred to train the slow learner continuously should properly managed. Moreover, applications to specific domains should take into account the inherent challenges. Lastly, we adopt the contrastive SSL approach to train the DualNet's slow learner in this work. Future work includes designing a slow objective tailored for continual learning.

---

Acknowledgement
The first author is supported by the SMU PGR scholarship. We thank the anonymous Reviewers for helpful discussions during the submission of this work.
References
[1] Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuyte- laars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Conference on Computer Vision (ECCV), pages 154,154, 2018.
[2] Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free free continual learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1225--12263, 2019.
[3] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processing Systems, pages 11816-11825, 2019.
[4] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8): 1798 1828, 2013.
[5] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple baseline. In 34th Conference on Neural Information Processing Systems (NeurIPS 2020), 2020.
[6] Fabio M Carlucci, Antonio D' I'Inoocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 2229-22222222 2238, 2019.
[7] Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), pages 532-547, 2018.
[8] Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. International Conference Learning Representations (ICLR), 2019.
[9] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv: 10486, 2019.
[10] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597-1607. PMLR, 2020.
[11] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15750-15758, 2021.
[12] Matthias Delange. Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
[13] Tom Diethe, Tom Borchert, Eno Thereska, Borja Balle, and Neil Lawrence. Continual learning in practice. arXiv preprini arXiv: 05200,, 2019.
[14] Rodney Douglas, Christof Koch, Misha Mahowald, KA Martin, and Humbert H Suarez. Recurrent excitation in neocortical circuits. Science, 269(5226)9881999999 1995.
[15] Vincent Dumoulin, Ethan Perez, Nathan Schucher, Florian Strub, Harm de Vries, Aaron Courville, and Yoshua Bengio. Feature-wise transformations. Distill, 2018. doi: 10.23915/ distill.00011. I. http://alllllaatooo/eeeeeeeeeeee
10

---

[16] Dumitru Erhan, Aaron Courville, Yoshua Bengio, and Pascal Vincent. Why does unsupervised pre-training help deep learning? In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 201-208. JMLR Workshop and Conference Proceedings, 2010.
[17] Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.0873 2017.
[18] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adap- tation of deep networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pages 1126-113 1135. JMLR. org, 2017.
[19] Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135, 1999.
[20] Alexander Gepperth and Cem Karaoguz. A bio-inspired incremental learning architecture for applied perceptual problems. Cognitive Computation, 8(5)9224-33, 2016.
[21] Sean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent: A new approach to self-supervised learning. Advances Neural Information Processing Systems, 2020.
[22] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pultern recognition, pages 770-778, 2016.
[23] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2020.
[24] Xu He, Jakub Sygnowski, Alexandre Galashov, Andrei A RUsu, Yee Whye Teh, and Razvan Pascanu. Task agnostic continual learning via meta learning. arXiv preprint arXiv: N5tiv:990..0522 05201, 2019.
[25] Khurram Javed and Faisal Shafait. Revisiting distillation and incremental classifier learning. In Asian conference on computer vision, pages 3-17. Springer, 2018
[26] Khurram Javed and Martha White. Meta-learning representations for continual learning. In Advances in Neural Information Processing Systems, pages 1818-1828, 2019.
[27] Ronald Kemker and Christopher Kanan. Fearnet: Brain-inspired model for incremental learning. In International Conference on Learning Representutions, 2018
[28] Ronald Kemker, Mare McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018
[29] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barne Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 2017.
[30] Dharshan Kumaran, Demis Hassabis, and James L McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in cognitive sciences, 2027):5 2016.
[31] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436 444, 2015.
[32] Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. International Conference Machine Learning, pages 3925-3934, 2019.

---

[33] Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning, 8(3-4):233-321 1992.
[34] Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun. Mnemonics training: Multi- class incremental learning without forgetring. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12245-12254, 2020.
[35] Vincenzo Lomonaco and Davide Maltoni. Core50: new dataset and benchmark for continuous object recognition. In Proceedings of the Isl Annual Conference on Robot Learning, Proceedings of Machine Learning Research, pages 17-26. PMLR, 2017.
[36] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, pages 6467-6476, 2017.
[37] James L McClelland, Bruce L McNaughton, and Randall C O'RReilly. Why there comple- mentary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
[38] Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint 1rXiv: 0299999990999... 2018
[39] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv: 0000000778..... 2018
[40] German Parisi, Jun Tani, Cornelius Weber, and Stefan Wermter. Lifelong learning of spatiotem- poral representations with dual- dual-memory recurrent self-organization. Frontiers in neurorobotics, 12:78, 2018.
[41] German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.
[42] Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with general conditioning layer. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
[43] Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven CH Hoi. Contextual transformation networks for online continual learning. International Conference on Learning Representations (ICLR), 2021.
[44] Dushyant Rao, Francesco Visin, Andrei Rusu, Razvan Pascanu, Yee Whye Teh, and Raia Hadsell. Continual unsupervised representation learning. Advances in Neural Information Processing Systems, 2019.
[45] Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi. Learning multiple visual domains with residual adapters. In Advances in Neural Information Processing Systems, pages 506-516, 2017.
[46] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H I Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
[47] James Requeima, Jonathan Gordon, John Bronskill, Sebastian Nowozin, and Richard E Turner. Fast and flexible multi-task classification using conditional neural adaptive processes. In Advances in Neural Information Processing Systems, pages 7959-7970, 2019.
[48] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Ger- ald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. International Conference on Learning Representations (ICLR), 2019.
[49] Hippolyt Ritter, Aleksandar Botev, and David Barber. Online structured laplace approximations for overcoming catastrophic forgetting. Advances in Neural Information Processing Systems, pages 2018.
12

---

[50] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins. Hubert Soyer. James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks, arXiv preprini arXiv: /606. 04671, 2016.
[51] Doyen Sahoo, Quang Pham, Jing Lu, and Steven C. H. Hoi. Online deep learning: Learning deep neural networks on the fly. In Proceedings of the Twenty-Sevntth International Joint Conference on Artificial Intelligence, IJCAI-18, 2018.
[52] Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. Proceedings of the 35th International Conference on Machine Learning- Volume 80, pages 4548-4557. JMLR. org, 2018.
[53] Gido M van de Ven and Andreas S Tolias. Generative replay with feedback connections as general strategy for continual learning. arXiv preprint arXiv: 10635, 2018.
[54] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. In Advances in neural information processing systems, pages 3630-3638, 2016.
[55] Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1)377557 1985.
[56] Johannes von Oswald, Christian Henning, João Sacramento, and Benjamin F Grewe. Continual learning with hypernetworks. International Conference on Learning Representations (ICLR), 2020.
[57] Jachong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. International Conference on Learning Representations (ICLR), 2018
[58] Yang You, Igor Gitman, and Boris Ginsburg. Large batch training convolutional networks. arXiv preprint (tth::770.03800 2017.
[59] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. arXiv preprint arXiv:2103.03230. 2021.
[60] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 3987-3995. JMLR. org, 2017.
[61] Michael Zhang, James Lucas, Jimmy Ba, and Geoffrey E. Hinton. Lookahead optimizer: k steps forward, step back, Advances in Neural Information Processing Systems, 2019.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section and further discussions provided the Appendix.
(c) Did you discuss any potential negative socictal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A] No theoretical results are provided in this work.
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments...
13

---

(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes] The URL is provided in the abstract.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See the Appendix.
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes] We report the mean and standard deviation over five runs in our experiments.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See the Appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A] No existing assets are used in this work.
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A] No crowsourcing research with human subjects are involved in this work.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "j0iHYZ_vDHP", "reviewer": "Reviewer_RowY", "summary": "This paper introduces a model for continual learning based on example replay. Motivated by the Complementary Learning Systems theory in neuroscience, the authors propose to use, instead of a single backbone network, two different models: the slow and the fast learner. The slow learner is composed of a standard backbone architecture, such as ResNet, and it is optimized via self-supervised learning with examples from a memory buffer. The fast learner performs an adaptation of representations provided by the slow learner (based on masking) and embeds a classifier. The fast learner is supervised with class labels. Experiments and ablation studies are carried out on two datasets, namely miniImagenet and CORE-50, both in the presence and in the absence of task labels at inference time.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper seems solid and it is very well written. \nMost equations seem correct and visuals help understanding the modeling choices.\nThe idea of exploiting self-supervised learning in continual learning is fairly new to my knowledge, and so it is the idea of using multiple networks with different objectives.\nThe experimental results seem really encouraging.\n\nHowever, I have some doubts about the experimental setting and the validation of the proposed components that I think are worth raising:\n1) Not all the components hereby introduced are properly validated. Specifically, the feature adaptation transforming representations from the slow learner to the ones of the fast learner (eq. 5 and 6) is also not properly validated. How can we be sure this step is necessary? I think a good baseline could be using features from the slow learner directly, without any adaptation, and applying a classifier on top of them.\n2) I am unsure why the authors decide to optimize only one epoch per task. In my experience, that sort of constraint (unfortunately fairly popular in CL literature) introduces some sample efficiency problems and confounds them with catastrophic forgetting. As such, methods that are more sample efficient can perform better, but not because of their continual learning capabilities. Similar considerations are also discussed in the DER paper [1] (appendix, section F.3). In the context of this paper, this issue is even more serious as the training involves some optimization choices that might improve sample efficiency. Specifically: i) the look-ahead classifier, requiring more forward-backward updates for every example (boosting the performances significantly, Tab. 2) and ii) the fact that the self-supervised learning objective is optimized n times (with n=3 in the comparisons with state of the art). \nThis is further testified by the fact that DualNet outperforms Offline training, that is considered the upper bound in continual learning settings (Tab. 2 in supplementary material). \nIn summary: to enable a fair comparison, is the number of updates performed by DualNet the same as the one allowed to competitors such as ER, DER, or CTN? Or even better, are the positive results consistent in longer optimizations, where catastrophic forgetting is even more threatening and sample efficiency problems are ruled out?\n\t\nEquations:\n3) in eq. 3, the plus should be a minus, as L_BT is a loss term that requires minimization. Also, there shouldn't be a learning rate involved in the look-ahead phase? Do the authors use a learning rate of 1?\n4) Eq. 7 should be revised. pi(x) denotes a softmax over the input, without any function being involved. Also, what is the k index in \\hat{y}_k referring to?\n\n5) If classification is used to train the slow learner, what is the difference with respect to DER++? There seems to be a significant gap from Tab. 2 (classification-SGD, 67.50) against DER++ in Tab. 1 (62.32). Can the authors provide an intuition for this gap? The two methods seem pretty similar to me in formulation.\n6) the authors may consider discussing Meta Experience Replay [2] in their related work, as the REPTILE algorithm [3] employed in such a paper is similar in spirit to the look-ahead optimization used hereby.\n\nOverall, I consider this a borderline submission. I appreciate the idea of using self-supervised learning on the buffer, that seems to bring benefits. However, I have several concerns about experiments, the one about sample efficiency being the most critical.\n\n---\nEDIT: after discussions with the authors, I will increase my score from 5 to 7.\n\nReferences:\n[1] Buzzega, Pietro, et al. \"Dark experience for general continual learning: a strong, simple baseline.\" NeurIPS 2020.\n[2] Riemer, Matthew, et al. \"Learning to learn without forgetting by maximizing transfer and minimizing interference.\" ICLR 2019.\n[3] Nichol, Alex, Joshua Achiam, and John Schulman. \"On first-order meta-learning algorithms.\" arXiv preprint arXiv:1803.02999 (2018).\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, there is a discussion in the supplementary material.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "gJBbtEyun3", "reviewer": "Reviewer_xfxT", "summary": "Inspired by Complementary Learning Systems (CLS), the paper proposes a framework for performing continual learning which is composed of self-supervised learning stream (aka slow learner) that conducts representation learning and of supervised learning stream (fast learner) that attains knowledge from labeled samples via adapting slow learner's representation. The DualNet outperforms other methods on CORE50 and miniImageNet.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "(+) The paper considers both multi-head and single-head (no task identifier) evaluations.\n\n(+) The paper offers a concrete philosophy for continual learning. DualNet is based on CLS which is what CTN was based on.\n\n(-) There are many criterion for choosing the appropriate SSL. It would be nice to have performance comparison between the different SSL and discussion as to why one would perform better than the other. Table 2 compares only to SimCLR but not SimCLR v2, BYOL, MoCo.\n\n(-) The proposed network is evaluated on only two dataset whereas many continual learning papers evaluate their algorithm on at least 3 (Permuted MNIST Alternating CIFAR 10 and CIFAR 100, Multiple datasets learning). The proposed network should be validate for much wider set of datasets.\n\n(-) There are many reference that are missing in the paper. Sebastian Lee · Sebastian Goldt · Andrew Saxe, Continual Learning in the Teacher-Student Setup: Impact of Task Similarity ICML 2021 Abhishek Kumar · Sunabha Chatterjee · Piyush Rai Bayesian Structural Adaptation for Continual Learning ICML 2021", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "Yes.", "needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "Grl2KrTspC", "reviewer": "Reviewer_qVwg", "summary": "This work proposes an Architectural-based method for online continual learning, suitable both for the Task-aware and Task-free scenarios. Inspired by biologic suggestions, the author separate a continual learner in a slow network (which learns generic representations in a contrastive fashion) and a fast network (which utilises the feature generated by the former to produce a classification response).\nExperiments show state-of-the-art accuracy in the presented setting.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strong points of this work:\n- I find it clearly written and easy to understand (with all the information and the pseudo-code provided in the supplementary material, I feel confident I could re-implement this work from scratch);\n- I find the idea to combine a separated representation learner with a lean feature adaptation model to be both novel (other works propose dual structures, but not in this way) and convincing.\n- The results include a very up-to-date roster of competitors, although not necessarily fair as I explain below.\n\nWeaknesses and suggestions:\n- The main issue that could be brought up about this paper is that competitors are not as sample-efficient as the proposed method. If I understand correctly, in its slow-learning branch, DualNet can optimise memory data repeatedly between batches from the continual input stream, while ER and DER++ only apply one update for each input batch (as they are not necessarily designed with the online setting in mind - Buzzega et al. perform multiple epochs on the same task). It could be interesting to add more sample-efficient competitors, which should be especially effective in the online scenario, e.g. MER [Riemer et al, 2019 in the paper] and GSS [1]\n- While I found the ablation section interesting, I feel that reproducing the same experiments in a task-free setting would make the results of Tables 2,3,4 much more interesting. In general, the task-aware setting is considered to be simplistic and less indicative of good performance w.r.t. the task-free one [1, 2].\n- A very minor point, one of the first CL models that came to my mind while reviewing this work is Fearnet [3], which is not cited and also borrows from similar biological premises. However, the latter is not necessarily an up-to-date reference, so I only indicate it here as a suggestion.\n\nThe following points did not affect my judgment, but I feel that they should be fixed:\n- The first citation of Kirkpatrick et al., 2017 at line 34 seems out of place, as EWC is not a method that uses an episodic memory.\n- The citation style for this paper is not the one that is found in typical NeurIPS 2021 submissions, namely using increasing numbers between brackets [X].\n- The following typos could be fixed:\nline 158: e.g. seems to be followed by a slightly larger space (the space after the last period might need escaping)  \nEquation 7: what does index k denote?  \nline 191: missing space between estimations and the citations  \nline 255: there's a misplaced u character after the citation  \nTable 2: The results for Barlow Twins, (SGD, ACC) have /- rather than \\plusminus  \nTable 3 caption: awarae -> aware  \nTable 4: \\rho=10 should have a percentage  \nline 327: to label all of them --> to entirely label  \n\n[1] Aljundi et al., Gradient based sample selection for online continual learning, NeurIPS 2019  \n[2] Farquhar et al., Towards Robust Evaluations of Continual Learning, ICML workshop 2018  \n[3] Kemker et al., FearNet: Brain-Inspired Model for Incremental Learning, ICLR 2018\n\n\n--- AFTER REBUTTAL ---\n\nI thank the authors for engaging in the rebuttal process. I especially found the experiments provided as a response to reviewer RowY to be interesting and meaningful, so I strongly encourage the authors to include them in their final revision for this paper.\nI confirm my previous rating and recommend acceptance of this work.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This is theoretical Continual Learning work that only refers to in-the-lab classification problems. I believe that the limitations of such a work are implicitly defined by the specific evaluation setting detailed in sec. 4.1. Discussion of societal impacts are not applicable due to the abstract nature of this work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper provides a dual-learning mechanism that uses a novel combination of a separated representation learner with a lean feature adaptation model.  The reviewers found their approach convincing and that it performs well against current competitors. They also found this paper to be well written, and after discussion on various technical aspects, quite compelling.  The one slightly negative review provided little justification for their points and chose not to engage with the authors after the authors provided a detailed rebuttal against the points raised in the more negative review.
Decision: Accept (Poster)