DualNet: Continual Learning, Fast and Slow

1 Introduction . Humans have the remarkable ability to learn and accumulate knowledge over their lifetime to perform different cognitive tasks . Interestingly , such a capability is attributed to the complex interactions among different interconnected brain regions [ 14 ] . One prominent model is the Complementary Learning Systems ( CLS ) theory [ 37 , 30 ] which suggests the brain can achieve such behaviors via two learning systems of the “ hippocampus '' and the “ neocortex . '' Particularly , the hippocampus focuses on fast learning of pattern-separated representation of specific experiences . Via the memory consolidation process , the hippocampus ’ s memories are transferred to the neocortex over time to form a more general representation that supports long-term retention and generalization to new experiences . The two fast and slow learning systems always interact to facilitate fast learning and long-term remembering . Although deep neural networks have achieved impressive results [ 31 ] , they often require having access to a large amount of i.i.d data while performing poorly on the continual learning scenarios over streams of task [ 19 , 29 , 36 ] . Therefore , the main focus of this study is exploring how the CLS theory can motivate a general continual learning framework with a better trade-off between alleviating catastrophic forgetting and facilitating knowledge transfer . In literature , several continual learning strategies are inspired from the CLS theory principles , from using the episodic memory [ 36 ] to improving the representation [ 26 , 44 ] . However , such techniques mostly use a single backbone to model both the the hippocampus and neocortex , which binds two representation types into the same network . Moreover , such networks are trained to minimize the supervised loss , they lack a separate and specific slow learning component that supports general representation learning . During continual learning , the representation obtained by repeatedly 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . performing supervised learning on a small amount of memory data can be prone to overfitting and may not generalize well across tasks . Consider that in continual learning , unsupervised representation [ 20 , 40 ] is often more resisting to forgetting compared to supervised representation , which yields little improvements [ 25 ] ; we propose to decouple representation learning from the supervised learning into two separate systems . To achieve this goal , in analogy to the slow learning system in neocortex , we propose to implement the slow general representation learning system using Self-Supervised Learning ( SSL ) [ 39 ] . Note that recent SSL works focus on the pre-training phase , which is not trivial to apply for continual learning as it is extensive in both storage and computational cost [ 28 ] . We argue that SSL should be incorporated into the continual learning process while decoupling from the supervised learning phase into two separate systems . Consequently , the SSL ’ s slow representation is more general and can capture the intrinsic characteristics from data , which facilitates better generalization to both old and new tasks . Inspired by the CLS theory [ 37 ] , we propose DualNet ( for Dual Networks , depicted in Figure 1 ) , a novel framework for continual learning comprising two complementary learning systems : a slow learner that learns generic features via self-supervised representation learning , and a fast learner that adapts the slow learner ’ s features to quickly attain knowledge from labeled samples via a novel per-sample based adaptation mechanism . During the supervised learning phase , an incoming labeled sample triggers the fast learner to make predictions by querying and adapting the slow learner ’ s representation . Then , the incurred loss will be backpropagated through both learners to consolidate the current supervised learning pattern for long-term retention . Concurrently , the slow learner is always trained in the background by minimizing an SSL objective using only the memory data . Therefore , the slow and fast networks learning are completely synchronous , allowing DualNet to continue to improve its representation power even in practical scenarios where labeled data are delayed [ 13 ] or even limited , which we will demonstrate in Section 4.6 . Lastly , we focus on developing DualNet for the online continual learning settings [ 36 , 2 ] since it is more challenging to optimize deep networks in such scenarios [ 51 , 3 ] . In the batch continual learning setting [ 46 ] , the model is allowed to revisit data within the current task and can achieve good representations when learning the current task . In summary , our work makes the following contributions : 1 . We propose DualNet , a novel continual learning framework comprising two key components of fast and slow learning systems , which closely models the CLS theory . 2 . We propose a novel learning paradigm for DualNet to efficiently decouple the representation learning from supervised learning . Specifically , the slow learner is trained in the background with SSL to maintain a general representation . Concurrently , the fast learner is equipped with a novel adaptation mechanism to quickly capture new knowledge . Notably , unlike existing adaptation techniques , our proposed mechanism does not require the task identifiers . 3 . We conduct extensive experiments to demonstrate DualNet ’ s efficacy , robustness to the slow learner ’ s objectives , and scalability to the computational resources . 2 Method . 2.1 Setting and Notations . We consider the online continual learning setting [ 36 , 8 ] over a continuum of data D = { xi , ti , yi } i , where each instance is a labeled sample { xi , yi } with an optional task identifier ti . Each labeled data sample is drawn from an underlying distribution P t ( X , Y ) that represents a task and can suddenly change to P t+1 , indicating a task switch . When the task identifier t is given as an input , the setting follows the task-aware setting where only the corresponding classifier is selected to make a prediction [ 36 ] . When the task identifier is not provided , the model has a shared classifier for all classes observed so far , which follows the task-free setting [ 7 , 2 ] . We consider both scenarios in our experiments . A common continual learning strategy is employing an episodic memoryM to store a subset of observed data and interleave them when learning the current samples [ 36 , 9 ] . FromM , we use M to denote a randomly sampled mini-batch , and MA , MB to denote two views of M obtained by applying two different data transformations . Lastly , we denote ϕ as the parameter of the slow network that learns general representation from the input data and θ as the parameter of the fast network that learns the transformation coefficients . 2.2 DualNet Architecture . DualNet learns the data representation independent of the task ’ s label , which allows for better generalization capabilities across tasks in the continual learning scenario . The model consists two main learning modules ( Figure 1 ) : ( i ) the slow learner is responsible for learning a general , taskagnostic representation ; and ( ii ) the fast learner learns with labeled data from the continuum to quickly capture the new information and then consolidate the knowledge to the slow learner . DualNet learning can be broken down into two synchronous phases . First , the self-supervised learning phase in which the slow learner optimizes a Self-Supervised Learning ( SSL ) objective using unlabeled data from the episodic memory M. Second , the supervised learning phase happens whenever a labeled sample arrives , which triggers the fast learner to first query the representation from the slow learner and adapt it to learn this sample . The incurred loss will be backpropagated into both learners for supervised knowledge consolidation . Additionally , the fast learner ’ s adaptation is per-samplebased and does not require additional information such as the task identifiers . Note that DualNet uses the same episodic memory ’ s budget as other methods to store the samples and their labels , but the slow learner only requires the samples while the fast learner uses both samples and their labels . 2.3 The Slow Learner . The slow learner is a standard backbone network ϕ trained to optimize an SSL loss , denoted by LSSL . As a result , any SSL objectives can be applied in this step . However , to minimize the additional computational resources while ensuring a general representation , we only consider the SSL loss that ( i ) does not require additional memory unit ( such as the negative queue in MoCo [ 23 ] ) , ( ii ) does not always maintain an additional copy of the network ( such as BYOL [ 21 ] ) , and ( iii ) does not use handcrafted pretext losses ( such as RotNet [ 16 ] or JiGEN [ 6 ] ) . Therefore , we consider Barlow Twins [ 59 ] , a recent state-of-the-art SSL method that achieved promising results with minimal computational overheads . Formally , Barlow Twins requires two views MA and MB by applying two different data transformations to a batch of images M sampled from the memory . The augmented data are then passed to the slow net ϕ to obtain two representations ZA and ZB . The Barlow Twins loss is defined as : LBT ≜ ∑ i ( 1− Cii ) 2 + λBT ∑ i ∑ j ̸=i C2ij , ( 1 ) where λBT is a trade-off factor , and C is the cross-correlation matrix between ZA and ZB : Cij ≜ ∑ b z A b , iz B b , j√∑ B ( z A b , i ) 2 √∑ B ( z B b , j ) 2 ( 2 ) with b denotes the mini-batch index and i , j are the vector dimension indices . Intuitively , by optimizing the cross-correlation matrix to be identity , Barlow Twins enforces the network to learn essential information that is invariant to the distortions ( unit elements on the diagonal ) while eliminating the redundancy information in the data ( zero element elsewhere ) . In our implementation , we follow the standard practice in SSL to employ a projector on top of the slow network ’ s last layer to obtain the representations ZA , ZB . For supervised learning with the fast network , which will be described in Section 2.4 , we use the slow network ’ s last layer as the representation Z . In most SSL methods , the LARS optimizer [ 58 ] is employed for distributed training across many devices , which takes advantage of a large amount of unlabeled data . However , in continual learning , the episodic memory only stores a small number of samples , which are always changing because of the memory updating mechanism . As a result , the data distribution in the episodic memory always drifts throughout learning , and the SSL loss in DualNet presents different challenges compared to the traditional SSL optimization . Particularly , although the SSL objective in continual learning can be easily optimized using one device , we need to quickly capture the knowledge of the currently stored samples before the newer ones replace them . In this work , we propose to optimize the slow learner using the Look-ahead optimizer [ 61 ] , which performs the following updates : ϕ̃k ←ϕ̃k−1 − ϵ∇ϕ̃k−1LBT , with ϕ̃0 ← ϕ and k = 1 , . . . , K ( 3 ) ϕ←ϕ+ β ( ϕ̃K − ϕ ) , ( 4 ) where β is the Look-ahead ’ s learning rate and ϵ is the Look-ahead ’ s SGD learning rate . As a special case of K = 1 , the optimization reduces to the traditional optimization of LBT using SGD . By performing K > 1 updates using a standard SGD optimizer , the look-ahead weight ϕ̃K is used to perform a momentum update for the original slow learner ϕ . As a result , the slow learner optimization can explore regions that are undiscovered by the traditional optimizer and enjoys faster training convergence [ 61 ] . Note that SSL focuses on minimizing the training loss rather than generalizing this loss to unseen samples , and the learned representation requires to be adapted to perform well on a downstream task . Therefore , such properties make the Look-ahead optimizer a more suitable choice over the standard SGD to train the slow learner . Lastly , we emphasize that although we choose to use Barlow Twins as the SSL objective in this work , DualNet is compatible with any existing methods in the literature , which we will explore empirically in Section 4.3 . Moreover , we can always train the slow learner in the background by optimizing Equation 1 synchronously with the continual learning of the fast learner , which we will detail in the following section .

Summary: This paper introduces a model for continual learning based on example replay. Motivated by the Complementary Learning Systems theory in neuroscience, the authors propose to use, instead of a single backbone network, two different models: the slow and the fast learner. The slow learner is composed of a standard backbone architecture, such as ResNet, and it is optimized via self-supervised learning with examples from a memory buffer. The fast learner performs an adaptation of representations provided by the slow learner (based on masking) and embeds a classifier. The fast learner is supervised with class labels. Experiments and ablation studies are carried out on two datasets, namely miniImagenet and CORE-50, both in the presence and in the absence of task labels at inference time.