Contextual Transformation Networks for Online Continual Learning

Continual learning methods with fixed architectures rely on a single network to learn models that can perform well on all tasks . As a result , they often only accommodate common features of those tasks but neglect each task ’ s specific features . On the other hand , dynamic architecture methods can have a separate network for each task , but they are too expensive to train and not scalable in practice , especially in online settings . To address this problem , we propose a novel online continual learning method named “ Contextual Transformation Networks ” ( CTN ) to efficiently model the task-specific features while enjoying neglectable complexity overhead compared to other fixed architecture methods . Moreover , inspired by the Complementary Learning Systems ( CLS ) theory , we propose a novel dual memory design and an objective to train CTN that can address both catastrophic forgetting and knowledge transfer simultaneously . Our extensive experiments show that CTN is competitive with a large scale dynamic architecture network and consistently outperforms other fixed architecture methods under the same standard backbone . Our implementation can be found at https : //github . com/phquang/Contextual-Transformation-Network . 1 INTRODUCTION . Continual learning is a promising framework towards building AI models that can learn continuously through time , acquire new knowledge while being able to perform its already learned skills ( French , 1999 ; 1992 ; Parisi et al. , 2019 ; Ring , 1997 ) . On top of that , online continual learning is particularly interesting because it resembles the real world and the model has to quickly obtain new knowledge on the fly by levering its learned skills . This problem is important for deep neural networks because optimizing them in the online setting has been shown to be challenging ( Sahoo et al. , 2018 ; Aljundi et al. , 2019a ) . Moreover , while it is crucial to obtain new information , the model must be able to perform its acquired skills . Balancing between preventing catastrophic forgetting and facilitating knowledge transfer is imperative when learning on a stream of tasks , which is ubiquitous in realistic scenarios . Thus , in this work , we focus on the continual learning setting in an online learning fashion , where both tasks and data of each task arrive sequentially ( Lopez-Paz & Ranzato , 2017 ) . In the literature , fixed architecture methods employ a shared feature extractor and a set of classifiers , one for each task ( Lopez-Paz & Ranzato , 2017 ; Chaudhry et al. , 2019a ; b ; Aljundi et al. , 2019a ) . Although using a shared feature extractor has achieved promising results , the common and global features are rather generic and not well-tailored towards each specific task . This problem is even more severe when old data are limited while learning new tasks . As a result , the common feature extractor loses its ability to extract previous tasks ’ features , resulting in catastrophic forgetting . On the other hand , while dynamic architecture methods such as Rusu et al . ( 2016 ) ; Li et al . ( 2019 ) ; Xu & Zhu ( 2018 ) alleviate this problem by having a separate network for each task , they suffer from the unbounded growth of the parameters . Moreover , the subnetworks ’ design is not trivial and requires extensive resource usage ( Rusu et al. , 2016 ; Li et al. , 2019 ) , which is not practical in many applications . These limitations motivated us to develop a novel method that can facilitate continual learning with a fixed architecture by modeling the task-specific features . To achieve this goal , we first revisit a popular result in learning multiple tasks that each task ’ s features are centered around a common vector ( Evgeniou & Pontil , 2004 ; Aytar & Zisserman , 2011 ; Pentina & Lampert , 2014 ; Liu et al. , 2019b ) . This result motivates us to develop a novel framework of Contextual Transformation Networks ( CTN ) , which consists of a base network that learns the common features of a given input and a controller that efficiently transforms the common features to become task-specific , given a task identifier . While one can train CTN using experience replay , it does not explicitly aim at achieving a good trade-off between stability and plasticity . Therefore , we propose a novel dual memory system and a learning method that encapsulate alleviating forgetting and facilitating knowledge transfer simultaneously . Particularly , we propose two distinct memories : the episodic memory and the semantic memory associated with the base model and the controller , respectively . Then , the base model is trained by experience replay on the episodic memory while the controllers is trained to learn task-specific features that can generalize to the semantic memory . As a result , CTN achieves a good trade-off between preventing catastrophic forgetting and facilitating knowledge transfer because the task-specific features can generalize well to all past and current tasks . Figure 1 gives an overview of the proposed Contextual Transformation Network ( CTN ) . Interestingly , the designs of our CTN and dual memory are partially related to the Complementary Learning Systems ( CLS ) theory in neuroscience ( McClelland et al. , 1995 ; Kumaran et al. , 2016 ) . Particularly , the controller acts as a neocortex that learns the structured knowledge of each task . In contrast , the base model acts as a hippocampus that performs rapid learning to acquire new information from the current task ’ s training data . Following the naming convention of memory in neuroscience , our CTN is equipped with two replay memory types . ( i ) the episodic memory ( associated with the hippocampus ) caches a small amount of past tasks ’ training data , which will be replayed when training the base networks . ( ii ) the semantic memory ( associated with the neocortex ) stores another distinct set of old data only used to train the controller such that the task-specific features can generalize well across tasks . Moreover , the CLS theory also suggests that the interplay between the neocortex and the hippocampus attributes to the ability to recall knowledge and generalize to novel experiences ( Kumaran & McClelland , 2012 ) . Our proposed learning approach closely characterizes such properties : the base model focuses on acquiring new knowledge from the current task while the controller uses the base model ’ s knowledge to generalize to novel samples . In summary , our work makes the following contributions . First , we propose CTN , a novel continual learning method that can model task-specific features while enjoying neglectable complexity overhead compared to fixed architecture methods ( please refer to Table 4 ) . Second , we propose a novel objective that can improve the trade-off between alleviating forgetting and facilitating knowledge transfer to train CTN . Third , we conduct extensive experiments on continual learning benchmarks to demonstrate the efficacy of CTN compared to a suite of baselines . Finally , we provide a comprehensive analysis to investigate the complementarity of each CTN ’ s component . 2 METHOD . Notations . We denote φ as parameter of the base model that extracts global features from the input and θ as the parameter of the controller which modifies the features from φ given a task identifier t. The task identifier can be a set of semantic attributes about objects of that task ( Lampert et al. , 2009 ) or simply an index of the task , which we use in this work as a one-hot vector . A prediction is given as gϕt ( hφ , θ ( x , t ) ) , where gϕt ( · ) is the task Tt ’ s classifier with parameter ϕt such as a fully connected layer with softmax activation . And hφ , θ ( x , t ) is the final feature after transformed by the controller . We denoteDtrt as the training data of task Tt , Memt andMsmt as the episodic memory and the semantic memory of task Tt respectively . The episodic memory and semantic memory maintains two distinct sets of data obtained from task Tt . The episodic memory of task T1 , . . . , Tt−1 is denoted asMem < t ; similarly , Msm < t denotes the semantic memory of the first t− 1 tasks . Remark . Both theMemt andMsmt are obtained from Dtrt through the learner ’ s internal memory management strategy and contains distinct samples from each other such that their combined sizes do not exceed a pre-defined budget . 2.1 LEARNING TASK-SPECIFIC FEATURES FOR CONTINUAL LEARNING . Given a backbone network , one can implement the task-specific features by employing a set of task-specific filters and applying them to the backbone ’ s output . However , this trivial approach is not scalable , even for small networks . In the worst case , it results in storing an additional network per task , which violates the fixed architecture constraint . Since we want to obtain task-specific features with minimal parameter overhead , we propose to use a feature-wise transformation ( Perez et al. , 2018 ) to efficiently extract the task-specific features h̃ ( x , t ) from the common features ĥ ( x ) as follows : h̃ ( x ; t ) = γt ‖γt‖2 ⊗ ĥ ( x ) + βt ‖βt‖2 and { γt , βt } = cθ ( t ) , ( 1 ) where ⊗ denotes the element-wise multiplication operator , and cθ ( t ) is the controller implemented as a linear layer with parameter θ that predicts the transformation coefficients { γt , βt } given the task identifier t. Since the task identifiers are one-hot vectors , which are sparse and make training the controller difficult , we also introduce an embedding layer to map the task identifiers to dense , low dimensional vectors . For simplicity , we will use θ to refer to both the embedding and the linear layer parameters . In addition , instead of storing a set of coefficients { γt , βt } for each task , we only need a fixed set of parameter θ to predict these coefficients , which results in the fixed parameters in the controller . The coefficients { γt , βt } are ` 2-normalized and then transforms the common features ĥ ( x , t ) to become task-specific features h̃ ( x ; t ) . Finally , both feature types are combined by a residual connection before passing to the corresponding classifier gt ( · ) to make the final prediction : gϕt ( σ ( h ( x , t ) ) ) , and h ( x , t ) = ĥ ( x , t ) + h̃ ( x , t ) , ( 2 ) where σ ( · ) is a nonlinear activation function such as ReLU . Importantly , when the task-specific features are removed , i.e. , h̃ ( x , t ) = 0 , Eq . 2 reduces to the traditional experience replay . Lastly , for each incoming task , CTN has to allocate a new classifier , which is the same for all continual learning methods , and a new embedding vector , which is usually low dimensional , e.g . 32 or 64 . Therefore , CTN enjoys almost the same parameter growth as existing continual learning methods . 2.2 TRAINING THE CONTROLLER . While one can train CTN with experience replay ( ER ) , it does not explicitly address the trade-off between facilitating knowledge transfer and alleviating catastrophic forgetting . This motivates us to develop a novel training method that can simultaneously address both problems by leveraging the controller ’ s task-specific features . First , we introduce a dual memory system consisting of the semantic memoryMsmt associated with the controller and the episodic memoryMemt associated with the base model . We propose to train only the base model using experience replay with the episodic memory to obtain new knowledge from incoming tasks . The controller is also trained so that the task-specific features can generalize to unseen samples to the base model stored in the semantic memory . As a result , the task-specific features can generalize to both previous and current tasks , which simultaneously encapsulate both alleviating forgetting and facilitating knowledge transfer . Formally , given the current batch of data for task Tt as Bt , the training of CTN can be formulated as the following bilevel optimization problem ( Colson et al. , 2007 ) : Outer problem : minθ Lctrl ( { φ∗ , θ } ; Msm < t+1 ) Inner problem : s.t φ∗ = arg min φ Ltr ( { φ , θ } , Bt ∪Mem < t ) , ( 3 ) where φ∗ denotes the optimal base model corresponding to the current controller θ . Since every CTN ’ s prediction always involves both the controller and the base model , we use Ltr ( { φ , θ } , Bt ∪Mem < t ) to denote the training loss of the pair { φ , θ } on the data Bt ∪Mem < t . Similarly , Lctrl ( · ) denotes the controller ’ s loss . For simplicity , we omitted the dependency of the the loss on the classifiers ’ parameters and imply that the classifiers are jointly updated with the base model . Since we do not know the optimal transformation coefficients of any task , the controller is trained to minimize the classification loss of the samples via φ . We implement both the training and controller ’ s losses as the cross-entropy loss . Notably , Eq . 3 characterizes two nested optimization problems : the outer problem , which trains the controller to generalize , and each controller parameter θ parameterizes an inner problem that trains the base model to acquire new knowledge via experience replay . Moreover , only φ is trained in the inner problem , while only θ is updated in the outer problem . The bilevel optimization objective such as Eq . 3 has been successfully applied in other machine learning disciplines such as hyperparameter optimization , meta learning ( Franceschi et al. , 2018 ; Finn et al. , 2017 ) , and AutoML ( Liu et al. , 2019a ) . In this work , we extend this framework to continual learning to train the controller . However , unlike existing works ( Franceschi et al. , 2018 ; Finn et al. , 2017 ; Liu et al. , 2019a ) , our Eq . 3 has to be solved incrementally when a new data sample arrives . Therefore , we consider Eq . 3 as an online learning problem and optimize it using the follow the leader principle ( Hannan , 1957 ) . Particularly , we relax the optimal solutions of both the inner and outer problems to be solutions from a few gradient steps . When a new training data arrives , we first train the base model φ using experience replay for a few SGD steps with an inner learning rate α , each of which is implemented as : φ← φ− α∇φLtr ( { φ , θ } , Bt ∪Mem < t ) , ( 4 ) Then , we optimize the controller θ such that it can improve φ ’ s performance on the semantic memory : θ ← θ − β∇θLctrl ( { φ , θ } , Msm < t+1 ) , ( 5 ) where β is the outer learning rate . As a result , Eq . 3 is implemented as an alternative update procedure involving several outer updates to train θ , each of which includes an inner update to trainφ . Moreover , performing several updates per incoming sample does not violate the online assumption since we will not revisit that sample in the future , unless it is stored in the memories .

Summary: This paper proposes CTN (Contextual Trasnformer Networks) for online continual learning. In particular, the authors introduce a dual memory framework that contains an episodic memory for base networks and semantic memory for task controllers. The overall framework is optimized with bi-level optimization. In addition, the base network also uses a KL-divergence loss to prevent catastrophic forgetting. Experiments are conducted on multiple datasets and the authors demonstrated that the proposed framewok outperforms other alternative approaches.