Continual Learning Using Task Conditional Neural Networks

1 INTRODUCTION . The human brain can adapt and learn new knowledge in response to changing environments . Hoshi et al . ( 1998 ) demonstrate that we can continually learn different tasks while retaining previously learned variations of the same or similar phenomenon and give different reactions under different contexts . Similarly , Asaad et al . ( 2000 ) have found that our neurons are task-independent . Under different context , the neurons are fired selectively with respect to the stimulus . In contrast , most of the machine learning models , in a scalable way , are not capable of adapting to changing environments quickly and automatically using artificial neurons corresponding to different tasks . Consequently , conventional machine learning models tend to forget the previously learned task after learning a new task . This scenario is known as catastrophic forgetting or interference in machine learning ( McCloskey & Cohen ( 1989 ) ) . Catastrophic interference problem in machine learning is one of the hurdles to implement a general artificial intelligence learning systems ( Legg & Hutter ( 2007 ) ) . Unable to learn several tasks , a model should be trained with all the possible scenarios in advance . This requirement is intractable in practice and is not inline with the continual learning models ( Thrun & Mitchell ( 1995 ) ) . Continual machine learning algorithms change over time and adapt their parameters to data or learning goals changes . We refer to the learning goal or a specific part of the data with a learning goal as a task . The learning models are not often equipped with solutions to quickly adapt to the situations which they have seen before if their parameters have significantly changed over time by continual learning . A variety of continual learning methods have been proposed to solve the problems mentioned above . Shin et al . ( 2017 ) proposed a memory-based approach which is to replay the trained samples to solve the forgetting problem while learning a new task . Lee et al . ( 2017 ) ; Kirkpatrick et al . ( 2017 ) ; Zeng et al . ( 2018 ) leverage the regularisation methods reduce the representational overlap of different tasks . Dynamic network approaches assign extra neuron resources to new tasks ( Yoon et al . ( 2017 ) ) . Most of the existing solutions need to know all the task changes in advance , or this information is given manually to the model throughout the learning . Some other works detect in-task and out-task samples to inference the task identity ( Li et al . ( 2019 ) ; Lee et al . ( 2020 ) ) . In this work , we propose a novel method to overcome the catastrophic forgetting problem . We leverage Misture of Experts ( MoE ) ( Masoudnia & Ebrahimpour ( 2014 ) ) to train the task-specific models . One of the advantage of MoE is that we can use a shallower networks and utilise the capacity as much as possible for each expert . Theoretically , the potential capacity of the combined networks is infinite . In other words , we do not need to initialise a giant network at the beginning of the training to avoid learning further tasks without forgetting them . In each expert , we integrate the probabilistic neural networks ( Specht ( 1990 ) ) and conventional neural networks ( Haykin ( 1994 ) ) to produce the task likelihood . The final prediction of each expert will be weighted by the task likelihood and calculate the task conditional probabilities . We name our approach as Task Conditional Neural Network ( TCNN ) . 2 RELATED WORK . There are different approaches to address the forgetting problem in continual learning . Parisi et al . ( 2019 ) categorise these approaches into three groups : Regularisation , Memory Replay and Dynamic Network approaches . The regularisation approaches find the overlap of the parameter space between different tasks . One of the popular algorithms in this group is Elastic Weight Consolidation ( EWC ) proposed by Kirkpatrick et al . ( 2017 ) . EWC avoids significantly changing the parameters that are important to a learned task . It assumes the weights have Gaussian distributions and approximates the posterior distribution of the weights by the Laplace approximation . A similar idea is used in Incremental Moment Matching ( IMM ) proposed by Lee et al . ( 2017 ) . IMM finds the overlap of the parameter distributions by smoothing the loss surface of the tasks . Zeng et al . ( 2018 ) address the forgetting problem by allowing the weights to change within the same subspace of the previously learned task . Li & Hoiem ( 2018 ) address the problem by using the knowledge distillation ( Hinton et al . ( 2015 ) ) . They enforce the prediction of the learned tasks to be similar to the new tasks ( Parisi et al . ( 2019 ) ) . However , these models require advance knowledge of the training tasks and the task changes . Memory Replay methods mainly focus on interleaving the trained samples with the new tasks . A pseudo-rehearsal mechanism proposed by Robins ( 1995 ) reduce the memory requirement to store the training samples for each task . In a pseudo-rehearsal , instead of explicitly storing the entire training samples , the training samples of previously learned tasks are drawn from a probabilistic distribution model . Shin et al . ( 2017 ) propose an architecture consisting of a deep generative model and a task solver . Similarly , Kamra et al . ( 2017 ) use a variational autoencoder to generate the previously trained samples . However , this group of models are complex to train , and in real-world cases , the sampling methods do not offer an efficient solution for sporadic and rare occurrences . These models also often require advance knowledge of the change occurring . Dynamic Networks allocate new neurons to new tasks . Yoon et al . ( 2017 ) propose Dynamic Expandable Networks ( DEN ) to learn new tasks with new parameters continuously . Similarly , Serrà et al . ( 2018 ) also allocate new parameters to learn new tasks . Masse et al . ( 2018 ) propose dynamic networks Context Dependent Gating ( XdG ) which provides the task context information to train the model . However , this group of models require the task information to be given to the model explicitly . In other words , the model knows in advance which neurons should be activated to perform each test task . To identify which neurons or experts should be used during the test state , Aljundi et al . ( 2017 ) proposed the Expert Gate and Lee et al . ( 2020 ) proposed Continual Neural Dirichlet Process Mixture ( CN-DPM ) leverage generative models to achieve the task-free continual learning , which does not need the task identities in the test phase . For each task , the Expert Gate and CN-DPM need to train a generator to distinguish the in-task and out-task samples . As the complexity of the dataset grows , the extra demand including parameters and computational resources of training a generator increases . Different from them , we introduce the probabilistic layer added into the discriminative model . Our method saves the memory since we do not need an extra model which is a generator to recognise the task identity . Furthermore , our approach can save the computational resources ( the probabilistic layer can be simultaneously trained with classification tasks and we do not need to train a generator for each expert ) . van de Ven & Tolias ( 2019 ) define different scenarios to illustrate the demand of a model for the task information . Generally , the class-incremental scenario ( Class-IL ) represents the model does not need the task information in advance , task-incremental scenario ( Task-IL ) represents the model need the task information at inference phase . 3 TASK CONDITIONAL NEURAL NETWORK . In the TCNN , each expert is independent of others and has the same behaviour during the training and inference phases . One model contains multiple expert corresponding to a different task . In this section , we introduce our model by taking one expert as an example . 3.1 ESTIMATING THE TASK LIKELIHOOD . In this work , we refer to the task likelihood as P ( t = k|x ) to represent the sample x from the kth task . To estimate the task likelihood efficiently , we add a probabilistic layer into the expert . Different from the conventional neural networks , this model contains two heads performing estimating task likelihood and classification . The classification head is connected to a classification layer , which is similar to a conventional neural network . The task likelihood head is connected to the probabilistic layer . The probabilistic layers contains several kernels , each of which performs the function shown in Equation 1 . Where the f ( · ) is the function of hidden , zi is a vector representing the ith kernel , zi has the same dimension with the output from previous layer , Σ is the covariance matrix of z1 : n , where n is the number of the kernels in the probabilistic layer . Calculating the covariance matrix could be intractable due to the high dimension of z1 : n. Since the kernels can be viewed as different data patterns , we assume the kernels in the probabilistic layers are independent of each other . K ( f ( x ) , zi ) = exp [ − 1 2 ( f ( x ) − zi ) T Σ−1 ( f ( x ) − zi ) ] ( 1 ) The Equation 1 measures the similarity between the input and existing kernels acting as anchors . The output of the probabilistic layer is an n dimensional vector . Each element in the vector can be regarded as the similarity of the input to one of the existing kernel . The summation of these n similarities can be viewed as the task likelihood . To estimate the task likelihood , we perform a normalised summation shown in Equation ( 2 ) . The range of the task likelihood P ( t|x , · ) is from 0 to 1 . P ( t|x , · ) = ∑n i=1K ( f ( x ) , zi ) n∑ i=1 K ( f ( x ) , zi ) + 1− max j=1 ... n { K ( f ( x ) , zj ) } ( 2 ) While training a new expert , we jointly maximise the task likelihood and the classification likelihood by minimizing Equation ( 3 ) . Where { x , y } is the training sample set of task k , C is the number of classes in the task k , { x ( c ) , y ( c ) } is the samples of cth class , θS is the parameters in the hidden layers shared by classification and probabilistic layer , θD and θT are the parameters in the classification and probabilistic layers respectively , λ is a hyper-parameter to weight the task likelihood loss , N is number of samples . Lv ( θS , θD , θT ) = − 1 N ( C∑ c=1 y ( c ) logP ( y ( c ) |x ( c ) , θD , θS ) + λ ∗ logP ( t = k|x , θT , θS ) ) ( 3 ) 3.2 TRAINING AND INFERENCE . Each expert is corresponding to a single task . We augment each samples to generate different views and make these views converge to the kernels in the probabilistic layer . A view generated by different pre-defined functions is a variation of the input . In this work , we define several functions to get a different view of the inputs . Including Zero Component Analysis ( ZCA ) Pal & Sudeep ( 2016 ) , shift , rotation shear and flip . More specifically , we minimize the loss function ( 4 ) , where m is the number of agumented views , Lv ( θS , θD , θT ) is the loss for the vth view . After training all K experts , we augment the test data by the same functions used in training process of each expert , and estimate the P ( y|x ) by Equation ( 5 ) , where xv is the vth view after augmentation , t = k represents the kth expert is responsible for the input . Overall , we do the training and inference process by averaging several different views . The idea is inspired by contrastive learning ( Chen et al . ( 2020 ) ) . We take the in-task samples as positive pairs and out-task samples as negative pairs to train a good feature extractor . However , in the continual learning scenarios , the out-task samples is not accessible . Hence we average multiple different views of the in-task samples to learn an anchor to distinguish the in-task and out-task samples . L = 1 m m∑ v=1 Lv ( θS , θD , θT ) ( 4 ) P ( y|x ) ≈ 1 m m∑ v=1 P ( yv|xv ) ≈ 1 m K∑ k=1 m∑ v=1 P ( yv|xv , t = k ) P ( t = k|xv ) ( 5 )

Summary: This paper introduces Task Conditional Neural Network for continual learning. The method is based on mixture of experts. Each expert is independent of other experts and therefore the model is not susceptible to catastrophic forgetting.