Continual learning with neural activation importance

1 INTRODUCTION . Continual learning , or sequential learning , is a concept of online learning along multiple sequential tasks . The aim of continual learning is learning a set of related tasks that are observed irregularly or separately online . Therefore each task does not necessarily contain overlapped classes with others and , in worst case , different tasks consist of mutually disjoint classes . Therefore , in continual learning , one of the main challenges is training new tasks with new classes without catastrophic forgetting existing knowledge of prior tasks ( and their classes ) . A model adapts to a new task without access to some or entire classes of past tasks but keeping and maintaining acquired knowledge from new tasks ( Thrun , 1996 ) . Since the training of a neural network is influenced more by recently and frequently observed data , a machine learning model forgets what it has learned in prior tasks without continuing access to them in the current task . On the other hand , rigorous methods that maintain the knowledge of entire previous tasks are impractical in adapting new tasks . Thus , researchers have developed diverse methods to achieve both stability ( remembering past tasks ) and plasticity ( adapting new tasks ) in continual learning . There are three major types in previous continual learning methods ; 1 ) architectural approaches modifying the architecture of neural networks ( Yoon et al. , 2017 ; Sharif Razavian et al. , 2014 ; Rusu et al. , 2016 ) , 2 ) rehearsal approaches using sampled data from previous tasks ( Riemer et al. , 2018 ; Aljundi et al. , 2019 ; Gupta et al. , 2020 ) , and 3 ) regularization approaches freezing significant weights of a model calculating the importance of weights or neurons ( Li & Hoiem , 2017 ; Kirkpatrick et al. , 2017 ; Zenke et al. , 2017 ; Nguyen et al. , 2017 ; Aljundi et al. , 2018a ; b ; Zeno et al. , 2018 ; Ahn et al. , 2019 ; Javed & White , 2019 ; Jung et al. , 2020 ) . Most recent regularization methods have tackled the problem in more fundamental way with regularization approaches that utilize the weights of a given network to the hit . The basic idea of regularization approaches is to constrain essential weights of old tasks not to change . In general , they alleviate catastrophic interference by imposing a penalty on the difference of weights between the past tasks and the current task . The degree of the penalty follows the importance of weights or neurons with respective measurements . Significance of an weight stands for how important the weight is in solving a certain task . EWC ( Kirkpatrick et al. , 2017 ) introduces elastic weight consolidation which estimates parameter importance using the diagonal of the Fisher information matrix equivalent to the second derivative of the loss . However , they compute weights ’ importance after network training . SI ( Zenke et al. , 2017 ) measures the importance of weights in an online manner by calculating each parameter ’ s sensitivity to the loss change while training a task . To be specific , when a certain parameter changes slightly during training batches but its contribution to the loss is high ( i.e. , rapid change of its gradient ) , the parameter is considered to be crucial and restricted to be updated while learning future tasks . However , the accuracy of their method shows limited stability even with the same order of tasks of test data . Unlike Zenke et al . ( 2017 ) , MAS ( Aljundi et al. , 2018a ) assesses the contribution of each weight to the change of learned function . In other words , it considers the gradient of outputs of a model with a mean square error loss . Gradient itself represents a change of outputs with respect to the weights . The strength of the method lies in the scheme of data utilization . It considers only the degree of change of output values of the network , so that any data ( even unlabeled one ) is valid to compute the gradient of the learned function with regard to weights . VCL ( Nguyen et al. , 2017 ) is a Bayesian neural network based method . It decides weight importance through variational inference . BGD ( Zeno et al. , 2018 ) is another Bayesian neural network based apporoach . In this method , it finds posterior parameters ( e.g . mean and variance ) assuming that posterior and the prior distribution are Gaussian . These methods ( Kirkpatrick et al. , 2017 ; Zenke et al. , 2017 ; Aljundi et al. , 2018a ; Nguyen et al. , 2017 ; Zeno et al. , 2018 ) calculate and assign importance to weights directly as describe in Figure 1a . In order to alleviate interference across multiple tasks , weight importance based approaches such as EWC , SI , and MAS assign importance Ωk to each weight ωk in the network . However , in the case of convolutional neural networks , weights in the same convolutional filter map should have the same importance . Furthermore , since those methods basically consider the amount of change of weight , it is impossible to reinitialize weights at each training of a new task , which decreases the plasticity of the network . ( Additional explanation of weight reinitialization is discussed in section 2.2 ) UCL ( Ahn et al. , 2019 ) propose a Bayesian neural network based method to mitigate catastrophic forgetting by incorporating weight uncertainty measurement indicating the variance of weight distribution . They claim that the distribution of essential weights for past tasks has low variance and such stable weights during training a task are regarded as important weight not to forget . As illustrated in Figure 1b , they suggest a node based importance in neural network . First , the smallest variance among the weights incoming to and outgoing from a corresponding neuron decides the importance of the neuron , and then all those weights take the same neurons importance as their weight importance . Ahn et al . ( 2019 ) is applicable only to a Bayesian neural network and highly dependent upon hyper-parameters . Furthermore , it is computationally expensive to train compared to Zenke et al . ( 2017 ) and our proposed method . Jung et al . ( 2020 ) is another recent algorithm based on neuron importance . Its node importance depends on the average activation value . This idea is simple but powerful . Activation value itself is a measurement of neuron importance , and weights connected to the neuron get identical weight importance . This corresponds to the type in Figure 1c . One of our key observations in prior experimental evaluations is that the accuracy of each task significantly changes when we change the order of tasks , as it is also claimed and discussed in ( Yoon et al. , 2019 ) proposing a order robust method of continual learning in their architectural approach . Evaluating with fixed task order does not coincide with the fundamental aim of continual learning where no dedicated order of tasks is given . Figure 2 shows example test results of state of the art continual learning methods compared to our proposed method . Classification accuracy values of prior methods fluctuate as the order of tasks changes ( from Figure 2a to Figure 2b ) . Based on the observation , we propose to evaluate the robustness to the order of tasks in comprehensive manner in which we evaluate the average and standard deviation of classification accuracy with multiple sets of randomly shuffled orders . The key contributions of our work are as follows : • We propose a simple but intuitive and effective continual learning method introducing activation based neuron importance . • We propose a comprehensive experimental evaluation framework on existing benchmark data sets to evaluate not just the final accuracy of continual learning also the robustness of the accuracy along the changes of the order of tasks . Based on the evaluation framework , existing state-of-the-art methods as well as our proposed method are extensively evaluated . 2 PROPOSED METHOD . 2.1 NEURON IMPORTANCE WITH AVERAGE OF ACTIVATION VALUE . As French ( 1991 ) and French ( 1999 ) suggested , since activation overlap causes catastrophic forgetting , a successful network is required to build representations with as little overlap of activation as possible across continual tasks , called sparse representations , to mitigate mutual interference . If a network builds dense representation , a small amount of changes in the representation affects the stability of all trained classes since nodes and weights in a network are in casual relations . On the other hand , if a network produces distributed representations for all classes , alteration of representation for certain class does not affect others . In our method , we adopt NM-NI ( Node Measurement , Node Importance ) type in Figure 1c based in the measurement of activation value at each neuron . Similar to Jung et al . ( 2020 ) , we measure the average activation value of kth neuron as its neuron importance Ωk . Incoming weights connected to the same neurons have the same importance . In the case of convolutional neural networks , the average activation value corresponds to the average activation value of a feature map ( i.e. , global average pooling value ) . Average activation value itself , however , is not able to fully represent the characteristics of an essential neuron . Encoded features at each layer describe different aspects of an input image and , as a result , activation at each layer can not be evaluated together . Due to the difference in absolute average activation values across the layers , weights of earlier layers tend to be considered more essential as Figure 3 shows . If average activation value is used as neuron importance , method will prefer to keep the weights of earlier layers . Instead , we propose to use layer-wise average activation divided by respective standard deviation for importance measurement . Even though it is a simple extension of ( Jung et al. , 2020 ) , it prevents a particular layer from getting excessive importance compared to other layers , that , in turn , prevents our network from overfitting to a particular task keeping its plasticity regardless of task order . As Figure 19a shows , for each method , normalized average weight importance of each layer ( total 6 layers ) is calculated . Prior average activation based regularization term assigns around 57 % of total importance to layer 1 ( 57 % , 12 % , 10 % , 6 % , 8 % , 8 % , respectively for the 6 layers ) . On the other hand , our proposed regularization loss term assigns 26 % of total importance to layer 1 . Furthermore , our method avoids assigning excessive importance to certain layer ( 26 % , 16 % , 16 % , 15 % , 15 % , 12 % ) . Then , why this improves the continual learning performance regardless of task order ? In prior works , more weights of lower layers tend to be frozen in earlier tasks that eliminate the chance of upcoming tasks to build new low-level feature sets . Only a new task that is fortunately able to rebuild higher layer features based on the frozen lower layer weights from previous tasks could survive . On the other hand , ours keeps the balance of frozen weights in all layers securing more freedom of feature descriptions for new tasks in both lower and higher layers . Indeed , lower layer features such as edges are not class ( task ) dependent features . Therefore , freezing lower layer features actually is not preferable in continual learning . Even though lower level features are mostly changed over tasks , new task may find alternative lower level features keeping its plasticity . On the other hand , if the network fails to freeze some higher level features that are class ( task ) dependent , it may lose robustness . As we have seen , prior works spend most of their importance to lower layers losing its robustness . In order to encode such relation , we propose to use average and std . of activation values of each layer rather than using weighs directly . Our loss function is described as follows . Lt = L̃t + α ∑ l Ωtnk ( w t−1 l − w t l ) 2 , ( 1 ) where L̃t is loss of current task , t is task index , l is weight index , and Ωtnk indicates k th neuron importance . α is a strength parameter to control the amount of weights consolidation . Neuron importance is defined as follows . Ωtnk = 1 Nt Nt∑ i=1 fnk ( x ( t ) i ) σ + , σ = √√√√√ Nt∑i=1 { fk ( x ( t ) i ) − 1Nt Nt∑i=1 fk ( x ( t ) i ) } 2 Nt , ( 2 ) whereNt is the number of instances , k is neuron index , fk ( · ) is activation value , and i is instance index . We introduce to prevent the numerator from being zero when the standard deviation becomes zero .

Summary: This paper proposes a method that tackles the problem of catastrophic forgetting in continual neural networks by assigning importance to neuron activations while tasks are executed in sequence. Similar to previous research (Jung et al., 2020), the proposed method measures neuron importance using average activation values divided by corresponding standard deviation. This strategy is accompanied by weight re-initialization to guarantee that new tasks are fully learned. The method is tested in benchmark datasets for continual learning.