A comprehensive, application-oriented study of catastrophic forgetting in DNNs

1 INTRODUCTION . This article is in the context of sequential or incremental learning in Deep Neural Networks ( DNNs ) . Essentially , this means that a DNN is not trained once , on a single taskD , but successively on two or more sub-tasksD1 , . . . , Dn , one after another . Learning tasks of this type , which we term Sequential Learning Tasks ( SLTs ) ( see Fig . 1a ) , are potentially very common in real-world applications . They occur wherever DNNs need to update their capabilities on-site and over time : gesture recognition , network traffic analysis , or face and object recognition in mobile robots . In such scenarios , neural networks have long been known to suffer from a problem termed “ catastrophic forgetting ” ( CF ) ( e.g. , French ( 1999 ) ) which denotes the abrupt and near-complete loss of knowledge from previous subtasks D1 , . . . , Dk−1 after only a few training iterations on the current sub-task Dk ( see Fig . 1b compared to Fig . 1c ) . We focus on SLTs from the visual domain with two sub-tasks each , as DNNs show pronounced CF behavior even when only two sub-tasks are involved . trainD1 testD1 sub-taskD1 sub-taskD2 trainD2 testD2 test D1∪D2 0 E 2E testD1 ( a ) Training scheme 0 2 epoch 0.0 0.5 1.0 te st a cc ur ac y D1 D2 D1 D2 training D1 retraining D2 ( b ) with CF 0 2 epoch 0.0 0.5 1.0 te st a cc ur ac y D1 D2 D1 D2 training D1 retraining D2 ( c ) without CF Figure 1 : Scheme of incremental training experiments conducted in this article ( a ) and representative outcomes with ( b ) and without CF ( c ) . The sequential learning tasks used in this study only have two sub-tasks : D1 and D2 . During training ( white background ) and re-training ( gray background ) , test accuracy is measured on D1 ( blue , M ) , D2 ( green , 2 ) and D1 ∪ D2 ( red , # ) . The blue curve allows to determine the presence of CF by simple visual inspection : if there is significant degradation w.r.t . the red curve , then CF has occurred . 1.1 DISCUSSION OF RELATED WORK ON CF . The field of incremental learning is large , e.g. , Parisi et al . ( 2018 ) and Gepperth & Hammer ( 2016 ) . Recent systematic comparisons between different DNN approaches to avoid CF are performed in , e.g. , Serra et al . ( 2018 ) or Kemker et al . ( 2018 ) . Principal recent approaches to avoid CF include ensemble methods ( Ren et al. , 2017 ; Fernando et al. , 2017 ) , dual-memory systems ( Shin et al. , 2017 ; Kemker & Kanan , 2017 ; Rebuffi et al. , 2017 ; Gepperth & Karaoguz , 2016 ) and regularization approaches . Whereas Goodfellow et al . ( 2013 ) suggest Dropout for alleviating CF , the EWC method ( Kirkpatrick et al. , 2017 ) proposes to add a term to the energy function that protects weights that are important for the previous sub-task ( s ) . Importance is determined by approximating the Fisher information matrix of the DNN . A related approach is pursued by the Incremental Moment Matching technique ( IMM ) ( see Lee et al . ( 2017 ) ) , where weights from DNNs trained on a current and a past sub-tasks are “ merged ” using the Fisher information matrix . Other regularization-oriented approaches are proposed in Aljundi et al . ( 2018 ) ; Srivastava et al . ( 2013 ) and Kim et al . ( 2018 ) which focus on enforcing sparsity of neural activities by lateral interactions within a layer . Number of tested datasets In general , most methods referenced here are evaluated only on a few datasets , usually on MNIST ( LeCun et al. , 1998 ) and various derivations thereof ( permutation , rotation , class separation ) . Some studies make limited use of CIFAR10 , SVHN , the Amazon sentiment analysis problem , and non-visual problems such as data from Q-learning of Atari games . A largescale evaluation on a huge number of qualitatively different datasets is still missing1 . Model selection and prescience Model selection ( i.e. , selecting DNN topology and hyperparameters ) is addressed in some approaches ( Goodfellow et al. , 2013 ) but on the basis of a “ prescient ” evaluation where the best model is selected after all tasks have been processed , an approach which is replicated in Kirkpatrick et al . ( 2017 ) . This amounts to a knowledge of future sub-tasks which is problematic in applications . Most approaches ignore model selection ( Lee et al. , 2017 ; Srivastava et al. , 2013 ; Aljundi et al. , 2018 ; Kim et al. , 2018 ) , and thus implicitly violate causality . Storage of data from previous sub-tasks From a technical point of view , DNNs can be retrained without storing training data from previous sub-tasks , which is done in Goodfellow et al . ( 2013 ) and Srivastava et al . ( 2013 ) . For regularization approaches , however , there are regularization parameters that control the retention of previous knowledge , and thus must be chosen with care . In Kirkpatrick et al . ( 2017 ) , this is λ , whereas two such quantities occur in Lee et al . ( 2017 ) : the “ balancing ” parameter α and the regularization parameter λ for L2-transfer . The only study where regularization parameters are obtained through cross-validation ( which is avoided in other studies ) is Aljundi et al . ( 2018 ) ( for λSNI and λΩ ) but this requires to store all previous training data . This review shows that enormous progress has been made , but that there are shortcomings tied to applied scenarios which need to be addressed . We will formalize this in Sec . 1.2 and propose an evaluation strategy that takes these formal constraints into account when testing CF in DNNs . 1.2 INCREMENTAL LEARNING IN APPLIED SCENARIOS . When training a DNN model on SLTs , first of all the model must be able to be retrained at any time by new classes ( class-incremental learning ) . Secondly , it must exhibit retention , or at least graceful decay , of performance on previously trained classes . Some forgetting is probably unavoidable , but it should be gradual and not immediate , i.e. , catastrophic . However , if a DNN is operating in , e.g. , embedded devices or autonomous robots , additional conditions may be applicable : Low memory footprint Data from past sub-tasks can not be stored and used for re-training , or else to determine when to stop re-training . Causality Data from future sub-tasks , which are often known in academic studies but not in applications , must not be utilized in any way , especially not for DNN model selection . This point might seem trivial , but a number of studies such as Kirkpatrick et al . ( 2017 ) ; Goodfellow et al . ( 2013 ) and Srivastava et al . ( 2013 ) perform model selection in hindsight , after having processed all sub-tasks . Constant update complexity Re-training complexity ( time and memory ) must not depend on the number of previous sub-tasks , thus more or less excluding replay-based schemes such as Shin et al . ( 2017 ) . Clearly , even if update complexity is constant w.r.t . the number of previous sub-tasks , it should not be too high in absolute terms either . 1Although the comparisons performed in Aljundi et al . ( 2018 ) include many datasets , the experimental protocol is unclear , so it is uncertain how to interpret these results . 1.3 CONTRIBUTION AND PRINCIPAL CONCLUSIONS . The original contributions of our work can be summarized as follows : • We propose a training and evaluation paradigm for incremental learning in DNNs that enforces typical application constraints , see Sec . 1.2 . The importance of such an applicationoriented paradigm is underlined by the fact that taking application constraints into account leads to radically different conclusions about CF than those obtained by other recent studies on CF ( see Sec . 1.1 ) . • We investigate the incremental learning capacity of various DNN approaches ( Dropout , LWTA , EWC and IMM ) using the largest number of qualitatively different classification datasets so far described . We find that all investigated models are afflicted by catastrophic forgetting , or else in violation of application constraints and discuss potential workarounds . • We establish that the “ permuted ” type of SLTs ( e.g. , “ permuted MNIST ” ) should be used with caution when testing for CF . • We do not propose a method for avoiding CF in this article . This is because avoiding CF requires a consensus on how to actually measure this effect : our novel contribution is a proposal how to do just that . 2 METHODS AND DNN MODELS . We collect a large number of visual classification datasets , from each of which we construct SLTs according to a common scheme , and compare several recent DNN models using these SLTs . The experimental protocol is such that application constraints , see Sec . 1.2 , are enforced . For all tested DNN models ( see below ) , we use a TensorFlow ( v1.7 ) implementation under Python ( v3.4 and later ) . The source code for all processed models , the experiment-generator and evaluation routine can be found on our public available repository 2 . FC A normal , fully-connected ( FC ) feed-forward DNN with a variable number and size of hidden layers , each followed by ReLU , and a softmax readout layer minimizing cross-entropy . CONV A convolutional neural network ( CNN ) based on the work of Cirean et al . ( 2011 ) . It is optimized to perform well on image classification problems like MNIST . We use a fixed topology : two conv-layers with 32 and 64 filters of size 5×5 plus ReLU and 2×2 max-pooling , followed by a fc-layer with 1024 neurons and softmax readout layer minimizing a cross-entropy energy function . EWC The Elastic Weight Consolidation ( EWC ) model presented by Kirkpatrick et al . ( 2017 ) . LWTA A fully-connected DNN with a variable number and size of hidden layers , each followed by a Local Winner Takes All ( LWTA ) transfer function as proposed in Srivastava et al . ( 2013 ) . IMM The Incremental Moment Matching model as presented by Lee et al . ( 2017 ) . We examine the weight-transfer techniques in our experiments , using the provided implementation . D-FC and D-CONV Motivated by Goodfellow et al . ( 2013 ) we combine the FC and CONV models with Dropout as an approach to solve the CF problem . Only FC and CONV are eligible for this , as EWC and IMM include dropout by default , and LWTA is incompatible with Dropout . 2.1 HYPER-PARAMETERS AND MODEL SELECTION . We perform model selection in all our experiments by a combinatorial hyper-parameter optimization , whose limits are imposed by the computational resources available for this study . In particular , we vary the number of hidden layers L ∈ { 2 , 3 } and their size S ∈ { 200 , 400 , 800 } ( CNNs excluded ) , the learning rate 1 ∈ { 0.01 , 0.001 } for sub-task D1 , and the re-training learning rate 2 ∈ { 0.001 , 0.0001 , 0.00001 } for sub-task D2 . The batch size ( batchsize ) is fixed to 100 for all experiments , and is used for both training and testing . As in other studies , we do not use a fixed number of training iterations , but specify the number of training epochs ( i.e. , passes through the whole dataset ) as E = 10 for each processed dataset ( see Sec . 2.2 ) , which allows an approximate comparison of different datasets . The number of training/testing batches per epoch , B , can be calculated from the batch size and the currently used dataset size . The set of all hyper-parameters for 2https : //gitlab.informatik.hs-fulda.de/ML-Projects/CF_in_DNNs a certain model , denoted P , is formed as a Cartesian product from the allowed values of the hyperparameters L , S , 1 , 2 and complemented by hyper-parameters that remain fixed ( E , batchsize ) or are particular to a certain model . For all models that use dropout , the dropout rate for the input layer is fixed to 0.2 , and to 0.5 for all hidden layers . For CNNs , the dropout rate is set to 0.5 for both input and hidden layers . All other hyper-parameters for CNNs are fixed , e.g. , number and size of layers , the max-pooling and filter sizes and the strides ( 2 × 2 ) for each channel . These decisions were made based on the work of Goodfellow et al . ( 2013 ) . The LWTA block size is fixed to 2 , based on the work of Srivastava et al . ( 2013 ) . The model parameter λ for EWC is set to λ1/ 2 ( set but not described in the source code of Kirkpatrick et al . ( 2017 ) ) . For all models except IMM , the momentum parameter for the optimizer is set to µ = 0.99 ( Sutskever et al. , 2013 ) . For the IMM models , the SGD optimizer is used , and the regularizer value for the L2-regularization is set to 0.01 for L2-transfer and to 0.0 for weight transfer .

Summary: The paper presents a study of the application of some well known methods on 9 datasets focusing on the issue of catastrophic forgetting when considering a sequential learning task in them.  In general the presentation of concepts and results is a bit problematic and unclear. Comments, such that the paper presents ' a novel training and model selection paradigm for incremental learning in DNNs ' is not justified.  A better description of the results, e.g., in Table 3 should be presented, as well a better linking with the findings; a better structure of the latter would also be required to improve consistency of them. Improving these could make the paper candidate for a poster presentation.