OvA-INN: Continual Learning with Invertible Neural Networks

In the field of Continual Learning , the objective is to learn several tasks one after the other without access to the data from previous tasks . Several solutions have been proposed to tackle this problem but they usually assume that the user knows which of the tasks to perform at test time on a particular sample , or rely on small samples from previous data and most of them suffer of a substantial drop in accuracy when updated with batches of only one class at a time . In this article , we propose a new method , OvA-INN , which is able to learn one class at a time and without storing any of the previous data . To achieve this , for each class , we train a specific Invertible Neural Network to extract the relevant features to compute the likelihood on this class . At test time , we can predict the class of a sample by identifying the network which predicted the highest likelihood . With this method , we show that we can take advantage of pretrained models by stacking an Invertible Network on top of a features extractor . This way , we are able to outperform stateof-the-art approaches that rely on features learning for the Continual Learning of MNIST and CIFAR-100 datasets . In our experiments , we reach 72 % accuracy on CIFAR-100 after training our model one class at a time . 1 INTRODUCTION . A typical Deep Learning workflow consists in gathering data , training a model on this data and finally deploying the model in the real world ( Goodfellow et al. , 2016 ) . If one would need to update the model with new data , it would require to merge the old and new data and process a training from scratch on this new dataset . Nevertheless , there are circumstances where this method may not apply . For example , it may not be possible to store the old data because of privacy issues ( health records , sensible data ) or memory limitations ( embedded systems , very large datasets ) . In order to address those limitations , recent works propose a variety of approaches in a setting called Continual Learning ( Parisi et al. , 2018 ) . In Continual Learning , we aim to learn the parameters w of a model on a sequence of datasets Di = { ( xji , y j i ) } ni j=1 with the inputs x j i ∈ X i and the labels y j i ∈ Yi , to predict p ( y∗|w , x∗ ) for an unseen pair ( x∗ , y∗ ) . The training has to be done on each dataset , one after the other , without the possibility to reuse previous datasets . The performance of a Continual Learning algorithm can then be measured with two protocols : multi-head or single-head . In the multi-head scenario , the task identifier i is known at test time . For evaluating performances on task i , the set of all possible labels is then Y = Yi . Whilst in the single-head scenario , the task identifier is unknown , in that case we have Y = ∪Ni=1Yi with N the number of tasks learned so far . For example , let us say that the goal is to learn MNIST sequentially with two batches : using only the data from the first five classes and then only the data from the remaining five other classes . In multi-head learning , one asks at test time to be able to recognize samples of 0-4 among the classes 0-4 and samples of 5-9 among classes 5-9 . On the other hand , in single-head learning , one can not assume from which batch a sample is coming from , hence the need to be able to recognize any samples of 0-9 among classes 0-9 . Although the former one has received the most attention from researchers , the last one fits better to the desiderata of a Continual Learning system as expressed in Farquhar & Gal ( 2018 ) and ( van de Ven & Tolias , 2019 ) . The single-head scenario is also notoriously harder than its multi-head counterpart ( Chaudhry et al. , 2018 ) and is the focus of the present work . Updating the parameters with data from a new dataset exposes the model to drastically deteriorate its performance on previous data , a phenomenon known as catastrophic forgetting ( McCloskey & Cohen , 1989 ) . To alleviate this problem , researchers have proposed a variety of approaches such as storing a few samples from previous datasets ( Rebuffi et al. , 2017 ) , adding distillation regularization ( Li & Hoiem , 2018 ) , updating the parameters according to their usefulness on previous datasets ( Kirkpatrick et al. , 2017 ) , using a generative model to produce samples from previous datasets ( Kemker & Kanan , 2017 ) . Despite those efforts toward a more realistic setting of Continual Learning , one can notice that , most of the time , results are proposed in the case of a sequence of batches of multiple classes . This scenario often ends up with better accuracy ( because the learning procedure highly benefits of the diversity of classes to find the best tuning of parameters ) but it does not illustrate the behavior of those methods in the worst case scenario . In fact , Continual Learning algorithms should be robust in the size of the batch of classes . In this work , we propose to implement a method specially designed to handle the case where each task consists of only one class . It will therefore be evaluated in the single-head scenario . Our approach , named One-versus-All Invertible Neural Networks ( OvA-INN ) , is based on an invertible neural network architecture proposed by Dinh et al . ( 2014 ) . We use it in a One-versus-All strategy : each network is trained to make a prediction of a class and the most confident one on a sample is used to identify the class of the sample . In contrast to most other methods , the training phase of each class can be independently executed from one another . The contributions of our work are : ( i ) a new approach for Continual Learning with one class per batch ; ( ii ) a neural architecture based on Invertible Networks that does not require to store any of the previous data ; ( iii ) state-of-the-art results on several tasks of Continual Learning for Computer Vision ( CIFAR-100 , MNIST ) in this setting . We start by reviewing the closest methods to our approach in Section 2 , then explain our method in Section 3 , analyse its performances in Section 4 and identify limitations and possible extensions in Section 5 . 2 RELATED WORK . Generative models Inspired by biological mechanisms such as the hippocampal system that rapidly encodes recent experiences and the memory of the neocortex that is consolidated during sleep phases , a natural approach is to produce samples of previous data that can be added to the new data to learn a new task . FearNet ( Kemker & Kanan , 2017 ) relies on an architecture based on an autoencoder , whereas Deep Generative Replay ( Shin et al. , 2017 ) and Parameter Generation and Model Adaptation ( Hu et al. , 2018 ) propose to use a generative adversarial network . Those methods present good results but require complex models to be able to generate reliable data . Furthermore , it is difficult to assess the relevance of the generated data to conduct subsequent training iterations . Coreset-based models These approaches alleviate the constraint on the availability of data by allowing the storage of a few samples from previous data ( which are called coreset ) . iCaRL ( Rebuffi et al. , 2017 ) and End-to-end IL ( Castro et al. , 2018 ) store 2000 samples from previous batches and rely on respectively a distillation loss and a mixture of cross-entropy and distillation loss to alleviate forgetting . The authors of SupportNet ( Li et al. , 2018 ) have also proposed a strategy to select relevant samples for the coreset . Gradient Episodic Memory ( Lopez-Paz et al. , 2017 ) ensures that gradients computed on new tasks do not interfere with the loss of previous tasks . Those approaches give the best results for single-head learning . But , similarly to generated data , it is not clear which data may be useful to conduct further training iterations . In this paper , we are challenging the need of the coreset for single-head learning . Distance-based models These methods propose to embed the data in a space which can be used to identify the class of a sample by computing a distance between the embedding of the sample and a reference for each class . Among the most popular , we can cite Matching Networks ( Vinyals et al. , 2016 ) and Prototypical Networks ( Snell et al. , 2017 ) , but these methods have been mostly applied to few-shot learning scenarios rather than continual . Regularization-based approaches These approaches present an attempt to mitigate the effect of catastrophic forgetting by imposing some constraints on the loss function when training subsequent classes . Elastic Weight Consolidation ( Kirkpatrick et al. , 2017 ) , Synaptic Intelligence ( Zenke et al. , 2017 ) and Memory Aware Synapses ( Aljundi et al. , 2018 ) all seek to prevent the update of weights that were the most useful to discriminate between previous classes . Hence , it is possible to constrain the learning of a new task in such a way that the most relevant weights for the previous tasks are less susceptible to be updated . Learning without forgetting ( Li & Hoiem , 2018 ) proposes to use knowledge distillation to preserve previous performances . The network is divided in two parts : the shared weights and the dedicated weights for each task . When learning a new task A , the data of A get assigned “ soft ” labels by computing the output by the network with the dedicated weight for each previous task . Then the network is trained with the loss of task A and is also constrained to reproduce the recorded output for each other tasks . In Rannen et al . ( 2017 ) , the authors propose to use an autoencoder to reconstruct the extracted features for each task . When learning a new task , the features extractor is adapted but has to make sure that the autoencoder of the other tasks are still able to reconstruct the extracted features from the current samples . While these methods obtain good results for learning one new task , they become limited when it comes to learn several new tasks , especially in the one class per batch setting . Expandable models In the case of the multi-head setting , it has been proposed to use the previously learned layers and complete them with new layers trained on a new task . This strategy is presented in Progressive Networks ( Rusu et al. , 2016 ) . In order to reduce the growth in memory caused by the new layers , the authors of Dynamically Expandable Networks ( Yoon et al. , 2018 ) proposed an hybrid method which retrains some of the previous weights and add new ones when necessary . Although these approaches work very well in the case of multi-head learning , they can not be adapted to single-head and are therefore not included in benchmarks with OvA-INN . 3 CLASS-BY-CLASS CONTINUAL LEARNING WITH INVERTIBLE NETWORKS .

Summary: This paper tackles continual learning problem with stacks of invertible network blocks. Similar as the ensemble idea, the proposed method learns an invertible network for each new object class. During test, for each class, each learnt network outputs a norm and the one with the smallest norm is the predicted class. The proposed model has shown performance improvements in classification accuracy on MNIST and CIFAR100 datasets in incremental class tasks.