OvA-INN: Continual Learning with Invertible Neural Networks
Authors: HOCQUET Guillaume
BICHLER Olivier
QUERLIOZ Damien
Venue:  (2020)
Keywords: Deep Learning
Continual Learning
Invertible Neural Networks

Abstract: In the field of Continual Learning, the objective is to learn several tasks one after the other without access to the data from previous tasks. Several solutions have been proposed to tackle this problem but they usually  assume that the user knows which of the tasks to perform at test time on a particular sample, or rely on small samples from previous data and most of them suffer of a substantial drop in accuracy when updated with batches of only one class at a time. In this article, we propose a new method, OvA-INN, which is able to learn one class at a time and without storing any of the previous data. To achieve this, for each class, we train a specific Invertible Neural Network to output the zero vector for its class. At test time, we can predict the class of a sample by identifying which network outputs the vector with the smallest norm. With this method, we show that we can take advantage of pretrained models by stacking an invertible network on top of a features extractor. This way, we are able to outperform state-of-the-art approaches that rely on features learning for the Continual Learning of MNIST and CIFAR-100 datasets. In our experiments, we are reaching 72% accuracy on CIFAR-100 after training our model one class at a time.

Under review as a conference paper at ICLR 2020
OVA-INN: CONTINUAL LEARNING WITH INVERTIBLE NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
In the field of Continual Learning, the objective is to learn several tasks one after the other without access to the data from previous tasks. Several solutions have been proposed to tackle this problem but they usually assume that the user knows which of the tasks to perform at test time on a particular sample, or rely on small samples from previous data and most of them suffer of a substantial drop in ac- curacy when updated with batches of only one class at a time. In this article, we propose new method, OvA-INN, which is able to learn one class at time and without storing any of the previous data. To achieve this, for each class, we train a specific Invertible Neural Network to extract the relevant features to compute the likelihood on this class. At test time, we can predict the class of a sample by iden- tifying the network which predicted the highest likelihood. With this method, we show that we can take advantage of pretrained models by stacking an Invertible Network on top of a features extractor. This way, we are able outperform state- of-the-an art approaches that rely on features learning for the Continual Learning of MNIST and CIFAR-100 datasets. In our experiments, we reach 72% accuracy on CIFAR-100 after training our model one class at a time.
INTRODUCTION
A typical Deep Learning workflow consists in gathering data, training a model on this data and finally deploying the model in the real world (Goodfellow et al. 2016). If one would need to update the model with new data, it would require to merge the old and new data and process training from scratch on this new dataset. Nevertheless. there are circumstances where this method may not apply. For example, it may not be possible store the old data because of privacy issues (health records, sensible data) or memory limitations (embedded systems, very large datasets). In order to address those limitations, recent works propose a variety of approaches in setting called Continual Learning (Parisi et al. 2018)
In Continual Learning, we aim to learn the parameters w of a model on a sequence of datasets D. = {(2 yd >) with the inputs z! E X and the labels y E y to predict p(y") (,,,"",)) x²) for an unseen pair y"), The training has to be done on each dataset, one after the other, without the possibility to reuse previous datasets. The performance of Continual Learning algorithm can then be measured with two protocols: multi-head or single-head. In the multi-head scenario, the task identifier is known at test time. For evaluating performances labels is then y = yr Whilst in the single-head scenario, the task identifier is unknown, in that case we have y L yi with N the number of tasks learned so far. For example, let us say that the goal is to learn MNIST sequentially with two batches: using only the data from the first five classes and then only the data from the remaining five other classes. In multi-head learning, one asks at test time to be able to recognize samples of 0-4 among the classes 004 and samples of 5-9 among classes 5-9. On the other hand, in single-head learning, one can not assume from which batch a sample is coming from, hence the need to be able to recognize any samples of (-9 among classes 0-9. Although the former one has received the most attention from researchers, the last one fits better to the desiderata of a Continual Learning system expressed in Farquhar & Gal (2018) and (van de Ven & Tolias 2019). The single-head scenario is also notoriously harder than its headi-head counterpart (Chaudhry et al. 2018) and is the focus of the present work.
task i, the of all possible

---

Under review as a conference paper at ICLR 2020
Updating the parameters with data from new dataset exposes the model to drastically deteriorate its performance on previous data, a phenomenon known as catastrophic forgetring (McCloskey & Co-
hen ) To alleviate this problem. researchers have proposed variety of approaches such stor-
ing a few samples from previous datasets (Rebufti et al. (017). adding distillation regularization (Li & Hoiem 2018); updating the parameters according to their usefulness on previous datasets (Kirk- patrick et al. [201]). using a generative model to produce samples from previous datasets (Kemker & Kanan 2017). Despite those efforts toward more realistic setting of Continual Learning, one can notice that, most of the time, results are proposed in the case of sequence of batches of multiple classes. This scenario often ends up with better accuracy (because the learning procedure highly benefits of the diversity of classes to find the best tuning of parameters) but it does not illustrate the behavior of those methods in the worst case scenario. In fact, Continual Learning algorithms should be robust in the size of the batch of classes.
In this work, we propose to implement a method specially designed to handle the case where each task consists of only one class. It will therefore be evaluated in the single-head scenario. Our approach, named One-versus- All Invertible Neural Networks (OvA-INN), is based on an invertible neural network architecture proposed by Dinh al. (014). We use it in One-versus-All strategy : each network is trained to make a prediction of a class and the most confident one on a sample is used identify the class of the sample. In contrast to most other methods, the training phase of each class can be independently executed from one another.
The contributions of our work are (i) a new approach for Continual Learning with one class per batch; (ii) a neural architecture based on Invertible Networks that does not require to store any of the previous data; (iii) state-of-the-an results on several tasks of Continual Learning for Computer Vision (CIFAR- 100, MNIST) in this setting.
We start by reviewing the closest methods to our approach in Section 2 then explain our method in Section 3 analyse its performances in Section 4 and identify limitations and possible extensions in Section
2 RELATED WORK
Generative models Inspired by biological mechanisms such as the hippocampal system that rapidly encodes recent experiences and the memory of the neocortex that is consolidated during sleep phases, natural approach is to produce samples of previous data that can be added to the new data to learn new task. FearNet (Kemker & Kanan| (017)) relies on an architecture based on an autoencoder, whereas Deep Generative Replay (Shin et al. (017] and Parameter Generation and Model Adaptation (Hu et al., 2018) propose to use generative adversarial network. Those methods present good results but require complex models to able to generate reliable data. Furthermore, it is difficult to assess the relevance of the generated data to conduct subsequent training iterations.
Coreset-based models These approaches alleviate the constraint the availability of data by allowing the storage of a few samples from previous data (which are called coreset). iCaRL (Rebuffi) ett al. 2017) and End-to-end IL (Castro et al.) 2018) store 2000 samples from previous batches and rely on respectively a distillation loss and a mixture of cross-entropy and distillation loss to alleviate forgetting. The authors of SupportNet (Li et al. 2018) have also proposed a strategy select relevant samples for the coreset. Gradient Episodic Memory (Lopez-Paz et al.] 2017)) ensures that gradients computed on new tasks do not interfere with the loss of previous tasks. Those approaches give the best results for single-head learning. But, similarly to generated data, it is not clear which data may be useful to conduct further training iterations. In this paper, we are challenging the need of the coreset for single-head learning.
Distance-based models These methods propose to embed the data in space which can be used to identify the class of a sample by computing a distance between the embedding of the sample and reference for each class. Among the most popular, we can cite Matching Networks (Vinyals al.) 2016) and Prototypical Networks (Snell et al. (2017). but these methods have been mostly applicd to few-shot learning scenarios rather than continual.
2

---

Under review as a conference paper at ICLR 2020
reguularization-basee approaches These approaches present an attempt to mitigate the effect of catastrophic forgetting by imposing some constraints on the loss function when training subsequent classes. Elastic Weight Consolidation (Kirkpatrick al. (2017). Synaptic Intelligence (Zenke al. 2017) and Memory Aware Synapses (Aljundi et al. 2018) all seek to prevent the update of weights that were the most useful discriminate between previous classes. Hence, it is possible constrain the learning of new task in such a way that the most relevant weights for the previous tasks are less susceptible to be updated. Learning without forgetting (Li & Hoiem 2018) proposes use knowledge distillation to preserve previous performances. The network is divided in two parts the shared weights and the dedicated weights for each task. When learning a new task A, the data of A get assigned "soft" labels by computing the output by the network with the dedicated weight for each previous task. Then the network is trained with the loss of task A and is also constrained to reproduce the recorded output for each other tasks. In [annen et al. (2017), the authors propose to use an autoencoder to reconstruct the extracted features for each task. When learning a new task, the features extractor is adapted but has to make sure that the autoencoder of the other tasks are still able to reconstruct the extracted features from the current samples. While these methods obtain good results for learning one new task, they become limited when it comes to learn several new tasks, especially in the one class per batch setting.
Expandable models In the case of the multi-head setting, it has been proposed to use the pre- viously learned layers and complete them with new layers trained new task. This strategy is presented in Progressive Networks (Rusu et al. (016). In order to reduce the growth in memory caused by the new layers, the authors of Dynamically Expandable Networks (Yoon al. 2018) proposed an hybrid method which retrains some of the previous weights and add new ones when necessary. Although these approaches work very well in the case of multi-head learning, they can not be adapted to single-head and are therefore not included in benchmarks with OvA-INN.
3 CLASS-BY-CLASS CONTINUAL LEARNING WITH INVERTIBLE NETWORKS
3.1 MOTIVATIONS AND CHALLENGE
We investigate the problem of training several datasets in a sequential fashion with batches of only one class at a time. Most approaches of the state-of-the-art rely on updating a features extractor when data from new class is available. But this strategy is unreliable in the special case we are interested in, namely batches of data from only one class. With few or no sample of negative data, it is very inefficient to update the weights of a network because the setting of deep learning normally involves vast amounts of data to be able to Iearn to extract valuable features. Without enough negative samples, the training is prone to overfit the new class. Recent works have proposed to rely on generative models to overcome this lack of data by generating samples of old classes. Nevertheless, updating network with sampled data is not as efficient as with real data and, on the long run, the generative quality of early classes suffer from the multiple updates.
3.2 OUT-OF-DISTRIBUTION DETECTION FOR CONTINUAL LEARNING
Our approach consists in interpreting a Continual Learning problem as several out-of- distrribbuooon (OOD) detection problems. OOD detection has already been studied for neural networks and can be formulated as binary classification problem which consists in predicting if an input x was sampled from the same distribution as the training data or from a different distribution (Lee et al., 2017] Liang et al. 2017). Hence, for each class, we can train network to predict if an input x is likely to have been sampled from the distribution of this class. The class with the highest confidence can be used the prediction of the class of æ. This training procedure is particularly suitable for Continual Learning since the training of each network does not require any negative sample.
Using the same protocol as NICE (Dinh al. 2014), for class i, it is possible to train neural network f; to fit a prior distribution and compute the exact likeliiel I, on a sample x
4 (x) = log(p(f(zz))
To obtain the formulation of log-llieeiihonl as expressed in Equation I the network fo has to respect some constraints discussed in Section 3.3 Keeping the same hypothesis NICE, we consider the
3

---

Under review as a conference paper at ICLR 2020
22
y0.n/ /
fi
f2
22m/
+
//1/2.1
Figure 1: Forward pass in an invertible block. x: is split in 20m//2 and Zn/2:nn fi and fz can be any type of Neural Networks as long as the dimension of their output dimension the same as their input dimension. In our experiments, we stack two of these blocks one after the other and use fully-connected feedforward layers for fi and f2.
case where p is distribution with independent components pa
p(f(z)) II pa(Saa(>))
In our experiments, we considered Pa be standard normal distributions. Although, it is possible to learn the parameters of the distributions, we found experimentally that doing decreases the results. Under these design choices, the computation the log-likelihood becomes
4(x) = > Log(pp((fr.((t) = Σ hal(t) + Σ
-
log
= [lf(())11 (3)
2m
where B log (22x)) is constant term.
Hence, identifying the network with the highest log-likelihood is equivalent to find the network with the smallest output norm.
3.3 INVERTIBLE NEURAL NETWORKS
The neural network architecture proposed by NICE is designed to operate a change of variables between density functions. This assumes that the network is invertible and respect some con- straints to make it efficiently computable.
An invertible block (see Figure consists in splitting the input o into two subvectors ø1 and 22 of equal size; then successively applying two (non necessarily invertible) networks fi and following the equation:
yi (42) + xi 1/2 = f2( ) + 22;
(4)
and finally, concatenate y and y2- The inverse operation be computed with
J2 1/2 f2(11)
(5)
V1 fx22))
These invertible equations illustrate how Invertible Networks operate bijection between their input and their output.
3.4 CONTINUAL LEARNING SETTING
We propose to specialize each Invertible Network to specific class by training them output vector with small norm when presented with data samples from their class. Given a dataset X; of class i and Invertible Network ft, objective is to minimize the loss L.

---

Under review as a conference paper at ICLR 2020
((2))
Σ 1 AC
Once the training has converged, the weights of this network won't be updated when classes will be added. At inference time, after learning classes, the predicted class y" for sample æ is obtained by running each network and identifying the one with the smallest output
y" = arg min (z)(1 1 I...t
As it is common practice in image processing, one can also use a preprocessing step by applying a fixed pretrained features extractor beforehand.
EXPERIMENTAL. RESULTS
We compare our method against several state-of-the-ar baselines for single-head learning on MNIST and CIFAR-100 datasets.
4.1 IMPLEMENTATION DETAILS
Topology of OvA-INN Due to the bijective nature of Invertible Networks, their output size is the same their input size, hence the only way change their size is by changing the depth or by compressing the parameters of the intermediate networks f1 and fz. In our experiments, these networks are fully connected layers. To reduce memory footprint, we replace the square matrix of parameters W of size X by product of matrices AB of sizes compressing factor for the first and second block m = 16 for MNIST and m 32 for CIFAR-100). More details on the memory cost can be found in Appendix
and X n (with
Regularization When performing learning class at time, the amount of training data be highly reduced: only 500 training samples per class for CIFAR-100. To avoid overfitting the training set, found that adding weight decay regularization could increase the validation accuracy. More details on the hyperparameters choices can be found in Appendix B
Rescaling As ResNet has been trained on images of size 224 x 224, we rescale CIFAR-100 100 images to match the size of images from Imagenet.
4.2 EVALUATION ON MNIST
We start by considering the MNIST dataset (eCunn et al. 1998 as it is a common benchmark that remains challenging the case of single-head Continual Learning.
Baselines
Generative models:
Parameter Generation and Model Adaptation (PGMA) (Hu et al. 2018
Deep Generative Replay (DGR) (Shin et al. 2017)
Coreset-based models:
iCaRL (Rebuffi et al. 2017)
SupportNet (Li et al. 2018)
For Parameter Generation and Model Adaptation (PGMA) (Hu et al. 2018) and Deep Generative Replay (DGR) (Shin et al. 2017), we report the results from the original papers; whereas we use the provided code of SupportNet compute the results for iCaRL and SupportNet with the conventional architecture of two layers of convolutions with poolings and a fully connected last layer. have also set the coreset size
800 samples.
5

---

Under review as a conference paper at ICLR 2020
Table Comparison of accuracy and memory cost in number of parameters (and memory usage for storing samples if relevant) of different approaches on MNIST at the end of the Continual Learning. The Learning type column indicates the number of classes used each training step.
Model
Accuracy (%)
Memory cost
Learning type
PGMA (Hu et al. 2018)
81.7
6,000k
2 by 2
SupportNer (l.1 et al. 2018)
89.9
940k
by
DGR (Shin et al. 2017)
95.8
12,700k
2 by
iCaRL (Rebuth et at. 2017)
96.0
940k
2 by
OvA-INN (this work)
96.4
520k
1 by 1
Analysis
We report the average accuracy over all the classes after the networks have been trained on all batches (See Table [T. Our architecture does not use any pretrained features extractor common to every classes (contrarily to our CIFAR-100 experiment) each sample is processed through an Invertible Network, composed of two stacked invertible blocks.
Our approach presents better results than all the other reference methods while having smaller cost in memory (see Appendix A and being trained by batches of only one class. Also, our architec- ture relies simple fulne-connected layers (as parts of invertible layers) whilst the other baselines implement convolutional layers.
4.3 EVALUATION ON CIFAR-100
We now consider a more complex image dataset with greater number of classes. This allows us to make comparisons in the case of long sequence of data batches and illustrate the value of using a pretrained features extractor for Continual Learning.
Baselines
Distance-batt based model:
Nearest prototype implementation of the method consisting in computing the mean vector (prototype) of the output of a pretrained ResNet32 for each class at train time. Inference is performed by finding the closest prototype to the ResNet output of given sample.
Generative model:
pretrained ResNet48 features extractor. FearNet is trained with a warm-up phase. Namely, the network is first trained with the all the first 50 classes of CIFAR-100, and subsequently learns the next 50 classes one by in continual fashion.
FearNet (Kemker & Kanan
2017
Coreset-based models:
iiCaRL (Rebuffi et al. 2017) retrains ResNet32 architecture on new data with distillation loss. End-to-end IL (Castro et al. 2018) retrains a ResNet32 architecture on new data with a cross- entropy together with distillation loss.
Analysis
The data is provided by batch of classes. When the training batch (D;) is completed, the accu- racy of the classifier is evaluated on the test data classes from all previous batches (DI..., D.). We report the results from the literature with various size of batch when they are available.
OvA-INN uses the weights of ResNet32 pretrained on ImageNet and never update them. FearNet also pretrained weights from ResNet. iCaRL and End-to-End IL use this architecture but retrain from scratch at the beginning and fine-tune with each new batch.
The performance of the Nearest prototype baseline proves that there is high benefit in using pre- trained features extractor on this kind of dataset. FearNet shows better performance by taking ad- vantage of warm-up phase with 50 classes. Still, outperform all the other approaches, reaching 72% accuracy after training on 100 classes. We can that the performances of methods retraining ResNet from scratch (iCaRL. and End-to-End II.)
that OvA-INN is able to clearly

---

Under review as a conference paper at ICLR 2020
100
90
80
70
Accuracy (%)
60
50
OvA-INN (1 by 1)
40
FearNet by
Nearest prototype (1 by 1)
30
End-to-End (10 by 10)
iCaRL (10 by 10)
20
End-to-End IL (2 by 2)
10
iCaRL (5 by
iCaRL (2 by 2)
0
20
40
60
80
100
Number of classes
Figure 2: Comparison of the accuracy of several Continual Learning methods on CIFAR-100 100 with various batches of classes. FearNet's curve has no point before classes because the first 50 classes are learned in non-continous fashion.
quickly deteriorate compared those using pretrained parameters. Even with larger batches of classes, the gap is still present.
can be surpising that at the end of its warm-up phase, FearNet still has accuracy bellow OvA- INN, even though has been trained on all the data available at this point. It should be noted that FearNet training an autoencoder and uses its encoding part as a features extractor (stacked on the ResNet) before classifying a sample. This can diminish the discriminative power of the network since it also constrained to reproduce input (only single autoencoder is used for all classes).
To further understand the effect of an Invertible Network on the feature space of sample, we propose to project the different features spaces in 2D using t-SNE (Maaten & Hinton 2008). We project the features of the five first classes of CIFAR-100 test set (see Figure 3). Classes that are already well represented in a cluster with ResNet features (like violet class) are clearly separated from the clusters of Invertible Networks. Classes represented with ambiguity with ResNet features (like light green and red) are better clustered in the Invertible Network space.
DISCUSSION
limiting factor in our approach the necessity to add a new network each time one wants to learn new class. This makes the memory and computational cost of OvA-INN linear with the number of classes. Recent works networks merging could alleviate the memory issue by sharing weights (Chou al. 2018) relying weights superposition (Cheung et al. 2019). This being said, showed that Ova-INN was able to achieve superior accuracy on CIFAR- 100 class-by-class training than approaches reported in the literature, while using less parameters.
Another constraint of using Invertible Networks is to keep the size of the output equal to the size of the input. When wants apply features extractor with high number of output channels, can have a very negative impact on the memory consumption the invertible layers. Feature Selection Feature Aggregation techniques may help alleviate this issue (Tang al. 2014).

---

Under review as a conference paper at ICLR 2020
(a)
(c)
(d)
(e)
(f)
Figure 3: t-SNE projections of features spaces for five classes from CIFAR- 100 test set (colors are given by the ground truth). (a): features space before applying Invertible Networks (black crosses the clusters centers). (b). each features space after the Invertible Network of each class. The samples of a class represented by network are clustered around the zero vector (black cross) whilst the samples from other classes appear further away from the cluster. Another visualization highlighting the differences between OvA-INN and Nearest Prototype is presented in Appendix D
Finally, we can notice that our approach highly dependent on the quality of the pretrained fea- tures extractor. In our CIFAR-100, we had to rescale the input to make it compatible with ResNet. Nonetheless, recent research works show promising results training features extractors in very efficient ways (Asano et al., 2019). Because it does not require to retrain its features extractor, we can foresee better performance class-by-class learning with OvA-INN as new and more efficient features extractors are discovered.
As future research direction, one could try to incorporate our method Reinforcement Learning scenario where various situations can be learned separately in first phase (each situation with its own Invertible Network). Then during a second phase where any situation can appear without the agent explicitly told in which situation it is in, the agent could rely previously trained Invertible Networks to improve its policy. This setting is closely related to Options in Reinforcement Learning. Also, in regression setting, one add fully connected layer after intermediate layer of an Invertible Network and use it to predict the output for the trained class. At test time, one only need to read the output from the regression layer of the Invertible Network that had the highest confidence.
CONCLUSION
In this paper, we proposed new approach for the challenging problem of single-head Continual Learning without storing any of the previous data. On top of fixed pretrained neural network, we trained for each class Invertible Network to refine the extracted features and maximize the log- likelihood on samples from its class. This way, we show that we can predict the class of sample by running each Invertible Network and identifying the with the highest log- lieliiill This setting allows us to take full benefit of pretrained models, which results in very good performances on the class-by-class training of CIFAR-10 compared to prior works.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Rahaf Aljundi, Francesca Babiloni. Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pp. 139-154, 2018.
Yuki M Asano, Christian Rupprecht, and Andrea Vedaldi. Surprising effectiveness of few-image unsupervised feature learning. arXiv preprint 19X1:1994..1132,, 2019.
Francisco M Castro, Manuel J Marín-liménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 233-248, 2018.
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Rieman- nian walk for incremental learning: Understanding forgetting and intransigence. arXiv preprint 1rXh::1801.10112 2018.
Brian Cheung, Alex Terekhov, Yubei Chen, Pulkit Agrawal, and Bruno A. Olshausen. Superposition of many models into one. CoRR, abs/1902.05522, 2019. URL
wwwp:/ii.orraa.o.. /bs/
19C2.05522
Yi-Min Chou, Yi-Ming Chan, Jia-Hong Lee, Chih- Chiu, and Chu-Song Chen. Unifying and merging well-trained deep neural networks for inference stage. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 2049-
2056. International Joint Conferences Artificial Intelligence Organization, 7 2018. doi: 10.24963/ijcci 201.i220012233
http://66...rg/012226/36//////// 20c.cii.1111. 22200122220 /283
Laurent Dinh, David Krueger, and Yoshua Bengio. NICE: Non-lines independent components estimation. arXiv preprint arXiv:1410.8516, 2014.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning, arXiv preprint 10X0::8805.09733. 2018
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. 2016.
Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting via model adaptation. 2018.
Ronald Kemker and Christopher Kanan. FearNet: Brain-inspired model for incremental learning.
arXiv preprint arXiv:171..1066. 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1:Xi::4412.6980 2014.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, pp. 201611835, 2017.
et al. Overcom-
Yann LeCun, Leon Bottou, Yoshua Bengio, Patrick Haffner, al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1
1998.
Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confideence-allbrateed classifiers for detecting out-t--distribution samples. arXiv preprint arXiv:1711.09325, 2017.
Yu Li, Zhongxiao Li, Lizhong Ding, Peng Yang, Yuhui Hu, Wei Chen, and Xin Gao. SupportNet: solving catastrophic forgetting in class incremental learning with support data. arXiv preprint vrXiv:1806.0294/ 2018.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEFE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935-294) 2018
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of duitrf--iittiiutiinn image detection in neural networks. arXiv preprint /rXiv 2017.

---

Under review as a conference paper at ICLR 2020
David Lopez-Paz et al. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, pp. 6467-6476, 2017.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of machine learning research, (NNv))2579-2605, 2008.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. 24:109-166 165, 1989.
German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. arXiv preprint arXiv: 2018.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Detito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Amal Rannen, Rahaf Aljundi, Matthew B Blaschko, and Tinne Tuytelaars. Encoder based lifelong learning. pp. 1920-1328, 2017.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. iCaRL: Incremental classifier and representation learning. pp. 5533-5542, 2017.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1oXv::660.04671,, 2016.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. pp. 2990-2999, 2017.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. pp- 4077-4087, 2017.
Jiliang Tang. Salem Alclyani, and Huan Liu. Feature selection for classification: A review. Data classification: Algorithms and applications, pp. 37, 2014.
Gido M. van de Ven and Andreas S. Tolias. Three scenarios for continual learning. CoRR, httgg///s/1110..0000
abs/1 abs/1904.00734,, 2019.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. pp. (630-3638, 2016.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 2017.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. 2018.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. pp. 3987-3995, 2017.
A MEMORY USAGE
A.1 MNIST
OvA-INN uses blocks with 2 layers (f1 and f2) for 10 classes. The weight matrix of each layer W is product of two matrices A and B of size 392 x 16 and 16 x 392. The memory required for OvA-INN is:
SONAINNNMIST (392 16 2 392)
2 2 10 517440
We set the coreset size of iCaRL and SupportNet to = 800 with each image of size 28 x 28, the convolutional network is composed of a layer of 64 channels with 5 x 5 kernel, layer of 32
10

---

Under review as a conference paper at ICLR 2020
channels with 5 x 5 kernel, fully-connected layer with 100 channels applied input of size 7 x 7 and a final layer of 10 channels:
MINSTT 28 28 800 + (6x5+1) 32 (5x 64 7x 100 (100 + 1) x 10- = 944406
A.2 CIFAR-100
Since every method rely ResNet32 (around 20M parameters) to compute their features (except FearNet which uses ResNet48), We do not count the features extractor in the memory consumption.
OvA-INN uses 2 blocks with 2 layers (fi and f2) for 100 classes. The weight matrix of cach layer W is product of two matrices and of size 256 x 32 and 32 x 256. The memory required is
SONA-INN.CHAR (256 x 32 x 2 256) 2 x 100 6656000
We use the default coreset size = 2000 iCaRL and End-to-End IL with each image of size 32 32:
SCCARLCCFAAR = 32 x 32 x 3 x 2000 6144000
B HYPERPARAMETERS SETTINGS
Our implementation is done with Pytorch (Paszke et al. 2017). using the Adam optimizer (Kingma & Ba 2014) and scheduler that reduces the learning rate by factor of 0.5 when the loss stops improving. We use the resize transformation from torchvision with the default bilinear interpolation.
Table 2: MNIST Hyperparameters
Table 3: CIFAR-100 Hyperparameters
Hyperparameter
Value
Hyperparameter
Value
Learning Rate
0.002
Learning Rate
0.002
Number of epochs
200
Number of epochs
1000
Weight decay
0.0
Weight decay
0.0002
Patience
20
Patience
30
Table 4: t-SNE Hyperparameters
Hyperparameter
Value
Perplexity
15.0
Principal Components
Steps
400
C TASK-BY-TASK LEARNING
We provide additional experimental results on the multi-head learning of CIFAR100 with 10 tasks of 10 classes each. The training procedure of OvA-INN does change from the usual single-head learning but, test time, the evaluation is processed by batches of 10 classes. The accuracy score is the average accuracy over all tasks. We report the results from Yoon et al. 2017). Although our approach is able to match state-of-the-art results in accuracy, it should be noticed that it is drastically more memory and time consuming than the other baselines.
Model
Accuracy (%)
EWC (Kirkpatrick et al.
2017
81.34
Progressive Networks (Rusu et al. 2016
88.19
DEN Yoon et al. 2017)
92.25
OvA-INN
92.58

---

Under review as a conference paper at ICLR 2020
D VISUALIZATION
We highlight the differences between OvA-INN and Nearest Prototype when classifying 20 classes of CIFAR-100 in Figure 4
Figure 4: top: t-SNE projection of the features space before applying Invertible Networks (black crosses are the clusters centers) for 20 classes from CIFAR-100 test set (colors are given by the ground truth). bottom: in blue and yellow are the samples correctly and wrongly classified by both Nearest Prototype and OvA-INN, in green the samples better classified by OvA-INN than Nearest Prototype and orange the samples better classified by Nearest Prototype than OvA-INN.
12


Reviews:
[{"review_id": "Skx6Uq7TKB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper tackles continual learning problem with stacks of invertible network blocks. Similar as the ensemble idea, the proposed method learns an invertible network for each new object class. During test, for each class, each learnt network outputs a norm and the one with the smallest norm is the predicted class. The proposed model has shown performance improvements in classification accuracy on MNIST and CIFAR100 datasets in incremental class tasks.\n\nDespite the linear increase of memory usage over number of tasks as authors pointed out in the conclusion, this is an elegant method for continual learning problems in incremental class tasks. The paper is very-well written and easy to follow. To my best knowledge, the formulations in the paper are correct and clear. It seems that there are sufficient details for reproduction. However, I have the following concerns which, I think, may lower the contribution of the paper, unless authors can help clarify.\n\n1. Since authors propose an invertible neural networks-based method for continual learning, instead of only focusing on incremental class, authors should also evaluate the proposed method in other continual learning tasks in object classification, for example, incremental domain and task, as defined in this review paper (https://arxiv.org/pdf/1810.12488.pdf).\n\n2. Though I would assume OVA-INN would outperform state-of-the-art regularization-based methods, it would be convincing to show quantitative results of these methods (e.g. inclusion of Elastic Weight Consolidation in Figure 2).\n\n3. I do not understand the term \"learning type\" in Column 4, Table 1. Please give the definition of this term.\n\n4. Figure 3 is an interesting visualization of the latent representation learnt by the model. What about prototypical networks? It would be great to provide side-by-side comparison with such visualizations for the clusters learnt by prototypical networks the few-shot continual learning settings.\n\n5. In Table 1, how much is the memory cost for nearest prototype method?\n\n6. How much training data has been used over multiple classes?\n\n7. Side note: it is unclear to me how this method can generalize to other tasks, e.g. regression problems and reinforcement learning problems. Authors can briefly discuss how the proposed method can be applied in these scenarios in the future work section.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SkeS0zb6Kr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The central idea of this paper is to converts the classification problem into conditional generative modeling, it trains one normalizing flow (NICE) for each class of the data to perform out of distribution detection (one versus all binary classification). The data are presented one class at a time without being able to look back.\n\nThis paper proves a good improvement over the baseline methods compared to. Especially on CIFAR100, which is considered a difficult task in the regime of continual learning. It is quite evident to me that this approach has a big advantage over the other incremental classification methods.\n\nI give a weak reject to this paper mainly because,\n\n1. For invertible network, the NICE [1] coupling structure is used in this paper, although it is described in this paper as minimizing the distance from zero vector, the loss is basically equivalent to NICE with a Gaussian distribution as the target distribution. This is because normalizing flow maximizes the log p(z) + log(|det(z/x)|), in the volume preserving case the log determinant is a constant, it reduces to maximizing log p(z), which is equal to L2 loss if Gaussian distribution is chosen for p(z).\n\nIn my opinion, the paper could be better presented from the angle of generative modeling / out of distribution (OOD) detection. From this point of view, it would be desirable to include comparisons to the other OOD detection literature. It has been reported that the log probability is not a very good measure for OOD detection [2] because if an input has \"simpler\" patterns it would be classified as in distribution although it is actually OOD, and it has been reported that volume preserving flow (as used in this paper), also suffer from the same problem. There're several works proposing alternatives [3,4]. The good results of this paper may be because of the pretrained feature extractor, using a pretrained feature extractor could be practically useful but has limited insight for future research. Also, using a fixed feature extractor is avoiding the forgetting problem instead of solving it, it wouldn't generalize to more general / realistic continual learning setting.\n\n2. The incremental classification setting in this paper is not a very practical assumption for continual learning. i.e. the task boundary are assumed to be available during training, and within each task the data are i.i.d shuffled. One of the goal of continual learning is to prevent catastrophic interference, training separate network for each task, in my opinion, is avoiding the problem rather than solving it. Although there're many papers in the continual learning regime assumes availability of task boundary, I think this setting doesn't bring too much insight to how we can eventually solve the general case continual learning. This been said, I think this paper could be better presented from the OOD detection point of view.\n\n\n[1] NICE: Non-linear Independent Components Estimation https://arxiv.org/abs/1410.8516\n[2] Do Deep Generative Models Know What They Don't Know? https://arxiv.org/abs/1810.09136 \n[3] WAIC, but Why? Generative Ensembles for Robust Anomaly Detection https://arxiv.org/abs/1810.01392 \n[4] DETECTING OUT-OF-DISTRIBUTION INPUTS TO DEEP GENERATIVE MODELS USING TYPICALITY https://arxiv.org/abs/1906.02994 ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "H1eIrA5k5r", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[update after rebuttal]\n\nI thank the authors for their detailed reply, answers to my questions, and updates to the paper. (I especially appreciate the substantial effort to address all points and improve the manuscript even given the strongly negative rating. I am not personally in favour of \"extremifying\" the rating system the way ICLR did this year; I think it discouraged many authors from working on their rebuttals, and caused many reviewers to have a sense of inertia to keep their bad scores. I'm glad the authors didn't fall into the first camp, and I'm doing my best not to fall into the second! I think you did a very good job of improving the paper).\n\nI think the manuscript has improved substantially; especially the clarifications about (not) using negative samples and additional detail on INNs with citations. \n\nI find some of the sentences still unclear, and strongly suggest having the text read over for grammar/clarity for a camera-ready version.\n\nBut overall I recommend acceptance of this work. \n\n------------\nPaper summary:\nThis paper does continual learning using an invertible neural network (INN) trained to recognize each class. At test time a new example is presented to all of the INNs and the INN whose prediction has the smallest norm is used to predict the class of the test example. They do experiments on MNIST and CIFAR-100\n\nPaper contributions: \n- Review of methods and evaluation settings for continual learning\n- Review of invertible neural networks \n- Experiments comparing the proposed method to several other continual learning methods \n- Examination of memory cost\n- Exploration of feature space of trained INNs\n\nReview summary & decision: \nThe proposed idea of using invertible networks for continual learning sounds interesting, and I think that this could be a good paper. The experimental evaluation is not consistent or complete enough for me to to tell, however, and the core motivation for the idea is not clearly explained. There are also some less critical, but still important aspects of the paper (related work, clarity of explanations, repetitiveness) which lead me to decide this paper is not currently ready for publication.\n\nReasons for decision: \n1. A lot of statements and decisions are made without being explained, or are unclear / innacurate\n      - In CL, the objective is to learn several tasks one after the other\" this is not the objective, it's the problem setting. The objective is classification accuracy (or some other metric) in this setting.\n      - The choice of datasets seems a little odd; why not CIFAR-10 as well?\n      - It's very misleading to say your method is able to learn one class at a time without storing data; unless I've misunderstood this is only after pretraining all of the invertible networks with labelled data.\n       - Why 500 examples per class for CIFAR-100, and how many for MNIST? What criteria was used to arrive at this number?\n      - in the abstract you state the definition of continual learning says you can't have access to the data from previous tasks, but then mention that previous works use samples from previous data. This seems contradictory\n      - I'm not extremely familiar with Li & Hoeim (learning without forgetting), but I don't think it's very accurate to call it distillation regularization\n      - In 4.1 I don't think it's accurate to call this compression; it's just having smaller size layers in between (unless I misunderstood).\n      - I'm not familiar with model superposition; it should be more explained since your claims of having lower memory cost rely on this, as far as I can tell.\n      - The experimental baselines seem inconsistent / not comparable, making it difficult to evaluate what is going on. E.g. some update pretrained features and some do not. Some (including yours) do offline pretraining on a subset of examples, with inconsistent amounts of data, and sometimes this is considered part of training (Fearnet) and sometimes not (yours). \n       - I don't understand the central motivation for using INNs. The only sentence that seems to talk about it is \"this way ... network won't be able to have an output similar to the outputs on data from its training set\", but this is not clear, not enough explanation, and seems somewhat innaccurate. It's not that it's not able, it's just not likely (I think; please correct me if I'm wrong!), and I don't see why this isn't also true of e.g. MLPs (i.e. for MNIST, train 10 MLPs each to classify [class i] vs [all classes except i] - In order to justify the proposed approach, the baseline described above (using MLPs instead of invertible NNs) would be very useful\n 2. The training regime for the INNs is not clear (are they shown negative (other class) examples? How many, how are they sampled? I didn't find any hyperparameters for the INN training in the appendix, even though it says they should be there.    \n 3. The single head setting seems very similar to (or maybe the same as?) open set learning; I think this should be mentioned and works in this area should be reviewed.\n 4. Section 3 is repetitive and unclear. It could be greatly shortened to make space for more experiments.\n 5. It would be nice to see computational cost as well as memory cost; this is important in many settings where continual learning would be deployed.\n\nFeedback/suggestions/nits (not necessarily part of decision assessment): \n1. Cite the definition of continual learning (e.g. with a reference to a textbook or review) \n2. A lot of the writing is unclear, wordy, and/or grammatically incorrect\n    - inconsistent verb tense\n         -  e.g. \"if one would need .... it is required\" should be \"if one would need .... it would require\" I'd suggest rewording this sentence entirely, because it's misleading - it says \"retrain on this new dataset (which sounds like train just on the new data), but I guess you mean retrain on all data including the new data. e.g. \"Updating the model with new data requires retraining on the full dataset (old + new data). However, there are.... method may not be applicable.\" \n         - \"we are reaching\" -> we reach    - frequent use of \"indeed\" when it doesn't make sense\n    - Section 3 repeats the intro, 3.1 and 3.2 are sort of saying the same thing, and are also sort of repetitive of the related work. The related work should all be in the related work section, and this section should just be about what _you_ are doing. Invertible neural networks should be reviewed in something like a 'background' section, which possibly could be in appendix. Right now that section has even more related work in it\n3. Include tsne hyperparameters in appendix \n4. \"appendix\" not \"annexe\" (annexes are on buildings :) ) \n5. when you define continual learning, bold it (not italicize) \n6. \"Y is then Y_i\" -> Y = Y_i\" \n7. Caption for Figure 3 is unclear; I don't understand what I'm supposed to take away from these images. I also don't see a black cross.\n\nQuestions:\n1. What is the motivation for using INNs specifically? (rather than e.g. normal MLPs). \n2. What is the training regime for the INNs?\n3. What is model superposition, is it expensive in computation or something other than memory, and if you don't do that what is the memory cost of your method?", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper is board-line but in the end below the standards for ICLR. Firstly this paper could use significant polishing. The text has significant grammar and style issues: incorrect words, phrases and tenses; incomplete sentences; entire sections of the paper containing only lists, etc. The paper is in need of significant editing.

This of course is not enough to merit rejection, but there are concerns about the contribution of the new method, experiment details, and the topic of study. The results are reported from either a single run or unknown number of runs of the learning system, which is not acceptable even if the we suspect the variance is low. The proposed approach relies on pre-training a feature extractor which in many ways side-steps the forgetting/interference problem rather than what we really need: new algorithms that processes the training data in ways the mitigate interference by learning representations. In general the reviewers found it very difficult to access the fairness of the comparisons dues do differences between how different methods make use of stored data and pre-training. The reviewers highlighted the similarity between the propose approach and recent work in angle of generative modeling / out of distribution (OOD) detection which suggests that the proposed approach has limited utility (as detailed by R1) and that OOD baselines were missing. Finally, the CL problem formulation explored here, where task identifiers are available during training and data is i.i.d, is of limited utility. Its hard to imagine how approaches that learn individual networks for each task could scale to more realistic problem formulations.

All reviewers agreed the paper's experiments were borderline and the paper has substantial issues. There are too many revisions to be done.
Decision: Reject