Discriminative Variational Autoencoder for Continual Learning with Generative Replay
Authors: Woo-Young Kang
Cheol-Ho Han
Byoung-Tak Zhang
Venue:  (2020)
Keywords: Continual learning
Generative replay
Variational Autoencoder

Abstract: Generative replay (GR) is a method to alleviate catastrophic forgetting in continual learning (CL) by generating previous task data and learning them together with the data from new tasks. In this paper, we propose discriminative variational autoencoder (DiVA) to address the GR-based CL problem. DiVA has class-wise discriminative latent embeddings by maximizing the mutual information between classes and latent variables of VAE. Thus, DiVA is directly applicable to classification and class-conditional generation which are efficient and effective properties in the GR-based CL scenario. Furthermore, we use a novel trick based on domain translation to cover natural images which is challenging to GR-based methods. As a result, DiVA achieved the competitive or higher accuracy compared to state-of-the-art algorithms in Permuted MNIST, Split MNIST, and Split CIFAR10 settings.

Under review as a conference paper at ICLR 2020
DISCRIMINATIVE VARIATIONAL AUTOENCODER FOR CONTINUAL LEARNING WITH GENERATIVE REPLAY
Anonymous authors Paper under double-blind review
ABSTRACT
Generative replay (GR) a method to alleviate catastrophic forgetting in continual learning (CL) by generating previous task data and learning them together with the data from new tasks. In this paper, we propose discriminative variational autoencoder (DiVA) to address the GR-based CL problem. DiVA has class-wise discriminative latent embeddings by maximizing the mutual information between classes and latent variables of VAE. Thus, DIVA is directly applicable to classifica- tion and class-conditional generation, which are efficient and effective properties in the GR-based CL scenario. Furthermore, translation to cover natural images which are challenging to GR-based methods. As result, DiVA achieved competitive or higher accuracy compared to state-of-the-art algorithms in Permuted MNIST, Split MNIST, and Split CIFARI0 10 settings.
novel trick based domain
INTRODUCTION
Even though deep learning models such as Szegedy et al. (2015); He et al. (2016); Xie et al. (2077)
have been showing remarkable performance on many image recognition tasks exceeding human accuracy, they suffer from the catastrophic forgetting problem. The catastrophic forgetting problem occurs when we want to learn several tasks sequentially without any continual learning (CL) technique (Ratcliff) (1990); French| (1999); Goodfellow et al. (2013)) Theoretically. the catastrophic forgetting problem can be completely avoided if we store all of the past data, but it is sometimes difficult to store all of the previous data due the limitation on memory capacity.
To handle the forgetting problem in deep learning. some studies suggested weight regularization (WR) techniques such EWC (Kirkpatrick et al. (201)), SI (Zenke et al.) (2017)), and IMM (Lee et al.) (2017)) In these approaches, each of them has regularization term constrain the update of some parameters that are important for previous tasks. Other researches such as GEM (Lopez-Paz et al. (2017)) VCL Nguyen et al. (218)) introduced an additional memory called coreset which consists of the small number of real images past tasks to further improve WR algorithm. We will call this approaches as experience replay (based methods in this paper. Specifically, ER-based methods are useful when we can access previous task data though the amount of past data is limited. Recently, generative replay (GR)-based approaches such as DGR (Shin et al. (2017)) and VGR (Farquhar & Gal (2018)) are also proposed to replay previous tasks with generative model instead of storing the small number of images into coreset. They showed that the generative model successfully approximates data distribution, this approach can be a strong way to alleviate the catastrophic forgetting problem.
In this paper, we propose a new efficient and effective model called discriminative variational autoencoder (DiVA) (Fig. [ to address the GR-based CL problem. GR-based approaches can address a continual learning setting where stream of real samples seen only once, which can not be considered in ER- based CL. algorithms. However, GR-based approaches have two shortcomings. First, they need an additional module, generative model, in addition to a classifier. Second, distribution discrepancy between real and generated samples makes GR- ba-based methods hard to address complex natural images such as CIFAR (Krizhevsky Hinton (2009) or ImageNet (Deng et al. (2009)). In this point of views, DiVA are clustered class-win wise discriminatively on the latent space by introducing mutual information maximization between latent variable and class condition Thus, Vi is directly applicable to
try to handle the shortcomings. Firstly, distributions of latent variables of

---

Under review as a conference paper at ICLR 2020
f
Prior Network
Classification Network
Classification Loss
5090010900
Inpul condition
8
Ouipeil condition
-ivedineeeeeee
Total Loss
Decoder Network
Reconstruction Loss
Encoder Network
Ouipul dhin
Iagul
Sample (0.1))
Figure 1: Overview of the discriminative variational autoencoder. The prior network and classification network can be implemented with simple neural network, e.g., one fully connected layer. The @ and @ represent element-wise addition and multiplication, respectively.
both classification and class conditional sample generation with one integrated model. Also, we apply simple but effective domain translation (DT) trick to mitigate the distribution discrepancy and thus DiVA can achieve better classification accuracy on past tasks. To the best our knowledge, we firstly introduce the DT trick to GR-based CL. problem. In summary, we describe three main contributions of our research as follows:
We suggest new type of conditional VAE in terms of mutual information maximization, which can be expressed as integrated classifier and learnable prior network. It makes discriminative class-wire wise distributions for each class on latent space.
We propose simple but effective DT trick, which significantly improve CL performance of BRsbased approaches on a natural image dataset such as CIFARIO.
By combining DiVA and DT, we achieved state-of-the-ars accuracy on the class incremental learning settings.
2 RELATED WORKS
VAE with GMM prior VAE (Kingma & Welling (013)) has been widely studied due to its concrete theoretical background and interesting latent representations. Similar to our mixture-like prior distribution, some researchers proposed learnable Gaussian mixture prior (Dilokthanakul et al. (016);; Nalisnick et al. (2016)) or hierarchical prior distributions (Tomezak & Welling (2018)) because the unit Gaussian assumption prior in VAE is an unsupervised learning task while we consider explicit supervision tasks. Wang et al. (2017) also proposed GMM VAE with class-specific and randomly fixed priors for supervised task. In this case, the random initialization can map similar patterns to far regions on latent space, which could not be suitable for the main desiderata, smoothness, of representation learning (Bengio et al. (2013)). On the other hand, since we consider learnable priors, DIVA satisfies the smoothness condition (Fig, 2(d).
strong constraint. They mainly address
Conditional VAE There are several ways to implement conditional VAE (CVAE). The easiest and intuitive way is to concatenate one-hot vectored class conditions onto latent variables which are outputs of an encoder network (Doersch (2016)) This method sets prior distributions for latent variables of all classes a unit Gaussian. Since the CVAE always needs data-label pairs for concatenation, other researches propose to infer the class label with an independent heavy classifier (Kingma et al. (2014); (Sohn et al. (2015)). As result, they make their conditional generative model work with unlabeled data. However, concatenation makes feature representation unnecessarily sparse and requires additional weight parameters for feeding class conditions. In contrast, do not need the heavy classifier and redundant weights as we model class-wise discriminative latent representations.
Generative replay Recently, studies which use GR have been actively proposed to mitigate catas- trophic forgetting in the field of CL. This approach generates previous task data when task
2

---

T2,,,,,,.... ENY where Under review as a conference paper at ICLR 2020
1
6
/ 6
0
4
5
O
4
5
6
3
3
9
2
3
6
4
8
5
9
a
6
2
9
0
2
3
8
9
6
7
0
4
/
5
9
8
6
4
7
8
9
6
f
7
5
4
5
7
8
9
C
5
8
8
2
3
5
6
7
9
9
8
8
5
8
2 3
4
5
6
7
3
8
7
0
4
5
9
5
2
0
2
4
6 3
5
9
8
2
3
8
4
8
2 0
4
9
7
9
0
7
(a) Standard VAE
(b) Standard CVAE
(c) CDVAE
(d)
DiVA
Figure Visualization of the latent space (top) and generated samples (bottom). this visualization. we randomly sampled 1000 images from MNIST test set. We used the t-SNE (Maaten & Hinton (2008) to reduce the dimension of latent vectors for plotting.
data is coming, then optimizes them concurrently. In the DGR and VGR, they use a GAN- generator generate previous task data unconditionally. Since DGR and VGR consider generator and a classifier as separate modules, they alternately optimize them. Also, other researchers propose GR-based algorithms using VAE such RtF (van de Ven & Tolias (011)), CDVAE (Mundt et al. (2019)), and DGDMN (Kamra et al.] (2017). RIF and CDVAE are closely related to our work in that they consider classifier integrated VAE and address CL problem. However, they both assume the prior distribution as unit Gaussian, which contradicts discriminative loss. Also, recently, Hu al. (2019) (PGMA) proposes a parameter generation and model adaptation trick mitigate the catastrophic forgetting problem.
3 PRELIMINARY
Variational Autoencoder Considering we have observations X Ia E Rd, the goal of the VAE is to maximize marginal log likelihoods of each observation log p(21, 22, 2(21, .... ZN) Ex= log p(x,) with respect both variational parameters and genera- tive parameters. However, this includes an intractable term when we use a complex model. Because the intractable term is always positive,
optimize the variational lower bound instead of the
marginal log likelihood as follows:
log p(x.) =D [go (2 a.) apl (2/2:)] + L(0, x1;;)
cC,0, 00; xx4) [en(( po (x:|=)] D (Krloo(ala:)))l(()))
where (2(2)2;; is the variational posterior (the encoder) and po (xi|=) is the decoder. In general, both the encoder and the decoder can be modeled with neural networks. The L(0, 8') is called as the evidence lower bound (ELBO).
Mutual Information In information theory, mutual information of two random variables, X and Y. a measure of the mutual dependence between the two variables. The mutual information can be decomposed into marginal entropy and conditional entropy as follows:
T(XY)) H()) H(X H(Y) H(Y
The meaning of this definition is intuitive: if observe Y then how much uncertainty of X changes. X and Y are completely independent, i.c. H(X = (X), then the I(XY) = On the contrary, if X is determined by observed then the T T(X; Y") is larger than zero.
3

---

Under review as a conference paper at ICLR 2020
4 DISCRIMINATIVE VARIATIONAL AUTOENCODER
In Fig 2 unconditional generative models such as standard VAE and CDVAE generate some am- biguous samples. Thus, should consider conditional generative model for sampling class-wise explicit images to handle the incremental learning problem using a GR-based approach (further details are discussed at Sec Here, consider classifier integrated VAE. If latent variables for input images are located class-wise discriminatívely on the latent space, we can conduct both class prediction and conditional generation. Fortunately, we can directly model the relationship between latent variables and class condition c by maximizing mutual information between them. The optimization of DiVA is as follows:
max E (x)x)) [log po (x/2)) Dx Dx D +T(c:2)
(1)
8.0' 000'0
This can be equivalently rewritten as:
max (e po 6,0".0.6 ...
(e (en (log Por (c/z)]
(2)
where, is weighting parameter of the mutual information loss and (xlx) [log por(clz)] is cross-entropy for multivariate Bernoullis or L2 loss for continuous-vall valued conditions. Also, 0, 8' Φ. and ø' mean network parameters of the encoder, the decoder, the prior network, and the classifier, respectively. When the encoder and decoder networks are sufficiently complex, it is enough to implement each the prior and classification network as one fully-connected layer. Detailed derivations described Sec
4.1 MUTUAL INFORMATION MAXIMIZATION
In this section, we describe how maximizing [(z;c) with ELBO makes our DiVA. Vincent et al. (2010): Chen et al. (2016) showed that minimization of reconstruction error is related to maximization of mutual information between two variables. Similarly, we maximize mutual information between class condition e and latent variable 2 by minimizing a reconstruction loss. Here, we will start from [c;;z = (z) (zle)). Then, we will optimize"
max [c;; z) max H()) (zlcc)
(3)
max (z) + E.g...) log ((2|c)].
Now, we derive the conditional entropy H (z)c) o) as follows:
(z|c)) - Σ g(c, z)log g(zlc).
(4)
= Σ g(c) Σ ((1cc)log q(zlc),
we introduce p(clz), then Eq. (4 can be rewritten as,
((zlc)
H(zlc) Σ g(c) Σ a(zlc) log p(clz) + log
p(clz)
(5)
= =- (E log p(c|2)] + De [Ā([c)) p(clz)))
Since DKLl [((()) || p(c|2)] is always positive and q(c) constant in our dataset, we can get lower bound of the mutual information as follows:
T(c;2) 2 H(z) + Egge ej [log p(ckz)
(6)
where, Egizle; llog p(c|z)] can be viewed as class reconstruction in autoeneoding manner, minimizing L2 or cross-entropy. In CDVAE, the authors directly optimize Eg(xis p(c|z)) impose class information on z so that classifier can be integrated with VAE as follows:
(x)x)) [log pa (xlz)) - Igo + (2)))/()l(()))(1 (z)x)]lp(z))
(e)) (logós (c|z)]
0,0'
For continuous variahles, can simply change the Σ f.
4

---

Under review as a conference paper at ICLR 2020
Algorithm Train DIVA with domain translation
procedure Train CyGAN, di)
procedure Train CyGANICyGAN, DiVA, X.) Domain A < X; training set X, real images
Input: T [T_) _i)= where T = (X,.)))
// Learning all lasks
for i = 0, .... until given epoch do:
for all i = 0, ... N do: // for all tasks
Domain_B _ t sample from DiVA // Task, optimise CyGAN(Domain + Domain B)
// Training DiVA
for all (x,y) y) in Xi, Y,, until convergence do:
end for end procedure
sampling & // randInt for past task labels
& <- sampling from DiVAprev (f)
& (r) if dt s2r else
procedure Test(DiVA, CyGAN, i, dt) for = O, .... do: //until current task for all x in X, do:
minimize LDIVA (a2 a] [y, g) using Eq.
end for
TesVDDiV,, CyGAN, i. dt)
if ş i: // for past tasks
Train CaGAn_CyyANNYNNN DIVA, X2) DiVApree + DiVA
CyGAN:r2s(x) if dt == r2s else
prediction +- DiVA(x) classification end for end for end procedure
end for return DiVA, CyGAN end procedure
However, since they assume unit Gaussian p(z) as a prior for the variational posterior g(z/x), relation between q(z|c) and ((z(x) is not considered. Thus, latent space has low density regions because becomes discriminative within unit Gaussian, which causes ambiguous sample generation (Fig. [
In contrast, explicitly consider the relation between ((z]c) and (<([x) as follows. Firstly, since corresponds to a class among a finite label set, we can simply assume ((z[c) as a class-conditional Gaussian distribution, i.e. I ~ (z)lee, (())) Then, we can replace the unit Gaussian prior in Eq. with the class conditoonii Gaussian (z|cc). which corresponds to the second term in Eq ② Now. since we minimize D [ Eggele) [[gg)(][))] As result, latent variable of our DiVA holds information of not only input but also class с without low density regions: each class condition has its own one mode compact Gaussian distribution. Furthermore, H (z) in Eq. ( can be indirectly maximized to hold information of x by maximizing the ELBO in Eq. (2) To summarize, our modeling for the probability of the latent variable given its class has three virtues:
in our loss, we can optimize Eg(+|s) [log p(c|z)] instead of
Class is well-separable in the latent space.
The distribution of the latent variable given its class is simply modeled.
The latent variable has enough information reconstruct the image.
The first two virtues are related to the class condition; classification network and prior network, respectively. The last virtue is achieved by the standard VAE.
DOMAIN TRANSLATION
In the GR-based CL approach, distributions of generated and real images should nearly close to work well. However, there is always discrepancy between generated and real images, even with current state-of-the-an art generative models. For this reason, it is known that GR- methods are hard to be expanded to a more complex dataset than MNIST variants such as CIFAR 10 or ImageNet (Lesort al. 2a).. Here, narrow the distribution discrepancy, we define the two domain, real domain X' and sample domain V. and their bidirectional domain translation, G : X' + y and F y + X. In this paper, we call G as a real to sample (r2s) and F sample to real (s2r) translation, respectively. We consider two conditions for this translation mappings. First, we should translate only the style as keeping outline patterns of given images. Second, we should consider an unpaired domain translation between real and generated images because the generated images are sampled randomly. In that sense, we use CycleGAN (Zhu et al. (2017)) which satisties the conditions. When we consider r2s mode, we apply r2s translation to test set images of past tasks and classify them. While, when we consider s2r mode, we translate generated past task samples on the training phase and train our model to classify them. The purpose of both r2s and s2r translation is to narrow the gap between what our model memorizes and what our model looks currently. Detailed definition of this translation is described in Appendix C, and the training process is explained in Algorithm 1.
5

---

Under review as a conference paper at ICLR 2020
Table 1: The averaged accuracy of tasks after training of all the 5 tasks done. WR, ER, and GR mean weight regularization, experience replay, and generative replay, respectively.
Approach
WR only
WR
ER
GR
Model EWC
SI
IMM
VCL
GEM
RdF
OCDVAE
DGR
PGMA
DIVA
Permuted
94.14
93.94
96.09
96.33
96.19
97.31
94.54
96.77
96.01
MNIST
Split
19.8
19.67
67.25
82.71
92.20
92.56
93.24
94.8
81.7
97.92
MNIST
Tacc
Permutee
Pormuted
Pormated
Average
VLS
DAA
Taacs
Talks
Teahs
Taal
Teats
Ta0x
100
21
DBC VOL-
15
Figure 3: Learning curve for both Permuted (top) and Split MNIST (bottom). Numbers written next to the 'VCL- meaa the size of random coreset data Nguyen et al. (2018) per each task.
6 EXPERIMENTS
In this section, firstly, we discuss the importance of conditional generation for continual learning. Then, we analyze the performance of our model on Permuted MNIST and Split MNIST. Finally, we discuss the effectiveness of our domain translation trick for GR-based methods address CIFARIO dataset. Since most the GR-based approaches did not report results on Split CIFAR10 setting, we firstly reproduce Split MNIST results and then expand Split CIFAR10 10 setting based open-source codes. More details such as hyperparameters and network structures of our model are described in Appendix A.
6.1
CONDITIONAL VS UNCONDITIONAL GENERATION
The quality of generated images is erucial in continual learning with the generative replay. In the generative replay, a model learns new task over both the generated previous task data and the real current task data. generated images have bad quality, then classifier and a generator of the model degrade over the previous task data. Also in learning the next new task, the degraded generator generates poorer images, and the classifier degrades repeatedly. Therefore generating images of good quality prevents the model from degrading in continual learning. Recently, this fact is also studied by Lesort et al. (0185)) Optimizing the VAE according to the conditional likelihood of data generates sharper and To double-check whether the conditional generation is effective, we generated a few MNIST samples conditionally unconditionally (Fig. 2). Samples generated conditionally were shown to be more recognizable. Such images of good quality generated conditionally will eventually prevent the VAE from degrading in continual learning.
realistic image samples (Sohn et al.) (2015): van den Oord et al. (2016)).
6.2 INCREMENTAL LEARNING WITH MNIST
In the Hsu et al. 2018), they categorized various incremental learning tasks as three general cases: incremental task, incremental domain, and incremental class learning. In this section, we address the incremental domain and class learning based on MNIST dataset. Since MNIST dataset is simple for generative model to make highly realistic samples, do not apply the domain translation trick to MNIST variant settings. The detailed explanation about each setting is described in Appendix D. and recommend to read Appendix D for deeper understanding of settings and reported results.

---

Under review as a conference paper at ICLR 2020
Table 2:
Ablation experiment for effectiveness of domain translation on Split CIFAR1O setting.
tasks w/o DT
tasks w/ DT
5 tasks w/o DT
tasks w/ DT
DGR
48.71
70.33
21.75
29.75
RIF
48.13
67.71
21.53
38.27
OCDVAE
48.48
67.84
33.15 21.64
DIVA
48.34
78.93
28.52
42.11
Permuted MNIST In this task, pixels of each image are shuffled with task-wise fixed permutations (Goodfellow et al. (2013); Kirkpatrick et al.] (2017). We applied five different permutations to original MNIST data to make 5 Permuted MNIST tasks. Most of the CL algorithms show high accuracy in this task. Our DiVA shows slightly low accuracy than others. This is because Permuted MNIST looks random noises and does not have spatial correlations between neighbor pixels. Thus, it is hard to generate such images with a CNN architecture exactly. For experience replay (ER) based methods such as VCL or GEM, set memory size for ER to 200 per each class; This memory size is also used for the Split MNIST setting. This memory size corresponds to the additional memory requirement for DIVA, decoder network, described at Table 4 in A Appendix. While this task is addressed in most of the continual learning studies, Farquhar& & Gal (2018) insightfully pointed out drawbacks of the Permuted MNIST task in that unrealistic large differences each permuted digits lead to artificially lessened forgetting. Thus, we also report results for Split MNIST setting in the next subsection.
Split MNIST Split MNIST is more challenging than Permuted MNIST task in that added new tasks could have similar patterns with old tasks (more realistic differences). In this experiment, we consider the five subsets of the MNIST data: the first subset consists of [0, 1 ], the second subset composed of (2, 3), and the same manner for remaining subsets. Also, we set our network with a single-headed form of network Farquhar & Gal (2018), which makes predictions over all classes (digits to 9) with one output head; a multi-headed network (Nguyen et al. (2018); Zenke et al. 2017); Lopez-Paz et al. () for Split MNIST comprises multiple output heads and uses a task id guiding to a specific head for its task, which is an easier setting than single-headed Split MNIST. We can see that WR only approaches such as EWC, SI, and IMM fail to prevent catastrophic forgetting for previous task data, see Figure 3 This is because they do not have any explicit terms to discriminate the old and similar new tasks in their loss function. Also considering minimization of cross entropy with softmax, values of newly added output nodes should be bigger than nodes for old tasks. Concurrently, weights connected to nodes for old tasks are trained to output small values. That is why WR only approaches fail in the Split MNIST setting without reference samples of the previous task. From T the VCL and GEM use previous real data stored in coreset memory. So they can maintain performance for previous tasks to some degree while generative replay approaches can effectively mitigate the catastrophic forgetting problem in this setting. This means that a generator can generate samples which are almost same with real images, GR-based approaches could be strong way to mitigate catastrophic forgetting. Furthermore, our proposed model achieves higher accuracy than other GR-based methods. This is because our model conditionally generates previous samples, which makes more class-wise clear samples than other unconditional generative models.
6.3 INCREMENTAL LEARNING WITH CIFARIO
GR-based methods can outperform other CL approaches such as WR or ER in the challenging situation that an agent can observe each sequential task only once. The assumption of the advantage is that generated images should be highly similar to real images. However, there is a discrepancy between generated and real data distributions depending on the complexity of images. As a result, GR-based algorithms with natural image dataset are suffering from catastrophic forgetting more than ER-based methods. In this section, we provide strong potential for GR-basee approaches to be able to extend natural image datasets.
Split CIFAR10 This setting is the same as the Split MNIST task for CIFAR10 dataset. We divide CIFAR1O data into two or five subsets to make 2 5 class incremental tasks. To reduce the diserepancy of real and sample distributions, we used the r2s domain translation trick based on CycleGAN ZZuu et al. (2077)) The results are reported in Table Without domain translation, all
7

---

Under review as a conference paper at ICLR 2020
Figure 4: From left to right, real, r2s translation, s2r translation, and sample images generated by DiVA are displayed.
GR-based approaches suffer from serious catastrophic forgetting problem. However, by applying r2s translation, all algorithms show significant improvement on both 2 and tasks Split CIFARIO settings. Especially, DiVA achieved the highest accuracy among the state-of-the-art GR algorithms. Since all other GR algorithms have similar network capacities with DiVA, we can conclude that class-wise clear sample generation is important for incremental learning. Example images of domain translation are illustrated in Figure g We observed that the domain translator successfully translates the domain of real images into the domain of sample images generated by DiVA. This simple but powerful domain translation trick can be applied to any other GR-based approaches. Also, considering the improved quality of s2r translation, we think this technique can be further studied for VAE-GAN hybrid model to take advantages of both models: diversity and high quality.
Comparison of Domain Translation Tricks Here, we compare possible domain translations described in Sec In addition to our proposed domain translation tricks,
Table 3: Comparison of domain transla- tion tricks with DiVA. Results are aver- aged after each setting is all done.
Razavi al. (2019) reported that applying reconstruction to test set images can significantly improve the CAS score (Ravuri & Vinyals (219)). Since CAS score measures accuracy for real test set images of a model that trained with generated samples from generative model, the re- construction process can be viewed as kind of domain translation. The results are Table 3 The experiments were conducted in two settings: Split CIFAR10 10 divided
2 tasks
5 tasks
Reconstruction CycleGAN s2r
75.89
38.31
72.41
19.54
CycleGAN r2s
78.93
42.11
into and subsets. Firstly, s2r translation showed lower accuracy than r2s translation. This is because that translating sample images to real style is still challenging even with additional domain translator. While, r2s translation could be achieved more easily by, for example, blurring. Also, r2s translation showed higher accuracy than the domain translation using reconstruction. This is because distributions of reconstructed real images and randomly generated samples also have a discrepancy. In contrast, we directly model the translation between generated samples and real images. Furthermore, domain translation with reconstruction can only be applied to autoencoder-basee architectures, while proposed methods can be applied any kind of GR models, including GANs.
CONCLUSION
In this paper, we addressed how to mitigate the catastrophic forgetting problem. To solve the forgetting problem, we considered a generative replay based approach which generates previous task data and used them as a reference for previous tasks. We proposed a new type of conditional generative model based on the VAE architecture, which can be viewed as a classifier integrated generative model. Thus, we can conduct both the classification and the class conditional generation with one model. Also, we apply a simple domain translation trick based on CycleGAN to overcome the weakness of generative replay based approach. As result, our DIVA outperforms other recent continual learning algorithms in Split MNIST and CIFAR 10 settings. We are sure that our DiVA has the potential to achieve higher accuracy than current reported results if powerful generative model based on VAE developed. Besides, we think that proposed domain translation trick can help other future continual learning researches which use the generative replay.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-1828. 2013.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets, In Advances in Neural Information Processing Systems, 2172-2180, 2016.
J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. CVPR09, 2009.
Nat Dilokthanakul, Pedro AM Mediano, Marta Garnelo, Matthew CH Lee, Hugh Salimbeni, Kai Arulkumaran, and Murray Shanahan. Deep unsupervised clustering with gaussian mixture varia- tional autoencoders. arXiv preprint arXiv:1611.02645 2016.
Carl Doersch. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908 2016.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09733, 2018.
Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, (4):128-113) 135, 1999.
Ian J Goodfellow. Mchdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investi- gation of catastrophic forgetting in gradient- baseed neural networks. arXiv preprint 1:Xiv::312.6111 2013.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition,
pp. 770-778, 2016.
Yen- Yen-Chang Hsu, Yen- Ben-Cheng Liu, and Zsolt Kira. Re-evaluating continual learning scenarios: categorization and case for strong baselines. arXiv preprint 18100. 12488, 2018.
Wenpeng Hu, Zhou Lin, Bing Liu, Chongyang Tao, Zhengwei Tao, Jinwen Ma, Dongyan Zhao, and Rui Yan. Overcoming catastrophic forgetting via model adaptation. In International Confer- ence Learning Representations, 2019. (tLLhttp:: /ttp::/:////eeeeeee/// www.:eeeeffooooee FYGvcoA5YX
Nitin Kamra, Umang Gupta, and Yan Liu. Deep generative dual memory network for continual learning, arXiv preprint urXiv:1710.10366 2017.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114, 2013.
arXiv preprint
Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised learning with deep generative models. In Advances in neurul information processing systems, pp. 3581-3589, 2014.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Baabska-Barwinska et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114 (13):3521-3526, 2017.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient- learning applied to document recognition. Proceedings of the IEEE, (6(11)::227-22222 1998.
Sang-Woo Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung- Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems, pp. 4655-4666 2017.

---

Under review as a conference paper at ICLR 2020
Timothée Lesort, Hugo Caselles- Dupré, Michael Garcia-Orti Andrei Stoian, and David Filliat. Generative models from the perspective of continual learning. arXiv preprint 101i::112.001111 2018a.
Timothée Lesort, Alexander Gepperth, Andrei Stoian, and David Filliat. Marginal replay vs condi- tional replay for continual learning. arXiv preprint arXiv: 1810. /2069, 2018b.
David Lopez-Paz et al. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, pp. 6467-6476, 2017.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, (NNo))2579--2655 2008
Martin Mundt, Sagnik Majumder, Iuliia Pliushch, and Visvanathan Ramesh. Unified probabilistic deep continual learning through generative replay and open set recognition. arXiv preprint arXiv: 12019, 2019.
Eric Nalisnick, Lars Hertel, and Padhraic Smyth. Approximate inference for deep latent gaussian mixtures. In NIPS Workshop on Bayesian Deep Learning, volume 2, 2016.
Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, and Richard E. Turner. Variational continual learning. In International Conference on Learning Representations, 2018. URL https:// net/ forum? 1d=
Roger Rateliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285, 1990.
Suman Ravuri and Oriol Vinyals. Classification accuracy score for conditional generative models. arXiv preprint 1905. 10887, 2019.
Ali Razavi, Aaron den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. arXiv preprint arXiv:I906. 00446, 2019.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2994-3003, 2017.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. In Advances in Neural Information Processing Systems, pp. 3483-3491, 2015.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, Going deeper with convolutions. In Computer Vision and Puttern Recognition, 2015. CVPR 2015. IEEE Computer Society Conference on. Cvpr, 2015.
Jakub Tomezak and Max Welling. Vae with a vampprior. In International Conference on Artificial Intelligence and Statistics, pp. 1214-1223, 2018
Gido M van de Ven and Andreas S Tolias. Generative replay with feedback connections as general strategy for continual learning. arXiv preprint arXiv:1809, 10635, 2018
Aaron van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelenn decoders. In Advances in Neural Information Processing Systems, pp. 4790-4798, 2016.
Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in deep network with local denoising criterion. Journal of machine learning research, 1 ((Dec):337133403 2010.
Liwei Wang, Alexander Schwing, and Svetlana Lazebnik. Diverse and accurate image description using a variational auto-encoder with an additive gaussian encoding space. In Advances Neural Information Processing Systems, pp. 3758-5768, 2017.
10

---

Under review as a conference paper at ICLR 2020
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pp. 5987-5995, IEEE, 2017.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. arXiv preprint
2017.
Jun- Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired using cycle-consistent adversarial networks. Proceedings of the IEEE international conference computer vision, pp. 2223-2232, 2017.
translation
A
APPENDIX
A. IMPLEMENTATION DETAILS
Here, describe the detailed network structure used in experiments. For the EWC, we stacked two hidden layers, where each layer consists of 800 hidden units with ReLU activation. Also, we applied the dropout to the input layer and hidden layers with 0.2 and 0.5, respectively. We trained this EWC with batch size 256 and 50 epochs for each task. For fine-tuning with respect to X, we conducted multiple runs with À 10 10 10 10 107 and selected the best value À = 10 For the VCL, we chose the same structure as Nguyen et al. Nguyen et al. (2018). For the DGR and VGR, we carefully referred to the reported accuracy from their original papers. Other results that we reported in [ are from Hu et al. (2019); Hsu et al. 2018); van de Ven & Tolias (2018). In our method, we found out that adding Gaussian noise to input images useful to improve our algorithm for Split MNIST task. Since our loss function includes reconstruction error, there are small differences in pixel values between real images and generated images. So, this error can pass to next tasks and will be accumulated. Thus, the added Gaussian noise plays positive role in training model to become robust to the error. Adding noise technique not necessary to reduce catastrophic forgetting for the prior-focused approaches because they do not include such generation process. GAN-base models such as the DGR and the VGR do not need to add noise too. This is because their loss function to generate image is just the classification error of discriminator, the pixel-wise mean-squared
Table 4: Implementation details of the DiVA for Permuted MINST and Split MNIST settings
Kernel
Stride/Padding Output dims
Network
Operation
BatchNorm
Activation
Input dims
Convolution
3 x I x 32
2/1
ReLU
Convolution
x x 32 64
2/1
ReLU
Convolution
3
x 128
ReLU
Encoder
Convolution
3 x x 128x 256
2/1
ReLU
Fully-connected
2 x 2 256
256
ReLU
Fully-connected
256
128
ReLU
Fully-connected
256
128
Sofiplus
Fully-connected
x x 128
ReLU
Convolution
3 128 x 128
2/1
RcLU
Convolution
3 x 128 x 64
2/1
ReLU
Decoder
Convolution
3 x 64 32
2/1
ReLU
Convolution
32
Sigmoid
Fully-connected
10
128
ReLU
Prior Net
Fully-connected
10
128
Softplus
Classifier
mully-connected
128
10
Softmax
Optimizer
Adam(Ir=0.001, betas=(0.9, 0.999), eps=lc-08)
Batch size
64
Hyper- parameters
Softplus
beta=1, threshold=20
X for DiVA
Permuted: 5c+3, Split: 2c+2
Gaussian noise
mu=0, std=0.3
Epochs per task
Permuted: [100, 100, 200, 200, 200)
Split: [100, 100, 200, 200, 200)

---

Under review as a conference paper at ICLR 2020
or binary cross entropy. Also, when conducted some experiments with this adding noise technique for the EWC and the VCL, there was no noticeable increase in final accuracy for 5 tasks incremental learning. The detailed structures of the DiVA for Permuted and Split MNIST settings are described in T and learning curves for Permuted and Split MNIST tasks are illustrated in Figure [ Also, for Split CIFARIO setting, we implemented our encoder network for DiVA based on
Depth, ffcten_fcctrr and dropRate are 28, 4, and 0.5, respectively. Also, we the latent dimension for mu and variance of DiVA as 1024 and stacked one fully connected layer for both prior and classification network. We did not apply random noise the Split CIFAR10 setting. The decoder network is doubled input and output channels than described in Table 4
For experimental results, used 60,000 images for training and 10,000 images for testing MNIST variant settings. Also, we used 50,000 images for training and 10,000 images for testing in CIFAR10 10 variant settings.
B. KL- AL-DIVERGENCE BETWEEN TWO GAUSSIAN DISTRIBUTIONS
Since we train both prior and posterior distributions for latent variables, we describe the detailed formulation of KL-divergence between two arbitrary Gaussian distributions here:
/ p(z)log p(x) da - / p(z))oog g(z) dx
K [L|g(=)llp(a)}
2 1og(2 + of
(/1 12)2
(1 log2Toi)
(8)
20²
log 02 +
o + (M1 u22
01
20
2
C. DOMAIN TRANSLATION
CycleGAN al. (0177) is image-to-image translation algorithm. It translates image from a source domain X' to target domain y' without paired examples (c.g.. photo of New York City and painting by Gogh). They proposed cyclie translation technique using GAN architecture to keep the consistency of shape and convert the style of an image. The loss function of CycleGAN consists of adversarial losses LGAN and a cycle consistency loss Laye as follows:
L(G, Fx, Dy) LGAN(G, Dy, X) + LGAN(G, DxxY,,) + XCrye F); where, LGAN (G, X,3) = Eyypaana (8) [log [yy))( + Exxpaara (x) [log(1 (y(G))))
[FAN v,, = Exxpaacc (x) [ogDx(x)) + E Ey-pasce(1) [log(1 - Dx(G(Y)))),
Coyo(G: F) = EExpanto [G(()) x||1] + E Eywpeera(w) [IG((F()) - =|li]
where, G and F mean the mapping functions G X - y and F y X. Also, À controls the relative importance the two objectives. As a result, CycleGAN aims to solve:
G. F arg min max L(G, F, Dx, Dy).
(10)
G.F Dr.D,
In our paper, the source domain X' and the target domain y' mean real images and generated images, respectively. We used codes for CycleGAN from https///gthub..co//arrrzz Saaitorrpp//yTorc-Cyyeeee and followed default hyperparameter settings. Between each task, we trained CycleGAN for 60 epochs.
D. DETAILED EXPLANATIONS OF INCREMENTAL LEARNING SETTINGS
There are two commonly used operations from datasets to evaluate continual learning algorithms: pixel permutation and class splitting. The typically used dataset is MNIST et al. (1998), which composed of 0 ~ 9 hand-writter digit classes. The Permuted MNIST setting includes ten digit classes, where each class consists of different pixel permutations of each digit class in MNIST dataset.
12

---

Under review as a conference paper at ICLR 2020
Incremental Task Learning (Mulci-hcaded)
Incremental Domain Learning (Singlc-headed)
Ineremental Class Learning
(Single-headod)
Task
Feature Extracter
Feature
Featare
y+
eir
olF
clf
null
null
Input
Clus
Class
Task
x-
2
3
Feature Extractor
2
3 0
Extractor
Feature
2
3
Peature Extractor
yz
y
y:
2
eir L
2
clf
2
null
Shared layers
Active classification bead
Inactive head
Figure 5: Three general incremental learning settings. This is based on Hsu al. (0188) van de Ven
Tolaa (0188)
Real
Real
Real data
Real data
data
label
data
label
data
label
DiVA
DiVA
DiVA
DiVA
Geuerated data (1)
Generated data (1:2)
Generated data (1:7-1)
data
label
data
label
data
label
Figure 6: The process for incremental learning with the DiVA. At first, the model learns real data. Then, it generates all previous task data learn task jointly. The number in () means each task.
This setting is widely used as incremental domain learning Kirkpatrick al. 2017 0) Lee ct al.
(2017
Lopez-Pat et al. (2017); Nguyen et al. (2018); Shin et al. (2017) Zenke et al. (2017: Hu ct al. despite some negative concerns that unrealistic substantial dificrences in each permuted digits lead to artificially lessened forgetting and, thus, making this setting less challenging Farquhar & Gal (2018)
2019
Next, the Split MNIST setting initially introduced in multi-headed from where the ten digits split into five two-class classification tasks (the model has five out heads for each task) Nguyen et al. (2018); Zenke al. (2017); Lopez-Paz al. (2017). At the multi-headed architecture, each task has its own task id and independent classification heads. So, when training and testing model, the task ids are given. This muati- split MNIST setting is also viewed as a relatively easy setting due the task id. Farquhar and Gal Farquhar & Gal (201 8) propose a hnade- variant which does not require task id, where model should predict all classes that have seen. This single-headed Split MNIST is known as incremental class learning. Both our Split MNIST and CIFAR settings consider this single-headed Split setting which is more challenging than the multi-headed Split setting. conceptual explanation of each setting is illustrated in Figure
E. CONTINUAL LEARNING PROCESS OF DIVA
Here, explain the whole process of DiVA for the incremental class learning task. At first, we train DiVA for task with real images and labels. When task comes, DiVA generates pseudo data and labels for task 1. Then, train Di with both task and task 2 jointly. When task 3 comes, DIVA
13

---

Under review as a conference paper at ICLR 2020
generates data and labels for task 1 and 2. Then, we train DiVA with pseudo data and labels for task and 2, and real data and labels for new task 3. The conceptual explanation illustrated at Figure 6
14


Reviews:
[{"review_id": "SkgI3OVTYH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors focus on alleviating the catastrophic forgetting problem in continual learning.  The authors propose a discriminative variational autoencoder (DiVA) to solve this problem under the generative replay framework. DiVA modifies the objective function of VAE by introducing an additional term that maximizes the mutual information between the latent variables and the class labels.  \n\nThe authors do not thoroughly explain the motivation of this paper. The authors do not explicitly define continual learning, incremental learning, and catastrophic forgetting problem. It is also not clear to me why these problems are important. \n\nThe idea that introduces labels in VAE is not novel. For example, Narayanaswamy et al. [1] also propose to utilize labels to VAE. I do not understand why making use of labels is important for solving the catastrophic forgetting problem and how the labels are useful in the generative replay process. It is also not clear to me how domain translation is relevant to continual learning. \n\nIn terms of modeling, since the input into the prior network has finite possible discrete values, we do not need a fully connected network to generate $\\hat{\\mu}_c$ and $\\hat{\\sigma}_c$. Instead, we can directly optimize $\\hat{\\mu}_c$ and $\\hat{\\sigma}_c$ for each $c$ as parameters.\n\nThe paper provides some good experimental results. But the problem settings are not clear to me. I do not understand how the model is trained to solve multiple tasks. Do the same model is trained for multiple tasks? Is each of the tasks trained sequentially or simultaneously? It is also not clear to me why CIFAR datasets involve two domains and how these domains are relevant in each of the tasks.\n\nIn summary, since DiVA gives a good experimental performance, the proposed method might be promising. However, it looks to me that the authors need to better explain the motivation of DiVA, the differences of DiVA from existing supervised VAE, and the experimental settings, before the acceptance of this paper.\n\nReferences\n[1]Narayanaswamy, Siddharth, T. Brooks Paige, Jan-Willem Van de Meent, Alban Desmaison, Noah Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. \"Learning disentangled representations with semi-supervised deep generative models.\" In Advances in Neural Information Processing Systems, pp. 5925-5935. 2017.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "S1eIP8MpYB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "-- This paper seeks to combine several ideas together to propose an approach for image classification based continual learning tasks. In this effort, the paper combines previously published approaches from generative modeling with VAEs, mutual information regularization and domain adaptation. \n\nI am a making a recommendation for reject for this paper with the main reason being that I believe the primary derivations for their method appear flawed. \n\n--In the main section describing the approach (Section 4), the authors start with a claim that Equation 1 and 2 are equal; I don’t believe 1 and 2 are equal.\n\n--In Section 4.1, it appears that they are instead making a claim about Equation 2 being a bound for equation 1; but even this derivation appears to have a problem. The following is the concern:\n\n--In the second line of Equation 5, the KL term appears to be measuring a distance between distributions on two different variables; z|c and c|z. If one were to interpret the second one as the unnormalized distribution on z defined via the likelihood for c given z; even this has an issue because then the expression for KL where we plug the unnormalized density in place of the normalized need not be positive which is something they need to derive their bound.\n\n--Another issue is that the regularization lambda should apply to both the terms in the bound but in Equation (7) only appears selectively for one of the two terms. \n\nIt is also not clear how the loss function proposed differs from that of the CDVAE, etc.  If the novelty is in applying to continual learning and new datasets, it is not clear that this is sufficient.\n\nAdditional feedback for authors (not part of the main decision reasoning):\n\n- What is dt in Algorithm 1 description?\n\nFigure 1:\n-typo “implmented”\n-What’s the 3d plot supposed to represent?\nDoesn't the classification loss have a dependency on the input condition?\n\n--What does a \"heavy classifier\" imply concretely? \n\n--“Redundant weights” seems like not a very strong constraint especially for a small cardinality label space (like 10, in the case of this paper).\n\n--The notation for the proposed parameters theta, theta’, phi, phi’ are not consistent with the notation in the intro section, where phi was used for the encoder and theta for the decoder. In later sections they use theta and theta’ for encoder/decoder resp.\n\n-- “When the encoder and decoder networks are sufficiently complex, it is enough to implement each the prior and classification network as one fully-connected layer” → what do the authors mean “when … networks are sufficiently complex” or do they actually mean when the “when the problem is simple enough”?\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "H1l4PVq4KH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper devises a pipeline that aims to address catastrophic forgetting in continual learning (CL) by the well-known generative replay (GR) technique. The key ingredient of the pipeline is a modern variational auto-encoder (VAE) that is trained with class labels with respect to a mutual information maximization criterion.\n\nThe paper does not follow a smooth story line, where an open research question is presented and a solution to this problem is developed in steps. The flowchart in Fig 1 is rather a system design consisting of many components, the functionality of which is not clearly described and existence of which is not justified. This complex flowchart does not even describe the complete task. It is in the end plugged into a continual learning algorithm which also performs domain transformation. All of these pieces are very well-known methods (e.g. VAEs, conditional VAEs, CL, catastrophic forgetting, domain transformation) in the literature and this paper puts them together in a straightforward way. Hence, I kindly do not think the outcome is truly a research result. It is more system engineering than science. \n\nThe next submission of the paper could choose one or few of these pieces as target research problems and develop a thoroughly analyzed novel technical solution for them. If this solution can be proven to improve a valuable metric (e.g. accuracy, interpretability, theoretical understanding, or computational efficiency) of a setup, it is then worthwhile being published.\n\nMinor: The abstract could be improved by providing more clear pointers to the presented novelty.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a method for continual learning with a variant of VAE. The proposed approach is reasonable but technical contribution is quite incremental. The experimental results are limited to comparisons among methods with generative replay, and experimental results on more complex datasets (e.g., CIFAR 100, CUB, ImageNet) are missing. Overall, the contribution of the work in the current form seems insufficient for acceptance at ICLR.
Decision: Reject