Title: DISCRIMINATIVE VARIATIONAL AUTOENCODER FOR CONTINUAL LEARNING WITH GENERATIVE REPLAY

Abstract: Generative replay (GR) is a method to alleviate catastrophic forgetting in continual learning (CL) by generating previous task data and learning them together with the data from new tasks. In this paper, we propose discriminative variational autoencoder (DiVA) to address the GR-based CL problem. DiVA has class-wise discriminative latent embeddings by maximizing the mutual information between classes and latent variables of VAE. Thus, DiVA is directly applicable to classification and class-conditional generation, which are efficient and effective properties in the GR-based CL scenario. Furthermore, we use a novel trick based on domain translation to cover natural images which are challenging to GR-based methods. As a result, DiVA achieved competitive or higher accuracy compared to state-of-the-art algorithms in Permuted MNIST, Split MNIST, and Split CIFAR10 settings.

1. INTRODUCTION:

Even though deep learning models such as Szegedy et al. (2015) ; He et al. (2016) ; Xie et al. (2017) have been showing remarkable performance on many image recognition tasks exceeding human accuracy, they suffer from the catastrophic forgetting problem. The catastrophic forgetting problem occurs when we want to learn several tasks sequentially without any continual learning (CL) technique (Ratcliff (1990) ; French (1999); Goodfellow et al. (2013) ). Theoretically, the catastrophic forgetting problem can be completely avoided if we store all of the past data, but it is sometimes difficult to store all of the previous data due to the limitation on memory capacity.

To handle the forgetting problem in deep learning, some studies suggested weight regularization (WR) techniques such as EWC (Kirkpatrick et al. (2017) ), SI (Zenke et al. (2017) ), and IMM (Lee et al. (2017) ). In these approaches, each of them has a regularization term to constrain the update of some parameters that are important for previous tasks. Other researches such as GEM (Lopez-Paz et al. (2017) ), VCL (Nguyen et al. (2018) ) introduced an additional memory called coreset which consists of the small number of real images of past tasks to further improve WR algorithm. We will call this approaches as experience replay (ER)-based methods in this paper. Specifically, ER-based methods are useful when we can access previous task data though the amount of past data is limited. Recently, generative replay (GR)-based approaches such as DGR (Shin et al. (2017) ) and VGR (Farquhar & Gal (2018) ) are also proposed to replay previous tasks with a generative model instead of storing the small number of images into coreset. They showed that if the generative model successfully approximates data distribution, this approach can be a strong way to alleviate the catastrophic forgetting problem.

In this paper, we propose a new efficient and effective model called discriminative variational autoencoder (DiVA) (Fig. 1 ) to address the GR-based CL problem. GR-based approaches can address a continual learning setting where a stream of real samples is seen only once, which can not be considered in ER-based CL algorithms. However, GR-based approaches have two shortcomings. First, they need an additional module, a generative model, in addition to a classifier. Second, distribution discrepancy between real and generated samples makes GR-based methods hard to address complex natural images such as CIFAR (Krizhevsky & Hinton (2009) ) or ImageNet (Deng et al. (2009) ). In this point of views, we try to handle the shortcomings. Firstly, distributions of latent variables of DiVA are clustered class-wise discriminatively on the latent space by introducing mutual information maximization between latent variable z and class condition c. Thus, DiVA is directly applicable to both classification and class conditional sample generation with one integrated model. Also, we apply a simple but effective domain translation (DT) trick to mitigate the distribution discrepancy and thus DiVA can achieve better classification accuracy on past tasks. To the best of our knowledge, we firstly introduce the DT trick to GR-based CL problem. In summary, we describe three main contributions of our research as follows:

• We suggest a new type of conditional VAE in terms of mutual information maximization, which can be expressed as integrated classifier and learnable prior network. It makes discriminative class-wise distributions for each class on latent space. • We propose a simple but effective DT trick, which significantly improve CL performance of GR-based approaches on a natural image dataset such as CIFAR10. • By combining DiVA and DT, we achieved state-of-the-art accuracy on the class incremental learning settings.

2. RELATED WORKS:

VAE with GMM prior VAE (Kingma & Welling (2013)) has been widely studied due to its concrete theoretical background and interesting latent representations. Similar to our mixture-like prior distribution, some researchers proposed learnable Gaussian mixture prior (Dilokthanakul et al. (2016) ; Nalisnick et al. (2016) ) or hierarchical prior distributions (Tomczak & Welling (2018) ) because the unit Gaussian assumption as prior in VAE is a too strong constraint. They mainly address an unsupervised learning task while we consider explicit supervision tasks. Wang et al. (2017) also proposed GMM VAE with class-specific and randomly fixed priors for a supervised task. In this case, the random initialization can map similar patterns to far regions on latent space, which could not be suitable for the main desiderata, smoothness, of representation learning (Bengio et al. (2013) ). On the other hand, since we consider learnable priors, DiVA satisfies the smoothness condition (Fig. 2 

(d)).:

Conditional VAE There are several ways to implement conditional VAE (CVAE). The easiest and intuitive way is to concatenate one-hot vectored class conditions onto latent variables which are outputs of an encoder network (Doersch (2016) ). This method sets prior distributions for latent variables of all classes as a unit Gaussian. Since the CVAE always needs data-label pairs for concatenation, other researches propose to infer the class label with an independent heavy classifier (Kingma et al. (2014) ; Sohn et al. (2015) ). As a result, they can make their conditional generative model work with unlabeled data. However, concatenation makes feature representation unnecessarily sparse and requires additional weight parameters for feeding class conditions. In contrast, we do not need the heavy classifier and redundant weights as we model class-wise discriminative latent representations.

Generative replay:

Recently, studies which use GR have been actively proposed to mitigate catastrophic forgetting in the field of CL. This approach generates previous task data when a new task 

3. PRELIMINARY:

Variational Autoencoder Considering we have observations X = {x 1 , x 2 , ..., x N } where x i ∈ R d , the goal of the VAE is to maximize marginal log likelihoods of each observation log p(x 1 , x 2 , ..., x N ) = N i=1 log p(x i ) with respect to both variational parameters and generative parameters. However, this includes an intractable term when we use a complex model. Because the intractable term is always positive, we can optimize the variational lower bound instead of the marginal log likelihood as follows:

log p(x i ) =D KL [q θ (z|x i )||p θ (z|x i )] + L(θ, θ ; x i ) ≥L(θ, θ ; x i ) = E q θ (z|xi) [log p θ (x i |z)] -D KL [q θ (z|x i )||p(z)],

where q θ (z|x i ) is the variational posterior (the encoder) and p θ (x i |z) is the decoder. In general, both the encoder and the decoder can be modeled with neural networks. The L(θ, θ ) is called as the evidence lower bound (ELBO).

Mutual Information:

In information theory, mutual information of two random variables, X and Y, is a measure of the mutual dependence between the two variables. The mutual information can be decomposed into marginal entropy and conditional entropy as follows:

I(X; Y ) = H(X) -H(X|Y ) = H(Y ) -H(Y |X).

The meaning of this definition is intuitive: if we observe Y then how much uncertainty of X changes. If X and Y are completely independent, i.e. H(X|Y ) = H(X), then the I(X; Y ) = 0. On the contrary, if X is determined by observed Y, then the I(X; Y ) is larger than zero.

In Fig 2, unconditional generative models such as standard VAE and CDVAE generate some ambiguous samples. Thus, we should consider a conditional generative model for sampling class-wise explicit images to handle the incremental learning problem using a GR-based approach (further details are discussed at Sec.6.1.). Here, we consider a classifier integrated VAE. If latent variables for input images are located class-wise discriminatively on the latent space, we can conduct both class prediction and conditional generation. Fortunately, we can directly model the relationship between latent variables z and class condition c by maximizing mutual information between them. The optimization of DiVA is as follows:

EQUATION

This can be equivalently rewritten as:

EQUATION

where, λ is a weighting parameter of the mutual information loss and E q θ (z|x) [log pφ (c|z)] is a cross-entropy for multivariate Bernoullis or L2 loss for continuous-valued conditions. Also, θ, θ , φ, and φ mean network parameters of the encoder, the decoder, the prior network, and the classifier, respectively. When the encoder and decoder networks are sufficiently complex, it is enough to implement each the prior and classification network as one fully-connected layer. Detailed derivations are described at Sec.4.1.

4.1. MUTUAL INFORMATION MAXIMIZATION:

In this section, we describe how maximizing I(z; c) with ELBO makes our DiVA. Vincent et al. (2010) ; Chen et al. (2016) showed that minimization of reconstruction error is related to maximization of mutual information between two variables. Similarly, we maximize mutual information between class condition c and latent variable z by minimizing a reconstruction loss. Here, we will start from I(c; z) = H(z) -H(z|c). Then, we will optimize1 :

EQUATION

Now, we derive the conditional entropy H(z|c) as follows:

EQUATION

If we introduce p(c|z), then Eq. ( 4) can be rewritten as,

EQUATION

Since D KL [q(z|c) || p(c|z)] is always positive and q(c) is a constant in our dataset, we can get a lower bound of the mutual information as follows:

EQUATION

where, E q(z|c) [log p(c|z)] can be viewed as class reconstruction in autoencoding manner; minimizing L2 or cross-entropy. In CDVAE, the authors directly optimize E q(z|x) [log p(c|z) ] to impose class information on z so that a classifier can be integrated with VAE as follows: However, since they assume unit Gaussian p(z) as a prior for the variational posterior q(z|x), relation between q(z|c) and q(z|x) is not considered. Thus, latent space has low density regions because z becomes discriminative within unit Gaussian, which causes ambiguous sample generation (Fig. 2(c) ).

EQUATION

In contrast, we explicitly consider the relation between q(z|c) and q(z|x) as follows. Firstly, since c corresponds to a class among a finite label set, we can simply assume q(z|c) as a class-conditional Gaussian distribution, i.e. q ∼ N (z|µ c , σ 2 c ). Then, we can replace the unit Gaussian prior in Eq. ( 1) with the class-conditional Gaussian q(z|c), which corresponds to the second term in Eq (2). Now, since we minimize D KL [q(z|x)||q(z|c)] in our loss, we can optimize E q(z|x) [log p(c|z)] instead of E q(z|c) [logp(c|z) ]. As a result, latent variable z of our DiVA holds information of not only input x but also class c without low density regions: each class condition has its own one mode compact Gaussian distribution. Furthermore, H(z) in Eq. ( 6) can be indirectly maximized to hold information of x by maximizing the ELBO in Eq. (2). To summarize, our modeling for the probability of the latent variable given its class has three virtues:

• Class is well-separable in the latent space.

• The distribution of the latent variable given its class is simply modeled.

• The latent variable has enough information to reconstruct the image.

The first two virtues are related to the class condition; classification network and prior network, respectively. The last virtue is achieved by the standard VAE.

5. DOMAIN TRANSLATION:

In the GR-based CL approach, distributions of generated and real images should nearly close to work well. However, there is always a discrepancy between generated and real images, even with current state-of-the-art generative models. For this reason, it is known that GR-based methods are hard to be expanded to a more complex dataset than MNIST variants such as CIFAR10 or ImageNet (Lesort et al. (2018a) ). Here, to narrow the distribution discrepancy, we define the two domain, real domain X and sample domain Y, and their bidirectional domain translation, G : X → Y and F : Y → X . In this paper, we call G as a real to sample (r2s) and F as a sample to real (s2r) translation, respectively. We consider two conditions for this translation mappings. First, we should translate only the style as keeping outline patterns of given images. Second, we should consider an unpaired domain translation between real and generated images because the generated images are sampled randomly. In that sense, we use CycleGAN (Zhu et al. (2017) ) which satisfies the conditions. When we consider r2s mode, we apply r2s translation to test set images of past tasks and classify them. While, when we consider s2r mode, we translate generated past task samples on the training phase and train our model to classify them. The purpose of both r2s and s2r translation is to narrow the gap between what our model memorizes and what our model looks currently. Detailed definition of this translation is described in Appendix C, and the training process is explained in Algorithm 1. 

6. EXPERIMENTS:

In this section, firstly, we discuss the importance of conditional generation for continual learning. Then, we analyze the performance of our model on Permuted MNIST and Split MNIST. Finally, we discuss the effectiveness of our domain translation trick for GR-based methods to address CIFAR10 dataset. Since most of the GR-based approaches did not report results on Split CIFAR10 setting, we firstly reproduce Split MNIST results and then expand to Split CIFAR10 setting based on open-source codes. More details such as hyperparameters and network structures of our model are described in Appendix A.

6.1. CONDITIONAL VS UNCONDITIONAL GENERATION:

The quality of generated images is crucial in continual learning with the generative replay. In the generative replay, a model learns a new task over both the generated previous task data and the real current task data. If generated images have bad quality, then a classifier and a generator of the model degrade over the previous task data. Also in learning the next new task, the degraded generator generates poorer images, and the classifier degrades repeatedly. Therefore generating images of good quality prevents the model from degrading in continual learning. Recently, this fact is also studied by Lesort et al. (2018b) . Optimizing the VAE according to the conditional likelihood of data generates sharper and more realistic image samples (Sohn et al. (2015) ; van den Oord et al. ( 2016)).

To double-check whether the conditional generation is effective, we generated a few MNIST samples conditionally or unconditionally (Fig. 2 ). Samples generated conditionally were shown to be more recognizable. Such images of good quality generated conditionally will eventually prevent the VAE from degrading in continual learning.

6.2. INCREMENTAL LEARNING WITH MNIST:

In the Hsu et al. (2018) , they categorized various incremental learning tasks as three general cases: incremental task, incremental domain, and incremental class learning. In this section, we address the incremental domain and class learning based on MNIST dataset. Since MNIST dataset is simple for a generative model to make highly realistic samples, we do not apply the domain translation trick to MNIST variant settings. The detailed explanation about each setting is described in Appendix D, and we recommend to read Appendix D for a deeper understanding of settings and reported results. Farquhar & Gal (2018) insightfully pointed out drawbacks of the Permuted MNIST task in that unrealistic large differences in each permuted digits lead to artificially lessened forgetting. Thus, we also report results for Split MNIST setting in the next subsection.

Split MNIST Split MNIST is more challenging than Permuted MNIST task in that added new tasks could have similar patterns with old tasks (more realistic differences). In this experiment, we consider the five subsets of the MNIST data: the first subset consists of {0, 1}, the second subset is composed of {2, 3}, and in the same manner for remaining subsets. Also, we set our network with a single-headed form of network Farquhar & Gal (2018) , which makes predictions over all classes (digits 0 to 9) with one output head; a multi-headed network (Nguyen et al. ( 2018 2017)) for Split MNIST comprises multiple output heads and uses a task id guiding to a specific head for its task, which is an easier setting than single-headed Split MNIST. We can see that WR only approaches such as EWC, SI, and IMM fail to prevent catastrophic forgetting for previous task data, see Figure 3 . This is because they do not have any explicit terms to discriminate the old and similar new tasks in their loss function. Also considering minimization of cross entropy with softmax, values of newly added output nodes should be bigger than nodes for old tasks. Concurrently, weights connected to nodes for old tasks are trained to output small values. That is why WR only approaches fail in the Split MNIST setting without reference samples of the previous task. From Table 1 , the VCL and GEM use previous real data stored in coreset memory. So they can maintain performance for previous tasks to some degree while generative replay approaches can effectively mitigate the catastrophic forgetting problem in this setting. This means that if a generator can generate samples which are almost same with real images, GR-based approaches could be a strong way to mitigate catastrophic forgetting. Furthermore, our proposed model achieves higher accuracy than other GR-based methods. This is because our model conditionally generates previous samples, which makes more class-wise clear samples than other unconditional generative models.

6.3. INCREMENTAL LEARNING WITH CIFAR10:

GR-based methods can outperform other CL approaches such as WR or ER in the challenging situation that an agent can observe each sequential task only once. The assumption of the advantage is that generated images should be highly similar to real images. However, there is a discrepancy between generated and real data distributions depending on the complexity of images. As a result, GR-based algorithms with a natural image dataset are suffering from catastrophic forgetting more than ER-based methods. In this section, we provide strong potential for GR-based approaches to be able to extend to natural image datasets.

Split CIFAR10 This setting is the same as the Split MNIST task for CIFAR10 dataset. We divide CIFAR10 data into two or five subsets to make 2 or 5 class incremental tasks. To reduce the discrepancy of real and sample distributions, we used the r2s domain translation trick based on CycleGAN (Zhu et al. (2017) ). The results are reported in Table 2 . Without domain translation, all GR-based approaches suffer from serious catastrophic forgetting problem. However, by applying r2s translation, all algorithms show significant improvement on both 2 and 5 tasks Split CIFAR10 settings. Especially, DiVA achieved the highest accuracy among the state-of-the-art GR algorithms. Since all other GR algorithms have similar network capacities with DiVA, we can conclude that class-wise clear sample generation is important for incremental learning. Example images of domain translation are illustrated in Figure 4 . We observed that the domain translator successfully translates the domain of real images into the domain of sample images generated by DiVA. This simple but powerful domain translation trick can be applied to any other GR-based approaches. Also, considering the improved quality of s2r translation, we think this technique can be further studied for a VAE-GAN hybrid model to take advantages of both models: diversity and high quality. Comparison of Domain Translation Tricks Here, we compare possible domain translations described in Sec.5.

In addition to our proposed domain translation tricks, Razavi et al. (2019) reported that applying reconstruction to test set images can significantly improve the CAS score (Ravuri & Vinyals (2019) ). Since CAS score measures accuracy for real test set images of a model that trained with generated samples from a generative model, the reconstruction process can be viewed as a kind of domain translation. The results are at Table 3 . The experiments were conducted in two settings: Split CIFAR10 divided into 2 and 5 subsets. Firstly, s2r translation showed lower accuracy than r2s translation. This is because that translating sample images to real style is still challenging even with an additional domain translator. While, r2s translation could be achieved more easily by, for example, blurring. Also, r2s translation showed higher accuracy than the domain translation using reconstruction. This is because distributions of reconstructed real images and randomly generated samples also have a discrepancy. In contrast, we directly model the translation between generated samples and real images. Furthermore, domain translation with reconstruction can only be applied to autoencoder-based architectures, while proposed methods can be applied to any kind of GR models, including GANs.

7. CONCLUSION:

In this paper, we addressed how to mitigate the catastrophic forgetting problem. To solve the forgetting problem, we considered a generative replay based approach which generates previous task data and used them as a reference for previous tasks. We proposed a new type of conditional generative model based on the VAE architecture, which can be viewed as a classifier integrated generative model. Thus, we can conduct both the classification and the class conditional generation with one model. Also, we apply a simple domain translation trick based on CycleGAN to overcome the weakness of a generative replay based approach. As a result, our DiVA outperforms other recent continual learning algorithms in Split MNIST and CIFAR10 settings. We are sure that our DiVA has the potential to achieve higher accuracy than current reported results if a powerful generative model based on VAE is developed. Besides, we think that proposed domain translation trick can help other future continual learning researches which use the generative replay.

error or binary cross entropy. Also, when we conducted some experiments with this adding noise technique for the EWC and the VCL, there was no noticeable increase in final accuracy for 5 tasks incremental learning. The detailed structures of the DiVA for Permuted and Split MNIST settings are described in Table 4 , and learning curves for Permuted and Split MNIST tasks are illustrated in Figure 3 . Also, for Split CIFAR10 setting, we implemented our encoder network for DiVA based on https://github.com/xternalz/WideResNet-pytorch. Depth, widen_factor, and dropRate are 28, 4, and 0.5, respectively. Also, we set the latent dimension for mu and variance of DiVA as 1024 and stacked one fully connected layer for both prior and classification network. We did not apply random noise to the Split CIFAR10 setting. The decoder network is doubled input and channels than described in Table 4 . For experimental results, we used 60,000 images for training and 10,000 images for testing in MNIST variant settings. Also, we used 50,000 images for training and 10,000 images for testing in CIFAR10 variant settings.

B. KL-DIVERGENCE BETWEEN TWO GAUSSIAN DISTRIBUTIONS:

Since we train both prior and posterior distributions for latent variables, we describe the detailed formulation of KL-divergence between two arbitrary Gaussian distributions here: 

EQUATION

EQUATION

where, G and F mean the mapping functions G : X → Y and F : Y → X . Also, λ controls the relative importance of the two objectives. As a result, CycleGAN aims to solve:

EQUATION

In our paper, the source domain X and the target domain Y mean real images and generated images, respectively. We used codes for CycleGAN from https://github.com/aitorzip/PyTorch-CycleGAN and followed default hyperparameter settings. Between each task, we trained CycleGAN for 60 epochs.

D. DETAILED EXPLANATIONS OF INCREMENTAL LEARNING SETTINGS:

There are two commonly used operations from datasets to evaluate continual learning algorithms: pixel permutation and class splitting. The typically used dataset is MNIST LeCun et al. (1998) , which is composed of 0 ∼ 9 hand-written digit classes. The Permuted MNIST setting includes ten digit classes, where each class consists of different pixel permutations of each digit class in MNIST dataset. (2017) . At the multi-headed architecture, each task has its own task id and independent classification heads. So, when training and testing a model, the task ids are given. This multi-headed split MNIST setting is also viewed as a relatively easy setting due to the task id. Farquhar and Gal Farquhar & Gal (2018) propose a single-headed variant which does not require task id, where a model should predict all classes that have seen. This single-headed Split MNIST is known as incremental class learning. Both our Split MNIST and CIFAR settings consider this single-headed Split setting which is more challenging than the multi-headed Split setting. A conceptual explanation of each setting is illustrated in Figure 5 .

E. CONTINUAL LEARNING PROCESS OF DIVA:

Here, we explain the whole process of DiVA for the incremental class learning task. At first, we train DiVA for task 1 with real images and labels. When task 2 comes, DiVA generates pseudo data and labels for task 1. Then, we train DiVA with both task 1 and task 2 jointly. When task 3 comes, DiVA

:

For continuous variables, we can simply change the to .

Figures and Tables:

Figure 1: Overview of the discriminative variational autoencoder. The prior network and classification network can be implemented with a simple neural network, e.g., one fully connected layer. The ⊕ and ⊗ represent element-wise addition and multiplication, respectively.

Figure 2: Visualization of the latent space (top) and generated samples (bottom). In this visualization, we randomly sampled 1000 images from MNIST test set. We used the t-SNE (Maaten & Hinton (2008)) to reduce the dimension of latent vectors for plotting.

Figure 3: Learning curve for both Permuted (top) and Split MNIST (bottom). Numbers written next to the 'VCL -' mean the size of random coreset data Nguyen et al. (2018) per each task.

); Zenke et al. (2017); Lopez-Paz et al. (

Figure 4: From left to right, real, r2s translation, s2r translation, and sample images generated by DiVA are displayed.

al. (2017) is an image-to-image translation algorithm. It translates an image from a source domain X to a target domain Y without paired examples (e.g., a photo of New York City and a painting by Gogh). They proposed a cyclic translation technique using a GAN architecture to keep the consistency of shape and convert the style of an image. The loss function of CycleGAN consists of adversarial losses L GAN and a cycle consistency loss L cyc as follows:

Figure 5: Three general incremental learning settings. This is based on Hsu et al. (2018); van de Ven & Tolias (2018)

The averaged accuracy of tasks after training of all the 5 tasks are done. WR, ER, and GR mean weight regularization, experience replay, and generative replay, respectively.

Table content: <table><tr><td>Approach</td><td/><td>WR only</td><td/><td>WR + ER</td><td/><td/><td>GR</td></tr><tr><td>Model</td><td>EWC</td><td>SI</td><td>IMM</td><td>VCL GEM</td><td>RtF</td><td colspan="2">OCDVAE DGR PGMA DiVA</td></tr><tr><td>Permuted MNIST</td><td colspan="5">94.14 93.94 96.09 96.33 96.19 97.31</td><td>-</td><td>94.54</td><td>96.77</td><td>96.01</td></tr><tr><td>Split MNIST</td><td>19.8</td><td colspan="4">19.67 67.25 82.71 92.20 92.56</td><td>93.24</td><td>94.8</td><td>81.7</td><td>97.92</td></tr></table>

Ablation experiment for effectiveness of domain translation on Split CIFAR10 setting. In this task, pixels of each image are shuffled with task-wise fixed permutations(Goodfellow et al. (2013);Kirkpatrick et al. (2017)). We applied five different permutations to original MNIST data to make 5 Permuted MNIST tasks. Most of the CL algorithms show high accuracy in this task. Our DiVA shows slightly low accuracy than others. This is because Permuted MNIST looks random noises and does not have spatial between neighbor pixels. Thus, it is hard to generate such images with a CNN architecture exactly. For experience replay (ER) based methods such as VCL or GEM, we set memory size for ER to 200 per each class; This memory size is also used for the Split MNIST setting. This memory size corresponds to the additional memory requirement for DiVA, a decoder network, described at Table4in Appendix. While this task is addressed in most of the continual learning studies,

Table content: <table><tr><td/><td colspan="4">2 tasks w/o DT 2 tasks w/ DT 5 tasks w/o DT 5 tasks w/ DT</td></tr><tr><td>DGR</td><td>48.71</td><td>70.33</td><td>21.75</td><td>29.75</td></tr><tr><td>RtF</td><td>48.13</td><td>67.71</td><td>21.53</td><td>38.27</td></tr><tr><td>OCDVAE</td><td>48.48</td><td>67.84</td><td>21.64</td><td>33.15</td></tr><tr><td>DiVA</td><td>48.34</td><td>78.93</td><td>28.52</td><td>42.11</td></tr><tr><td>Permuted MNIST</td><td/><td/><td/><td/></tr></table>

Comparison of domain translation tricks with DiVA. Results are averaged after each setting all done.

Table content: <table><tr><td/><td colspan="2">2 tasks 5 tasks</td></tr><tr><td>Reconstruction</td><td>75.89</td><td>38.31</td></tr><tr><td>CycleGAN s2r</td><td>72.41</td><td>19.54</td></tr><tr><td>CycleGAN r2s</td><td>78.93</td><td>42.11</td></tr></table>

Peer review (iclr2020):
In this paper, the authors focus on alleviating the catastrophic forgetting problem in continual learning. The authors propose a discriminative variational autoencoder (DiVA) to solve this problem under the generative replay framework. DiVA modifies the objective function of VAE by introducing an additional term that maximizes the mutual information between the latent variables and the class labels. The authors do not thoroughly explain the motivation of this paper. The authors do not explicitly define continual learning, incremental learning, and catastrophic forgetting problem. It is also not clear to me why these problems are important. The idea that introduces labels in VAE is not novel. For example, Narayanaswamy et al. [1] also propose to utilize labels to VAE. I do not understand why making use of labels is important for solving the catastrophic forgetting problem and how the labels are useful in the generative replay process. It is also not clear to me how domain translation is relevant to continual learning. In terms of modeling, since the input into the prior network has finite possible discrete values, we do not need a fully connected network to generate and . Instead, we can directly optimize and for each as parameters. The paper provides some good experimental results. But the problem settings are not clear to me. I do not understand how the model is trained to solve multiple tasks. Do the same model is trained for multiple tasks? Is each of the tasks trained sequentially or simultaneously? It is also not clear to me why CIFAR datasets involve two domains and how these domains are relevant in each of the tasks. In summary, since DiVA gives a good experimental performance, the proposed method might be promising. However, it looks to me that the authors need to better explain the motivation of DiVA, the differences of DiVA from existing supervised VAE, and the experimental settings, before the acceptance of this paper. References [1]Narayanaswamy, Siddharth, T. Brooks Paige, Jan-Willem Van de Meent, Alban Desmaison, Noah Goodman, Pushmeet Kohli, Frank Wood, and Philip Torr. *Learning disentangled representations with semi-supervised deep generative models.* In Advances in Neural Information Processing Systems, pp. 5925-5935. 2017.