BooVAE: Boosting Approach for Continual Learning of VAE
Authors: Evgenii Egorov
Anna Kuzina
Evgeny Burnaev
Venue: NeurIPS 2021 Poster (2021)
Keywords: Continual Learning
Variational Autoencoder
VAE
Catastrophic Forgetting
Entropy regularisation
Boosting
Functional regularisation

Abstract: Variational autoencoder (VAE) is a deep generative model for unsupervised learning, allowing to encode observations into the meaningful latent space. VAE is prone to catastrophic forgetting when tasks arrive sequentially, and only the data for the current one is available. We address this problem of continual learning for VAEs. It is known that the choice of the prior distribution over the latent space is crucial for VAE in the non-continual setting. We argue that it can also be helpful to avoid catastrophic forgetting. We learn the approximation of the aggregated posterior as a prior for each task. This approximation is parametrised as an additive mixture of distributions induced by an encoder evaluated at trainable pseudo-inputs. We use a greedy boosting-like approach with entropy regularisation to learn the components. This method encourages components diversity, which is essential as we aim at memorising the current task with the fewest components possible. Based on the learnable prior, we introduce an end-to-end approach for continual learning of VAEs and provide empirical studies on commonly used benchmarks (MNIST, Fashion MNIST, NotMNIST) and CelebA datasets. For each dataset, the proposed method avoids catastrophic forgetting in a fully automatic way.

Boo VAE: Boosting Approach for Continual Learning of VAE
Evgenii Egorov University of Amsterdam egorov. gvoroveevgenyy@aaruu
Anna Kuzina
Evgeny Burnaev Skoltech, AIRI
Vrije Universiteit www.zziaayaande...uu
burnaev@skklteech. ru
Abstract
Variational autoencoder (VAE) is deep generative model for unsupervised learn- ing, allowing to encode observations into the meaningful latent space. VAE is prone catastrophic forgetting when tasks arrive sequentially, and only the data for the current one is available. We address this problem of continual learning for VAEs. It is known that the choice of the prior distribution over the latent space is crucial for VAE in the non-continual setting. We argue that it can also be helpful avoid catastrophic forgetting. We learn the approximation of the aggregated posterior as a prior for each task. This approximation is parametrised as additive mixture of distributions induced by encoder evaluated at trainable peeudo-inpute We use a greedy boosting- boose approach with entropy regularisation to learn the components. This method encourages components diversity, which essential as we aim at memorising the current task with the fewest components possible. Based on the learnable prior, we introduce an end-to-end approach for continual learning of VAEs and provide empirical studies on commonly used benchmarks (MNIST, Fashion MNIST, NotMNIST) and CelebA datasets. For each dataset, the proposed method avoids catastrophic forgetting in fully automatic way.
Introduction
Variational Autoencoders (VAEs) Kingma Welling 2013) are deep generative models used in various domains (Lee et al. 2017 Zhou et al. (020). The VAE model consists of deep neural networks (DNNs): an encoder (inference network) and decoder (generative network). DNNs are known to reduce their quality previously learned tasks when trained on data from new task. Several directions to address this problem catastrophic forgetting were suggested. But this phenomenon is mainly considered without attention to the specific propenties of the VAE. We want discuss current approaches to continual learning and formulate requirements for the ideal solution.
Dynamic architecture approach adds task-specific last layers (multi- -eadt) to encoder and decoder for each task (Rusu et al. 2016: Nguyen et al. 2018: Li & Hoiem 2018). We suppose that practical applications of VAE require the common latent space, which is violated in multi-heads. Using multi- heads requires deciding which head to apply to the new data and when to expand the architecture. They reduce reuse of similarities between tasks. Hence, suppose that the approach should keep the static architecture for both encoder and decoder.
The generative replay (Shin et al., 2017 Rao et al., 2019) uses a "teacher" generative model for generating "fake" data that mimies former training examples. Then the "student" model trained on joint "fake" and new data. This approach is conceptually simple, model-agnostic and overcomes forgetting. However, these benefits come with a computational price. We need retrain model while generating the dataset from all the past tasks, asses samples quality and the task-balance. Thus the approach should avoid generative replay).
"These authors contributed equally
I The work was done while the author was at the Skoltech, Moscow, 2019
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Weight penalty approach (Liu et al. 2018 Kirkpatrick et al. 2017) forms the trust-region around the optimum of the previous task for protecting parameters. This approach preserves the architecture and avoids generative replay. However, for DNNs a change in the weights is poor proxy for the difference in the outputs (Benjamin et al. 2018). It is an even more critical issue for VAE model as consists of pair DNNs: encoder and decoder. Thus the approach should link the data-space and the latent-space.
We propose a novel continual learning approach for the VAEs. For each task, we expand current prior to get the approximation of an aggregated posterior over the whole data. We parametrise approximation as an additive mixture of distributions induced by encoder evaluated at trainable pseudo-inputs. These pseudo-inputs link the data-space and latent-space and help to memorise knowledge about past tasks. The problem of matching the aggregated posterior is ill-posed since we observe only its empirical version. As a solution, we use greedy boosting-like approach with entropy regularisation. This method encourages components in the learned approximation to be diverse, which essential as we aim at memorising the current task with the fewest components possible. The proposed approach is orthogonal to other methods mentioned above and can be applied in combination with them. Our main contributions are the following:
We relate the approximation of optimal prior, the aggregated posterior and the continual learning task for VAE model. We find optimal additive perturbation in order to approximate optimal prior distribution. We derive the algorithm of effective approximation of the optimal prior for the continual learning framework.
We use this result and present Boosting Approach for Continual Learning of VAE (Boo framework for training VAE models in the continual framework with static architecture.
We empirically validate the proposed algorithm on commonly used benchmarks (MNIST, MNIST. NotMNIST) and CelebA for disjoint sequential image generation tasks. The proposed generative model could be efficiently used in a generative replay for discrimi- native models. We train both generative and discriminative models incrementally, avoiding retraining the generative model for each task from scratch or storing several generative models. We provide code at httpss: // //ggttubb. com AKKuzinnaaaa BooVAAE
2 Background
Variational Autoencoders (VAEs) Let po( 2) be the joint distribution of observed variable x E RERD and hidden latent variable 2, with the distribution (z). Given the distribution x p~pee we aim to find f which maximizes the marginal log-likelihood Ey,(() [log / Po z 2)dz]. Since the marginal likelihood is often intractable, we solve this optimization problem by the variational inference (Jordan et al. 1999). Variational autoencoders (VAE) (Kingma & Welling 2013) amortize inference with go [2((>) ()) variational posterior (encoder) and pa(ar z) as likelihood (decoder). The encoder and decoder are parameterized by neural networks with parameters Φ, O. Nets are optimized simultaneously via maximisation of the evidence lower bound (ELBO) objective:
(x) (log po (x) (x) log ( x)) S EEp log /
C(O, o, A) A)
pa (z, 2 2)
(1)
()(=(=)
Typically, instead of the density Pe (w) given a dataset (x: ) and consider empirical
distribution / Σ (x)).
Optimal Prior and Aggregated Posterior Hoffman & Johnson (016); Goyal et al. (017) discuss the choice of the prior distribution over latent space T(2) and observe that default choice of the Gaussian prior significantly restricts the expressiveness of the model. Tomczak & Welling (018) proposes to use a prior that optimizes the ELBO The solution of this problem the aggregated variational posterior:
((z) (2)
For an empirical pe( ((2))(z)))))))) >A go(2(10,) and Tomezak & Welling (0118) proposes a para-
metric approximation to avoid over-fitting and reduce computational costs: T(2) K Σk k= qs (z))). Parameter (ux { KxD is optimized simultaneously with (0, Φ) by the BLB
maximization, and K is a hyper-parameter of the algorithm. The additive structure of the optimal prior n" (2) over
the dataset points motivates us to consider such a prior distribution for continual learning.

---

Continual learning framework In the continual learning framework, we do not have access to the whole dataset. We define the sequence of tasks be solved t= ........ T. Subsets of the data for each task DD DI..... D' arrive sequentially and we have access only to the data of the current task. For each task I, we consider the empirical distribution p'. (xx)
12 Σ 5x,, (x)).
JE ED!
VAE Algorithm (Proposed Method)
We start from defining an optimal prior for the VAE model in the continual learning framework. Next, we find the optimal additive expansion of the current prior to match the innovation coming from the new task. We use obtained result and provide general algorithm for training the VAE model in the continual learning framework. It works as an iterative Minorization- cetoriiattionMMaiizzztttii algorithm. In the minorization step, we expand the current approximation of the prior and learn pseudo-inpur -initss. At the Maximization step, we update parameters of the encoder and the decoder with the prior being fixed.
3.1 Optimal prior in continual learning
In this section, we derive an optimal prior for the VAE model in the continual framework and provide the algorithm to it approximation. We provide skipped technical details in Sup.dA.] We start from the following decomposition of the E
C(O, Φb m) 11 p. (=(()) log po z) - (KT. [Kll
where (c) Uo (z x)) is the aggregated variational posterior. As KL-divergence is non-negative, global maximum of Eq. over ı reached when:
("()) (=)= (e)
We assume that for the sequence of the two tasks, mixture of two distributions: (x(x)= = a + (=(x),, a (0; 1). This assumption holds
write the data distribution the discrete
for empirical data distribution, which is of the most interest for us: (=(x)= 3/2 E Sa., (x)= |D*]
(D 1D11+ (x)
|D*|
|D* p2 (2r). Then we can decompose the ELBO as following:
C(0, (0 (x) log (ogp)(z)2) (s(=/=)
E
KL x) (Klla///llllalllaa)aXa//a//////a
(5)
1.2
where aj 02 =1 and (z) Ep: (z) go x). Keeping only terms with dependence over the prior distribution, we conclude to the optimization problem over the probability density space P:
min a KL((I())| + (1 a) KL KL (L()(()) =(*)P
(6)
Hence, if we can store the data for both tasks we can use the same optimal prior Egg go (z(z), where expectation is taken with respect to a (x) (1-a) p²(x), However, in continual setting we don't
have access to the (z), neither it is reasonable to store di (z). Thus after Task 1, we consider using the approximation of the aggregated variational posterior of the first task of (z) This modification of the problem (6) leads to the following prior:
x (z) = (1- arg min =(2)EP
(7)
Next, we consider that the optimal prior for the first task n² (z) a good start for new prior after observing the new task. Hence, we propose using an additive expansions of the current prior to match innovation induced by the new task. Then the objective of the problem 7 transforms to:
min a KT (=)[(1 - 3)x (z) (1(=))+++1+11--111))))))) (L. (1-)) (h()))))
(8)
KEP
The optimization problem X is bi- co-cco over h and B. We consider the Functional Frank-Wolfe
algorithm (informally, "boosting") Waan et 2015) for the corresponded objective:
Fa.9 (=.)
(9)
=a (L (10_))000---_((((()0))))))>aa))0
3

---

Task Available Data Space: data points D
Task Available Data Space: data points x D',
pscudo inputs u C U Er
pseudo inputs trl
2 (x)x)
(x/x))
Go (2/px)
Po(x/x)
Prior over latent space:
Prior over space!
9 (z(z rx) sa N Σ 2 (z/x)
(1- a''' a' K
Σ (s(2))) (z)) uEt
1 No
Eg(1e)) MEDSE
()(x)))
K
UEU
Figure I: We propose to expand prior distribution in order to match new information from the coming task. We parametrize each component in the prior distribution with encoder, evaluated on the trainable pseudo-inputs. These pseudo inputs store information about the data at the correspondent task.
We aim find h the projection of the functional gradient the probability density space and then optimize over 3 with the stochastic gradient method, while fixed. The functional gradient of
the objective 8 with respect to perturbation h is a (z) +(1 a) (2)
(=)
(a(x) in the following sense:
Fa. 9i 2 (2(2); h(=]]
= =Fol0((=), (2) (z) -3 (ʃ h(z))
(z)
+ (1-a)
02(2)
-1)+
(10)
(2)
We consider projection to the probability space as the solution of the optimization problem:
max log f h()z) НEP
of π (2)
(2(x) x (z)
/ [x22) log
(z) π (2)
+(1 (1-a) (2(z) a (z)
+(1
(1-a)
dz max
dz.
REP
(1I)
We use the lower bound in the 11 in order to use Monte-Carlo estimates of the gradients. The problem is linear over h(z) and the solution is degenerate distribution. To this end, we add the entropy regularization H [ - h(z) log h(z) dz and obtain the final objective:
aqi (z) + a-l-))g222
min KL kEP
h
(12)
(2)
As far as we update the initial prior r (x) = (z) with the mixture component h: (1) 3)r BB((z,, we can improve current approximation by finding the new component, solving the same optimization problem, but using o instead of T² The objective (2) encourages new component to be different from the already constructed approximation:
ag (z) + (x))))
KL h
KL (1)
h(z)) log dz
π (2)
match the lurgel prior
different from the current approximation
(13)
To this end, we have defined the algorithm of learning optimal prior for VAE in continual setting. Now we will use to formulate the algorithm of VAE training in continual learning framework.
3.2 Boo Algorithm
In this section, we formulate the algorithm for continual learning BooVAE (short for Boosting VAE). It works as the iterative Minorization-Maxinie
algorithm. In the minorization step, we use the obtained optimization problem (12) to learn a new component the prior. These steps are alternated with ELBO maximization steps until the desired number of components in prior is reached. From that point further on, only model parameters are updated until convergence. As at maximization step follows the usual routine used in training VAEs, approach VAE model. The derivations of applications of BooVAE to the VAE with flow-based prior presented A.2. These steps are summarized in Alg. [T) and idea of prior learning for continual setting is illustrated in Fig.([] Now, we are going to consider them in more details.
be easily used on top of any

---

Prior Update Step At this step we expand the current approximation of the optimal prior distribu- tion. Subsets DD ,.... D' arrive sequentially and may come from different domains. We have ac- cess only to one subset of data and current prior at time but aim at learning the prior distribution [DST LEEDST qo (z(z))). At the task we start from the current prior (z)) (z). We expand current approximation with most K components. At each step k we add new component / to the cur- rent approximation r² to move it towards an optimal prior St. Following (Tmczaak & Welling 2018), we select the parametric family R of the compo- nents h as learnable pscudo-inputs u to the encoder: h(=)=9<(z(uu
Algorithm 1 BooVAE algorithm
input Current task & dataset D: J input Maximal number of components K
Prior to approximate S= + ai). Initialize prior r' =-'- 0* Ø arg m ,(r', 00
k=1
while not converged and k: < K do
h" =rrg min KL h A
BER
8 arg min KL KL [3h+ (1- (1-8"
BE(0:1)
y' 3" (1- 8'
k= k+1
0", o" =arg max [(r', 00 end while
This choice connects parameters of the prior with the data-space. We conclude with a two step procedure for adding a new component:
output r', 00'00"
1. Train new component:
h arg min KL RER
h
2. Train component weight:
3 = arg min KL +1- B)r" x=5)]
3E(0;1)
As it was already mentioned, we define optimal prior to be equal to aggregate posterior. Therefore, it stores all the information about training dataset. The optimal prior for tasks 1: can be expressed with prior from the previous step (e.g. aggregate posterior over tasks 1 - 1) and training dataset from the current task only:
DDSt-1 n S5--1
D DD
π St
(z)=
(z)+
Σ ED.
g0(2|<n))
(14)
DS
D
Since we don't have access to the data from the previous tasks, we suggest using trained prior r- as proxy for the corresponding part of the mixture. We use random subset from the current task M DD containing N₂ observations to estimate aggregate posterior of the current task.
DSt-1
(=|Hu) St-l
D
1
Σ (z(z)z).
(15)
IDS
DE
N.
ZE.M
Such formulation allows algorithm not to forget information from the previous task, which is stored in the prior distribution and pseudo-inputs {u}S+ The prior distribution performs regularization for the VAE model. As the budget of components learning per-task is reached, we perform component pruning by weights optimization, see Supp.
ELBO Maximization Step. At this step parameters of the encoder and decoder 0, áb are updated. We add regularization term, to ensure that the model remembers the information, which is stored in the ineudo-inputs. Given the fixed mean- parameeress of prior distribution from the previous task r-1, we keep its components during training the task as well as distribution of their fixed decoded peaam-aarametes/ p
Ren)(0) KLyym -Er
Rdec(() = Σ
KLaym [pe(2|=)\p (xx x)]
(16)
where (plgm (KL(()))))))))))))))_ (LL())) We use symmetric KL-divergence, since we have observed that it help to avoid encoder and decoder to memorizing delta-function centered in the pscudo-input and results in more diverse samples from the previous tasks. Final objective for the maximization step over o, for the task t is the following:
E, Dn. [log po 2) KT (2)2))))))) (=))] - X ((enn()) (0) + Racc(())
(17)
x~<(x a))
5

---

4 Related Work
The approximation of the unnormalized distribution with the sequential mixture models has been considered previously in several studies. Several works (Miller al. 2017: Gershman et al. 2012) perform direct optimization of ELBO with respect to the new component's parameters. Unfortunately, it leads to unstable optimization problem. Therefore, other works consider functional Wolfe framework, where subproblems are linearized (Wang et al.) 2015). At each step, KL-divergence functional is linearized at the current approximation point by its convex perturbation, obtaining tractable optimization subproblems over distribution space for each component. Guo al. (2016) suggested using concave log-det regularization for gaussian base learners. Egorov et al. (2019). Locatello et al. (2018) used satropy-based regularization. In our work, come up with similar optimization algorithm. We optimize different objective for different propose: the weighted sum of exclusive KL to approximate optimal prior for VAE, while mentioned works approximate inclusive KL, in order to approximate posterior distriubtion.
Boosting density approximation.
Continual learning for VAE. Lesort et al., 2019) compare EwC, generative replay and random coresets. We use a single pair of encoder and decoder for all the tasks unless otherwise stated. [guyyen al. (2018) propose to use multi-head VAE architecture, where separate encoder is trained for each task, and an extra head is added to the decoder. We believe that even though this model was shown to produce good results, it has several crucial limitations. Firstly, it makes scalability of the method very poor, since the number of parameters in the models increases dramatically with the number of tasks. Secondly, it requires task labels to be known both during training and validation to the suitable "head" for each data point. Finally, it limits the amount of information we can share between tasks and thus may lead to worse performance for the tasks, which has fewer data available and have a lot of similarities with other tasks. We have observed the latter phenomenon on the CelebA dataset, where all the tasks are supposed to generate images with faces but with different hair color. Recent work (Rao et all 2019) use learnable mixture as prior distribution. However, this reached by also expanding the architecture of encoder with multi-heads and using self-replay overcome forgetting. Achille et al. propose similar ideas of encoder expansion and use also use self-replay. However, our goal is provide orthogonal approach, which avoids self-replay and multi-heads, but can be combined with them.
5 Experiments
We empirically evaluation our algorithm on both disjoint image generation and classification tasks. In the latter case we suggest using VAE learned in the continual setting for generative replay (Shin et al. 2017).
5.1 Disjoint image generation task
Setup. We consider an experimental setup, where each task contains objects of I class (e.g. one digit for MNIST dataset). The resulting generative model, trained on all the classes one-by-one is supposed to generate images from all the classes. We perform experiments on MNIST, notMNIST. fashion MNIST and CelebA datasets. Each of MNIST datasets has 10 classes with digits, characters and pieces of clothing correspondingly. For the CelebA dataset we consider tasks based on the attributes: black, blond, brown and gray hair. For all the datasets we use VAE (Kingma & Welling 2013) model with gaussian posterior. A complete description of the experimental setup, architectures and hyperparameters can be found in Supp. (B.5).
Compared methods. As baseline we consider standard VAE model. We expect it to suffer from catastrophic forgetting and use it as a lower bound on the model performance. We refer to this model as Standard. Our method has some similarities with the regularization based continual learning algorithms, which add quadratic penalty on the weights. The difference is that we use prior, defined as mixture of the encoded pseudo-inputs, to avoid catastrophic forgetting both for the encoder and the decoder, Eq. { In the experiments we compare with such regularization-bast Elastic Weight Consolidation (EWC) (Kirkpatrick et al. 2017) and Variational Continual Learning (VCL) (Nguyen et al. 2018). Our approach uses pseudo-inputs to approximate an optimal prior distribution for each task. This procedure can also be seen as coreset selection (Huggins et al. 2016
approaches as

---

AIMSST
FadIooWNIST
IvSSINIST
Standard
VCL
MoG
Coneet
Boo VCL
13545
tasks
t25ks
Figure 2: Diversity of generated samples, estimated as a KL between discrete distribution with the equal probability for each class and empirical distribution of samples from VAEs. The lower is the better. We conclude that Boo VAE outperforms other approaches.
Standard
EWC
VCL
McG
Random Coreset
Boo VCL
Lanks
Londos
Figure 3: NLL on the full test dataset averaged over runs after continually training on 10 tasks. Lower is better. We observe that BooVAE performance comparable or better than of other methods.
99
9
9
9
9
9
9 4
0
0 2
8
2
8
2
MNIST
9 9
9
9
9
9
9
9
9
0
0
6
4
8
6
6
9
NOEMNIST FashionMINIST
CAQ AAGGG
D
Standard
EWC
Coreset (100 comp.)
McG
Boo
Beo VOL
Figure 4: Samples from the VAEs after the last task is learned. BooVAE keep sampling different
tasks, while other approaches suffer from catastrophic forgetting.
Bachem et al. 2015). To validate the quality of the learned coresets, we perform the comparison with random coresets, which also used in (Nguyen et al., 2018). We store random subsample the training dataset from previous tasks and add them to each batch during training to avoid catastrophic forgetting. Moreover, we can learn the prior distribution components the latent space instead of the data space we do with the pseudo-inputs, We have conducted experiments, where the prior is trained as mixture of Gaussian's (with diagonal covariance) the latent space. We refer this approach as MoG. Just as we do in BooVAF, in MoG we learn new mixture of components for each task and use the regularization from Eq. to avoid catastrophic forgetting.
Metrics. To evaluate the performance of the VAE approach on grey scale images, we a standard metric negative log- eileliilldood (NLL) the test set. NLL is calculated by importance sampling
Table I: FID values for CelebA dataset averaged over 5 runs, the lower is the better. Each row corresponds to the FID for cumulative learned tasks of different hair types. BooVAE outperform other approaches, including multihead.
#T
EWC
Multihead EWC
Multihead + VCL
Random Coreset cat
Random Coreset (90)
Boo :40 oxsp)
35.8 (0.8)
37.7 (1.1)
35.8 (0.6)
(1.0)
36.4 (1.3)
38.1 (1.9)
27.5 (0.6)
2
61.4 (1.7)
(1.3)
58.7 (0.6)
65.54 (0.2)
59.6 (0.4)
59.8 (0.8)
45.7 (2.4)
40.7 (0.6)
45.2 (0.8)
(0.6)
42.18 (0.4)
39.7 (0.8)
(0.1
33.4 (1.4)
(0.6)
75.7 (0.8)
48.7 (0.9)
75.16 (2.0)
48.1
47.4
43.1 (0.2)

---

vany io Scown Alllnd Blace
EWC
Multihead + EWC
Coreset (80 comp.)
Beo (40 comp.)
Figure 5: Samples from VAE trained on CelebA. For each model rows correspond to cumulatively learned tasks of different hair types.
method, using 5000 samples for each test observation:
K
po(z)z))p(z)) go(2|2))
llg ~ log K
(0(2))
NLL measures the quality of the reconstructions that VAE produces. is also essential to evaluate the diversity of generated images with respect to all the tasks in the continual setting. our experiments. each task contains a new class. Thus, we expect our model to generate images from all the classes t=1,... .... I' in the same proportion as they appear in the training dataset. For the dataset with balanced classes, this proportion is equal to T.. We assess the diversity using the sum of KL-divergences between T pairs of Bernouli distributions:
Σ KL (p:lpi) P: Be (+), De Be E No
where is empirical distribution of the generated classes, N₁- number of generated images from the class t. To estimate N,, we train the classification neural network to achieve high accuracy and use it to classify images generated by the model. We use 10 samples in total calculate the metric. For the CelebA dataset, we used Frechet Inception Distance (FID) (Heusel et al. 2017) over 10 samples, which is supposed measure both quality and diversity of generated samples. FID rely heavily on the implementation of Inception network (Barratt & Sharma 2018) we use PyTorch version of the Inception V3 network (Paszke al. 2017).
Results on (s In Fig. 2 we provide results for three MNIST datasets. Both figures depict values averaged over five runs. We report mean values, standard deviations and comparison with the multihead architectures in Supp. [.I). We evaluate the performance of VAE on the test dataset after each new task is added. The x-axis denotes a total number of tasks seen by the models (and thus, the total number of tasks in the test dataset). The flatter the line is, the less forgetting we observe as number of tasks increases. We provide numbers for each task separately in Supp.(B.2) Notice that BooVAE can be combined with any other weight-regularizatte observed that EWC does not improve performance a lot (see Supp. (upp.(B.I) therefore we report only results for the combination of BooVAE with VCL, which gives the best performance in terms of NLL.
method. We have
We observe that, based on KL metric, pure BooVAE produces the most diverse samples. We plot several samples from the model after training it on tasks Fig. ④ For MoG, Random Corset and BooVAE we use the same amount of components equal to 15 for each task. It is worth mentioning that the closer the random corset size is the size of the training data, the better the performance should be. In the limit case, we can store all the data from the previous tasks, guaranteeing the absence of catastrophic forgetting. Our goal is reduce the amount of stored information, thus we used a pretty small number of components. In Supp. B.4 we explore, how large the random corset should be to get the performance comparable to BooVAE.
Results on CelebA We conduct experiments on CelebA dataset for several reasons. Firstly, we want to show that our method works not only on small-scale images, such as MNIST but also on higher-dimensional data. Secondly, since the classes differ only by the hair color in this setting, we may see the advantages of the shared architecture more clearly. In the CelebA dataset there much fewer faces with grey hair, compared to other colors. Therefore, information from other

---

0.9
0.0
Accuracy
Accuracy
VAE: tasks
VAE: tanks
WAE: 5 tasks
VAE lasks
0.6
Baseline
0.-
Baseliee
replay
replay
5
Number tasks
Number of tasks
(a) MNIST
(b) Fashion MNIST
Figure 6: We consider continual learning experiment for MNIST and Fashion MNIST datasets, where we split the dataset in task, containing pairs of disjoint classes, ie. '0/1', '2/3', etc. We train both Boo and classification DNN in continual setting, using VAE for generative replay avoid catastrophic forgetting in classification.
classes is essential obtain good results on these images. We compare the FID values as new tasks are added to the models in Table(T). The BooVAE outperforms other approaches, including model with multihead architecture. Notice that multihead fails on the last task, which has much fewer observations, compared to others. This highlights the benefit of the shared architecture, as Multi-head approach limits the amount of shared information between tasks. Samples from the different VAEs can be found Fig. We provide samples in Supp.
5.2 Generative Replay for Discriminative Model with continual VAE
Motivation Common approach to mitigate catastrophic forgetting in discriminative models deep generative replay (Shin et al. 2017). The method is based on the recollection of the past knowledge, such as the data of past classes, by generating it from the trained generative model. However, since continual learning for generative models was limited, it proposed to re-train the generative model from scratch when a new task arrives. Since we propose the method for the continual learning of the generative model, we can avoid this. We can continually train VAE along with the discriminative model.
Setting We perform the continual learning experiment using the MNIST and Fashion MNIST datasets. We apply SplitMNIST setting when the dataset is split into five binary classification tasks. E.g. the first task containing digits '0" and "I', the second task - digits '2' and '3', and on. We perform similar splitting into the five tasks for the Fashion MNIST. We train MLP with two hidden layers, LeakyReLU activations and Dropout layers. For each task, we add a new classification head (one fully connected layer) and train for 200 epochs with batch size 500.
Results The mean accuracy across all tasks is reported in Fig. We provide two models for comparison. Exact replay setting reuses all the training data from the previous task, thus, giving an upper bound on the generative replay's performance. In the baseline method, we do not use any information about the previous task. This gives us a lower bound on the performance, i.e. if replay buffer is not used. We report two versions of the generative replay with VAE. In the first one, consider each class as a separate task and train VAE in the continual setting (VAE: 10 tasks) and sample images from the last available model. We use prior components to label the image classes. For example, along with the first classification task on MNIST (with digits '0" and '1') we train VAE firstly on '0's and then I's. As result, have separate components in the prior distribution for both tasks. Sampling latent vector from these components decoding them gives us the replay buffer for the next classification task. In the second setting, combine classes like in splitMNIST setting, i.e. each task contains two classes (VAE: 5 tasks). In this setting, samples from the prior have to be classified in order to be used in classification model. We follow the approach from Shin al. (2017). using classier from the previous step for this purpose. Results that we observe are comparable with the MNIST results in Shin et 2017 while our approach avoids full re-training of the generative model.
AAtthors provide only plot (Fig. 2 in their paper), therefore are not able to report exact numbers for comparison

---

density
component(s) learned
component(s) learned
ccmponent(s learned
Figure The mixture of 12 Gaussian is approximated by 2-component Gaussian mixture with Eq. This toy experiment illustrates the intuition of our approach, where we hope to efficiently approximate prior x (z) = (m) (a) given only empirical version of the target density N > (2(2 x,,
Conclusion
We propose novel algorithm for continual learning of VAEs with the static architecture by incorpo- rating the data-driven information about new task into the prior over the latent space. We leverage the specific structure of the VAE model and match new data innovation with the additive aggregated posterior expansion. The boosting- like approach allows us to reduce the number of components in the approximation of the optimal prior distribution without the loss of performance. We empirically validate performance of our algorithm and compare with other approaches. That being said, the proposed algorithm is orthogonal to them and could be easily combined. We would like to finalize with additional comments on performance evaluation and approach intuition.
Model perfomance In the continual learning setting important evaluate the evolution of the metrics for each task while new task arrives. Hence in Sections (B.I-
(B.3) we provide more
detailed overview of models performance and report NLL and diversity metric after each additional
task is trained. We discuss how the performance changes on the whole test dataset as we keep training in continual learning setting. Moreover, we report and discuss these metries for each task separately B.2 Finally, we visually study the samples from the model on each step in Section (B.3).
in Section
Approach intuition Sec.
We provide toy-example visualization in Fig. to illustrate equations at
Standaed prior, ([jole)pp(s) 0.20
Boo prior, KL]p(s()p(e)) 0.193
Prior
Aggregated postettx
Asu-aaated
Figure 8: Visualization of the prior density and aggregate posterior for the VAE with 2-d latent space. In left we present results for the VAE with (= /) prior and in the right with the proposed Boo prior. By visual inspection and KL-diiergence comparison we conclude that the proposed prior matches the aggregated posterior better that standard normal prior.
Acknowledgments
Authors are thankful to Jakub Tomezak for feedback and inspiring words, to Kirill Neklyudov and Dmitry Molchanov for important discussions during 2019 and to the reviewers who provided detailed feedback. The work of Evgeny Burnaev was supported by Ministry of Science and Higher Education grant No.
10

---

References
Achille, A., Eceles, T., Matthey, I.., Burgess, P., Watters, N., Lerchner, A., Higgins, I., and UCLA, D. Life-long disentangled representation learning with cross-domain latent homologies.
Bachem, O., Lucic, M., and Krause, A. Coresets for nonparametric estimation-the case of dp-means. In International Conference on Machine Learning, pp. 209-211 217. PMLR, 2015.
Barratt, S. and Sharma, R. A note the inception score. arXiv preprint arXiv: /811009777 2018.
Benjamin, A., Rolnick, D., and Kording, K. Measuring and regularizing networks in function space. In International Conference on Learning Representations, 2018.
Egorov, E., Neklydov, K., Kostoev, R., and Burnaev, E. Maxentropy pursuit variational inference. In International Symposiun Neural Networks, pp. 409-417. Springer, 2019.
Gershman, S. J., Hoffman, M. D., and Blei, M. Nonparametric variational inference. In Proceed- ings of the 29th International Coference on International Conference on Machine Learning, pp. 235-242 242. Omnipress, 2012.
Goyal, P., Hu, Z., Liang, X., Wang, C., and Xing, E. P. Nonparametric variational auto-encoders for hierarchical representation learning. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5094, 2017.
Guo, F., Wang, X., Fan, K., Broderick, T., and Dunson, D. B. Boosting variational inference. arXiv preprint Xrv:: /61 1.55559, 2016.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S. Gans trained by two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems, pp. 6626-6637, 2017.
Hoffman, M. D. and Johnson, M. Elbo surgery: yet another way to carve up the variational evidence lower bound. In Workshop Advances in Approximate Bayesian Inference, NIPS, volume 1, pp. 2, 2016.
Huggins, J. H., Campbell, T., and Broderick, T. Coresets for scalable bayesian logistic regression. arXiv preprint arXiv: (005:0660000642. 2016.
Jordan, M. I., Ghahramani, 7., Jaakkola, T. S., and Saul, L. K. An introduction variational methods for graphical models. Machine learning, 1732)3883-23,, 1999.
Kingma, D. and Welling, M. Auto-encoding variational bayes. arXiv preprint 131261131,.6114, 2013.
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Baabska-Barwinska.. A., et al. Overcoming catastrophic forgetting neural networks. Proceedings of the national academy of sciences, [14(13)3351-33566 2017.
Lee, W., Song, K., and Moon, I.-C Augmented variational autoencoders for collaborative filtering with auxiliary information. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pp. 1139-1148, 2017.
Lesort, T., Caselles-Dupré, H., Garcia-Ortiz, M., Stoian, A., and Filliat, D. Generative models from the perspective of continual learning. In 2019 International Joint Conference on Neural Networks
(ICCN), pp. 1-8. IEEE, 2019.
Li, Z. and Hoiem, D. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2944 2018.
Liu, X., Masana, M., Herranz, L.. Van de Weijer, J.. Lopez, A. M., and Bagdanov, A. D. Rotate your networks: Better weight consolidation and less catastrophic forgetting. In 2018 24th International Conference Pattern Recognition (ICPR), pp. 2262 2222.226. IEEE, 2018.
Locatello, F., Khanna, R., Ghosh, J., and Ratsch, G. Boosting variational inference: an optimization perspective, In International Conference on Artificial Intelligence and Statistics, pp. 464 472. PMLR, 2018
11

---

Miller, A. C., Foti, N. J., and Adams, R. P. Variational boosting: Iteratively refining posterior approximations. Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 2420-2429. JMLR. org, 2017.
Nguyen, C. V., Li, Y., Bui, T. D., and Turner, R. E. Variational continual learning, In International Conference on Learning Representations, 2018.
Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., Veto,, Z., Lin, 2., Desmaison, A., Antiga, L., and Lerer, A. Automatic differentiation in pytorch. 2017.
Rao, D., Visin, F., Rusu, A., Pascanu, R., Teh, Y. W., and Hadsell, Continual unsupervised representation learning. Advances in Neural Information Processing Systems, 32:7647-7657, 2019.
Rusu, A. A., Rabinowitz, C.,, Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsell, R. Progressive neural networks. arXiv preprint arXiv: (eo.:666664711 2016.
Shin, H., Lee, J. K.,, Kim, J.. and Kim, J. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990-2999, 2017.
Tomezak, J. and Welling, M. Vae with a vampprior. In International Conference on Artificial Intelligence and Statistics, pp. 1214-1223. PMLR, 2018.
Wang, C.,, Wang, Y.. Schapire, R., al. Functional frank-wolfe boosting for general loss functions. arXiv preprint arXiv:1510002558, 2015.
Zhou, I.., Deng, W., and Wu, X. Unsupervised anomaly localization using vae and veta-vae. arXiv preprint ar 20055 10686, 2020.
12

---

[Yes], [No], [N/A]
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Limitations are illustrated in Sec and Sup B
(c) Did you discuss any potential negative societal impacts of your work? [N/A] Our method is general algorithm for VAE continual learning
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Please see Sup A
(b) Did you include complete proofs of all theoretical results? [Yes] Please see Sup A
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL.) [Yes] We provide code and readme in attached .zip
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We provide all hyperparameters and details of the experiments in B.3
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] We provide error bars and number of runs in the main text and in Sup B
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Please see Sup
4. If you using existing assets (e.g., code, data, models) curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material or URL? [No] We provide only code written by ourselves.
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "yq4HFbTlWkJ", "reviewer": "Reviewer_T9Fg", "summary": "This manuscript proposes a continual learning algorithm for VAE.  The encoder q(z|x) and decoder p(x|z) are apparently untouched when new tasks are learned; instead, continual learning here focuses on the prior pi(z).  The prior is adapted to approximate, as well as possible, the average of q(z|x) over all tasks observed until the present.  This is done by an algorithm similar to coreset selection, but instead of selecting data from each task, the algorithm learns representative vectors, u, from each task such that the average of q(z|u) matches the cumulative sum of q(z|x) as well as possible.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n\nThe idea is quite innovative.  It is a significant improvement over coreset selection, the algorithm that it most resembles. \n\nWeaknesses\n\nWhy is it reasonable to update only the prior?  Why is it not necessary to also update the encoder and decoder in order to represent new tasks?\n\nIf there is a new set of U vectors selected for each new task, then it seems that space complexity grows linearly with the number of tasks observed.  This problem is also suffered by coreset selection and by multi-head VAE, but not by VCL; since VCL performs so well in your experiments, it would be interesting to compare their performance and their space complexity.\n\nEq. (10) is only correct up to O(beta).  Beta is not a noise term, it is the interpolation constant, so its size is important to the problem; why is ignoring O(beta) an acceptable approximation?\n\nEq. (11) is true if the denominator is \\pi_\\lambda^{1,2}(z), because of Eq. (7), but it is not true if the denominator is \\pi_\\lambda^{1}(z) as shown.\n\nThe notation is difficult to follow; there are so many different versions of the prior, and some of them seem to mean exactly the same thing.  For example, if r^{t}(z)=\\pi_\\lambda^{t}(z), then why is the r^{t}(z) notation needed?  Indeed, why use h(z) instead of \\pi_\\lambda^{2}(z)?  With all of these variables floating around, a table listing the variables would be useful.\n\nThere are frequent minor English mistakes, but they do not obscure understanding .  For example, \"asses\" -> \"assess,\" \"Maximization\" capitalized in the middle of a sentence, \"to it approximation\", \"additive expansions\" -> \"additive expansion\", \"this reached.\"\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Descriptions of the relationship of this work to prior art overstate the potential impact of this work.  This is especially the case on page 1: (1) The manuscript tries to say that learning a new head for each new task is theoretically impossible because the latent space should not change, but since Nguyen et al. have shown that multi-head continual VAE works pretty well, this argument seems self-defeating. (2) The manuscript tries to say that generative replay is too expensive, but since generative replay can be accomplished by something as simple as sampling from a memorized coreset, this argument also seeems self-defeating.\n\n", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "UeRxESqk5H8", "reviewer": "Reviewer_1sJx", "summary": "This work introduces the greedy-boosting approach to approximate the posterior for tasks in a continual learning setup for VAE. Its main contributions is the proposed relationship between the approximation of optimal prior, aggregated prior and the learning task at hand, which leads to the introduction of the BooVAE method.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is an interesting extension of [VAE with a VampPrior, Jakub M. Tomczak, Max Welling] with the application to continual learning. The idea of parametrizing each task's prior with own variables is novel and original, although a bit incremental. The concept of leveraging pseudo-inputs to approximate an optimal prior distribution per task looks fresh to me. I like the relationship between the proposed method and AdaBoost in terms of Beta coefficient computation. I believe the paper is over-formalized and simple intuitions behind the method should be introduced to increase its impact.\n\nBelow I outline the major comments:\n\nMotivation for VAE selection missing\nIn terms of the significance, I miss the motivation of focusing on VAE in terms of solving for continual learning. The proposed approach relates only to a single generative model, and it is not straightforward to me how to generalize this method to other generative models that can be used in generative replay, e.g. GANs.\n\nDataset selection\nI would recommend extending the dataset pool with Omniglot and validating the performance of the method when the dataset changes (for instance train on MNIST, validate on FashionMNIST, etc. This becomes a standard of continual learning evaluation pipeline and gives more visibility in the algorithm performance.\n\nRequirements formulation\nOne of the requirements formulated is related to a generative replay and it claims that the generative replay methods need to assess (there's a missing 's') samples quality and task-balance, which leads to a computational overload. This requirement ends with the claim that the proposed approach should avoid generative replay. If I understand correctly this concerns the learning of a VAE (not a general continual learning setup), as in the evaluation section the authors use generative replay as a baseline. This should be clarified.\n\nELBO Maximization Step\nThe authors introduce the function that allows to train VAE parameters, but it remains unclear if the ELBO is correctly defined here. The authors claim that they introduce a symmetric KLD, yet I'm unsure if it doesn't conflict with the lower bound. \n\nMinorization-Maximization algorithm\nThe authors use this term as if it is a well known method, while it seems to be the one introduced in this work. Please clarify that in the text.\n\nNLL as an evaluation metric\nNLL is used in the evaluation to compare the quality of generations, yet the results presented in Fig. 3 look a bit suspicious, because of the high variance within the subsequent tasks. Why is that?\n\nOne-class per task protocol\nThe authors evaluate their method under the assumption of disjoint classes par task. While this assumption is frequently used, it does not often correlate with the real-life conditions. Furthermore, the proposed method does not seem to provide any countermeasures against different splits. Following:\n[Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335, 2019]\nI believe the authors should evaluate their method using non-uniform and more complex class splits per task. This validation should provide a more thorough analysis of the proposed method.\n\nMinor comments:\n\n114 the following (approximated) optimal prior: (approximated or optimal - we cannot have both approximation and optimal solution\n\n118 h looks like a density function, yet this symbol is introduced ad hoc without additional conditions for this function in (8), the notation should be fixed - does P \nmean a set of density functions? It is unclear to me.\n\n125 - 126 To this end, we add the entropy regularization - motivation missing\n\n157 Train new component - it is unclear why h is part of R, not P as in the following equation.\n\n190 gaussian -> Gaussian\n\n212 empirically evaluation - empirically evaluate\n\nFig. 4 - keep -> keeps\n\na/the - please double check the work for the article corrections (e.g. 162 a/the training dataset\n\nNaming in figures - the authors use \"Boo\" to describe their method, instead of BooVAE as introduced in the text. Why is that? I think it should be consistent.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This part is completely missing (together with the conclusions section). ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "MRpRRYi-iVR", "reviewer": "Reviewer_3uR8", "summary": "This paper proposes a new approach for the continual learning problem of VAE. The key idea is to use trainable pseudo-inputs [Tomczak & Welling, 2018] to approximate the prior distribution of previous tasks using an entropy-based regularization method [Egorov et al., 2019], and aggregate the prior distribution to match the data points of the new task. On this basis, the proposed BooVAE algorithm is broadly evaluated on multiple datasets, and shown to mitigate catastrophic forgetting without using multi-head VAE architecture and generative replay.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros: The paper presents a sufficient literature review. Experimental results (especially qualitative results) show that the proposed approach is effective in dealing with catastrophic forgetting and improving sample diversity.\n\nCons: \n1. Although the empirical results are good, the technical novelty of this paper remains unclear to me. If I understand correctly, the proposed method can be viewed as a combination of the existing VampPrior method and the entropy-based regularization technique from [Egorov et al., 2019], which improves the efficiency and diversity of the learned prior components.\n2. In Fig. 3, compared with Random Coreset, the proposed method is shown to only achieve remarkable improvement when integrated with VCL.\n3. It would be good if the proposed approach can be compared with more recent and competitive baseline models, such as CURL (Rao 2019), both in performance and computational efficiency. \n\nA minor concern: It is suggested that the authors add a Conclusion section at the end of the paper to further summarize the contribution.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5 hours"}, "extra_text": {}}, {"review_id": "Gk7d_paDy4v", "reviewer": "Reviewer_w1MQ", "summary": "The authors propose a novel approach to tackle catastrophic forgetting. In more detail, they focus on variational autoencoders (VAEs) and investigate the role of the prior in the aforementioned setting. This work -- which is built on top of (Tomczak & Welling, 2018) that introduces a learnable prior through pseudo-inputs -- provides a definition of “optimal prior” in continual learning as well as an algorithm to update it when new tasks arrive. The authors evaluate the proposed approach on image generation (MNIST, its variants, and CelebA) and, moreover, as a generative memory that mitigates the forgetting of a discriminative model. ", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I feel that this is a valid and strong submission:\n- The motivations and the ideas are clear and well-stated.\n- The paper remarkably differs from the current trend, which mainly focuses on classification.\n- The approach appears grounded from a theoretical perspective.\n- I appreciated the writing style: the section on Related work is concise and well-shaped; the conclusions lack but the reader does not miss it at all. \n- The experimental evaluation is extensive and well-structured: the setups and protocols are clear; competitors and baselines adopted by the authors provide insightful suggestions and remarks to the reader; last but not least, the results are good.  \n\nMy main concern regards Section 3, which I found particularly hard to follow. Honestly, this may partially be due to my limited knowledge of variational inference; however, I have a strong background in the closest related works (i.e. Vamp-Prior, VCL, and a plethora of CL methods). On the other hand, I think the authors should improve the readability of this section:\n- Figure 1 is not insightful at all and does not provide an understanding of the approach.\n- I read Section 3.2 several times, but still, I have only a vague idea about the functioning of the BooVAE algorithm.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See above.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission proposes a learnable prior for continual learning of a VAE to retain performance on previous tasks.  The reviewers were unanimous that the paper is above the threshold for acceptance at NeurIPS.  Quoting from Reviewer T9Fg "The idea is quite innovative. It is a significant improvement over coreset selection, the algorithm that it most resembles."  The reviewers also appreciated both the theoretical and empirical results in the paper.  From Reviewer w1MQ "The experimental evaluation is extensive and well-structured: the setups and protocols are clear; competitors and baselines adopted by the authors provide insightful suggestions and remarks to the reader; last but not least, the results are good."  Some concerns remained, including some language typos, and clarity of presentation (comments by T9Fg, w1MQ - section 3, and 1sJx).
Decision: Accept (Poster)