Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This manuscript proposes a continual learning algorithm for VAE.  The encoder q(z|x) and decoder p(x|z) are apparently untouched when new tasks are learned; instead, continual learning here focuses on the prior pi(z).  The prior is adapted to approximate, as well as possible, the average of q(z|x) over all tasks observed until the present.  This is done by an algorithm similar to coreset selection, but instead of selecting data from each task, the algorithm learns representative vectors, u, from each task such that the average of q(z|u) matches the cumulative sum of q(z|x) as well as possible.
main_review: Strengths

The idea is quite innovative.  It is a significant improvement over coreset selection, the algorithm that it most resembles. 

Weaknesses

Why is it reasonable to update only the prior?  Why is it not necessary to also update the encoder and decoder in order to represent new tasks?

If there is a new set of U vectors selected for each new task, then it seems that space complexity grows linearly with the number of tasks observed.  This problem is also suffered by coreset selection and by multi-head VAE, but not by VCL; since VCL performs so well in your experiments, it would be interesting to compare their performance and their space complexity.

Eq. (10) is only correct up to O(beta).  Beta is not a noise term, it is the interpolation constant, so its size is important to the problem; why is ignoring O(beta) an acceptable approximation?

Eq. (11) is true if the denominator is \pi_\lambda^{1,2}(z), because of Eq. (7), but it is not true if the denominator is \pi_\lambda^{1}(z) as shown.

The notation is difficult to follow; there are so many different versions of the prior, and some of them seem to mean exactly the same thing.  For example, if r^{t}(z)=\pi_\lambda^{t}(z), then why is the r^{t}(z) notation needed?  Indeed, why use h(z) instead of \pi_\lambda^{2}(z)?  With all of these variables floating around, a table listing the variables would be useful.

There are frequent minor English mistakes, but they do not obscure understanding .  For example, "asses" -> "assess," "Maximization" capitalized in the middle of a sentence, "to it approximation", "additive expansions" -> "additive expansion", "this reached."


limitations_and_societal_impact: Descriptions of the relationship of this work to prior art overstate the potential impact of this work.  This is especially the case on page 1: (1) The manuscript tries to say that learning a new head for each new task is theoretically impossible because the latent space should not change, but since Nguyen et al. have shown that multi-head continual VAE works pretty well, this argument seems self-defeating. (2) The manuscript tries to say that generative replay is too expensive, but since generative replay can be accomplished by something as simple as sampling from a memorized coreset, this argument also seeems self-defeating.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a new approach for the continual learning problem of VAE. The key idea is to use trainable pseudo-inputs [Tomczak & Welling, 2018] to approximate the prior distribution of previous tasks using an entropy-based regularization method [Egorov et al., 2019], and aggregate the prior distribution to match the data points of the new task. On this basis, the proposed BooVAE algorithm is broadly evaluated on multiple datasets, and shown to mitigate catastrophic forgetting without using multi-head VAE architecture and generative replay.
main_review: Pros: The paper presents a sufficient literature review. Experimental results (especially qualitative results) show that the proposed approach is effective in dealing with catastrophic forgetting and improving sample diversity.

Cons: 
1. Although the empirical results are good, the technical novelty of this paper remains unclear to me. If I understand correctly, the proposed method can be viewed as a combination of the existing VampPrior method and the entropy-based regularization technique from [Egorov et al., 2019], which improves the efficiency and diversity of the learned prior components.
2. In Fig. 3, compared with Random Coreset, the proposed method is shown to only achieve remarkable improvement when integrated with VCL.
3. It would be good if the proposed approach can be compared with more recent and competitive baseline models, such as CURL (Rao 2019), both in performance and computational efficiency. 

A minor concern: It is suggested that the authors add a Conclusion section at the end of the paper to further summarize the contribution.

limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The authors propose a novel approach to tackle catastrophic forgetting. In more detail, they focus on variational autoencoders (VAEs) and investigate the role of the prior in the aforementioned setting. This work -- which is built on top of (Tomczak & Welling, 2018) that introduces a learnable prior through pseudo-inputs -- provides a definition of “optimal prior” in continual learning as well as an algorithm to update it when new tasks arrive. The authors evaluate the proposed approach on image generation (MNIST, its variants, and CelebA) and, moreover, as a generative memory that mitigates the forgetting of a discriminative model. 
main_review: I feel that this is a valid and strong submission:
- The motivations and the ideas are clear and well-stated.
- The paper remarkably differs from the current trend, which mainly focuses on classification.
- The approach appears grounded from a theoretical perspective.
- I appreciated the writing style: the section on Related work is concise and well-shaped; the conclusions lack but the reader does not miss it at all. 
- The experimental evaluation is extensive and well-structured: the setups and protocols are clear; competitors and baselines adopted by the authors provide insightful suggestions and remarks to the reader; last but not least, the results are good.  

My main concern regards Section 3, which I found particularly hard to follow. Honestly, this may partially be due to my limited knowledge of variational inference; however, I have a strong background in the closest related works (i.e. Vamp-Prior, VCL, and a plethora of CL methods). On the other hand, I think the authors should improve the readability of this section:
- Figure 1 is not insightful at all and does not provide an understanding of the approach.
- I read Section 3.2 several times, but still, I have only a vague idea about the functioning of the BooVAE algorithm.  
limitations_and_societal_impact: See above.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This work introduces the greedy-boosting approach to approximate the posterior for tasks in a continual learning setup for VAE. Its main contributions is the proposed relationship between the approximation of optimal prior, aggregated prior and the learning task at hand, which leads to the introduction of the BooVAE method.

main_review: The paper is an interesting extension of [VAE with a VampPrior, Jakub M. Tomczak, Max Welling] with the application to continual learning. The idea of parametrizing each task's prior with own variables is novel and original, although a bit incremental. The concept of leveraging pseudo-inputs to approximate an optimal prior distribution per task looks fresh to me. I like the relationship between the proposed method and AdaBoost in terms of Beta coefficient computation. I believe the paper is over-formalized and simple intuitions behind the method should be introduced to increase its impact.

Below I outline the major comments:

Motivation for VAE selection missing
In terms of the significance, I miss the motivation of focusing on VAE in terms of solving for continual learning. The proposed approach relates only to a single generative model, and it is not straightforward to me how to generalize this method to other generative models that can be used in generative replay, e.g. GANs.

Dataset selection
I would recommend extending the dataset pool with Omniglot and validating the performance of the method when the dataset changes (for instance train on MNIST, validate on FashionMNIST, etc. This becomes a standard of continual learning evaluation pipeline and gives more visibility in the algorithm performance.

Requirements formulation
One of the requirements formulated is related to a generative replay and it claims that the generative replay methods need to assess (there's a missing 's') samples quality and task-balance, which leads to a computational overload. This requirement ends with the claim that the proposed approach should avoid generative replay. If I understand correctly this concerns the learning of a VAE (not a general continual learning setup), as in the evaluation section the authors use generative replay as a baseline. This should be clarified.

ELBO Maximization Step
The authors introduce the function that allows to train VAE parameters, but it remains unclear if the ELBO is correctly defined here. The authors claim that they introduce a symmetric KLD, yet I'm unsure if it doesn't conflict with the lower bound. 

Minorization-Maximization algorithm
The authors use this term as if it is a well known method, while it seems to be the one introduced in this work. Please clarify that in the text.

NLL as an evaluation metric
NLL is used in the evaluation to compare the quality of generations, yet the results presented in Fig. 3 look a bit suspicious, because of the high variance within the subsequent tasks. Why is that?

One-class per task protocol
The authors evaluate their method under the assumption of disjoint classes par task. While this assumption is frequently used, it does not often correlate with the real-life conditions. Furthermore, the proposed method does not seem to provide any countermeasures against different splits. Following:
[Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335, 2019]
I believe the authors should evaluate their method using non-uniform and more complex class splits per task. This validation should provide a more thorough analysis of the proposed method.

Minor comments:

114 the following (approximated) optimal prior: (approximated or optimal - we cannot have both approximation and optimal solution

118 h looks like a density function, yet this symbol is introduced ad hoc without additional conditions for this function in (8), the notation should be fixed - does P 
mean a set of density functions? It is unclear to me.

125 - 126 To this end, we add the entropy regularization - motivation missing

157 Train new component - it is unclear why h is part of R, not P as in the following equation.

190 gaussian -> Gaussian

212 empirically evaluation - empirically evaluate

Fig. 4 - keep -> keeps

a/the - please double check the work for the article corrections (e.g. 162 a/the training dataset

Naming in figures - the authors use "Boo" to describe their method, instead of BooVAE as introduced in the text. Why is that? I think it should be consistent.
limitations_and_societal_impact: This part is completely missing (together with the conclusions section).
review_title: null

Meta-review: The submission proposes a learnable prior for continual learning of a VAE to retain performance on previous tasks.  The reviewers were unanimous that the paper is above the threshold for acceptance at NeurIPS.  Quoting from Reviewer T9Fg "The idea is quite innovative. It is a significant improvement over coreset selection, the algorithm that it most resembles."  The reviewers also appreciated both the theoretical and empirical results in the paper.  From Reviewer w1MQ "The experimental evaluation is extensive and well-structured: the setups and protocols are clear; competitors and baselines adopted by the authors provide insightful suggestions and remarks to the reader; last but not least, the results are good."  Some concerns remained, including some language typos, and clarity of presentation (comments by T9Fg, w1MQ - section 3, and 1sJx).
Decision: Accept (Poster)