Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: Summary: This paper proposes a unified model for continual learning and aims to address the following problems:
Out-of-train-domain dataset recognition
Catastrophic forgetting
The out-of-domain or open set recognition model is not only used to detect outliers but also for sampling “representative data” of previous tasks for forward (and backward) transfer. 

COMMENT(S)/QUERIES:
------------------------------
 
While the experiments do justice to the contributions mentioned in the paper, I believe that certain sections need clarifications and expansion (while some can be dismissed). 

1. Given my limited knowledge in the literature on this topic, I would have appreciated a proper related work section. 

2. I believe equation (1) could have been moved to supplementary for reference since equation (2) is the only important equation in the paper. 
I don’t believe Equation (1) is really used for testing purposes. It’s just that the data is sent through the probabilistic encoder and then classified. There is no need for reconstruction of the data point.

3. It’s difficult to follow the flow of the method. Shouldn’t generative replay (algorithm 3) be placed before open set recognition of unknown and uncertain inputs (algorithm 2), since the latter is probably just used during the test, while the former affects the training procedure directly (implicit data augmentation)?

4. Major concern: It’s somewhat strange to observe that the VAE model is able to generate multiclass data so fluidly with a simple gaussian prior. This kind of challenges the belief that current generative models are unable to capture all modes perfectly. A small note about why multimodal prior was not used (which intuitively and mathematically makes more sense) and also a statement about the average time to generate multiclass multi-modal data using algorithm 3 would have been nice. 

5. In section 3.2, “For our single-head expanding classifier this ensures..”. While having a single-head expanding classifier is listed as one of the important contributions, it hasn’t been given enough justice w.r.t to the implementation details. Is it like adding a completely new classifier during the training of the new task?
The entire section on hyper-parameters could’ve been moved to the supplementary if space was a major constraint but compromising on details about an important contribution only weakens the paper. 

I especially like figure 2 in the experiments, where the importance of Weibull CDF outlier rejection prior is highlighted.


Typo: 
incorrect opening inverted comma for the word background in the introduction section (page 1)

OVERALL COMMENT
-----------------------------

The paper combines the generative, and discriminative models into one framework for multiple important tasks. While the contributions are clear in the introduction, the presentation of the paper is somewhat too complicated at a couple of places. It does not do full justice to explaining its more vital components and some parts of the method section feel like a jigsaw puzzle, where readers are heavily expected to “figure out on their own”. 
With proper presentation though, I can envision this paper contributing positively to unified frameworks in general.

Due to the above reasons. I am giving it a score of 6.

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper combines replay and openMax approach to help continual learning.  The results shows robustness on different dataset include image and audio in the continual learning condition, where the new come data has a different distribution but the model still able to maintain reasonable quality for the previously and newly come examples. To my understanding, this approach was not ground-breaking but seems a reasonable combinations.

I'm learning to give weak reject for this paper because of it's poorly written. (1) It's very hard to align the contribution claimed by the paper and previous work in the introduction section. I highly suggest the author re-write this part and has a separate section about related work and explicit describe the difference compare to others. (2) The contribution seems over-claimed, it said it's a unified framework, but I don't understand what it has been unified. (3) In the experimental part, it use audioMNIST. Any reason to use this dataset? There are more well-defined audio task such as TIMIT for phoneme classification or Aurora for digital recognition. They are more easy to understand since they have well established benchmark.

Given my limited knowledge on this the literature of this topic, I'm happy to change the score if the written being improved and the following question being addressed.

(1)  Introduction. I take most of space to describe previous work, but hard to find out what's difference of this paper. My understanding it combines A and B and apply it to C. But it claim it's a unified framework. 
(2) "We fully share our model across tasks and automatically expand the linear classifier with additional units when encountering new classes, thus not requiring explicit task labels." I cannot link "automatic" with the proposed method. Is that doable because of the proposed framework? 
(3) Why use AudioMNIST which is an unusual task for audio?
(4) For the giant Table 1, I suggest link each acronym with the reference paper. So it can easily get how it associate with different approach. Highlight some numbers can also help the reader understand what's going on in this giant table.
(5) Can the author give me some insights, what these KL loss demonstrated in the table? I feel since you use the beta-vae version, the kl scale is depend on different approach, not really comparable for these different models.
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper tackles the problem of catastrophic forgetting when data is organized in a large number of batches of data (tasks) that are sequentially made available. To avoid catastrophic forgetting, the authors learn a VAE that generates the training data (both inputs and labels) and retrain it using samples from the new task combined with samples generated from the VAE trained in the previous tasks (generative replay). In this way, there's no need to store all past data and even the first learned batch keeps being refreshed and should not be forgotten.

I like that this paper uses a single global probabilistic model instead of separate discriminative and generative ones. Unfortunately, there are several things that left me unconvinced about this paper:

1) Presentation of the paper

- Variables x, y, z are introduced and talked about without explanation. The graphical model or factorization assumptions are not even mentioned until after the loss has been defined. A normal flow is to first describe the model and what the involved variables mean, and then talk about what the loss for learning it should be, not the other way around.

- Text contradicting the equation: "In order to balance the individual loss terms, we normalize according to dimensions and weight the KL divergence with a constant of 0.1". But equation (2) shows a loss with no weighting. I'm assuming the text is correct, but then a beta should be added to the equation in front of the KL divergence.

- Tables and figures are inconveniently far from where they are referenced in the text.

2) Theoretical inconsistencies

Although the system might work overall, two things seem to be technically incorrect:

- The decoder and classifier are expected to approximate the distribution of training data according to the authors (for valid generative replay). This is not true in a beta-VAE. The weighting of the KL that the authors introduce is going to bias the learned generator towards the high probability regions. This is not a sound mechanism to achieve an as-faithful-as-possible (limited by the expressiveness of the encoder-decoder architectures) approximation to the training data.

- A Weibull distribution is used to model the same data, again, in a different way. I.e., there are two different probabilistic models modeling the same data in inconsistent ways and one or the other is used depending on the part of the system. (As an example, q(z) could be arbitrarily multimodal as far as the encoder is concerned, but the Weibull seems to force one mode per class. But regardless of this, both models are inconsistent.)

- Similarly, the proposed rejection sampling scheme of OCDVAE is not consistent with the theory of VAEs and it's a post-hoc tweak that is not theoretically expected to provide a pdf of data with lower KL divergence to the true data pdf.

3) Experiments

Finally, the experimental results do not look very compelling, it seems to be overall worse than the baselines in the two image datasets and slightly better in the audio dataset, so it's unclear that this approach is superior.
review_title: Official Blind Review #1

Meta-review: This paper presents a unified probabilistic approach for deep continual learning by combining generative and discriminative models into one framework that solves the following problems: catastrophic forgetting, and identifying out of distribution and open set examples. The method termed, OCDVAE in the paper achieves closer or better to SOTA results in different evaluation tasks. 

The reviewers had several concerns about the presentation of the paper and some errors in the equations, all of which seem to have been fixed in the latest upload made by the authors. Blind review #3 was delayed as the original reviewer refused to review the paper and this review was then obtained by someone else after the new upload of the paper, so this review looks at the new version of the paper. I would recommend the authors to incorporate suggestions provided by reviewer #3 in the final version of the paper including expanding on the related work section. 

However, as of now I recommend to reject the paper.
Decision: Reject