Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: The paper proposes an elegant method for task-free continual learning problems. It has nicely pointed out that the conventional continual learning algorithms had the limitation of knowing the task boundaries. Followings are my summary. 

Summary: 
By applying DPM, the authors proposed a method of automatically determining whether to add a new expert for a new task or train the existing experts. While the Dirichlet Process Mixture (DPM) is not new, applying such nonparametric method to continual learning is new. The experimental results are impressive given the single-epoch setting. 

Pros:
1. Good experimental results for the task-free setting, in which no information about task boundaries is given. Particularly, even with smaller memory-usage than the experience replay (ER) methods, the proposed method achieves better results. 
2. Many past work should suffer from increased number of tasks due to the model capacity limit, but the proposed method efficiently expands the model capacity. 
3. Writing flow is good and is easy to follow. 

Cons & Questions: 
1. I am not sure whether the proposed methods should work well for "all" cases. Is there any cases in which the proposed DPM would fail?
2. What happens when you actually know the task boundaries? Would following the framework with known number of experts also excel other methods?
3. Can you apply this to the RL setting? 

review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Summary: The paper proposes to use a Bayesian nonparametric mixture model for task-free (without explicit task labels) continual learning. The main idea is to use an expansion-based model where the number of mixture components (experts) adapts to the training data/tasks. Specifically, a Dirichlet Process Mixture Model (DPMM) consisting of a set of neural network experts is used. Empirical results demonstrate improved performance on three different datasets over some of the baselines. 

I find the methodological contribution in the paper to be somewhat limited since the main idea of the model was initially proposed in the prior work (cited in the main paper): Dahual Lin - “Online Learning of Nonparametric Mixture Models via SVA”. In fact, the paper claims its contribution is expansion-based task-free continual learning. However, this “task-free characteristic” is the contribution of SVA based inference. Nevertheless, I do like how the existing SVA based inference has been adapted from an online learning setting to a more general continual learning setting by using various approximations/tricks (like short-term memory with wake-sleep training, point estimates). 

Pros:
- The idea of using a nonparametric model for CL is interesting and can lead to follow-up work.
- Results show that the approach works well.
- The code has been released.

Overall I am inclining towards voting for acceptance if the authors could address my following questions:

- Could you comment on the creation of test data? It is not clear to me how the model is evaluated if the task-boundaries are not known a priori. Shouldn’t the evaluation be based on tasks? How are you evaluating catastrophic forgetting? I am interested to know what was the test accuracy for the task for which the training data was seen early on during the training.

- It seems to me that the method works on the assumption that the number of data points for each task is at least M (size of STM) and moreover, that these data points appear together sequentially. The method should be sensitive to the size of the STM. How are you choosing M? Would the framework work if data points for each task do not appear together?

- Assuming you have clear task boundaries, how would you adapt this framework? Was the model compared to other methods that assume known task-boundaries like (VCL, EWC, Memory Replay)? 

Other comments:

- The method is inspired by a Bayesian framework but calling it Bayesian wouldn’t be fair since only a point estimate is being learned for parameters. This is important to distinguish since there are other methods that are fully Bayesian like Nguyen et. al. “Variational Continual Learning”  (although such methods may have other pros and cons)

- The samples from the base distribution of the posterior (v) are not iid anymore due to lateral connections b/w the representations. Do you think the theoretical result in Appendix B that the number of clusters is upper bounded by O(alpha*logN) is still valid?

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper proposes Continual Neural Dirichlet Process Mixture Model (CN-DPM) to solve task-free continual learning. The core idea is to employ Dirichlet process mixture model to create novel experts in online fashion when task distributions change.  The proposed method is validated on various tasks and demonstrated to perform well compared to the other baselines.

Overall I find this paper to be well-written and the experiments are conducted thoroughly. The method is compared to proper baselines in various settings, and the paper describes detailed experimental settings and architectural choices to help readers willing to reproduce. I’ve gone through the appendix and they provide enough additional experiments to support the author’s claim.

The main algorithm itself cannot be considered to be novel. DPM or other Bayesian nonparametric models have been extensively used for the problems requiring to adapt the model size according to the change of data. Nevertheless, the application of DPM in task-free continual learning context seems to be considered as a contribution.

I have much experience in implementing Bayesian nonparametric models with parametric distributions and compared various methods to conduct the posterior inference of them. In my experience, even for the low-dimensional parametric models, the posterior inference algorithms for DPM usually suffer from local optima, and the sequential methods such as SVA depends heavily on the data processing order. According to the experimental setting presented in the paper, the algorithm goes through a single pass over the data stream, yet still able to reasonably train (deep) neural networks and identify mixture components jointly. Do you have any intuition about how this becomes feasible?

I don’t fully understand why generative modeling is required. In page 4 the authors stated that the generative model prevents catastrophic forgetting. But in my understanding, using expert-specific parameters is the part that prevents catastrophic forgetting, not the generative model itself. Learning generative model in online fashion may work well in simple structured data such as MNIST,  but I highly doubt that the generative model could be trained properly for CIFAR10 or CIFAR100, especially in online setting. My concern is that learning generative model part may even impede the discriminative learning. Could you elaborate more on this?

Another minor concern is the way the concentration parameter alpha is selected. The authors stated that they chose proper value of alpha according to the number of tasks known in advance. I think this does not make sense. Alpha should also be inferred along with other parameters, or fixed to non-informative value if the performance of the algorithm is not very sensitive to the choice of alpha.

I think it would be more helpful to show how the task-assignment p(z=k|x) is learned. For instance, the clustering accuracy according to p(z=k|x) against the ground-truth task label can be measured, or at least qualitatively show what examples were assigned to each task. 
review_title: Official Blind Review #2

Meta-review: This paper proposes an expansion-based approach for task-free continual learning, using a Bayesian nonparametric framework (a Dirichlet process mixture model).

It was well-reviewed, with reviewers agreeing that the paper is well-written, the experiments are thorough, and the results are impressive. Another positive is that the code has been released, meaning it’s likely to be reproducible.

The main concern shared among reviewers is the limited novelty of the approach, which I also share. Reviewers all mentioned that the approach itself isn’t novel, but they like the contribution of applying it to task-free continual learning. This wasn’t mentioned, but I’m concerned about the overlap between this approach and CURL (Rao et al 2019) published in NeurIPS 2019, which also deals with task-free continual learning using a generative, nonparametric approach. Could the authors comment on this in their final version?

In sum, it seems that this paper is well-done, with reproducible experiments and impressive results, but limited novelty. Given that reviewers are all satisfied with this, I’m willing to recommend acceptance. 


Decision: Accept (Poster)