Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This work proposes an online variational Bayesian (VB) approach to continual learning. The prior over neural network functions is both over the neural network structure and parameter values, where the structure is modelled by an Indian Buffet process (IBP) and the weights are drawn from a Gaussian.
Similarly, the approximate posterior is assumed to be IBP and factorised Gaussian as well and inference is performed through variational inference and reparametrization of the respective distributions.

The approach is similar to VCL in that it uses online VB for learning, however, the prior and approximate posterior is more general in that it also considers the neural network structure as a random variable (prior and posterior). 

Theory:
The approach is theoretically sound and well motivated; the paper is presented well and easy to follow.
My main concern is that that the second paragraph of the paper motivates with scenarios where the ability to *adapt to dynamically changing environments or evolving data distributions* is essential. However, online VB assumes iid data. That is, the online algorithm should infer the posterior over all tasks rather than adapting to dynamically changing data distributions. Inference is sequentially, but the ordering of the task should in theory not matter - it only matters in practice as we perform approximations. 
See e.g. [1] (ICLR 2020) for an approach that explicitly adapts (through forgetting) the distribution over neural network weights. It could be possible to extend this work to similar adaptation mechanisms, although for multi-task learning such adaptation/forgetting may not be desirable. I would appreciate a few comments on this and I think it should also be discussed shortly in the paper. 

Experimental evaluation:
The experimental section considers scenarios that are very common in the CL literature. Unfortunately these are not the most interesting or insightful, as these are variants of MNIST. But since most related work considers these settings as well, the choice is justified. 
The results are quite strong. I find the results on classification from the latent space of the unsupervised learning approach especially convincing and interesting (Table 1). 

Related work:
The relation to [2] needs to be discussed in more detail. What exactly is the difference if the IPB is put on the activations rather than weights? What are pros and cons? I am aware that there are additional experiments in the supplementary material comparing to Kessler. Why do you think your approach outperformed the one of Kessler?

Another interesting aspect is that the coreset does not help much, which is in contrast to VCL. Do you think this is because the performance is already high? Or because the coreset selection algorithms (k-center, random) are unsuitable?

[1] Continual Learning with Bayesian Neural Networks for Non-Stationary Data, ICLR 2020
[2] Hierarchical indian buffet neural networks for bayesian continual learning

review_title: Official Blind Review #3

final_score: rating: 4: Ok but not good enough - rejection
review_content: I found it difficult to evaluate this paper because the paper does not say which continual learning problem it is solving in the supervised learning case. Is it solving the class-incremental learning or task-incremental learning problem? Without knowing that, it is hard to make an assessment because the two problems are usually solved in very different ways and their evaluation protocols are different too. I tried to guess but get confused. 

The paper writes in the evaluation section “To gauge the effectiveness of our model towards preventing catastrophic forgetting, we report (i) the test accuracy of first task after learning each of the subsequent tasks; and (ii) the average test accuracy over all previous tasks 1, 2, . . . t after learning each task.” And it also writes earlier “Omitting the task id t for brevity.” I am confused with these two statements. Do you need task id during training and testing? If you are solving the class-incremental learning, task id should not be used in training or testing, at least not testing. How do you do (i)? Do you only use the test instances of the first task? Do you restrict those instances to be classified into only the classes in the first task, or do you allow them to be classified to future classes in future tasks? For class-incremental learning, one should be getting the accuracy of all classes learned so far rather than each task. So, I am guessing that you are doing task-incremental learning. (ii) also gives me the same impression. If that is the case, the following systems are expected to be compared: Uncertainty-based Continual Learning with Adaptive Regularization (NIPS-2019) and Overcoming Catastrophic Forgetting with Hard Attention to the Task (Serrà et al., 2018). IBP is related to the mechanisms in (Serrà et al., 2018) and (Adel et al. 2020). It is desirable to have them compared. 

If you are doing class-incremental learning, more recent baselines should be compared. The baselines used in your experiments are old. 

Learning a Unified Classifier Incrementally via Rebalancing. CVPR 2019. 
Overcoming catastrophic forgetting for continual learning via model adaptation. ICLR, 2019. 
Large scale incremental learning. CVPR 2019
Random path selection for continual learning. NeurIPS 2019
Continuous learning of context-dependent processing in neural networks. Nature Machine Intelligence, 2019.
itaml : An incremental task-agnostic meta-learning approach, CVPR, 2020
Continual learning with hypernetworks. ICLR, 2020

In the experiment, varying the number of tasks for each dataset is also desired to show the generality of the proposed approach. 

review_title: This paper proposed a Bayesian nonparametric approach to continual learning in the context of deep neural networks. It combines structure learning and online bayes. The framework uses the Indian Buffet Process and is an extension of the VCL work in (Nguyen et al. 2018). However, I don’t know which continual learning problem it is solving and thus it is hard for me to evaluate. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:  The paper proposes a continual learning framework based on Bayesian non-parametric approach.  The hidden layer is modeled using Indian Buffet Process prior.  The inference uses a structured mean-field approximation with a Gaussian family for the weights, and Beta-Bernoulli for the task-masks.  The variational inference is done with Bayes-by-backprop on a common ELBO setup.  The experiments show less diminishing accuracy on the increment of tasks on five datasets for the discriminative problem, and for generation the methods learn one digit or character at a time on MNIST and notMNIST datasets.


Pros:
- The paper shows a structured mean-field approximation of the continual learning problem, and train a single hidden layer of 200 units on the discriminative and generative settings.
- The paper is easy to follow.

Cons: 
- The structure of the network is rather shallow.  The authors also mentioned the challenge in their conclusion.
- The experiments were done on rather simple datasets.  The paper introduces a proof of concept instead of showing the capabilities of the proposal in complex scenarios.
- The mixture of the structured mean-field approximation seems straightforward and builds extensively on previous work.  The novelty may be on making the network train using this approach, but the proposal uses a rather simple layer configuration (one layer).

Comments:
- In Section 5.1 you mention that you report the mean accuracies on the different tasks (Fig. 2).  How many versions of the tasks do you generate and average over?  Adding error bars would help to see the variance of the methods over the tasks.

Minor comments:
- Typo P6 par2 "tries to adapts"
- Typo P7 par 3 "comapred"

Overall rating:
The idea of applying Bayesian non-parametric for continuous learning is interesting, and the authors show a simple implementation on simple datasets.  The evaluated tasks are extremely related and in a general continuous learning setup this method may not work.  More extensive and complex experiments (i.e., more complex databases and setups for discrimination as well as for generation) may shed some light on the process.  Due to all these issues I rate the paper as a 5.
review_title: Interesting applicaton of Bayesian non-parametrics for continuous learning but the experiments could be more challenging

final_score: rating: 7: Good paper, accept
review_content: The authors present a new structure-learning approach to Continual Learning, by modelling each hidden layer using a nonparametric Bayesian prior, a technique inspired by recent work on learning sparse NNs. Using this technique, a sparse subset of available weights is used for each task, selectively allowing knowledge sharing between subsequent task while reducing catastrophic forgetting on the deactivated connections. 

This makes for an overall very convincing submission, reflected by my minor criticism. Well placed among other recent publications on CL in top venues. 

Pros (in no particular order):
- The method is principled and follows naturally and elegantly in the VCL framework. The presentation follows a clear narrative that is easy to follow.
- The method comes with a task detection mechanism.
- Both discriminative and generative modelling are naturally supported in the same framework.
- The Appendix discusses all necessary details to a level well-above standard in the literature. Analysis covers most of the interesting questions that can naturally be asked about this method. 
- Presented results are overall strong and cover standard evaluation in the literature. I was pleased to see experiments in an application other than supervised image classification (here unsupervised learning). 
- A simple heuristic for dynamic expansion is introduced, a worthwhile direction for future research. 

Cons (in no particular order):
- This method requires storage of a binary matrix for task and each layer in the network. However, the authors show that space complexity grows logarithmically with the number of tasks, which is likely to make this an acceptable trade-off. 

Author feedback:
- IMHO the main contribution of this work is its ideas on structure learning in the context of Continual Learning. This is however not reflected in the title.
- The paper would benefit from clearly stating why structure learning for CL is a worthwhile direction to pursue. Well-written papers clearly state why the proposed direction is a worthwhile method of investigation. 
- Figure 3 is missing axis labels
review_title: Automated probabilistic structure learning for CL.

Meta-review: This paper proposes a Bayesian non-parametric method for task-incremental continual learning. It is more general than previous work in that it considers the network structure as a random variable and works for both supervised and unsupervised settings. Experimental results show that the proposed method outperforms prior work in the proposed tasks.

Pros:
- It is well motivated.
- It's theoretically sound.
- It can do task inference.
- It outperforms other methods in the proposed tasks.

Cons:
- The experimental setup was not very challenging, because the dataset(MNIST) was simple and the network was shallow.
- There was no ablation study to analyze the contributions of the algorithm to the performance.
- There is not enough experiments to support the advantage of task inference.
- The paper did not compare with the SOTA task-incremental learning algorithms HAT and DEN.

The main concerns of reviewers are on the experimental section as listed in cons and the difference from previous work. The authors explained that their method has the advantages over previous work
that it can do task inference. R3 agreed with the advantage and suggested more experiments in this direction should be performed.
The authors conducted additional experiments suggested by the reviewers including comparison with HAT. They also uploaded a revised version to incorporate the comments from the reviewers.

I think the paper is well motivated and the idea of applying Bayesian non-parametric for continuous learning is interesting. It could potentially motivate interesting future work on CL. However, the main advantages/contributions are not well presented and supported by the experiments. So at present time I believe there is much room for the authors to improve their paper before publication. I hope that the authors will be able to address the feedback they received to make this submission get where it should be.
Decision: Reject