Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: The authors propose a novel method for continual learning with neural networks based on a Bayesian approach. The idea consists in working with Bayesian neural networks, using the Bayes by back-prop approach in which a factorized Gaussian variational distribution is used to approximate the true posterior. To address the continual learning setting, the authors propose to multiply the learning rate of the mean parameters in the posterior approximation by the corresponding standard deviation parameter in the posterior approximation, while the learning rate for the variance parameters in the posterior approximation is not changed. The authors also consider a version of his method which freezes the mean and variance variational parameters when the signal to noise ratio is high. The proposed method is evaluated in exhaustive experiments, showing state-of-the-art results.

Clarity:

The paper is clearly written and easy to read. The method proposed is well described and it would be easy to reproduce.

Quality:

The proposed method is well justified and the experiments performed clearly illustrate the gains with respect to previous methods.

Novelty:

The proposed method is novel up to my knowledge. The methodological contributions do not seem very sophisticated, but the experiments show that the proposed method, despite being very simple, works very well in practice.

Significance:

The experiments show that the proposed method achieves state of the art results when compared with a very large number of baselines. This indicates that the proposed method will be relevant to the community. In my opinion, this work is highly significant.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: ** post rebuttal start **

After reading reviews and authors' response, I decided not to change my score.
I am happy with the author's response addressing my concerns (mainly about the fairness on the size of the model), so I recommend its acceptance. I believe it is a good addition to the community of continual learning.

** post rebuttal end **


- Summary:
This paper proposes to use a way to improve continual learning performance by taking "Bayes-by-backprop" method. They claim that the uncertainty can naturally be measured by estimating (log of) the standard deviation, and it is indeed useful to judge the importance of each learnable parameter. Experimental results on several benchmarks show that their method outperforms few state-of-the-art methods.


- Decision and supporting arguments:
Weak accept.

1. The proposed method is simple but effective. However, It is still questionable whether \sigma is the best measure of the weight importance. An ablation study with different choices of the importance measure (maybe \mu can also be incorporated as well as \sigma?) would be good to see.

2. Survey and comparison with memory-based methods are limited. Though memory-based methods require some memory to keep the experience, the proposed method also requires additional memory for \sigma; it essentially doubles the model capacity, assuming that \sigma is solely for measuring the weight importance. In particular, when it comes to large-scale models, memory for storing some important experiences would be small compared to the memory to store the model.
Here are some papers about recently proposed memory-based methods, which are not cited:

Castro et al. End-to-End Incremental Learning. In ECCV, 2018.
Wu et al. Large Scale Incremental Learning. In CVPR, 2019.
Lee et al. Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild. In ICCV, 2019.

3. Comparison should include the model capacity as in Table 1(b). Again, compared to the conventional non-Bayesian model, half of the model capacity is used for computing \sigma (uncertainty), I wonder it causes a performance drop when the model capacity is the same over all compared methods. If they used the same model architecture and just doubled the number of learnable parameters for \sigma, then it is obviously unfair.


- Comments:
1. Pruning is not beneficial in terms of the performance. I hope to see some quantitative benefits obtained by introducing pruning. In Table 1(b), why doesn't pruning reduce the number of parameters?

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: **** Post Rebuttal ****

I have read the author's response and other reviewers' comments. In light of comments by other reviewers, I am increasing the score. The paper reports decent empirical results in some challenging settings which might be useful to the continual learning community. 

**** End ****

The paper presents a simple yet effective way to avoid catastrophic forgetting in a continual learning setting. The proposed approach is referred to as UCB - "Uncertainty Guided Bayesian Neural Networks". The main idea of the approach is to weight the learning rate of each parameter in the neural network by the standard deviation of its posterior distribution. This leads to regularizing parameters that are "important" to tasks seen earlier and thus avoiding forgetting.  Results indicate an improvement over other baselines. However, I do not see any analysis of the method that explains this improvement. I do not recommend acceptance.

Cons:

- My main concern with the paper is that it fails to justify the superiority of the method over other baselines. The numbers reported in the paper do seem good, but I don't see an explanation of why this is the case. What are the drawbacks of EWC, VCL or HAT that the proposed method solves? Why using uncertainty to define importance works better than using online VI in VCL or fisher information in EWC? There is no discussion in the paper about that. Without such a discussion it seems that the model was run a number of times and the best score was reported out of all those runs (especially because the improvement is only marginal). 

- I am not sure why weighting the learning rate would be a good idea? Having high uncertainty may increase the learning rate arbitrarily. Is there a constraint on the standard deviation? Does having a very high weight for learning rate not cause instability during optimization? I think the method would be very sensitive to the initialization of the standard deviation. 

Overall I think the idea of using uncertainties for continual learning is interesting. But from where it stands, I am not fully convinced that this method should do better than existing approaches.
review_title: Official Blind Review #2

Meta-review: While prior work has shown the potential of using uncertainty to tackle catastrophic forgetting (e.g. by appropriate updates to the posterior), this paper goes further and proposes a strategy to adapt the learning rate based on the uncertainty. This is a very reasonable idea since, in practice, learning rate control is one of the simplest and most understood techniques to fight catastrophic forgetting. 
The overall approach ends up being a well-motivated strategy for controlling the learning rate of the parameters according to a notion of their "importance". Of course now the question is if this work uses a good proxy for "importance" so further ablation studies would help, but the current results already show a clear benefit. 

Decision: Accept (Poster)