Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper amends the gradient update rule for continual learning using a natural-gradient-style formulation in order to regularise the trajectory during learning to forget previous task(s) less. They show experiments where this 'co-natural gradient' update rule improves some baselines. They also provide experiments showing the benefits of this update rule for low-resource finetuning settings.

Although the idea seems reasonable and interesting, I feel like this paper needs work both in the theory and experiments. Figure 1 is a nice visualisation of the key take-away point of the paper.

Theory: the authors take the natural-gradient updates from batch learning and just modify it so that the KL term is now for the previous task(s) instead of the current one. Although this may seem reasonable, I would appreciate some analysis as to what this implies or means.

Experiments: 
- Split CIFAR from Chaudhry et al. (2018b) uses 10 tasks, why does this paper use 20 tasks? 
- Previous works usually find that for EWC, large values of the \lambda hyperparameter provide best results. This corresponds to lower forgetting of previous tasks. The hyperparameter range in Appendix A.2.3 is only over small values of \lambda (by orders of magnitude). 
- Why do the authors only allow 1 epoch per task for Split CIFAR? This probably results in early stopping: the new tasks are not able to reach their new optimal points (with or without regularised trajectories). This seems to go against the intuition provided by Figure 1, where the authors are showing that changing the trajectory results in a better local minimum being found. 
In fact, by adding another regularisation term, it is unsurprising that co-natural gradient updates have less forgetting, as the extra regularisation term probably means the trained parameters are even closer to the previous parameters.

-------------------
EDIT: Score changed to 'Weak Accept' following discussion with the authors.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: The paper proposed a novel regularization methods for continual learning. The authors introduce co-natural gradients, which is an incremental development of natural gradient methods, note that co-natural gradients use Fisher information to regularize the trajectory of the gradients which will be optimal on both tasks.

The method can be applied on existing regularization-based continual learning approach such as EWC, or ER. And with co-natural gradients, the model can perform better on continual learning problem.

I think that the performance of finetuning + co-gradient is too natural, and not much meaningful. And the tradeoff between accuracy and forgetting for baselines is hard to compare on table 1 since the tradeoff depends on the hyperparameters. 

And it is required to apply on heterogeneous datasets to evaluate the performance when problems are really different. (like MNIST->CIFAR100->Omniglot->Imagenet, and so on). In the paper, the experiments are performed on very similar problems (split a single dataset).

Also, it would be great to show an illustration like figure 1, on real dataset, like split CIFAR.


review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: The paper puts forward a new regularization based continual learning method that explicitly regularizes the optimization trajectory by constraining in the distribution space. The paper is well written, and preliminary empirical results are promising. 
 
[Model]
 After surveying previous work, I am not sure if the paper is really novel: 

First of all, adding KL-based constrains to alleviate model forgetting has already been widely explored in prior arts, as the authors also acknowledged in the paper. The proposed regularizer has a close relationship with the EWC, especially with the EWC++. EWC++ encourages the KL-divergence between two distributions learned at successive tasks to be minimized, while the authors proposed to regularize the KL-divergence between two nearby updates, which leads to the well-known natural gradient descent.

Natural gradient updates require to calculate the Fisher based on the current curvature, which is computationally expensive in practice. The authors further proposed to use a static Fisher estimated at the previous task for fast approximation. However, the approximation makes the algorithm not a natural gradient descent approach, nor a valid KL-regularized optimization problem. The theoretical implications of the static Fisher approximation are not discussed in the paper.
 
[Experiments]
The authors may consider comparing with EWC++, which is a closely related baseline. 
The experiment results have shown that the co-nature gradient method help from time to time. Understandably, the co-nature gradient-based optimization has the add-on benefit for any continual learning tasks. Still, it would be much better to see how it can help more state-of-the-art methods like LwM, LwF, etc, and especially on a few more challenging datasets.


review_title: Official Blind Review #2

Meta-review: The submission proposes a 'co-natural' gradient update rule to precondition the optimization trajectory using a Fisher information estimate acquired from previous experience. This results in reduced sensitivity and forgetting when new tasks are learned. 

The reviews were mixed on this paper, and unfortunately not all reviewers had enough expertise in the field. After reading the paper carefully, I believe that the paper has significance and relevance to the field of continual learning, however it will benefit from more careful positioning with respect to other work as well as more empirical support. The application to the low-data-regime is interesting and could be expanded and refined in a future submission. 

The recommendation is for rejection.
Decision: Reject