Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper tackles the problems of continual learning and catastrophic forgetting in neural networks. It uses methods inspired by Hebbia learning and complementary learning system (CLS), where there are slow and fast weights in the model. 

The paper addresses an important topic. I find the idea of fast/slow weights to be a refreshing and different approach compared to previous work on catastrophic forgetting. 

I had trouble following the details of the DHS Softmax, however. This may stem from my inexperience with Hebbian Learning, but here are some questions/suggestions:

- I am a bit confused exactly what is referred to as post/pre-synaptic connection, what is penultimate layer, etc. A figure might be helpful. 

- It also might be helpful to write out an equation for the standard Softmax so it can be compared to Eq 2. 

- Related to above, I am confused what is indexed by i,j in Eq 4. Compared to Eq 1, where theta only has one index (k), in Eq 4 theta has two indices (i,j). 

- In terms of motivation, can you explain why this Hebbian strategy is applied only to the final softmax? 

Other questions:

- Does it seem like DHS Softmax is not as strong by itself but works best in conjunction with others, such as EWC? I do not quite follow how they complement each other intuitively. 

- Are there any hyperparameters required for DHS Softmax? It seems to be no? 


review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: The authors introduce DIFFERENTIABLE HEBBIAN CONSOLIDATION,a new framework for continual learning that can be implemented in the usual differentiable programming setups. This framework is motivated in terms of complementary learning system (CLS) theory which features an episodic memory module. The method is shown to be easily implemented as seen in their pytorch pseudocode (authors also suggest code will be released). Additionally, authors show the method leads to significant  improvements over simple baselines, and can complement other task-specific hebbian-based learning paradigms


As an alien to the continual learning literature, my judgment is mostly positive. First, the paper is well-written, and the method is well-motivated in terms of cognitive principles with a nice commentary about underlying neurobiological substrates. The description of alternative methods seems exhaustive (but see below). The resulting algorithm is simple and doesn't seem to entail a significant computational burden. Importantly, it is differentially programing-friendly. Results show improvements over a baseline. However, I have two significant criticisms regarding the experimental section that I hope the authors will address.

1)Authors comment on two paradigms to hebbian-based continual learning: the task-specific and theirs, based on CLS. Authors show their method complement improvements due to task-specific approaches (which is great), but proper comparisons to other CLS-based approaches seem missing. In other words, I would hope a much better statement than "
However, all of these methods were focused on rapid learning on simple tasks or metalearning over a distribution of tasks or datasets. Furthermore, they did not examine learning a large
number of new tasks while, alleviating catastrophic forgetting in continual learning" Perhaps the authors may try find experimental cases where comparison to such approaches are possible, and show the empirical results? I am concerned the finetuned MLP baseline could be a bit weak.
2)The authors should try more experiments. They showed many MNIST variants and a 5-vision dataset. However, results in other papers in the literature are shown in much more sophisticated contexts (e.g. learning to play games). I understand this could exceed the computational capabilities of authors, but  why, for example, not to try on a different dataset, as CIFAR? (as the Zenke et al paper did).

MInor: On section 3, page 4 authors mention the number of labels d but that number appears undefined
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper addresses the continual learning setting, and aims to mitigate catastrophic forgetting, with results on Permuted MNIST, Split MNIST, Vision Datasets Mixture, and their own class-imbalanced version of the Permuted MNIST dataset. The authors propose to augment differentiable plastic weights - a general neural network component - with class-specific updates (similarly to prior work, such as the Hebbian softmax) at the final layer of a neural network, prior to a softmax. While well-motivated in terms of the background and methodology (indeed, this is a simple way to prevent interference in fast weights), and nicely explored experimentally with lots of examinations into the workings of the method, the weak results on the simpler continual learning settings lead me to consider this a weak reject.

The authors show that fast weights can be applied in the continual learning setting, but alone they do not perform that well on the more challenging datasets, with mixed results on how much better they are as compared to a naive fine-tuning baseline, and they definitely lag behind synaptic consolidation methods. The authors' method combined with synaptic consolidation methods perform the best, but not much beyond the effect of the synaptic consolidation methods themselves. The authors are encouraged to evaluate their method with a large amount of classes, e.g. as done by iCaRL with CIFAR-100 and ILSVRC, with class-incremental training, to show if their method (a) scales (which I would anticipate, given the class-conditional Hebbian update) and (b) can deal with an alternative continual learning setting; a further resource is the work done around CORe50, which I would consider encapsulates more current thinking and practices around continual learning.
review_title: Official Blind Review #1

Meta-review: The reviewers agreed that this paper tackles an important problem, continual learning, with a method that is well motivated and interesting. The rebuttal was very helpful in terms of relating to other work. However, the empirical evaluation, while good, could be improved. In particular, it is not clear based on the evaluation to what extent more interesting continual learning problems can be tackled. We encourage the authors to continue pursuing this work.
Decision: Reject