Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors in the paper propose a complementary system trained using local learning rules and bio-inspired mechanisms inspired from basal ganglia, such as lateral inhibition and gating based on context, task selection using context memory. The authors propose a decomposed learning mechanism which eliminates the need for weight transport mechanisms. The methods proposed in the paper combine multiple mechanisms to demonstrate continual learning capabilities on image recognition datasets.
strengths_and_weaknesses: **Strengths**

* The back propagation-alternative approaches are closer to the biological neural networks and provide an avenue for efficient learning
* The treatment of local knowledge propagation among the hidden layers is novel.

**Weaknesses**
* The writing/presentation of ideas need to be improved for clarity
* The experimental setup is not clear/reproducible
* Comparison with other local learning-based continual learning approaches is missing
questions: 1. Based on the algorithm, the initial prediction states are being computed using a top-down approach. So, how is $z^l$ being computed for the initial state, is it a random initialization or is it directly the one hot encoded label? If the z values for the intermediate labels are initialized to 0, how would we compute the $z^l_u$ values for the corresponding layers based on equation 1.
2. "Concretely, our approach could be classified as task incremental learning (Domain-IL) [21] with the exception that no task descriptors are used at both training and test time (an aspect of Class-IL [21])" - This sentence needs to be clarified and rephrased. Task IL is not Domain IL.
3. Isn’t there an overhead for storing the task-specific context memory. And would that not also increase/grow with the number of tasks? there are other replay-based approaches which also operate with very limited and small buffer sizes or there are other generative based models like brain inspired replay, so the point regarding the proposed model being more memory efficient needs to be discussed in the paper.
4. The learning rule decomposing the weight update towards local layers without transmitting the error backwards has been already proposed by other surrogate gradient-based mechanisms. So, however using local knowledge only for learning is novel. How would you compare the proposed approach to the other surrogate gradient-based approaches?
5. The proposed weight update rule uses modulation factors as an optimizer factor for updating the weights. The modulation factors are similar to some of the previously proposed approaches

    * Beaulieu, S., Frati, L., Miconi, T., Lehman, J., Stanley, K. O., Clune, J., & Cheney, N. (2020). Learning to continually learn. arXiv preprint arXiv:2002.09571.

    * Tsuda, B., Tye, K. M., Siegelmann, H. T., & Sejnowski, T. J. (2020). A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex. Proceedings of the National Academy of Sciences, 117(47), 29872-29882.

    * Imam, N., & Cleland, T. A. (2020). Rapid online learning and robust recall in a neuromorphic olfactory circuit. Nature Machine Intelligence, 2(3), 181-191.

    * Madireddy, S., Yanguas-Gil, A., & Balaprakash, P. (2020). Neuromodulated neural architectures with local error signals for memory-constrained online continual learning. arXiv preprint arXiv:2007.08159.

What is the role of the modulation factors? Is it just for inducing weight stability or are these factors necessary for continual learning? An ablation analysis would enable the reader to understand the role of the modulation factors? Generally modulation is considered necessary for continual learning and is helpful for selective attention in the network. Are the modulation factors performing a similar role?

6. The experimental setup in Section 4 is not clear. In Table 2, the accuracies for multi-head models looks inaccurate? Do you mean task incremental learning scenario for multihead or class Incremental learning? Class-IL based models are generally not categorized as multi-headed models? Please clarify the description and check the correctness in the accuracies for compared works. Additionally, newer works which demonstrate superior performance need to be compared. You can find some of the works in
    * Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., & Sanner, S. (2022). Online continual learning in image classification: An empirical survey. Neurocomputing, 469, 28-51.

limitations: limitations have not been discussed in detail but potential negative societal impacts are briefly touched upon
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose a biologically inspired/plausible neural network for continual learning. The proposed method leverages a context mechanism to learn multiple tasks while minimising forgetting; and compare favourably against baselines across standard benchmark tasks.
strengths_and_weaknesses: The formalism based on predictive coding is attractive because of its biological plausibility as well as its flexibility. At face value, the model demonstrates very impressive empirical results by outperforming backprop-based neural networks -- which is not an easy feat. However, the experiments section is missing some important analysis in my view. 

Regarding the experiments, I understand that no meta-parameter optimisation is done for the baseline methods, which makes the comparison a bit unfair, but it is not the end of the world. However, I do not understand why the original learning rates are altered and set to 0.01 for all baselines (see Question 2). 

Many continual learning papers display learning curves across multiple tasks (eg, the EWC paper). This is very helpful for identifying the degree of learning speed and forgetting. In addition, it is helpful to compare the learning curves across papers, which gives some form of reassurance that the implementations or hyperparameter choices are correct/decent.

In terms of novelty, I would like to learn more about the differences between the proposed approach,  "Learning to Adapt by Minimizing Discrepancy", and "Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations" (Question 1). In fact, this should ideally be discussed in "Related Work".

The paper is written clearly and is easy to understand overall. It would be helpful to show how the update equations can be derived from Equation 6. Because, it seems to me that just differentiating the loss wrt weights result in backprop without zeroing/stopping some gradients. Also, the line 20 in Algorithm 2 should refer to Equation 4 I believe.

I am open to bumping up my rating if my concerns are adequately addressed.
questions: 1 - It seems to me that the main difference between the proposed approach and "Learning to Adapt by Minimizing Discrepancy" is the addition of contexts. Could the authors discuss the similarities and the differences?

2 - Why do all the baselines use same learning rate (\lambda = 0.01)? This seems quite high for conventional neural networks used in conjunction with conventional preprocessing.

3 - The proposed model has two independent learning components: "predictive" weights and contexts. Such models, that aren't trained end-to-end, can be challenging to train due to hyperparameter sensitivity. Can the authors comment on this?
limitations: Limitations are adequately addressed.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This work uses a generative predictive coding model to perform continuous learning. The goal of such models is ti perform learning and inference on multiple tasks by using the same network, with the hope that (1) training on a new dataset won't make the network forget previously learned information, and (2) that the network will actually use previously learned information to perform better inference on the new task. To this end, the authors propose a generative PC model, conceptually similar to the one proposed by Rao and Ballard, with multiple differences:

1) Feedforward and feedback weights are different;

2) Multiple regularizers are used to improve the performance (modulation factors);

3) To make continual learning possible, learnable task embedding vectors, that depend on specific tasks and are separated using a contrastive loss, are added. 

The method is then tested on multiple benchmarks, with good results. As a disclaimer, I am not an expert on continual learning, hence I'm judging the experiments based on the reported results. After a couple of google searches, it seems to me that the authors have well picked the baselines and datasets, but I may be wrong here. 
strengths_and_weaknesses: Pros:

The idea of the method is nice, and the experimental results confirm the intuition of the authors. It is original, as I'm not aware of any application of generative predictive coding networks (PCNs) to continual learning. The paper reads ok: it is well written, but sometimes the large number of details makes the text flow heavy. To conclude, this work could be significant to two different communities: computational neuroscience, more in detail the branch that tries to tackle machine learning problems this way, and continual learning. 

Cons: 

While being a generative PCN, the literature about these models is not discussed properly: although some of the relevant work is actually cited In block in line 197, a proper discussion feels needed. Particularly, a proper comparison with the models that inspired your architecture would raise multiple interesting questions that would be nice to discuss in the paper, that I will state in the next paragraph.
questions: Above, I have highlighted the three differences between generative PCNs and your proposed method above: Feedback connections, task embedding vectors, and regularisers. Of course, task embedding vectors are essential for continual learning, and are probably the main reason your model works well.  However, why have you decided to implement the other two differences? so, more in detail:

1) How would a model with symmetric connections behave?

2) How important is the regularization in the final performance? 

I'm asking this, as I have the feeling that removing one of the two, would harm performance, but I'm having troubles understanding why. Hence, I would like to read a discussion that better tackles this. Even better, I would like to see experiments on one task made by using a network where  (1) and/or (2) are removed, and how the performance compare with the final model. 


Minor:

Equation 5 is not that clear to me, what is M?

WTA seems to only depend on the value and not on the absolute value, right? Why this choice?

Feedback weights are called "recurrent weights". As a reader, I often associate "recurrent" with "lateral connections", and not feedback ones. Maybe changing the name would make this clear.

Equation 4: shouldn't it be g^l_{t+1} instead of g_l_t.


limitations: Well and largely discussed in the supplementary material.
review_title: null

Meta-review: metareview: This paper provides a biologically-inspired method based on predictive coding to address the dangers of catastrophic forgetting in continual learning, while encouraging models to leverage similar data from their past when learning from present information. The reviewers agreed the paper was interesting and worth publishing, although there was a spread in their enthusiasm. That said, the consensus is clear enough that I am happy to recommend acceptance.
confidence: Certain
Decision: Accept