Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a new framework, PC graphs, which can train on arbitrary network structures without the limitation of backpropagation.
It is inspired by neuroscience and implemented by predictive coding. The authors show how to use this framework to solve different tasks, e.g., classification, generation, reconstruction and denoising. 
strengths_and_weaknesses: Strengths:
1. It is very interesting to leverage biological insights to design a new training framework.
2. The authors analyze how this framework is applied to solve different tasks, and the relationships between classical neural networks, like the feedforward network.
3. The paper is well-written and easy to follow.


Weaknesses:
1. The theoretical analysis of PC graphs is missing, such as time complexity, speed of convergence, and so on.
2. The experiments are limited in some datasets, or qualitative. The accuracy of MNIST is ~92%, which is much lower than simple CNNs.

questions: 1. From the accuracy of MNIST, the performance of PC graphs is still a issue, which needs to be improved. The authors should show the audience its potential ability to surpass artificial neural networks.
2. The training time of PC graphs, or convergence situation of different graphs, is not mentioned in the paper.
3. Could author use some quantitative evaluation metrics to show the performance of PC graphs in generation, reconstruction and denoising tasks?
limitations: not available
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents the use of predictive coding (PC) to perform various learning and inference tasks. In particular, the paper proposes PC graphs that generalize PC to arbitrary network structures. The flexibility of PC graphs is then demonstrated experimentally using a variety of network structures and learning tasks.
strengths_and_weaknesses: - Originality: While the idea of PC graphs in the paper is not new, this paper however focuses on showing how a single PC graph model can be extensively used in various ways to learn different tasks.

- Quality: The PC graph model is technically valid though calling it a biologically plausible model seems slightly misleading since the existence of error nodes is biologically questionable. The claim for the flexibility of the model is well supported by the experiments on various learning settings.

- Clarity: The paper is clearly written and well-organized.

- Significance: While the paper does not advance state-of-the-art results, the paper clearly presents the flexibility of PC graphs in various learning tasks through experiments. However, the overall practicality/utility of the model is still hard for readers to assess since 1) the potential computational overhead of the method is only briefly illustrated in the supplementary material, which is not convincing and 2) experimental results are mostly qualitative with some quantitative comparison against a few simple baselines. Additional knowledge of its computational efficiency and demonstrated effectiveness would instill more trust into the method.

questions: - Corruption noise variance values of 0.2 and 0.5 are quite small. How would a larger choice affect the analysis? Would the method still work in that case?
- P. 6, line 211: Missing SVHN dataset
limitations: The authors do not discuss the limitations and societal impact of their work.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper describes an approach to training arbitrary neural topologies based on predictive coding (PC). The nodes of the graph corresponding to the inputs are clamped to the observed values, and the remaining nodes are trained locally to minimise the discrepancy between their value and their prediction of the value itself (which is a weighted function of the values of its neighbors). The authors describe the general principle, and then perform a few experiments on MNIST and FashionMNIST showcasing how the model can be used for different applications depending on which nodes are initialised or updated.
strengths_and_weaknesses: Note: see further comments to the authors for the updated evaluation.

Let me premise that I am not familiar with the literature on PC and its applications to deep learning, and I have learned about most papers by following the references listed in the paper itself. The paper is well written, and the authors makes a good case for the strengths of the approach (locality) and in particular its neuroscience background. The experiments are interesting, and all the diagrams in the paper are very clear (e.g., how different masks over the connectivity can results in different known formulations).+

Experiments are done on very simple CV datasets with no comparisons. However, I think this is expected of papers that are trying to modify the way neural networks are trained at a fundamental level. Still, the authors are not even mentioning (or comparing) to the literature they are citing on PC, which is strange.
questions: Note: see further comments to the authors for the updated evaluation.

Upon reading several of the papers mentioned in the bibliography, I am unsure about the novelty which is claimed in the paper. In particular, the authors state that they propose an extension to “PC graphs” allowing PC to be run on arbitrary graph topologies, but this was already done in the majority of cited papers, including:

https://proceedings.neurips.cc/paper/2021/hash/1fb36c4ccf88f7e67ead155496f02338-Abstract.html

The above paper also provides examples about using the network as an associate memory, which encompasses half of the experiments done in this paper. The algorithm which is used is also equivalent (gradient descent for T steps on values and weights after fixing a single input example). Since this is my major concern, I am currently rating this as a borderline paper, as it appears to be a collection of previous known results.

A few additional questions I have:

•	The authors should be more comprehensive in describing the comparisons of their method with Boltzmann machines and Hopfield networks, since they share a number of common characteristics.
•	The convergence guarantees of the algorithms wrt the loss (sum of local errors squared) is not discussed.
•	In general, how efficient is the training compared to a standard backpropagation?
limitations: NA
review_title: null

Meta-review: metareview: This work presents the use of predictive coding (specifically PC graph) as a way to overcome some of the limitations of the commonly used backpropagation approach in deep learning. The initial reviews have raised some concerns regarding the paper, but these were sufficiently addressed in rebuttal to result in one reviewer recommending acceptance and others leaning towards acceptance. Taking this into account, and since I also think this line of work would contribute well to the audience of NeurIPS, I recommend acceptance and I would like to encourage the authors to take into account the reviewer comments, and the points discussed in their rebuttal, when preparing the camera ready version of the paper.
confidence: Certain
Decision: Accept