Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: A number of different algorithms have been proposed as more biologically plausible alternatives to backpropagation.  While different works have shown how several of these proposals approximate backprop by converging to it in some limit, the authors here outline an overall framework of energy-based models that encompasses several predictive coding proposals, equilibrium propagation, and contrastive Hebbian learning, showing how they all can be formulated as approximations to backprop.  By breaking the global energy into components corresponding to the internal energy of the network and the network's cost function, the authors show that different algorithms can approximate backprop by finding equilibrium points near the minimum of the internal energy (by, e.g., employing weak feedback).  They finally show that other algortihms can be derived from their approximation, with "PC-nudge" described as an example. 
strength_and_weaknesses: Strengths

1. The authors gather previously disparate theoretical results together under the same umbrella of energy-based models.

2. Several versions of predictive coding, along with equilibrium propagation and contrastive Hebbian learning are shown how, under the same framework, they can be derived as approximations to backprop.

3. A novel algorithm, "PC-nudge," is shown to be derivable from the proposed framework

Weaknesses

No major weaknesses of the manuscript were found


clarity,_quality,_novelty_and_reproducibility: The novelty of the present work stems from the amalgamation of several lines of work under one framework.  The manuscript is fairly clear and staightforward
summary_of_the_review: Several results are gathered and combined by showing how a number of proposed biologically plausible alternatives to backprop can be shown to approximate backprop when formulated as energy-based models. The work then leads to an example of a novel algorithm that can be derived using this common framework
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper unifies several approximation to backpropagation through the energy-based models view. 
strength_and_weaknesses: ### Strengths

The paper unifies several backprop approximation under the same energy-based framework.

This framework leads to different approximations.

### Weaknesses

I don’t understand what the take-home message is. Predictive coding and equilibrium propagation were already formulated in a similar fashion, but what can we do with it now? I'd guess deriving more plausible algorithms would be interesting, but it's not really discussed.

Contrastive Hebbian learning narrowly fits the proposed framework. Eq. 18 seems to be just the gradient of the loss (since $I$ is zero in that case), which is not what CHL is doing. 



clarity,_quality,_novelty_and_reproducibility: ### Clarity

Overall, the paper is well-written. Some minor comments:

Both figures (especially Fig. 1) should be exported in either a vector format or much higher resolution.

Fig.3B: PC-nudge is completely hidden behind backprop dynamics.

Sec. 2.3, first line: capitalize Hebbian.

Before Eq. 15: If we assume that the gradients of each component of the energy with respect to the weights -is- continuous and differentiable -> are.

Just before Sec. 3.4: capitalize Hopfield.

Eq. 23: the norms should not be squared in cosine similarity.

### Quality 

The technical contributions are somewhat limited as the paper mainly re-writes the three existing algorithms in a different form. The experimental contributions are very limited as the proposed algorithm, PC-nudge, is tested on a single network and only on MNIST. It's not clear if the reasonable alignment with backprop will hold for large networks/harder tasks. Also, what's the takeaway from this algorithm?

### Novelty

The connection between different algorithms is, to my knowledge, novel. All relevant literature is cited.

### Reproducibility

The code is not provided but it's implied that it will be. The authors can attach a zip file with the code to the submission.


**UPDATE**: the code is now provided.
summary_of_the_review: Potentially interesting connection between algorithms, but I'm not sure what we can do with it. I'm willing to discuss it further, however.

**UPDATE**: updated the score from 5 to 6 post-rebuttal.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper starts from a PCN and expands around the limit where the output is not tethered to the target. Perturbing in the parameter $\lambda$, which may be interpreted as the precision parameter deciding how tightly would the target be tied to the output, builds a useful bridge to many other approaches to training neural networks.
strength_and_weaknesses: Strengths:
Analysis of the small $\lambda$ limit
The PC-nudge model
Discussion of contrastive Hebbian learning, starting from PC.

Minor issues:
In equations (3) and (4) the indexing conventions of $W_l$ and $x_l$ seems to be inconsistent with the definition of $\epsilon_l$ in the previous page.
Under equation(5), the sentence mentioning “..neurons which are co-active at the clamped phase are strengthened…” is confusing. What does 'neurons being strengthened' mean? What is really strengthened? Synaptic strengths? That seems to have the opposite effect. A clearer explanation here would help.

clarity,_quality,_novelty_and_reproducibility: Overall, this is a well-written paper, needing some small modifications.
summary_of_the_review:  I found this paper useful for relating predictive coding networks with other competing approaches. It is a welcome addition to the literature.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Three types of biologically-plausible learning algorithms --- predictive coding, contrastive hebbian learning, and equilibrium propagation --- are reformulated as energy based models. This framework is used to provide a unifying description for when these algorithms closely approximate gradient descent (backprop), and the authors use this new perspective to suggest a new variant of predictive coding called PC-Nudge. Experiments with a multi-layer neural network on MNIST are used to support the proposed ideas. 
strength_and_weaknesses: Strengths:
- The paper is well-written and clear.
- The new energy-based model perspective is a way to compare and contrast three different non-backprop  learning algorithms, and understand how they deviate from backprop (gradient descent on the supervised loss). 

Weaknesses:
- It is not obvious to me that viewing these algorithms as gradient descent on an energy function is useful.
- One justification given for proposing this new framework is that it can be used to propose new algorithms.  PC-Nudge is proposed. However, this algorithm seems to be a minor variation on existing algorithms, and the experimental results on MNIST are not particularly interesting. 
- Performing gradient descent on an energy function is not necessarily biologically plausible, and this paper does not have much to say about the biological plausibility of these various algorithms. 
clarity,_quality,_novelty_and_reproducibility: The paper is clear, high-quality, and original. 
summary_of_the_review: This is a well-written paper that might be of interest to some in the community, but despite my familiarity with the algorithms discussed, I was unable to see how it is particularly useful to view them from the energy-based model perspective.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a theoretical framework which unifies all the disparate results in the literature concerning the conditions under which various ‘biologically plausible’ algorithms approximate the BP of error algorithm. The authors demonstrate that these convergence results arise from a shared and fundamental mathematical property of EBMs in the infinitesimal inference limit. The paper validates the proposed framework by deriving a new PC Nudge algorithm and testing it on the MNIST dataset. After the rebuttal, all 4 reviewers lean to accept the paper (particularly, 2 of them champion the paper, with each rating a score 8: accept) because the paper is interesting, novel, and makes a valuable contribution for relating predictive coding, equilibrium propagation and contrastive Hebbian learning. The rebuttal has addressed all the major concerns raised by reviewers. The AC agrees with the judgements of the reviewers and recommends accepting the paper.   

justification_for_why_not_higher_score: Even though the current version of the paper is good for publication, but it is not perfect and the current experiments for justification can be further improved and strengthened. 
justification_for_why_not_lower_score: The paper provides a novel theoretical framework that relates predictive coding, equilibrium propagation and contrastive Hebbian learning. All reviewers lean to accept it.
Decision: Accept: poster