Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper describes a variant of predictive coding, named incremental predictive coding (iPC), based on incremental EM, which it is argued should be considered a biologically plausible approach to learning in the brain.  The complexity of iPC is considered in relation to back-propagation (BP), and a CPU implementation is provided.  Further, the generalization performance is investigated on a number of datasets, and the algorithm is shown to perform well in comparison to BP and PC.
strength_and_weaknesses: Strengths:

-  The biological plausibility argument is interesting, and in general the argument is convincing that some form of 'localized EM' algorithm is more plausible than BP or PC alternatives, while retaining convergence and generalization properties.
-  The experimentation convincingly demonstrates that iPC should be considered a viable alternative to BP generally, at least for simple architectures and specialized hardware.

Weaknesses:

- I'm mainly concerned about the paper's novelty - essentially, iPC is equivalent to iEM applied to a hierarchical Gaussian model.  The theoretical properties are described elsewhere (e.g. Karimi 2019) and the biological plausibility argument is hard to evaluate, although likely to be worth pursuing further.
- There is little theoretical novelty, since the time-complexity analysis (Theorem 1) essentially follows simply by definition.  As discussed by the authors, the comparison of training-loss convergence rates in terms of 'non-parallel matrix multiplications' is an interesting result, but this is investigated solely empirically (Fig. 2 right).
clarity,_quality,_novelty_and_reproducibility: The clarity, quality and reproducibility are mainly good (I spotted a few typos - for instance, in Eq. 4, the conditional in the second expression should read 'p(x^(l-1) | x^(l))', and the Gaussian formulation in the third expression should include the prior and the x's).  As noted above, the novelty is an issue for me.
summary_of_the_review: An interesting investigation of an algorithm that may have relevance in neuroscience, and deserves further attention.  Potentially, the paper may be of interest to those working in neuroscience and optimization.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an incremental predictive coding method that performs faster on larger models than BP.
strength_and_weaknesses: I do like the idea and the demonstrated performance of the proposed method. That being said, there are a few weaknesses I wish that the author could address:
* 3.2 on CPU implementation, I do not fully understand what is the overhead for iPC. Besides, the time complexity is related to L, then why does the improvement over BP also scales with hidden dimension (is this the amount of neurons in hidden layers)?
* For the experimental results reported in table 1, is the PC trained with the same amount of epochs as iPC, so that it behaves worse on AlexNet as it is not yet converges? The comparison with PC states that the iPC converges faster, but my understanding is that he iPC is a more efficient way to approximate PC, I do not understand why it failed to scale with model complexity. Besides, is it possible to test on some more complex model or image data set to back the idea that iPC preforms better on complex image related tasks?


clarity,_quality,_novelty_and_reproducibility: This work is of good quality. 
summary_of_the_review: Overall, this is an interesting work. Some clarification on test results would make it more convincing.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper points out that the predictive coding implementation from Rao and Ballard is essentially expectation maximization, and therefore can be sped up by using the incremental (aka partial) variant of EM instead of the full version. The paper also argues that this is more efficient than backpropagation, at least for large networks and full-batch learning, and has better uncertainty calibration than models trained with backprop. 

strength_and_weaknesses: I have some issues with the claims and descriptions made in this paper. I want to acknowledge early that some of my issues may be misunderstandings, because I am still learning about predictive coding here. 

The high-level issue for me is: the background seems to be "predictive coding vs backpropagation", where predictive coding is essentially equivalent to expectation-maximization, but this is not the way I think about predictive coding. This sentence matches my understanding better: "PC is based on the assumption that brains implement an internal generative model of the world, needed to predict incoming stimuli (or data)." Expectation-maximization is one way to optimize a predictive coding model, and backpropagation is another, and there are more still, like Hebbian learning -- these are all optimization techniques, and choices here are orthogonal to the issue of predictive inference vs non-predictive inference. 

clarity,_quality,_novelty_and_reproducibility: > [main limitation of predictive coding being] "lack of efficiency"

I'm not sure about this. I think it's more like: predictive coding models are harder to train and do not work as well as feedforward alternatives. 

> "can match backpropagation in its most important property: generalization capabilities"

I'm not sure about this. I think the the common view is that the most important property of backpropagation is that it can manage credit assignment across long chains of modules/neurons, and enable learning of large nonlinear systems.

Notation-wise, it seems like almost every variable in the paper has a bar over it. Maybe all bars can be removed, to make things simpler. 

In Eq. 1, ybar is defined as "the generated vector", but also p(ybar, \thetabar) is described as "likelihood of the data given the causes", suggesting ybar is actually the data. Can this be clarified please?

Some citation issues -- parenthetical and non-parenthetical citations are mixed up, making  it hard to read. (e.g., "intractable Friston (2003)", and "Following (Rao ...),".

optimizingthe -> optimizing the
gradient descend -> gradient descent
 – the slow inference phase –there  -> ---the slow inference phase---there 

> "Note that this increased speed does not harm the final performance, as the iEM algorithm has been proven to converge to a stationary point of the loss function, the same as EM or backpropagation" 

This is not very convincing. I think what we really want to know is whether or not this works in practice. I think even incremental EM (without the predictive coding interpretation) is typically less stable than EM. 

> "This algorithm is able to obtain performance competitive to BP on image classification
tasks."

This is a problematic claim. There are many algorithmic components involved in applying BP on image classification tasks. I don't think such a general claim can be made accurately. 

> "we provide enough evidence that the speed up against BP in full batch training is theoretically possible using iPC"

I'm not sure -- what makes it "enough"? 

> "the concept of inference and iterations are not present in BP"

I'm not sure what is meant by this. These concepts are certainly present in BP-based models. Why not simply count the number of gradient steps?

> "we first prove this formally using the number of non-parallel matrix multiplications needed to perform a weight update as a metric"

I don't know what to do with this "proof". Is this useful? What exactly is meant by "full batch" anyway? Does this mean putting the whole dataset into memory? To me this seems totally impractical. 

> "This is still below the result of Theorem 1 due to the large overhead introduced in our implementation."

OK but is there any feasible way forward on this? To me it seems more like the theorem is not very useful. Also, I would like to know: What are the actual runtimes here? I understand the ratio is somewhat helpful for a comparison, but it obscures the actual values in play here. 

The descriptions in "setup of experiments" and "change of width" were a bit confusing. Why is everything sequential ("first we trained this, then we trained that")? Why does the order matter? Are you training the model, then adding more parameters, then continuing training?  (I expect not...)

Instead of Theorem 1 and the proof and so on, why not simply count the actual number of matrix multiplications? 

I am not sure about the comparison between BP and iPC in the experiments. Is the network architecture exactly the same? Normally with BP we have a feedforward architecture, without a generative interpretation. 

Sometimes the term "Z-IL" appears but I think it was never defined. It appears more in the discussion. What is this? 
summary_of_the_review: I enjoyed reading the paper and I think the topic is important, and a great fit for ICLR. I pointed out a variety of claims that appear questionable to me, which maybe stem from thinking about predictive coding as a model formulation (as I do, which I think is standard) vs. thinking about predictive coding as an optimization technique (as the paper does).
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper is basically an implementation of the prospective configuration algoithm (Yuhang Song, et al, Inferring neural activity before plasticity: A foundation for learning beyond backpropagation) where each layer of a heirarchical Gaussian generative model (with the further assumption that the covariance matrix of the gaussians is identity), is updated in parallel. Convergence guarantees to some local optima come from Neal and Hinton's incremental EM paper. 
strength_and_weaknesses: Strength: Empirical results show faster convergence
Weekness: There is nothing novel in this paper. Neither does this paper claim to introduce the prospective configuration algo (I am guessing it is done in another submission), nor does it need to show convergence (which follows from Neal and Hinton). It is therefore just an implementation paper. Had the experiments been large and comprehensive, there would be reason to consider this paper.
clarity,_quality,_novelty_and_reproducibility: The paper is very clearly written. But it lacks novelty
summary_of_the_review: The paper has very limited novelty. It is just an implementation of ideas found in other papers.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper points out that a learning algorithm called "predictive coding" in neuroscience is essentially maximum likelihood estimation in a basic latent variable model. As is well known, ML estimation in such models can conveniently be done with the EM algorithm, even though sometimes direct gradient-based learning can be faster. The authors then suggest to apply Neal and Hinton's classical incremental EM, and point out some advantages of that, in particular scalability to large data.

Neal and Hinton's incremental EM is well known in the probabilistic ML field, where ICLR is situated, even though this may not be the case for certain neuroscience circles, where maybe the proposed work would be better placed. For ICLR readers, there is simply nothing much new here. Looking at the call for papers, while ICLR encourages submission of "neuroscience applications", debates about "biological plausibility" and such are probably better placed elsewhere.

I'd also like to state that the interaction with the authors has been somewhat painful, and would strongly recommend to them to change their tone when interacting with AC and reviewers who may have different opinions to them. Use of capital letter words and strong claims are not appreciated, and probably less than helpful as a response to feedback.
justification_for_why_not_higher_score: See reviews. This paper may contain novelties in some neuroscience circles, but not enough for ICLR.
Decision: Reject