Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm
Authors: Eli Zachary Sennesh
Hao Wu
Tommaso Salvatori
Venue: NeurIPS 2024 poster (2024)
Keywords: bioplausible
predictive coding
neuroscience
variational Bayes
VAE
deep generative model
probabilistic graphical models
Primary area: probabilistic_methods
TL;DR: We design a nested Sequential Monte Carlo algorithm based on predictive coding in neuroscience, and demonstrate both its biological plausibility and its performance in deep generative modeling.

Abstract: Unexpected stimuli induce "error" or "surprise" signals in the brain. The theory of predictive coding promises to explain these observations in terms of Bayesian inference by suggesting that the cortex implements variational inference in a probabilistic graphical model. However, when applied to machine learning tasks, this family of algorithms has yet to perform on par with other variational approaches in high-dimensional, structured inference problems. To address this, we introduce a novel predictive coding algorithm for structured generative models, that we call divide-and-conquer predictive coding (DCPC); it differs from other formulations of predictive coding, as it respects the correlation structure of the generative model and provably performs maximum-likelihood updates of model parameters, all without sacrificing biological plausibility. Empirically, DCPC achieves better numerical performance than competing algorithms and provides accurate inference in a number of problems not previously addressed with predictive coding. We provide an open implementation of DCPC in Pyro on Github.

Divide-and-Conquer Predictive Coding: a Structured Bayesian Inference Algorithm
Eli Semnesh' Hao Wu', Tommaso Salvatori
Department of Psychology, Vanderbilt University, Nashville, TN, USA 2VERSES AI Research Lab, Los Angeles, USA
Vienna University of Technology. Vienna, Austria
. www.esh@vanderbblt.... yuhaomxhy@gmail com, tommaso salvatori@verses. .ai
Abstract
Unexpected stimuli induce "error" "surprise" signals in the brain. The theory of predictive coding promises to explain these observations in terms of Bayesian inference by suggesting that the cortex implements variational inference in a probabilistic graphical model. However, when applied to machine learning tasks, this family of algorithms has yet to perform on par with other variational approaches in high-dimensional, structured inference problems. To address this, we introduce novel predictive coding algorithm for structured generative models, that we call divide-and-come conquer predictive coding (DCPC); it differs from other formulations of predictive coding, as respects the correlation structure of the generative model and provably performs maximum-likelihood updates of model parameters, all without sacrificing biological plausibility. Empirically, DCPC achieves better numerical performance than competing algorithms and provides accurate inference in a number of problems not previously addressed with predictive coding. We provide an open implementation of DCPC in Pyro on Github.
1 Introduction
In recent decades, the fields of cognitive science, machine learning, and theoretical neuroscience have borne witness to flowering of successes in modeling intelligent behavior via statistical learning. Each of these fields has taken a different approach: cognitive science has studied probabilistic inverse inference [Chater al. 2006 Pouget al. 2013 Lake al., 2017] in models of each task and environment, machine learning has employed the backpropagation of errors [Rumelhart et al.] 1986]
Lecun al. 2015 Schmidhuber 2013), and neuroscience has hypothesized that predictive coding
(PC) Srinivasan el al. 1982 Rao and Ballard 1999 Friston| 20055 Bastos et al. [0122] Spratling
2017 Hutchinson and Barrett 2019 Millidge et 2021 may explain neural activity in perceptual
tasks. These approaches share in common a commitment to "deep" models, in which task processing
emerges from the composition of elementary units.
In machine learning, PC-based algorithms have recently gained popularity for their theoretical potential to provide more biologically plausible alternative backpropagation for training neural networks (Salvatori et al.) 2023 Song et al., 2024]. However, PC does not perform comparably in these tasks to backpropagation due limitations in current formulations. First, predictive coding for gradient calculation typically models every node in the computation graph with a Gaussian, and hence fails to express many common generative models. Recent work PC has addressed this by allowing approximating non non-Gaussian energy functions with samples [Pinchetti et al., 2022] (MAP)
Second, the Laplace approximation the posterior infers only maximum-a-posterre estimate and Gaussian covariance for each latent variable, keeping PC from capturing multimodal or correlated distributions. Third, this loose approximation the posterior distribution results in inaccurate, high-variance updates to the parameters of the generative model.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Predicioo Erro
Prediction Error
6
6
Prediction
Prediction
Figure I: Left: Classical PC learns a mean-field approximate posterior with prediction error layers. Right: Divide-and-conquer PC approximates the joint posterior with bottom-up and recurrent error.. Where classical predictive coding has layers communicate through shared error units, divide-and- conquer predictive coding separates recurrent from "bottom-up" error pathways to target complete conditional distributions rather than posterior marginal distributions.
In this work we propose new algorithm, divide-and-conquer and-conquer predictive coding (DCPC), for approxi- mating structured target distributions with populations of Monte Carlo samples. DCPC goes beyond Gaussian assumptions, and decomposes the problem of sampling from structured targets into local coordinate updates to individual random variables. These local updates are informed by unadjusted Langevin proposals parameterized in terms of biologically plausible prediction errors. Nesting the local updates within sonddeand-congee
Sequential Monte Carlo (Lindsten et al., 2017 Kuntz et al. 2024] ensures that DCPC can target any statically structured graphical model, while Theorem provides a locally factorized way to learn model parameters by maximum marginal likelihood.
DCPC also provides computational perspective on the canonical cortical microcircuit [Bastos et al. 2012 2020 Campagnola et al., [2022] hypothesis neuroscience. Experiments have suggested that deep laminar layers in the cortical microcircuit represent sensory imagery, while superficial laminar represent raw stimulus information [Bergmann et al.J 2024]; experiments
predictive coding paradigm specifically suggested that the deep layers represent" "predictions" while the shallow layers represent "prediction errors". This circuitry could provide the brain with its fast, scalable, generic Bayesian inference capabilities. Figure compares the computational structure of DCPC with that of previous PC models. The following sections detail the contributions of this work:
Section defines the sonide-and-oonqque predictive coding algorithm and shows how to use it as variational inference algorithm:
Section examines under what assumptions the cortex could plausibly implement DCPC, proving two theorems that contribute to biological plausibility:
demonstrates DCPC experimentally in head-to-heatd comparisons against recent generative models and inference algorithms from the predictive coding literature.
Section 5
Section 2 will review the background for Section 33 algorithm: the problem predictive coding aims to solve and line of recent work adressing that problem from which this paper draws.
2 Background
This section reviews the background necessary to construct the divide-and conquer predictive coding algorithm in Section 3 Let assume we have directed, acyclic graphical model with joint density split into observations x E x and latents E z, parameterized by some at each conditional density
po po z) = II (x Pa(x)) Il po(2 Pa(z)),
2Ex
where Pa(z) denotes the parents of the random variable = and Ch(z) denotes its children.
Empirical Bayes Empirical Bayes consists of jointly estimating, in light of the data, both the parameters 8* and the Bayesian posterior over the latent variables z, that is:
Po (z x) = Pg- (x, z)
0" = arg max po (x) = arg max
Po (x 2) dz,
8
ZE
Pa (x)

---

Typically the marginal and posterior densities have no closed form, so learning and inference algorithms treat the joint distribution as a closed-form unnormalized density over the latent variables; its integral then gives the normalizing constant for approximation
TT(Z) = (0((z)
(0(2) = (x, z):
Ze : Z
Ta(z) dz - (x):
Za
Neal and Hinton
1998 reduced empirical Bayes to minimization of the variational free energy:
No (z) g(z) 2 log Z()).
F(0. q) (x)
log
The ratio of densities in Equation is an example of weight used approximate distribution known only up to its normalizing constant. The proposal distribution q(z) admits tractable sampling, while the unnormalized targer density (z(z) admits tractable, pointwise density evaluation.
Predictive Coding Computational neuroscientists now often hypothesize that predictive coding (PC) can optimize the above family of objective functionals in local, neuronally plausible way [Millidge et al.) 2023] More in detail, it is possible to define this class of algorithms as follows:
Definition 1 (Predictive Coding Algorithm). Consider approximate inference model po (x(x z) using an algorithm A. Salvatori et al. (2023) calls A a predictive coding algorithm if and only if:
1. It maximizes the model evidence (og)o(x) by minimizing a variational free energy;
2. The proposal q(z) =I==e factorizes via a mean-field approximation; and
3. Each proposal factor is Laplace approximation (i.e. qu(z) == (Nu, E(n)))
Particle Algorithms In contrast predictive coding, particle algorithms approach empirical Bayes problems by setting the proposal to a collection of weighted particles (w* z²²) drawn from sampling algorithm meeting certain conditions (see Definition in Appendix B Any proposal meeting these conditions (see Proposition inn Appendix B and Naesseth et al. [2015], [Stites et al. [2021) defines a
free energy functional, analogous Equation in upper-bounding the model surprisal:
F(0,9) == E (w.2-)((;22) [- log w] = F(0, - llg 7(0).
This paper builds on the particle gradient descent (PGD) algorithm of Kuntz et al. 2023). that works as follows: At each iteration f, PGD diffuses the particle cloud gK (z) = R Σk=1 6zt (z) across the target log-density with a learning rate 77 and independent Gaussian noise: then updates the parameters / by ascending the gradient of the bogliike,hood, estimated by averaging over the particles.
The update rules are then the following:
2 ++1A = 2h + V log Ya: (z/th) + /2ngs
(3)
K
0 = 0¹ +
Va log 7 (z (2'))
(4)
K
The above equations target the joint density of entire graphical mode When the prior Po (z) factorizes into many separate conditional densities, achieving high inference performance often requires factorizing the inference network or algorithm into conditionals as well [Webb et al., 22018) Estimating the gradient of the entire log-joint, as in PGD and amortized inference Dasguupta et al. 2020 [Ptters et al.] 20244, also requires nonlocal backpropagation. To provide a generic inference algorithm for high-dime structured models using only local computations, Section 3 will apply Equation to sample individual random variables in a joint density, combine the coordinate updates via sequential Monte Carlo, and locally estimate gradients for model parameters via Equation 4
3
Predictive Coding
The previous section provided mathematical toolbox for constructing Monte Carlo algorithms based on gradient updates and a working definition of predictive coding. This section will combine those
Kuntz et al. 2023 also interpreted Equation 3 an update step along the Wasserstein gradient in the space of probability measures. Appendix extends this perspective to predictive coding of discrete random variables.

---

PC
LPC
MCPC
DCPC (ours)
Generative density
Gaussian
Differentiable
Gaussian
Differentiable
Inference approximation
Laplace
Gaussian
Empirical
Empirical
Posterior conditional structure
X
X
X
Table I: Comparison of divide-and-conquee predictive coding (DCPC) against other predictive coding
algorithms. DCPC provides the greatest flexibility: arbitrary differentiable generative models, an empirical approximation to the posterior, and sampling according to the target's 's conditional structure.
tools to generalize the above notion predictive coding, yielding the novel divide-and-conquer predictive coding (DCPC) algorithm. Given causal graphical model, DCPC will approximate the posterior with a population (z) of K samples, while also learning explaining the data. This will require deriving local coordinate updates and then parameterizing them terms of prediction enrors.
Let us assume we again have causal graphical model po(x, z) locally parameterized by and factorized (as in Equation T into conditional densities for each x E x and 2 E z. DCPC then requires two hyperparameters: a learning rate 1) E R+ and particle count
N+ and is initialized (at t 0) via population of predictions by ancestor sampling defined as z ~ II.e. po(2" Pa(2))).
DCPC aims to minimize the variational free energy (Equation 2). The optimal proposal q- for each random variable would equal, if it had closed form, the complete conditional density for that variable, containing all information from other random variables
qu (z' 2 1) (2(2; po(= Pa(=)) II véCh(=)
po(u Pa(u)).
We observe that the prediction errors S2 in classical predictive coding, usually defined as the precision weighted difference between predicted and actual value of variable, can be seen as the score function of Gaussian, where the score is the gradient with respect to the parameter z of the log-likelihood:
= V. log.N = (x 2);
When given the ground-truth parameter z, the expected score function Ex=p(x]ss V p(x =)] under the likelihood becomes zero, making score functions a good candidate for implementing predictive coding. We therefore define Eg in DCPC as the complete conditional's score function
Es = V. loggog (2; 2. = V log ( Pa(=)) + Σ
V Pog PPa())).
(6)
eECh(*)
This gradient consists of a sum of local prediction-error terms: one for the local "prior" on = and one for each local "likelihood" of a child variable. By defining the prediction error as a sum of local score functions, we write Equation 3 in terms s, (Equation 6) and the preconditioner Definition
(= 166 (/=))=) : (2' + + 211)
The resulting proposal now targets the complete conditional density (Equation 5), simultancously meeting the informal requirement of Definition I for purely local proposal computations while also "dividing and conquering" the sampling problem into lower-dimensional coordinate updates.
Since the proposal from which we sample by predictive coding is not the optimal coordinate update, we importance weight for the true complete conditional distribution that is optimal
7 (2t 2
(n(2 (==1,,t)) e
4
resampling with respect to these weights corrects for discretization error, yields particles distributed according to the true complete conditional, and estimates the complete conditional's 's normalizer
K
1
RESAMPLE (sl,ut) - (2 (x)))
K
The recursive step of "Divide and Conquer" Sequential Monte Carlo Lindsten al., 2017
Kuntz
al. 2024 exploits the estimates Z (2 ) to weigh the samples for the complete target density
Pot 1 (x,2²'
w
II 1
(8)
Il 70 (2:: ZEE

---

Algorithm Divide-and-Conquerr Predictive Coding for empirical Bayes
Require: learning rate 7 E Rt, particle count K e N. number of sweeps S N Require: initial particle vector z", initial parameters 80, observations x E N'
1: for E [1...77 ... T] do
Looop through predictive coding steps D. Loop through Gibbs sweeps over graphical model Loop through latent variables in graphical model
2: for 8 E [1..S] do
3:
for E z do
4:
62 +
Va Po (2 Pa(z))
> Local prediction error Children's prediction errors
5:
6, 1 E LvECh(2)
Vx Ppr 1(v) Pa(u))
Σx t x
6:
> Estimate precision of prediction errors
(e
7:
2' qn(2" |E (2212-1) 1
Sample coordinate update
u Tp' 1())
8:
Correct coordinate update by weighing
9:
2¹ RESAMPLE (2 ut)
Resample from true coordinate update
10:
2 - (Z)=)² + K Ek= wit
< Estimate coordinate update's 's normalizer
Pgt (xxa))))
11:
F t- K Ex log
1I
> Update free energy
ITe T (1))
0 t 0t x Ek
Voc Pg Pat
(x, zt.hk
Update parameters
13:
return x., 0T, F
b Output: updated particles, parameters, free energy
By Proposition
1. log-rransforming these weights estimates the free energy (Equation
Pgr (x, z')
F (z oC-1) = (xt1ot))) log
~ Ey | log wy. .]
q. (z' z 11
Theorem 3 in Appendix B shows that the gradient V p Eg Vo log Pa- (x, z²)] of the above estimator equals the expected gradient of the log-joint distribution. Descending this gradient 0¹ = 0r V 1 enables DCPC to learn model parameters 0.
The above steps describe a single pass of srride-and-conquee predictive coding over a causal graphical model. Algorithm[] (sswos the complete algorithm, consisting of nested iterations over latent variables z E z (inner loop) and iterations E T (outer loop). DCPC satisfies criteria (1) and (2) of Definition [ and relaxes criterion (3) to allow gradient-based proposals beyond the Laplace assumption. As with Pinchetti et al. [2022] and Oliviers et al. (2024), relaxing the Laplace assumption enables much greater flexibility in approximating the model's true posterior distribution.
4 Biological plausibility
Different works in the literature consider different criteria for biological plausibility. This paper follows the non-spiking predictive coding literature and considers an algorithm biologically plausible it performs only spatially local computations in a probabilistic graphical model [Whittington and Bogacz, 2017). 2017), without requiring global control of computation. However, while in the standard literature locality is either directly defined in the objective function [Rao and Ballard. 1999], or derived from a mean-field approximation to the joint density [Friston] 2005]. showing that the updates of the parameters of DCPC require only local information is not as trivial. To this end, in this section we first formally show that DCPC achieves decentralized inference of latent variables (Theorem) and then that also the parameters are updated via local information (Theorem 2).
Gibbs sampling provides the most widely-used algorithm for sampling from high- high-dimensional
probability distribution by local signaling. consists of successively sampling coordinate updates to individual nodes in the graphical model by targeting their complete conditional densities To(2 [2 |
x, 2.2). Theorem demonstrates that DCPC's coordinate updates approximate Gibbs sampling.
Theorem (DCPC coordinate updates sample from the true complete conditionals). Each DCPC coordinate update (Equation 7 for a latent E z samples from z complete conditional (the normalization of Equation 5.. Formally, for every measurable h Z I. resampled expectations with respect to the DCPC coordinate update equal those with respect to the complete conditional
(2)eg1 ( [hx)]]] = {
h(z) Te(2 zxz)) dz.
ZEZ
5

---

Proof. See Corollary 4.1 in Appendix B
We follow the canonical cortical microcircuit hypothesis of predictive coding [Bastos al. 2012
Gillon et al. 2023 or predictive routing [Bastos et al. 2020]. Consider a cortical column representing
2 E z; the B, a B. and ry frequency bands neuronal oscillations [Buzsáki and Draguhn 2004 could synchronize parallelizations (known to exist for simple Gibbs sampling in causal graphical model [Gonzalez et al. 201T)) of the loops in Algorithm [T] From the innermost to the outermost and following the neurophysiological findings of Bastos al.
2015 Fries [013], ny-band oscillations could synchronize the bottom-up conveyance of prediction errors (lines 4-6) from L2/3 of lower
cortical columns to L4 of higher columns, 3-band oscillations could synchronize the top-down conveyance of fresh predictions (implied in passing from 8 to 8 + in the loop of lines 2-9) from LS/6 of higher columns L1+L6 of lower columns, and O-band oscillations could synchronize complete attention-directed sampling of stimulus representations (lines 1-11). Figure5jin Appendix visualizes these hypotheses for how neuronal areas and connections could implement DCPC.
Biological neurons often spike to represent changes in their membrane voltage [Mainen and Sejnowski 1995 Lundstrom et al. 2008 Forkosh 2022]. and some have even been tested and found signal the temporal derivative of the logarithm of an underlying signal [Adler and Alon 2018 Borba et al. 2021). Theorists have also proposed models [Chavlis and Poirazi, 2021, Moldwin al.) 2021 under which single neurons could calculate gradients internally. In short, if neurons can represent probability densities, many theoretical proposals and experiments suggest they can, then they can will demonstrate that given the
likely also calculate the prediction errors used in DCPC. Theorem 2 "factorization" above, DCPC's model learning requires only local prediction errors.
Theorem 2 (DCPC parameter learning requires only local gradients in a factorized generative model). Consider graphical model factorized according Equation L with the additional assumption that the model parameters 0 E @ = IIrex O₂ x II θ₂ factorize disjointly. Then the gradient V F(0 q) of DCPC's free energy similarly factorizes into a sum of local particle averages
VoF = Eg V logpo(x.
Σ VE (x
Ve. log (1 Paa())*).
(9)
K
k=1
Proof. See Proposition 5 in Appendix
Our practical implementation of DCPC, evaluated the experiments above, takes advantage of Theorem 2 to save memory by detaching samples from the automatic differentiation graph in the forward ancestor- saapslng pass through the generative model.
Finally, DCPC passes from local coordinate updates to the joint target density via an importance resampling operation, requiring that implementations synchronously transmit numerical densities or log-densities for the freshly proposed particle population. While phase-locking oscillation may make this biologically feasible, resampling then requires normalizing the weights. Thankfully, divisive normalization appears ubiquitously throughout the brain [Carandini and Heeger 2012]. as well as just the type of "winner-take-all" circuit that implements a softmax function (e.g. for normalizing and resampling importance weights) being ubiquitous in crosstalk between superficial and deep layers of the cortical column [Liu] 1999 Douglas and Martin 2004].
cortical
Experiments
Divide-and-conquer predictive coding is not the first predictive coding algorithm to incorporate sampling into the inference process, and certainly not the first variational inference algorithm for structured graphical models. This section therefore evaluates DCPC's performance against both models from the predictive coding literature and against standard deep generative model. Each experiment holds the generative model, dataset, and hyperparameters constant except where noted.
We have implemented DCPC as variational proposal or "guide" program in the deep probabilistic programming language Pyro [Bingham et al., 2019]; doing so enables us to compute free energy and prediction errors efficiently in graphical models involving neural networks. Since the experiments below involve minibatched subsampling of observations unknown distribution, replace Equation g with subsampled form (see Welling and Tch [001
from a dataset ~ p(D) of

---

Inference algorithm
Dataset
NLL
Mean Squared Error +
MCPC
MNIST
144.6
(8.29 = 0.05) x 10
DCPC
MNIST
102.5 0.01
0.01 7.2 x 10
DCPC
EMNIST
160.8 0.05
3.3 x 10 3.5 10-9
DCPC
Fashion MNIST
284.1 + 0.05
0.03 + 2.7 10-5
Table 2: Negative likeliiol and mean squared error for MCPC against DCPC on held-out images
from the MNISTs. Means and standard deviations are taken across five random seeds.
for derivation) of the variational Sequential Monte Carlo gradient estimator [Naesseth et al. 2018
K
1 K
V ooF [D] (D
R (2,,0) my XBEB
log
W
(10)
B
We optimized the free energy in all experiments using Adam (Kingma and Ba 2014], making
sure to call detach( () after every Pyro sample(, operation to implement the purely local gradient calculations of Theorem and Equation [0] The first experiment below considers hierarchical Gaussian model on three simple datasets. The model consists of two latent codes above an observation.
Deep latent Gaussian models with predictive coding Oliviers al. 2024 brought together predictive coding with neural sampling hypotheses in a single model: Monte Carlo predictive coding (MCPC). Their inference algorithm functionally backpropagated the score function of log-likelihood, applying Langevin proposals to sample latent variables from the posterior joint density along the way. They evaluated MCPC's performance on MNIST with deep latent Gaussian model [Rezende et al.) 2014) (DLGM), Their model's, conditional densities consisted of nonlinearities followed by linear transformations parameterize the of each Gaussian conditional, with learned covariances. shows that the DLGM structure already requires DCPC to respect hierarchical dependencies.
Figure 2
We tested DCPC's performance elementary reconstruction and generation tasks by using to train this exact generative model, changing only the like- lihood from discrete Bernoulli to continuous Bernoulli Loaiza-Ganen and Cunningham 2019]. After training we evaluated with a discrete Bernoulli likelihood. Table shows that in terms of both surprise (negative log evidence, with the discrete Bernoulli likelihood) and mean squared reconstruction error, DCPC enjoys better average performance with lower standard deviation of performance, the latter by an order of magnitude. All experiments used a learning rate 77 0.1 and K particles.
1
22
Figure shows an extension of this experiment to EMNIST (Cohen et al. 2017) and Fashion MNIST [Xiao et al. 2017] well the original MNIST. with ground-truth images in the top row and their reconstructions from DCPC- inferred latent codes below. The ground-truth images
from 10% validation split of each data-set, on which DCPC only infers particles (z),(z).
Figure 2: Hierarchi- cal graphical model for DLGM's.
The above datasets do not typically challenge a new inference algorithm. The next experiment will thus attempt to learn representations color images, as in the widely-used variational autoencoder (Kingma and Welling 2013 framework, without an encoder network or amortized inference.
Image generation with representation learning
Zahid et al. 2024 have also recently designed and evaluated Langevin predictive coding (LPC), with differences from both MCPC and DCPC.
9
3
4
2
1
N
9
3
4
(a) MNIST
(b) EMNIST
(c) Fashion MNIST
Figure 3: Top: images from validation sets of MNIST (left), EMNIST (middle), and Fashion MNIST (right). Bottom: reconstructions by deep latent Gaussian models trained with DCPC for MNIST (left), EMNIST (middle), and Fashion MNIST (right), averaging over K = 4 particles. DCPC achieves quality reconstructions by inference over without training an inference network.

---

Algorithm
Likelihood
Resolution f
Sx Epochs 4.
FID 4
PGD
N
32 x 32
I I00
100 2.7
DCPC (ours)
N
32 32
100
82.7 0.9
LPC
DN
64 x 64
300 15 = 4500
120 (approximate)
VAE
DN
64 x 64
x 4500 = 4500
86.3 + 0.03
DCPC (ours)
DN
64 x 64
30 x 150 = 4500
79.0 + 0.9
Table 3: FID score comparisons on the CelebA dataset [Luu et al. 2015].
The score for LPC comes
from Figure 2 in Zahid et al. [2224) where they ablated warm-starts and initialized from the prior.
(a) Reconstructions of the CelebA validation set (b) Samples drawn de novo from the posterior pre- by generator network trained with DCPC. dictive distribution of the trained network.
Figure 4: Left: reconstructions from the CelebA validation set. Right: samples from the generative model. DCPC achieves quality reconstructions by inference over z with K 16 particles and no inference network, while the learned generative model captures variation in the data.
While MCPC sends prediction errors up through hierarchical model, LPC computed as its prediction error the log-joint gradient for all latent variables in the generative model. This meant that biological plausibility, and their goal of amortizing predictive coding inference, restricted them single-level decoder adapted from Higgins et al. [2017]. We evaluated with their discretized Gaussian likelihood, taken from Cheng et al. 2020 Ho al. 2020), learning the variance as in Rybkin et al. [202T].
We compare DCPC to LPC using the Frechet Inception Distance (FID) [Seitzer 2020) featured in Zahid et al. [024). holding constant the prior, neural network architecture, learning rate on 0, and number of gradient evaluations used to train the parameters 0 and latents z. /Zaid et al. (2044 evaluated a variety of scenarios and reported that their training could converge quickly when counted in epochs, but they accumulated gradients of @ over inference steps. We compare to the results they report after 15 epochs with 300 inference steps applied to latents initialized from the prior, equivalent to 15 x 300 = 4500 gradient steps on 0 per batch, replicating their batch size of 64. Since Algorithm I updates 0 only in its outer loop, we set S = 30 and ran DCPC for 150 epochs, with и) 0.001 and K = 16. Table 3 shows that DCPC outperforms LPC in apples-to-apple generative quality, though not the point of matching other model architecture by inference quality alone.
Figure 4 shows reconstructed images from the validation set (left) and samples from the posterior predictive generative model (right). There is blurriness in the reconstructions, often occurs with variational autoencoders, but DCPC training allows the network to capture background color, hair color, direction in which face is looking, and other visual properties. Figure reconstructions over the validation set, while Figure 4b shows samples from the predictive distribution.
Kuntz et al. [2023] also reported experiment on CelebA in terms of FID score, at the lower 32 32 resolution. Since they provided both source code and an exact mathematical description, we were able to run exact, head-to-heed head comparison with PGD. The line in Table 3 evaluuting DCPC with PGD's s example neural architecture at the 32 x 32 resolution (with similar particle count and learning rate) demonstrates significant improvement in FID for DCPC, alongside reduced FID variance.
Such vision Transformers, denoising diffusion models, etc.
8

---

Necessary Compute Resources The initial DLGM experiments on the MNIST-alike datasets were performed on a desktop workstation with 128GB of RAM and an NVIDIA Quadro P4000 with 8GB of VRAM. Experiments on CelebA were conducted on an NVIDIA DGX equipped with eight (8) NVIDIA A A100's, each with 80GB of VRAM. The latter compute infrastructure was also used for unpublished experiments, on several different datasets, in structured time-series modeling.
6 Related Work
Pinchetti et al. [002] expanded predictive coding beyond Gaussian generative models for the first time, applying the resulting algorithm to train variational autoencoders by variational inference and transformer architectures by maximum likelihood. DCPC, in turn, broadens predictive coding to target arbitrary probabilistic graphical models, following the broadening in Salvatori et al. [2022] to arbitrary deterministic computation graphs. DCPC follows incremental predictive coding (Salvatori et al., 2024) in quickly alternating between updates to random variables and model parameters, giving an incremental EM algorithm [Neal and Hinton [998]. Finally, Zahid et al. [2024] and Oliviers
ct al. [2024) also recognized the analogy between predictive coding 's prediction errors and the score functions used in Langevin dynamics for continuous random variables.
There exists large body of work on how neurobiologically plausible circuits could implement probabilistic inference. Classic work by (Shi and Griffiths| [2009) provided biologically plausible implementation hierarchical inference via importance sampling; DCPC proceeds from importance sampling as a foundation, while parameterizing the proposal distribution via prediction errors. Recent work by Fang et al. [2022] studied neurally plausible algorithms for sampling-bars based inference with Langevin dynamics, though only for Gaussian generative model of sparse coding. Golkar et al. [2022] imposed a whitening constraint on a Gaussian generative model for biological plausibility. Finally, Dong and Wu [2023] and Zahid et al. (2024] both suggest mechanisms for employing momentum to reduce gradient noise in a biologically plausible sampling algorithm; the former intriguingly analogize their momentum term to neuronal adaptation. To conclude, other works have already implemented predictive coding models for image generation tasks, a notable example being the neural generative coding framework Ororbia and Kifer (2022). Ororbia and Mali [0222]
7 Conclusion
This paper proposed divide-and-conquer predictive coding (DCPC), an algorithm that efficiently and scalably approximates Gibbs samplers by importance sampling; DCPC parameterizes efficient proposals for model's complete conditional densities using local prediction errors. Section 4 showed how Monte Carlo sampling can implement form of "prospective contiguration" (Song et al. 20224]
first inferring a sample from the joint posterior density (Theorem D and then updating the generative model without a global backpropagation pass ( TTheorem 2). Experiments in Section S showed
that DCPC outperforms the state of the art Monte Carlo Predictive Coding from computational neuroscience, head-to-heald
on the simple generative models typically considered in theoretical neuroscience; DCPC also outperforms the particle gradient descent algorithm of Kuntz et al. [2023) while under the constraint of purely local computation. DCPC's Langevin proposals admit the same extension to constrained sample spaces as applied in Hamiltonian Monte Carlo (Brubaker et al.. 2012]; our Pyro implementation includes this extension via Pyro's preexisting support for HMC.
DCPC offers number of ways forward. Particularly, this paper employed naive Langevin proposals, while Dong and Wul [2023). Zahid et al. (2024) applied momentum-basee preconditioning to take advantage of the target's 's geometry. Yin and Ao (2006) demonstrated that gradient flows of this general kind can also provide more ellicient samplers by breaking the detailed-balance condition necessary for the Metropolis- Metsopolis-Haattings algorithm, motivating the choice of SMC over MCMC to correct proposal bias. Appendix C discrete random variables. Future work could follow Marino et al. 2018). (Taniuuchi et al.) [2022] in using a neural network to iteratively map from particles and prediction errors to proposal parameters.
derives a mathematical background for an extension of DCPC to
7.1 Limitations
DCPC's main limitations are its longer training time, and greater sensitivity to learning rates, than tfate-of-the-art amortized variational inference trained end-to-end. Such limitations occur frequently in the literature on neuroscience-inspiee learning algorithms, as well as in the literature on particle- based algorithms with no parametric form. This work has no singular ethical concerns specific only to DCPC, rather than the broader implications and responsibilities accompanying advancements in biologically plausible learning and Bayesian inference.

---

Acknowledgments and Disclosure of Funding
E.S. was supported by Vanderbilt Brain Institute Faculty Funds, as well as the Vanderbilt Data Science Institute, through PI Andre Bastos. Hamed Nejat produced the laminar circuitry figure in the supplementary material in conjunction with E.S. H.W. and T.S. were supported through VERSES.A AL. E.S. would also like to thank Jan- Willem van de Meent, Karen Quigley, and Lisa Feldman Barrett for the training in approximate inference and predictive processing out of which this paper developed.
References
Miri Adler and Uri Alon. Fold-change detection in biological systems. Current Opinion in Systems Biology, 8:81-89, April 2018. ISSN 2452-3100. doi: 10..016fj.cibb22111..0000
Andre M Bastos, Julien Vezoli, and Pascal Fries. Communication through coherence with inter- areal delays. Current Opinion in Neurobiology, 31:173-180, April 2015. ISSN 09594388. doi: 1t000//6.cobb2014.1....
André M. Bastos, W Martin Usrey, Rick A Adams, George R Mangun, Pascal Fries, and Karl J Friston. Canonical microcircuits for predictive coding. Neuron, 76(4):695-711, 2012.
André M. Bastos, Mikael Lundqvist, Ayan S. Waite. Nancy Kopell, and Earl K. Miller. Layer and rhythm specificity for predictive routing. Proceedings of the National Academy of Sciences, 117 (49):31459-3146) December 2020. ISSN 0027-8424, 1091-6490. doi: 10.10733/mas..111111 201468811 17.
Johanna Bergmann, Lucy Petro, Clement Abbatecola, Min S Li, A Tyler Morgan, and Lars Muckli. Cortical depth profiles in primary visual cortex for illusory and imaginary experiences. Nature Communications, 15(1): 2024.
Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D. Goodman. Pyro: Deep universal probabilistic programming. Journal of Machine Learning Research, 20(28):1-6, 2019. URL http://jmlr org/ http://pppss///////// hatperrs/////181/4/// 18-433.. html
Cezar Borba, Matthew J Kourakis, Shea Schwennicke, Lorena Brasnic, and William C Smith. Fold change detection in visual processing. Frontiers in Neural Circuits, 15:705161, 2021.
Marcus Brubaker, Mathieu Salzmann, and Raquel Urtasun. A family of meme methods on implicitly defined manifolds. In Artificial intelligence and statistics, pages 161-172. PMLR, 2012.
György Buzsáki and Andreas Draguhn. Neuronal oscillations in cortical networks. Science, 304 19679) June 2004. ISSN 0036-8075, 1095-9203. doi: 10.1 wet////eence.0999900
Luke Campagnola, Stephanie C. Seeman, Thomas Chartrand, Lisa Kim, Alex Hoggarth, Clare Gamlin, Shinya Ito, Jessica Trinh, Pasha Davoudian, Cristina Radaelli, Mean-Hwan Kim, Travis Hage, Thomas Braun, Lauren Alfiler, Julia Andrade, Phillip Bohn, Rachel Dalley, Alex Henry, Sara Kebede, Alice Mukora, David Sandman, Grace Williams, Rachael Larsen, Corinne Teeter, Tanya L. Daigle, Kyla Berry, Nadia Dotson, Rachel Enstrom, Melissa Gorham, Madie Hupp, Samuel Dingman Lee, Kiet Ngo, Philip R. Nicovich, Lydia Potekhina, Shea Ransford, Amanda Gary, Jeff Goldy, Delissa McMillen, Trangthanh Pham, Michael Tieu, La' Akea Siverts, Miranda Walker, Colin Farrell, Martin Schroedter, Cliff Slaughterbeck, Charles Cobb, Richard Fllenbogen, Ryder P. Gwinn, C. Dirk Keene, Andrew L. Ko. Jeffrey G. Ojemann, Daniel L. Silbergeld, Daniel Carey, Tamara Casper, Kirsten Crichton, Michael Clark, Nick Dec, Lauren Ellingwood, Jessica Gloe, Matthew Kroll, Josef Sule. Herman Tung, Katherine Wadhwani, Krissy Brouner, Tom Egdorf, Michelle Maxwell, Medea McGraw, Christina Alice Pom, Augustin Ruiz, Jasmine Bomben, David Feng. Nika Hejazinia, Shu Shi. Aaron Szafer, Wayne Wakeman, John Phillips, Amy Bernard, Luke Esposito, Florence D. O'Ozzi, Susan Sunkin, Kimberly Smith, Bosiljka Tasic, Anton Arkhipov, Staci Sorensen, Ed Lein, Christof Koch, Gabe Murphy, Hongkui Zeng, and Tim Jarsky. Local connectivity and synaptic dynamics in mouse and human neocortex. Science, 375(6585)eab)558861 2022, doi: 10. /2t///ccince.aass8..... URL /tttps:///// ://::///wwww www. science. wwt.eee..ooo/////////e/ee/eeeeee 1126/ science. science.abj5861
10

---

Matteo Carandini and David Heeger. Normalization as canonical neural computation. Nature reviews neuroscience, 13(1):51 62, 2012.
Nick Chater, Joshua B Tenenbaum, and Alan Yuille. Probabilistic models cognition: Conceptual foundations. Trends in cognitive sciences, 10(7):287-29) 2006.
Spyridon Chavlis and Panayiota Poirazi. Drawing inspiration from biological dendrites empower artificial neural networks. Current Opinion in Neurobiology, 70:1-10, October 2021. ISSN 0959-4388. doi: 1...01//..oon.2221...
Zhengxue Cheng. Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: handwritten letters. In 2017 international joint conference neural networks (ICNN), pages 2921-2926. IEEE, 2017.
Extending mnist to
Ishita Dasgupta, Eric Schulz, Joshua B Tenenbaum, and Samuel J Gershman. A theory of learning to infer. Psychological review, 127(3):412, 2020.
Xingsi Dong and Si Wu. Neural Sampling in Hierarchical Exponential-familn Energy- based Models. In Advances in Neural Information Processing Systems, New Orleans, LA, 2023. Curran Associates Inc.
Rodney J Douglas and Kevan AC Martin. Neuronal circuits of the neocortex. Annu. Rev. Neurosci., 27(1):419-451 2004.
Michael Y-S Fang, Mayur Mudigonda, Ryan Zarcone, Amir Khosrowshahi, and Bruno A Olshausen. Learning and inference in sparse coding models with langevin dynamics. Neural Computation, 34 (8):1676-1700, 2022.
Oren Forkosh. Memoryless optimality: Neurons do not need adaptation to optimally encode stimuli with arbitrarily complex statistics. Neural Computation, 34(12):2374-2387 November 2022. ISSN 0899-76677, 1530-888X.
doi: 10. 10.11162neco_a_01151
Pascal Fries. Rhythms for cognition: Communication through coherence. Neuron, 88(1):220-22) 235,
October 2015. ISSN 0896-6273 doi: 10. n.016/..euron.2015.00 2015.09.34.
Karl Friston. A theory of cortical responses. Philosophical transactions of the Royal Society B: Biological sciences, 360 (45))4815)88888 2005.
Colleen J. Gillon, Jason E. Pina, Jérôme A. Lecoq, Ruweida Ahmed, Yazan N. Billch, Shiella Caldejon, Peter Groblewski, Timothy M. Henley, India Kato, Eric Lee, Jennifer Luviano, Kyla Mace, Chelsea Nayan, Thuyanh Nguyen, Kat North, Jed Perkins. Sam Seid, Matthew T. Valley, Ali Williford, Yoshua Bengio, Timothy P. Lillicrap, Blake A. Richards, and Joel Zylberberg. Learning from unexpected events in the neocortical microcireuit. bioRviv, 2023. doi: 10. 101/ 2021.01.15.426615. URL htt.::tts/////t www.:iiooivvvooo http:///wttttteeorrr///////////0 01. 426915
Siavash Golkar, Tiberiu Tesileanu, Yanis Bahroun, Anirvan Sengupta, and Dmitri Chklovskii. Constrained predictive coding biologically plausible model of the cortical hierarchy. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 14155-14169 Curran Associates, Inc., 2022. URL 0555665/4acc11887770868777778883 Panfeaapppr-oooeeeeeeeeeeeeeee .pt
neurips cc/ cctppper...p..ppeeeeeeee paperrapperrrr fi 1illes/paper/2022////aa
Joseph Gonzalez, Yucheng Low, Arthur Gretton, and Carlos Guestrin. Parallel gibbs sampling: From colored fields to thin junction trees. In Geoffrey Gordon, David Dunson, and Miroslav Dudík, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 324-332, Fort Lauderdale, FI., USA, 11-13 Apr 2011. PMLR. URL https: ttttss:/pproeeeding mlr, httpiiggssml....sssseeee press/
v15/ r15/gonzaleziaa. html
11

---

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. VAE: Learning basic visual concepts with constrained variational framework. In International Conference on Learning Representations, 2017. URL httpptttts//opeeeeeiiiie
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, page 6840-6851. Cur- Associates, Inc., 2020. URL https:://proceeddnggs nwwrips wwtuurps.ccpppppeee wttpccpppprr//2///aaaaeee
Benjamin Hutchinson and Lisa Feldman Barrett. The Power of Predictions: An Emerging Paradigm for Psychological Research. Current Directions in Psychological Science, 2019. ISSN 14678721. doi: 10.117700672141988992
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114 2013.
arXiv preprint
Juan Kuntz, Jen Ning Lim, and Adam Johansen. Particle algorithms for maximum likelihood training of latent variable models. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, volume 206, Valencia, Spain, April 2023. Proceedings of Machine Learning Research.
Juan Kuntz, Francesca R. Crucinio, and Adam M. Johansen. The divide-and-conquer sequential Monte Carlo algorithm: Theoretical properties and limit theorems. The Annals of Applied Probability, 34(IB):1469 1523, 2024. doi: 10. 10.1214/3-AAAP199 URL htttps://dat org/10. 1214/
23- AAP1966
Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. Building machines that learn and think like people. Behavioral and Brain Sciences, 40:e253, 2017. doi: 10.1017/501405555X 05004400055X05188332 16001837.
Yann Lecun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521((553):436 444, 2015. ISSN 14764687. doi: 10. 1038//attreee444ee 14539, Citation Key: Lecun2015.
F. Lindsten, A. M. Johansen, C. A. Naesseth, B. Kirkpatrick, T. B. Schön, J. A.D. Aston, and A. Bouchard-CGol Côté. Divide-and and--adddduuee with sequential monte carlo. Journal of Computational and Graphical Statistics, 36(2):445-44 4458, 2017. ISSN 15372715. doi: 10.1080/1061 1000006611000221112333566 arXiv: 1406.4993 Citation Key: Lindsten2017.
Shih-Chii Liu. A winner-take-all circuit with controllable soft max property. In S. Solla, T. Leen, and K. Müller, editors, Advances in Neural Information Processing Systems, volume 12. MIT Press, 1999. URL UtL.https::/pprcceeiinng wew.:eiiiiiinnnnnniineeee.. www.eeurr....... www.urips..cpppp.....a.pp httpa//ppp.ppppppp....es//////// wtt.ss/pppper/199/////5// 3e7 Baro22200aaa355laabb66462aa.aa.2
pdf
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Gabriel Loaiza-Ganem and John P Cunningham. The continuous bernoulli: fixing a pervasive error in variational autoencoders. In Advances in Neural Information Processing Systems, volume 32. neurips. https/ccpappppee/1////e/aaae 71179966800002355566222222bbbb. Abstract. html html
Curran Associates, Inc., 2019. URL https: http:::://////oeddns//ee
Brian N. Lundstrom, Matthew H. Higgs, William J. Spain, and Adrienne L. Fairhall. Fractional differ-
entiation by neocortical pyramidal neurons. Nature Neuroscience, 11(11):1335-1342 November 2008. ISSN 1726. doi: 10. (038/nn.2212)
Zachary F Mainen and Terrence J Sejnowski. Reliability of spike timing in neocortical neurons. Science, 268(5216):150311500 1995.
12

---

Joseph Marino, Yisong Yue, and Stephan Mandt. Iterative amortized inference. In 35th International Conference on Machine Learning. ICML 2018, volume 8, page 5444-5462, 2018. ISBN 978-1- 5108-6796-3. arXiv: 1800.09356 Citation Key: Marino2018a.
Beren Millidge, Anil Seth, and Christopher Buckley. Predictive coding: a theoretical and experi- mental review. arXiv preprint arXtv:2107.12279, 2021.
Beren Millidge, Yuhang Song, Tommaso Salvatori, Thomas Lukasiewicz, and Rafal Bogacz. A theoretical framework for inference and learning in predictive coding networks. In The Eleventh International Conference on Learming Representations, 2023. URL https://openreviier net/
"ooru??id2CCTvs _VM44
Toviah Moldwin, Menachem Kalmenson, and Idan Segev. The gradient clusteron: A model neuron that learns to solve classification tasks via dendritic nonlinearities, structural plasticity, and gradient descent. PLOS Computational Biology, 1009015, May 2021. ISSN 1553-7358. doi: 10.1371/jourmal.ppe 1009015.
Christian Naesseth, Fredrik Lindsten, and Thomas Schon. Nested sequential monte carlo methods. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 1292- 1301, Lille, France, 07-09 Jul 2015. PMLR. URI.hhtps:://// http:ttst////rrcediinn .ma.ediinnnmmm......mmmmmm.m wwtesemw......s.eeeeeeeeeemeeeee press/v37
maesseth15.her html
Christian Naesseth, Scott Linderman, Rajesh Ranganath, and David Blei. Variational sequen- tial monte carlo. In Amos Storkey and Fernando Perez-Cruz, editors, Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, volume 84 of Proceedings of Machine Learning Research, pages 968-999899 977. PMLR, 09-11 Apr 2018. URL https://pproceedingg mlr httpi/nnww.r...rrsreeeee Salpas//8///aesseee 1aa. html
Radford M. Neal and Geoffrey E. Hinton. A View of the EM Algorithm that Justifies Incremental, Sparse, and other Variants, page 355-368 NATO ASI Series. Springer Netherlands, Dordrecht, 1998. ISBN 778-94-011-50114 doi: 10.1007/978-94-00000 940011-5014--999 12. URL/attps://aoi org/ 10. /9700///77--0 04-0199-----00000 5014-004499999
Gaspard Oliviers, Rafal Bogacz, and Alexander Meulemans. Learning probability distributions of sensory inputs with monte carlo predictive coding. bioRxiv, 2024.
Alexander Ororbia and Daniel Kifer. The neural coding framework for learning generative models.
Nature communications, 13(1):2064, 2022.
Alexander Ororbia and Ankur Mali. Convolutional neural generative coding: Scaling predictive
coding to natural images. arXiv preprint arXiv:22 12.02277,1 2022.
Benjamin Peters, James DiCarlo, Todd Gureckis, Ralf Haefner, Leyla Isik, Joshua Tenenbaum, Talia Konkle, Thomas Naselaris, Kimberly Stachenfeld, Zenna Tavares, et al. How does the primate brain combine generative and discriminative computations in vision? ArXiv, 2024.
Luca Pinchetti, Tommaso Salvatori, Yordan Yordanov, Beren Millidge, Yuhang Song, and Thomas Lukasiewicz. Predictive coding beyond gaussian distributions. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neu- ral Information Processing Systems, volume 35, pages 1280-1293. Curran Associates, neurips ccppapprrp... fiwes/ ppts//pppp//////lll// Inc., 2022. URL https::/ Paper- Swwpperr-ooffeeence..o .dff 08510002300085110111666666666661
Alexandre Pouget, Jeffrey M Beck, Wei Ma, and Peter E Latham. Probabilistic brains: knowns and unknowns. Nature neuroscience. (709):1170-1778 2013.
Rajesh Rao and Dana Ballard. Predictive coding the visual cortex: functional interpretation of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79-87, 1999. ISSN 1097-6256. doi: 10. 1038/450. URL 1RR..DD..00B///0//CCcttt////// journal/v2/n/aaee http:////w//////////////a////a/e 1n0199 1w111199999.911 html
http::///wwnna..ueeeee com/neuro.
13

---

Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning. volume 4, page 3057-3070, Beijing, China, 2014. arXiv: 1401.4082 Citation Key: Rezende2014 ISBN: 9781 9781634393973.
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. Nature, 323(688)):533-555 1986.
Oleh Rybkin, Kostas Daniilidis, and Sergey Levine. Simple and effective vae training with calibrated decoders. In International conference on machine learning, pages 9179-9189. PMLR, 2021.
Tommaso Salvatori, Luca Pinchetti, Beren Millidge, Yuhang Song, Tianyi Bao, Rafal Bogacz, and Thomas Lukasiewicz. Learning on arbitrary graph topologies via predictive coding, Advances in neural information processing systems, 35:38232-38244 2022.
Tommaso Salvatori, Ankur Mali, Christopher L Buckley, Thomas Lukasiewicz, Rajesh PN Rao, Karl Friston, and Alexander Ororbia. Brain-inspired computational intelligence via predictive coding. arXiv preprini arXiv:2308.078777 2023.
Tommaso Salvatori, Yuhang Song, Yordan Yordanov, Beren Millidge, Cornelius Emde, Zhenghua Xu, Lei Sha, Rafal Bogacz, and Thomas Lukasiewicz. A stable, fast, and fully automatic learning algorithm for predictive coding networks. In International Conference on Learning Representations, 2024.
Jürgen Schmidhuber. Deep learning in neural networks: An overview. Neural Networks, 61:85-117, 2015. ISSN 18792782. doi: 10. wwt.e//...ee.2001111
Citation Key: Schmidhuber2015.
Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. httpp://ggichu...ommmeeet
pytorch- fid August 2020. Version 0.3.0.
Lei Shi and Thomas L. Griffiths. Neural implementation of hierarchical bayesian inference by importance sampling. In Advances in Neural Information Processing Systems, page 1669-1677, 2009. ISBN 978-1-61567-911 -91-99
Citation Key: Shi2009.
Yuhang Song, Beren Millidge, Tommaso Salvatori, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Inferring neural activity before plasticity as a foundation for learning beyond backpropagation. Nature Neuroscience, page 1-11, January 2024. ISSN 1546-1726. doi:
10.103884415993022-0115444
M. W. Spratling. A review of predictive coding algorithms. Brain and Cognition, 112:92-97, 2017. ISSN 10902147. doi: 10.10166j.bamm 2015...0115.1...1
M. V. Srinivasan, S. B. Laughlin, and A. Dubs. Predictive coding: A fresh view of inhibition in the retina. Proceedings of the Royal Society of London Biological Sciences, 216(1205):427-45, 1982. ISSN 09628452, doi: 10. 10.1098/rspb.199 1982.0085.
Sam Stites, Heiko Zimmermann, Hao Wu, Eli Sennesh, and an-Willem Van Meent. Learning proposals for probabilistic programs with inference combinators. 37th Conference on Uncertainty in Artificial Intelligence (UAI 2021), 2021.
Shohei Taniguchi, Yusuke Iwasawa, Wataru Kumagai, and Yutaka Matsuo. Langevin autoencoders for learning deep latent variable models. Advances in Neural Information Processing Systems, 35: 13277-13289, 2022. URL/atpss:/prrcceednggs
www.sennnniiiieeeeee http://dwwa..a.u.oooopoooppppppp files/ watss/appr//2/ 2022/ html
Stefan Webb, Adam Golinski, Rob Zinkov, Siddharth N, Tom Rainforth, Yee Whye Teh, and Frank Wood. Faithful inversion of generative models for effective amortized inference. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, ed- itors, Advances in Neurul Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL https: http:s:: www.rreneniip..eee wwwriip..c/ppppeeeee 69977188550443325544433353333333 Paper .df
14

---

Max Welling and Yee Whye Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th International Conference on Machine Learning, ICML 201 Bellevue, WA, USA, 201 1. Proceedings of Machine Learning Research.
James CR Whittington and Rafal Bogacz. An approximation of the emorr backpropagation algorithm in predictive coding network with local hebbian synaptic plasticity. Neural computation, 29(5): 1229 1262, 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv: 2017.
L. Yin and P. Ao. Existence and construction of dynamical potential in nonequilibrium processes without detailed balance. Journal of Physics A: Mathemattiaa and General, 39((7):8593, June 2006. ISSN 0305 doi: 10. 04158/00054440/3////
httpttps: /tdpss//dx... doi. org/10
Umais Zahid, Qinghai Guo, and Zafeirios Fountas. Sample as you infer: Predictive coding with langevin dynamics. In Proceedings of the 41st International Conference Machine Learning, volume 235, Vienna, Austria, 2024, Proceedings of Machine Learning Research.
15

---

Lower order area
L21
3
L2/3
E x
L4
E y
15
(())
L4
L6
((()
L5
Nela
Higher order area
Pu(e) L6
Figure Divide-and-conque predictive coding provides an algorithmic interpretation for some of the connections mapped in the canonical neocortical microcircuit [Bastos et 2012 2020 Campagnola (et al.) 2022]: : prediction errors (red) arrive through ascending pathways into the central laminar layer 4, which transmits them up to layers 2/3 (green). These layers combine the incoming errors with present posterior estimate (green 1,5 L.2/3 connection) to generate prediction errors for the next cortical area. Eventually the updated predictions flow back down the cortical hierarchy (blue).
A Further experiments and results
Alternate image generation/ representation learning As indicated in Section 2 this paper builds upon the particle gradient descent (PGD) algorithm; Kuntz et al. [2023] demonstrated the algorithm's performance by training generator network CelebA. Their network employed Gaussian likelihood with a fixed standard deviation of 0.01, and evaluated log-joint objective over 100 epochs exactly 10,000 subsampled data points. The paper then evaluated mean squared error inpainting task and the Frechet Inception Distance over data images.
When applied to the resulting target density, DCPC amounts to PGD with resampling step. Table 4 shows the results of training and evaluating the same model described above with DCPC. Since PGD trained for 100 epochs with batch size of 128, albeit on 10,000-image subsample of CelebA, we trained with the entire dataset for 100 epochs with batch-size 128.
Inference type
Log-joint
FID
VAE (K = 10)
-4.3 x 10a
PGD (K 10)
-3.8 105
100 = 2.7
DCPC (ours, K 10)
-1.8 x 106
82.7 0.9
Table 4: Log-joint probabilities and FID metrics show how DCPC performs against the original PGD.
We suspect the supplied code for log-joint calculation averages over either particles or batch items differently from how we have evaluated DCPC (e.g. call mean () without dividing by any further shape dimensions), accounting for the apparent order-of of-mannitude difference between log-joints.
At the request of reviewers, we have substituted simplified Figure in the main text for Figure showing how map DCPC onto laminar microcireuit structure.
B Importance sampling and gradient estimation proofs
Titsias 2023 introduced optimal estimators for preconditioning Langevin dynamics to adapt with the Fisher information of the target density. Definition the most basic estimator for that Fisher information, defined in terms of the score functions use prediction error..
16

---

Definition 2 (Bayesian Fisher estimator (Titsias 2023)). Denoting by the score function (from Equation 6) and letting À >> 0 be a fixed hyperparameter, the Bayesian Fisher estimator
Ix := (op(sl)))))
[(sss])
A R
(11)
is an empirical estimator of a Bayesian target density's Fisher information (the target density from which the prediction errors were derived) based on a cloud of K particles. When the particles are not yer distributed around a mode of the target (e.g. the score function does not have average of zero), substituting the empirical covariance for the expected outer product reduces the estimator's 's bias
ix(e = Cov (5
(12)
The above matrix does not describe the preconditioner that Titsias 2023 actually recommended applying Langevin proposal. Pefinition@@rovidde the fully normalized preconditioner.
Definition 3 (Predictive coding Fisher preconditioner [Titsias] 2023 ). Using Definition 2 to pa-
rameterize preconditioner for the Langevin dynamics proposal, the predictive coding Fisher preconditioner is the inverse of Equation 12 normalized to have an average eigenvalue of
Ix(E!:K)-
Ex(=!K) = [T((I(e)-K>-1
(13)
Definition 4 generalizes the definition of importance sampling, suitable for recursively constructing sequential Monte Carlo algorithms with changing target densities.
Definition 4 (Strict proper weighting for a density). Given an unnormalized density "f(()) with corresponding normalizing constant Z(0) and normalized density (z)
[ 70(2) dz ZEZ
To (z) = Val (Z)
7(0)
Z(0)
the random variables respect to (z(z)) if and only if for any measurable test function + R, the weighted expectation over the proposal q(w, z) equals the expectation under the target (z)
q(w,2) 2) are strictly properly weighted / |Naesseth et al. 2015 with
ff ZEZ
E Bww (w [wh(z)] =
h(zz) (z) dz.
(14)
The following two propositions come from the previous work by Wu et al. (2020), Stites et [202]]
and Zimmermann et al. [2021] The reader looking for foundations can see Naesseth al. [2015] or
Chopin and Papaspiliopoulos 2020).
Proposition (The free energy upper-bounds the surprisal). Given proposal go go(w, z) strictly
properly weighted (Definition 4 for the targer (yo(z), the variational free energy provides an upper
bound the target's surprisal
F(0.g) log Z(0)-
(15)
Proof. begin by writing out the free energy (Equation as an expectation of a negative logarithm
F(0. q) (g, log w]
Jensen's Inequality allows moving the expectation into the negative logarithm by relaxing the definition of the variational free energy from an equality to an upper bound
F(0, q) log Feg,w [w].
Setting h(2) = 1, strict proper weighting for an unnormalized density (Definition 4 says the expected weight will be the normalizing constant
E ( [w] 2(0)
which substitute back in obtain the desired inequality F(0,g) > -log Z(0).
17

---

Proposition 2 (Weighted expectations approximate the normalized target up to constant). Given a proposal 9 w. z) strictly properly weighted (Deffmition 4) for the target (o(( and measurable test function h 2 - TR, weighted expeciations under the proposal equal the target' 's normalizing constant times the test function's n's expectation under the normalized target
(1 [wh(z)] 2(0) [f(z)]
Proof. Strict proper weighting (Equation 14) states that weighted expectations under the proposal equal integrals over the unnormalized target, and by definition the normalized target equals the unnormalized density over its normalizing constant
Eu (( [uch(z)] = / h(z) (z(z) dz,
To := 7
2
2())
The second equation expresses the unnormalized target terms of the normalized one
Z(0) (o(z)) (z).
and substituting this expression into the definition of strict proper weighting leads to the desired result
[ h(z) 7e(z) dz
/ h(z) Z(0)To(z) dz,
JzEZ
-(6) /
h(z)) [z) dz
2E2
Ew Ew.zrg(wx)) [wh(z)] Z ((0)Em)(2) [l(z)]]
Proposition 3 (DCPC's free energy has pathwise derivative). The free energy F+1l
E.g [- log wht constructed by the population predictive coding algorithm (Algorithm has pathwise derivative as the expectation of the negative gradient of the log-joint density
Fottl Eg Vo log Pø (x, 2z,zz 22+1)]
Proof. The free energy has an expression in terms of Equation
JT+1 Egg [- log 1
.+1 wige
Por (x, z)
II (Z),))
I 7o (0 ;2
262
(0(2 m,ee)
Zo (z)=) r41
)+1
1+1.
+++
"
((=++1 (2()))
and writing out the free energy itself in full shows that many terms cancel
q(z 2²) = II
(( 1x
sh Ezt+1
Po (x, z)
7o (2 ++1 :212)
Jt+1
Eyyce++1p (2t)
log
II
I.ex 70 (2) 1+1, ;
K
a x {=(2(()
ZE
Po (x, 2)
Ueen 7062 212)
Exy (at)
log
Il 20 ;2)2))
II g( {x())
Pol (x, 2)
Exyeet+spp (at)
log
9 zt)
The proposal distribution q is a function of the random variable values themselves through the prediction errors, not of the parameters 0. The above expression therefore admits pathwise
18

---

derivative [Schulman et al. 2013 moving the gradient operator into the expectation
Pot (x, x 11)
V Vg.Ft+1
Vo B 1xt))
log
(((++1 x²)
Pot (x, 22 1 g( zzt)
V gr log
Begle 1 1|21)
V dr Ilgg et (x, x,,2*+1
log q |z']]]
V g+ log Pø (x, x²11) V log (ggl(zt) x')]
gz Fo+ (xt) V gr log (x, =²]
Proposition (DCPC coordinate updates are strictly properly weighted for the complete conditionals). Each DCPC coordinate update (Equation]) 7 for latent variable E z strictly properly weighted (Definition 4) for z's unnormalized complete conditional. For every measurable h
IR
f J
R 1a ( 2-RESMMLLE(S.... [b(=)]]
11()) (0() dz.
(16)
Proof. Expanding the outer expectation into an integral and replacing the Dirac delta with the expression for the local weights transforms Equation T6 into
70(2;3 E RESAMPLE(..... [h(z')] f 1 dz [ 2 (80)
,8
f
h(2) (0(2; dæ;
2EE
importance resampling also preserves strict proper weighting (see Naesseth et al. 2021 and Chopin and Papaspiliopoulos [2020) for proofs), and this yields
2015
Stites et al.
JEEZ E. RES-MMEMMPLE(...) [h(z')] (0(2) dz -f / h(2)) (2(2) dz
[ JEZZ
h(2') ro(<' de' = J
(02;2()) dz.
Corollary 4.1 (DCPC coordinate updates sample from the true complete conditionals). Each DCPC coordinate update (Equation 7 for a latent a samples from z's complete conditional (the normalization of Equation 5). Formally, for every measurable h Z - R, resampled expectations with respect to the DCPC coordinate update equal those with respect to the complete conditional
ff
Eng (xlet-1,et) [Bu -.E-Sfaa.. [M<=)]]]
1(2) mo(2 (xxe)) dz.
JEZ
Proof. Proposition in Appendix B provides lemma
Eg E B (.-r6:)):..2--RREMMPPPLL R...SAA
[M<>0]]
h(2) 70(2; 2xa)) de,
ZEZ
which we can apply by observing that resampling sums over self-normalized weights
Egman(alet-1,st) [Eund()) REESAMPL.. [hx)]]]
(xlgn(
Euvō(u)
E mda()) [b(=')]]
which is just a weighted sum that by Definition 4 is itself properly weighted
E Ee Buno(u))
By ( [h(=')]
E Be
Bund(u)
1 11(2))
Σu u
19

---

1
E (x Bu-
h(2) 70(2: dz
Σ u J3Ez
-e (,65)
Buwb(u)
Zofx; Z
11(2) TO(= x, dz
JZEZ
JEEz 1(2)) T0(2 x, 2i dz.
Proposition (DCPC parameter learning requires only local gradients in a factorized generative model). Consider graphical model factorized according to Equation with the additional assump- tion that the model parameters E θ IIrex TIrex0, O₂ x II O₂ share that factorization. Then the gradient F(, q) of DCPC's free energy similarly factorizes into a sum of local particle averages
VFF = E. |- log (x,,
Σ
By(v.Pa(o)leen)Patt))) 1 Vo, logpo, (v Pa(t))]
(E(x22
e. logpo, (v* Pa(u)*).
(E(X.2)
Proof. Proposition provides the lemma that vaF = Ey L- log (x( and applying the factorization of the generative model demonstrates that
V
Eg
V
log Pa(P))
(E(x2))
Since the proposal does not depend on any 0 and consists of a particle cloud, we can rewrite it as a mixture over the particles (after sampling performed)
V
log (o Pa(u)*),
K
k=1
(E(x22)
and then finally apply the assumption of this theorem that E e IIxex O2 x II.ee O., moving the gradient operation into the sum over individual random variables
Vo Po (o ))(u)**..
K
Jm (E(x,*)
C Extension to discrete sample spaces
Contemporaneously the work of Kuntz 2023 on partiele gradient descent, Sun al. [2023] derived a novel Wasserstein gradient flow and corresponding descent algorithm for discrete distributions. In their setting, each Wasserstein gradient step constructs a D-dimensional, finitely supported distribution over the C-Hamming ball of the starting sample, such that the distribution has DC possible states in total. Let _+ E Nc(z²) denote the resulting discrete random variable in the C-neighborhood around zf with respect to the Hamming distance. The update rule relies on simulating the gradient flow for time h, sampling from a Markov jump process at time
II q(2
22)))
dE(1...D
rate matrix Qa(z*) defined by the entire discrete variable 2f parameterizes the proposal distribution
gn ( 20) cxp (Qa(=*)h)-
(17)
the rate matrix will have nondiagonal entries at indices + j E [1 [1... CJ in the neighborhood No((2))
T 2/a.)
Qa(z")es wi.jg
To( 2/2,))
20

---

The above equation requires that V E [1...C] C), wa, JS E R and g(a) ag (2). The ratio of normalized target densities π will equal the ratio of unnormalized densities y
To 2ao,) No no za,i)
ro(4a, a 8)
Zantafa (/a,
no no(=ias=aa Fd.s) To Te(afaa 2/a.))
r0 (a(Fa, z
9
70 70(='as' 24
Based on the experimental recommendations of Sun et al. (202)), let wi, Ø5 = and anndg(a) = va. The matrix then simplifies to nondiagonal and diagonal terms
To (0(2 2 70 (
Qa(=))ss
= jtfi
(18)
Equations 17 and 18 give distribution descending the Wasserstein gradient of the free energy with
respect to particle cloud in a discrete sample space. Applying Equation 18 to (a 2,=)) yields factorization in log space
(= + ( Q(=))s (0 + (2
log Q(2))i) = (log no(0 + +1 - log 70(2' + x())))))
This difference can be written as a difference differences
log 70(2" + +, log (2 + jiz(())
(log ro + i: - log 70(= = (logno( + - (19)
Recent work efficient sampling for discrete distributions has focused approximating density ratios, such as the one in Equation 18 with series expansions parameterized by error vectors. When the underlying discrete densities consist of exponentiating differentiable energy function, in Grathwohl et al. [2021), these error vectors have taken the form of gradients and the finite-series expansions have been Taylor series. When they do not, Xiang al. [2023) showed how they take the form of finite differences and Newton's series
log 7(2)) - log ((2) A (log (z)) (z' z).
(20)
Discrete DCPC would therefore use finite differences as discrete prediction errors, breaking each discrete 2 E into dimensions and incrementing each dimension separately to construct a vector
A1 (z 1,,22)), f(=11, 2, 1, 1, ((1.DD- 2D 1)) e f(=), (21)
where é subtracts the scalar f2) from the vector elements and f 7 IR: is the target function. This would lead to defining the discrete prediction error as the finite difference
A lo (222222
(22)
Applying Equation to the two terms of Equation
19
we obtain the approximations
log (2" + i; xx) log (o
A
(log ((=' 22)
Rd =2(2
+ +; log 20(2';0 R A (logno(((iz(e)) ((=* + -) =2)
R (2(2 J,
log Q()((2)))) ( [2))T ()-
Discrete DCPC would thus parameterize its discrete proposal (Equation 17) in terms of es (Equa- tion 22 that Equation (
[8 out the (matrix) exponential of the (elementwise) exponential
Qa(Ez)i,5 exp (<=)d (ia ja)
gr(22 (h E2) exp (Q(e))))
21

---

Supplementary References
Nicolas Chopin and Omiros Papaspiliopoulos. An Introduction to Sequential Monte Carlo. Springer, 2020. ISBN 978-3-030-4784455 doi: 10.10077997333000044444444 Citation Key: Chopin2020 ISSN: 2197-568X.
Will Grathwohl, Kevin Swersky, Milad Hashemi, David Duvenaud, and Chris Maddison. Oops took a gradient: Scalable sampling for discrete distributions. In Proceedings of the 38th International Conference on Machine Learning, page 3831-3841. PMLR, July 2021. URL
https://prooeeeding.. mlr Bttes//vv33///raahooooo http://poeeeiiig....peeeee//r//s
html
John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel. Gradient estimation us- ing stochastic computation graphs. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran As- sociates, Inc., 2015. URL httppttt//////ooceeeeeee fiil:/eeeeeffeeddffffffeeaeaaeee .fp
neurips. cc/ wwtpapprrfiiiiiieceee files/ wwterrffllesspee/eeeeeeer /2015/
Haoran Sun, Hanjun Dai, Bo Dai, Haomin Zhou, and Dale Schuurmans. Discrete Langevin Samplers via Wasserstein Gradient Flow. In Proceedings of the 26th International Conference Artifi- cial Intelligence and Statistics, Valencia, Spain, April 2023. Proceedings of Machine Learning Research.
Michalis Titsias. Optimal preconditioning and fisher adaptive langevin sampling. In Advances in Neural Information Processing Systems, volume 37. Curran Associates, Inc., 2023. URL https: h http:////ppppeeeeggoo.oeo/icccap
html
Hao Wu, Heiko Zimmermann, Eli Sennesh, Tuan Anh Le, and Jan Willem van de Meent. Amortized population Gibbs samplers with neural sufficient statistics. In Proceedings of the 37th International Conference Machine Learning, 2020.
Yue Xiang, Dongyao Zhu, Bowen Lei. Dongkuan Xu, and Rugi Zhang. Efficient Informed Proposals for Discrete Distributions via Newton's Series Approximation.
Proceedings of The 26th International Conference Artificial Intelligence and Statistics, pages 7288-7310 -310. PMLR, April 2023. Utt.aatps://pprocedingg mlr wttpl/.peesssse//.eeeeeee xxw///000xxxag2aa..am html
ISSN: 2640-3498.
Heiko Zimmermann, Hao Wu, Babak Esmaeili, and Jan- Willem van de Meent. Nested variational inference. In M. Ranzato, A. Beygelzimer, Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 20423-20435. Cur- ran Associates, Inc., 2021. URL URL 200.1/21114220000000004444000000
wtw.roocceiinngn.nnrrr......... newrips..... http://wwwooooooooieeeee.pppeppa
pdf
22

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). ). In general, answering binary way, we
"[No] or "[NA] is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can found.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Section 7.1 discusses limitations specifically.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
23

---

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specing well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms
and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Proofs can be found in Appendix B
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Thcorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper includes the learning rates and particle counts, the necessary Justification: hyperparameters, for the original experiments, as well as releasing code to run them.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
24

---

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The paper provides a [link to the code in the abstract and introduction.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7/
gwwss///gueeeeeeee httpg//dddee/odeebbiiiii CCodeSoodieeeeeeeeee ssi mollicyy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
25

---

Answer: [Yes]
Justification: The paper notes the hyperparameters and datasets used, gives additional details in the appendices, and releases the code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Results are given in terms of sample averages and standard deviations, with the standard deviations serving as error bars. The paper notes the number of replicates over which averages taken.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2- 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The experiments section now includes a paragraph at the end documenting the specific compute resources used in experiments. Many of our experiments can be trained with less compute than we used, albeit more slowly.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
26

---

The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The paper does not rely human subjects, nor release any new dataset. Approximate Bayesian inference a well-understood problem whose downstream ethical impacts do not differ significantly from those of other machine-learning methods.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: We propose novel neuroscience-inspired learning algorithm, which does have immediate societal impacts, positive or negative.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: We do not use data that not already available online, nor do we release any models for task more complex than non-customizable image generation.
27

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, thcode thcode. com datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: The code is not well-documented by the standards of industry, being research code. We do not release any new datasets foundation models document.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
28

---

Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "RppR7qR2ZJ", "reviewer": "Reviewer_jNdT", "summary": "The paper introduces a novel algorithm called Population Predictive Coding (PPC)\nfor structured generative models. The PPC algorithm aims to enhance the \nperformance and biological plausibility of predictive coding\napproaches by respecting the correlation structure of generative models. The\npaper provides theoretical foundations, discusses the biological plausibility of\nPPC and performs empirical validation against previous predictive coding and\ndeep generative modeling algorithms.", "questions": "1. I might be missing something here, but why is the FID score for PPC missing\n   in table 3? On that note, have you considered using `cleanfid` instead of\n   `pytorch-fid` to obtain more stable FID score estimates?\n2. It is great that you implemented it all in the `Pyro` framework. However, the\n   submission seems to be lacking an (anonymous) code attachment. Will you make\n   the code publicly available with instructions and all relevant information\n   for reproducing your experiments?", "limitations": "Yes, the authors have discussed the limitations of their work and potential negative societal impact.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "### Originality\n\nThe proposed PPC algorithm is a novel combination of recent developments in\nMonte Carlo sampling and the predictive coding approach. The transfer of these\nnew techniques to PC to derive the PPC algorithm is a substantial contribution.\n\n### Quality\n\nThe technical derivation of the proposed method appears sound, but see my\ndisclaimer below. The empirical evaluation is thorough and the results are\nconvincing. Limitations, especially the higher computational costs are discussed\nopenly, which is great.\n\n### Clarity\n\nThe paper is generally well-written and structured, making the high-level\nconcepts accessible to readers with a background in computational neuroscience\nand machine learning. Especially the introduction and motivation are clear and\neasy to follow.\n\nThe technical sections (2 and 3) are complex and could benefit\nfrom additional diagrams, examples, and intuitive explanations. More detailed\nbreakdowns of the PPC algorithm steps would enhance comprehension, especially\nfor readers less familiar with predictive coding.\n\n### Significance\n\nThe results are important and have the potential to significantly impact the\nfield of predictive coding and structured Bayesian inference. The proposed\nalgorithm addresses a difficult task and improves upon previous work, advancing\nthe state of the art demonstrably. The PPC algorithm's biological plausibility\nhas implications for both machine learning models and our understanding of\nneural information processing.\n\nDisclaimer: As an expert in simulation-based inference and with a background in\ncomputational neuroscience, I found the high-level concepts and motivations of\nthe paper clear and compelling. However, my expertise in predictive coding is\nlimited, and I found the technical details in sections 2 and 3 challenging to\nfully understand. This review reflects my understanding based on the provided\nexplanations and my background knowledge.", "weaknesses": "See text box above.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the time and valuable feedback.\n\n> The technical sections (2 and 3) are complex.\n\nThank you for the pointer. We have heavily modified the explanations, restructuring the sections and adding sentences that lead to a more intuitive understanding of the algorithm. For more details, we refer to the general answer provided above, and to the updated manuscript present in the anonymous link.\n\n> Have you considered using cleanfid instead of pytorch-fid to obtain more stable FID score estimates?\n\nWe will add cleanfid to our calculations alongside our existing FID evaluator/implementation, for which we have been using torchmetrics by Lightning.ai.\n\n> It is great that you implemented it all in the Pyro framework. However, the submission seems to be lacking an (anonymous) code attachment. Will you make the code publicly available with instructions and all relevant information for reproducing your experiments?\n\nYes. We have [anonymized our Github repository](https://anonymous.4open.science/r/ppc_experiments-8AF5/README.md); please share and enjoy. Our training code uses the Lightning framework and our evaluations consist of running a Jupyter notebook all the way through.  We will make the code publicly available and include a README.md specifically listing the combination of shell-command (for training) and notebook (for evaluation) to produce each and every figure and numerical result in the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1arxsATCtT", "reviewer": "Reviewer_AbQA", "summary": "Predictive coding (PC) is a speculative but attractive theory of how certain parts of the brain—especially the so-called 'canonical' cortical microcircuit—might implement Bayesian inference, and hence focus processing effort on the 'surprising' rather than the 'predictable' features of (e.g., sensory) stimuli. PC requires a generative model and an optimization algorithm (i.e., given observations, what is the posterior associated with the model's parameters?). \n\nWhile there exists a rich literature related to how PC might be implemented in the brain, previous work has been biologically implausible in a few different senses. In particular, usually (i) a simple (usually Gaussian) generative model is assumed; and/or (ii) inference involves some kind of approximation; and/or (iii) the algorithm is not necessarily fully 'local'. \n\nThe authors propose a novel implementation of PC that they argue is biologically plausible. They call it \"population prediction coding\" (PPC). They also show that their algorithm can solve some practical machine learning tasks.", "questions": "1. Can the authors briefly summarize the 'new' features of their algorithm relative to other PC ideas?\n2. Why particle-based gradient descent as opposed to some other approach to empirical Bayes?\n3. Concretely, how might neurons learn log-density gradients?\n4. What is the FID score of PPC in Table 3?\n5. Is PPC strictly better than other PC algorithms, or is there some kind of tradeoff?", "limitations": "The authors discuss some limitations (line 296). I think they could have better discussed limitations related to whether neurons can perform the desired computations, and whether particle-based gradient descent is the only / most appropriate possibility here. I think possible tradeoffs related to (e.g., training) time could also be interesting to discuss.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The authors study an interesting topic and display a good grasp of the literature. There are a lot of interesting ideas referenced and discussed throughout.", "weaknesses": "My main concerns are that the core ideas of the paper are confusingly presented, that the 'biological plausibility' claim may be too strong, and that the model's performance is a bit underwhelming. \n\nA more minor concern is that figure real-estate is used strangely (why do 80 copies of the same picture take up a huge amount of space in Fig. 4?). It's 'cool', but why is the cortical microcircuit figure (Figure 1) there? It doesn't seem like any of its details are referenced in the main text. Much more helpful would be a diagram of the proposed PPC algorithm, or better yet, two side-by-side diagrams comparing the proposed PPC algorithm to a more 'vanilla' PC algorithm. \n\nA very minor quibble: \"PPC\" in neuroscience is already used to refer to \"probabilistic population codes\" (see Ma et al. 2006) as well as the \"posterior parietal cortex\", so using PPC here is a bit confusing.\n\nAnother relatively minor point. Does vanilla PC really *require* Gaussian densities and the Laplace approximation (Table 1)? My understanding is that the framework itself doesn't *require* this; rather, this is just a useful assumption people use to get something simple and workable. I could be wrong, though. Discussing this point in the paper would be very helpful.\n\n**Confusing presentation.** The meat of the work is in Sections 2 through 4, and I found these sections confusingly written. In Sections 2 and 3, this is probably in part because the authors have to both (a) discuss standard PC in detail, and (b) introduce their modifications to the standard idea. The way things were written made it hard for me to tell what's 'new' and what's not, so it would be extremely helpful if the sections of the text were very obviously separated between 'this is old PC' and 'this is a new modification to PC we are proposing'. As mentioned above, a diagram of vanilla PC vs the proposed modification to it would greatly help. \n\nIn a few places, the motivation for doing things a certain way is also not quite clear. For example, PPC involves using particle-based gradient descent; why? Why this particular approach to empirical Bayes and not some other one? Are there good reasons to believe this to be a good basis for a 'local' solution to empirical Bayes? This does not appear to be discussed in the text. Put differently, is it better to think of this as a *possible* way the brain could implement empirical Bayes, or for some reason a *really good candidate* for that?\n\n**Biological plausibility.** This term is fraught and my view here is one that others may not agree with. But I think the authors are overclaiming here when they discuss biological plausibility. While they acknowledge that it means different things to different people (line 145), I still think this is insufficient. \n\nThe authors really mean *local computations* when they talk about biological plausibility, as they make clear between lines 145 and line 152. But biologically plausible neural computations involve neurons, and hence I think it is reasonable to expect that there is some discussion of whether or not neurons can implement the computations required. Locality is a start, but it is not the end, and in general it can be quite a bit of work to convert the steps of an algorithm into something neurons can plausibly do. There are references suggesting that neurons *might* be able to learn the required log-density gradients (line 175), but this is more of an afterthought than a core part of the contribution of this work, and these thoughts are not well-developed. Without a more specific proposal for how neurons can learn log-density gradients, and some in-silico validation that the proposed strategy works, I really think the term \"biological plausibility\" should be removed in many places, or else used with extreme caveats included near each use. \n\nFinally, part of the goal of developing 'biologically plausible' algorithms is to make predictions about biology. What are the specific predictions here? If there are some interesting ones, how do they compare to the predictions of other variants of PC? This should be explicitly discussed.\n\n**Underwhelming performance.** First, I found the tasks used in Sec. 5 confusingly described. It would be helpful if the details of the reconstruction and generation tasks, as well as the training procedure, were better explained in the main text. Second, there are not that many comparisons shown in Table 2; are there other alternatives PPC could be compared to? Third, in Table 3, PPC's FID score is shown as a question mark; I am not sure if this is an error. I assume \"undefined\" would not be good. \n\nFinally, there are some issues related to training time and speed. Can plots analyzing these features be included? How do other PC approaches compare on these axes? (Is there some time - performance tradeoff, for example?)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the time and valuable feedback.\n\n> Does vanilla PC really require Gaussian densities and the Laplace approximation (Table 1)? My understanding is that the framework itself doesn't require this; rather, this is just a useful assumption people use to get something simple and workable. I could be wrong, though.\n\nWe searched the literature for a definition that would clarify whether vanilla PC requires or merely uses the Laplace approximation and Gaussian densities. The definition we found in Salvatori et al 2023, now included and discussed in the manuscript, says that they are required. On a somewhat more in-depth level, Bogacz 2017 and a number of papers by Friston appear to imply that the Free Energy Principle literature first constructs the VFE/ELBO as is typically done in machine learning, and then applies the Laplace approximation and Jensen’s Inequality to the VFE itself.  The resulting energy function provides a looser bound on the model evidence than the usual ELBO in machine learning, but (seemingly vitally to this earlier work) saves the resulting PC schemes from having to perform any Monte Carlo computations. Our use of up-to-date SMC and variational inference methods to construct our VFE/ELBO and evaluate its gradients appear a required ingredient for relaxing PC’s Laplace assumption.\n\n> The authors really mean local computations when they talk about biological plausibility … But biologically plausible neural computations involve neurons, and hence I think it is reasonable to expect that there is some discussion of whether or not neurons can implement the computations required.\n\nWe agree that the term “bio-plausibility” has been loosely adopted in the literature, often indeed to refer to purely local computations. In our case, we consider our list of minimal properties to consist of local computations and the lack of a global control or synchronization signal. As much of the field refers to predictive coding models as biologically plausible, we follow the mainstream view and keep this terminology for our variation upon PC. We have also added a discussion about biological plausibility in the manuscript that specifies where we are referring to local computations, as well as references to a number of papers showing that neurons across a variety of model organisms (up to and including primate cortex) can calculate derivatives of logarithms of underlying signals. We thank the reviewer for the opportunity to go into detail on an underappreciated question in the study of predictive coding algorithms.\n\n> There are some issues related to training time and speed. Can plots analyzing these features be included?\n\nYes. Given some time we can retrieve that data from the training logs and plot it. At an approximate estimate, since PPC has to load and unload per-batch particles from GPU memory at every iteration, it takes time to train that is more on par with minibatched mean-field variational approaches than amortized variational approaches, approximately 4x the time of amortized.  This gap may well shorten when working with “full” datasets instead of minibatching, as in cognitive modeling rather than deep learning tasks.\n\n> Can the authors briefly summarize the 'new' features of their algorithm relative to other PC ideas?\n\nRelative to contemporaries Langevin Predictive Coding and Monte Carlo Predictive Coding, to which we compare, we build a Gibbs sampling method out of the interpretation of predictive coding as gradient-based sampling. Relative to Pinchetti 2022, the first to generalize PC beyond Gaussian distributions and the Laplace assumption, we optimize a globally valid free energy/evidence lower bound, rather than a layer-wise one.  Relative to all previous PC methods, we optimize a tighter bound on the model evidence (by using Monte Carlo sampling instead of applying Jensen’s Inequality a second time under the Laplace assumption) and support approximate posterior distributions with no closed form at all, well beyond Gaussian approximate posteriors.\n\n> Why particle-based gradient descent as opposed to some other approach to empirical Bayes?\n\nOur motivation for choosing a gradient-based sampling method comes from our analogy between score functions in probability and prediction errors in neuroscience. We chose PGD in specific because it was a recent gradient-based particle method, and we chose particle methods out of a combination of neuroscientific and behavioral motivations: brains have been observed to solve non-conjugate Bayesian inference tasks. Also, compared to plain variational inference, both mean-field and amortized, particle methods achieve better likelihoods.  Tentatively, as one of our VAE baselines runs, it appears to be converging to a log-likelihood two orders of magnitude (10^2) away from what PPC achieves on the same experiment. We can give similar gaps for unpublished experiments.\n\n> Concretely, how might neurons learn log-density gradients?\n\nWe have included the below text in our revised manuscript:\nBiological neurons often spike to represent changes in their membrane voltage, and some have even been tested and found to signal the temporal derivative of the logarithm of an underlying signal. Theorists have also proposed models under which single neurons could calculate gradients internally. In short, if neuronal circuits can represent probability densities, as many theoretical proposals and experiments suggest they can, then they can likely also calculate the prediction errors used in DCPC.\n\n> Is PPC strictly better than other PC algorithms, or is there some kind of tradeoff?\n\nThere are two trade-offs: classical PC algorithms, being deterministic, can run more quickly than PPC/DCPC, and yet for the same reason, PPC/DCPC gives a far more expressive approximation to the true posterior distribution. This is the bullet we bite using particle algorithms and empirical distributions/particle clouds to represent the posterior.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "L8dIlSa3ob", "reviewer": "Reviewer_DHBv", "summary": "SETTING: *Biologically plausible* EM with variational inference in directed graphical models.  In particular, the authors aim to implement predictive coding with local parameter updates for learning.\n\n\nAPPROACH: \"Variational\" EM with sampling (sequential Monte Carlo, SMC) in place of a separate inference/recognition model---a method called \"particle gradient descent\" (PGD, Kuntz et al., AISTATS 2023).\n\nMore precisely, following Neal & Hinton, the authors formulate the objective as minimizing the free energy (i.e. maximizing the ELBo), an upper bound on the marginal cross entropy (the fit of the model to the data).  In place of the recognition model (as in Neal & Hinton, VAEs, etc.), PGD implements Langevin dynamics (LD) in latent space.  More precisely still, a single step of LD is interpreted as providing a (Gaussian) proposal density, for use in importance sampling, with the unnormalized posterior of the generative model as the target distribution.  The importance weights are then used to resample the particles (making this into an SMC algorithm).\n\nTo handle structured probabilistic graphical models, this entire procedure is embedded into a Gibbs sampler.  That is, the particle representation for a particular latent variable is computed with all other latent variables fixed; the sampling procedure is repeated for each latent variable in turn; and *this* entire procedure, which constitutes a single step of Gibbs sampling, is repeated S times.  Then the free energy and model parameters are updated once.  *This* entire update procedure is itself repeated T times, with the LD for each particle continuing from where it left off.\n\nThe MS aims to produce a biologically plausible algorithm, and identifies its steps with biologically plausible operations:\n        \n--The Gibbs sampling is proposed to be coordinated with cortical oscillations.\n        \n--Each cortical column is proposed to encode an individual latent variable, with columnar connectivity reflecting the structure of the directed graph.  Thus columns receive their parents' current (sampled) state via L1 or L6 in order to compute conditional probabilities; locally calculate the energy gradients, and pass it \"up\" (L2/3 ->) to their parents (-> L4) for accumulation.\n        \n--Parameter updates can all be performed locally.\n        \n--Computation of the joint from the local complete conditionals is coordinated by phase-locking to cortical oscillations (at some other frequency?).", "questions": "--It is difficult to determine precisely what aspects of the method are new and not from published work (especially Kuntz 2023, Kuntz 2024, Naesseth 2015).  Can the authors clarify this?  (I have only read these papers cursorily.)\n\n--Is the mapping to the cortical microcircuit and other neurological constraints supposed to be the (or a) main result of the MS?  These are certainly intriguing but they are not very constraining.  Can the authors make predictions for neuroscience experimentalists based on their mappings?  Can they explain empirical findings in the literature?\n\n--Is it necessary to write the free energy in terms of weights?  Why not just let the recognition model be the product over the normalized \"complete conditionals\" (with the normalizers computed with Langevin dynamics)?\n\n--The authors propose cortical oscillations as the mechanism for synchronizing a parallel form of Gibbs sampling (that is still guaranteed to have the right stationary distribution).  At what frequency do they hypothesize these oscillations to be?  Given a reasonable number of Gibbs sweeps, does this let the computation happen fast enough?\n\nMore generally, can the authors give some more detail on the time complexity of the algorithm and how they would expect it to scale to problems with (say) real graph structure?\n\n--Proposition 5 is repeated as Corollary 5.1, which then references Proposition 5 (should be 3).\n\n--\"PPC\" is already the name of a (very popular) proposed representation of probability distributions in populations of neurons (\"probabilistic population codes,\" Ma et al., Nature Neuroscience, 2006).  Perhaps the authors can invent another name.", "limitations": "N/A", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The algorithm provides an interesting approach to spatially localizing computations, a prerequisite for mapping to (real) neural circtuits.  Models learned with the algorithm can reconstruct images from simple data sets.", "weaknesses": "This MS has two major weaknesses:\n\n(1) The results are very thin/missing.  Indeed, the only results reported in the whole MS are reconstructions (Figs. 2 and 5, Table 2), and these are quantified only for MNIST (and cousins).  They are certainly not close to the performance of modern generative models.  \n\nSeveral additional results or their descriptions are missing:\n        \n--Table 3 is missing the FID score!\n\n--There is no Figure 3.\n\n--Figure 4 is not referenced in the text.\n  \n--Figure 5 refers to \"top\" and \"bottom,\" but there are four rows of images.  (And it is not clear visually which images are reconstructions of which other images.)\n\nI also expected to see an example with a graphical model that has some actual structure, since this is part of the appeal of the algorithm, but there is none.\n\n(2) It is difficult to determine precisely what aspects of the method are new and not from published work (especially Kuntz 2023, Kuntz 2024, Naesseth 2015).  If the algorithm exists in prior work and the contribution of this MS is supposed to be the mapping to biological circuits, then it does not provide much novelty at all (more on this in the questions below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the time and valuable feedback.\n\n> Novelty over previous work (Kuntz 2023;2024, Naesseth 2015)\n\nKuntz 2023 did not construct or evaluate importance weights, while Lindsten 2017/Kuntz 2024 and Naesseth 2015 did not propose a unique decomposition of a target model into Gibbs kernels as target densities for samplers, nor did they employ gradient-based proposals. Our DCPC algorithm starts from a neuroscientific motivation to derive a new sampling algorithm: first approximate the true Gibbs kernels using gradient-based proposals and SMC, then take the nested D&C-SMC step “up” to calculate importance weights for the entire joint density, then take the pathwise derivative of their negative logarithm to estimate gradients of the complete generative model’s free-energy. Finally, as far as we know, ours is the first sampling-based predictive coding algorithm to compute a free energy that properly upper-bounds the surprisal of the sensory data, as required by the Free Energy Principle and achieved in neuronal message-passing proposals.\n\n> Is the mapping to the cortical microcircuit and other neurological constraints supposed to be the (or a) main result of the MS? These are certainly intriguing but they are not very constraining. Can the authors make predictions for neuroscience experimentalists based on their mappings? Can they explain empirical findings in the literature?\n\nThe mapping to the cortical microcircuit is not intended to be the main result of the manuscript, only a suggestion. The primary contribution is the algorithm itself, which should stand or fall on its own.\n\n> The authors propose cortical oscillations as the mechanism for synchronizing a parallel form of Gibbs sampling. At what frequency do they hypothesize these oscillations to be? Given a reasonable number of Gibbs sweeps, does this let the computation happen fast enough?\n\nThe γ-band of cortical oscillations take place at 30–150 Hz and tend to correlate with bottom-up processing of oddball or deviant stimuli, in contrast to the top-down alpha/beta band in the 8–30 Hz range.  “High gamma” past 50 Hz is often more associated with “prediction error”. Without having the experimental evidence to break down gamma more finely, the oscillation frequency band we can associate with “prediction error” is faster than the frequency band associated with “prediction”. Picking the median of each frequency band, median alpha/beta would be 19 Hz while median gamma would be 90 Hz, more than 4x faster. We do suggest that if inference takes place 4x faster than generative posterior prediction, with a reasonably efficient inference algorithm, inference can converge fast enough.\n\n> More generally, can the authors give some more detail on the time complexity of the algorithm and how they would expect it to scale to problems with (say) real graph structure?\n\nAbstracting over the time complexity of automatic differentiation and SMC, our algorithm has linear asymptotic complexity in the number of nodes in a graph, and then multiplicative factors in the number of sweeps and the total number of inference steps in the training loop. Further asymptotic guarantees would require details of model structure (ie: log-concavity, etc).\n\n> The results are very thin/missing.\n\nWe have now updated the manuscript, and added multiple, generative experiments on the Celeb64 dataset. Note that, while the results are not as good as the ones of modern generative models, they perform as well (and sometimes outperform) these of bio-plausible learning algorithms, such as other predictive coding networks such as Oliver 2024: https://www.biorxiv.org/content/10.1101/2024.02.29.581455v1.full\nand the particle gradient descent antecedent to ours. For more details about the experiments, we refer to the paragraph Numerical and Figure Results on CelebA, provided as a general answer to all the reviewers.\n\n> Is it necessary to write the free energy in terms of weights? Why not just let the recognition model be the product over the normalized \"complete conditionals\" (with the normalizers computed with Langevin dynamics)?\n\nIt is indeed mathematically necessary to write the free energy in terms of importance weights targeting the generative joint density. By definition, a variational free energy is the expected value of the negative logarithm of an importance weight.  Equivalently, the VFE is the cross-entropy of the generative joint distribution, taken with respect to the proposal distribution (recognition model), minus the entropy of the recognition model.  The product of normalized complete conditionals (with the normalizers estimated by importance sampling) would only estimate the entropy of the recognition model, the second term.\n\n> Typos\n\nThank you for the pointers, they have now been addressed in the final version of the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9ESGerWQvl", "reviewer": "Reviewer_Cx3n", "summary": "This paper proposes a version of predictive coding that is biologically plausible (in the sense that all computations are local), and can perform both classification and generation (by operating in both directions). This version of predictive coding, dubbed Population Predictive Coding (PPC), overcomes the limitations of traditional PC by enabling a more general class of distributions in the latent layers.", "questions": "I would have liked to see the classification accuracy of the model on MNIST (as a percent). Predictive coding networks tend not to perform well on classification tasks.", "limitations": "Yes", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The work seems to have a rigorous probabilistic coverage of predictive coding.", "weaknesses": "Some notation needs to be defined to help the reader. For example:\n- eqn (1): the function Pa is not defined.\n- eqn (5): Ch(z) is not defined. I infer from Algorithm 1 that it probably means \"children of\", but it should be made clear when it is first used.\n\n\nAdditionally, there are a few things that could be fixed.\n- There is no figure 3.\n- figure 4 has 4 rows, but the caption only states what is in the \"Top\" and \"Bottom\". Please clarify.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the time and valuable feedback.\n\n> would have liked to see the classification accuracy of the model on MNIST (as a percent). Predictive coding networks tend not to perform well on classification tasks. \n\nIt is true that, when used as generative models, predictive coding networks do not perform well on image classification tasks. However, when used in a discriminative direction (where the goal is to generate the label, providing the image as a very precise (Delta) prior on the first layer), PC has been shown to perform as well as backprop in complex image classification tasks:\n\nhttps://arxiv.org/abs/2407.01163\n\nHowever, the study of discriminative PCNs is not the focus of our work, and we have hence not considered it in our tests. \n\n> Typos\n\nThank you for the pointers, they have now been addressed in the final version of the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank the four reviewers for their close and detailed engagement with our manuscript, stretching across the computational neuroscience material and the core contributions on Bayesian inference.

We see the reviewers divided on score but in broad agreement on the contributions, and needs for improvement, in the paper. Overall, the reviewers agree on the technical soundness of the PPC algorithm, with R1 (jNdT) recognizing the connections to recent work in Sequential Monte Carlo, R2 (AbQA) appreciating the application of particle gradient descent, R3 (DHBv) seeing links to variational EM, and R4 (Cx3n) recognizing that Bayesian inference can apply to both discriminative and generative tasks. The reviewers also share our view that biological plausibility is a desirable goal to achieve in an inference or training algorithm.  Finally, they would all like to see the core technical sections of the paper significantly clarified. For all the reviewers, we now have an [anonymized Github repository](https://anonymous.4open.science/r/ppc_experiments-8AF5/README.md). To address some concerns, especially the ones regarding clarity and typos, we have decided to append a [new version of the manuscript](https://anonymous.4open.science/r/ppc_experiments-8AF5/neurips2024_population_predictive_coding.pdf) in the anonymous link. We will describe the changes below, as well as addressing the common concerns. We will start with the easiest concern to answer and then move on to the less trivial matters.

> Name PPC clashes with an existing acronym

We thank the reviewers for the reminder to pick a name that does not clash with “probabilistic population codes/coding”, particularly since both “old” PPC and “our” PPC concern Bayesian inference in the cortex.  We have taken the suggestion to simply change the name, instead choosing “Divide & Conquer Predictive Coding” to emphasize the connections with Gibbs sampling and [D&C-SMC](https://projecteuclid.org/journals/annals-of-applied-probability/volume-34/issue-1B/The-divide-and-conquer-sequential-Monte-Carlo-algorithm--Theoretical/10.1214/23-AAP1996.short). We have updated our manuscript accordingly in all usages of the name and acronym.

> Numerical and Figure Results on CelebA

We thank the reviewers for pointing out the broken figure links and typos in our initial manuscript submission.  The supplementary page to this response provides a fixed results figure for the generator network on CelebA experiment at 64x64 resolution, showing reconstructions on the left and de novo generations/samples on the right. It also includes a consolidated table of FID scores, showing that in an apples-to-apples comparison with particle gradient descent, PPC achieves a clear and significant improvement in FID. The PPC FID score remains worse as against Langevin Predictive Coding, because we have only recently managed to clarify with the authors their neural architecture, momentum-based optimization method, and likelihood function. Their [“discretised Gaussian” likelihood](https://github.com/lucidrains/denoising-diffusion-pytorch/blob/ec0a1c7596f654d9b5d3952a63ce3301128f1979/denoising_diffusion_pytorch/learned_gaussian_diffusion.py#L43) has been noted to produce sharper images and better FID scores than the continuous Gaussian log-density that we employed in training.  Running an apples-to-apples experiment will require implementing the “discretised Gaussian” as a Pyro distribution and thus take time into the reviewer discussion period, though we will of course do so. Finally, the neural architecture used for LPC here turns out to have come from the original Beta-VAE with slight modifications (nonlinearities and the likelihood function), and so we can train a Beta-VAE on the same problem to disentangle the training algorithm from the underlying neural architecture.

> Clarifying the technical presentation and novelty

R1 would like to see additional diagrams, examples, and intuitive explanations in Sections 2 and 3 of the paper. R2 appreciates the difficult task facing us in summarizing work on empirical Bayes, predictive coding and our novel contributions in these core sections, but would find it helpful if the sections more obviously separated between old and new contributions. R3 would like clarification on which aspects of the DCPC/PPC method are novel to our work specifically, rather than applications of the previous literature. We have endeavored to clarify Section 2 by separating it into clear paragraph-marked subsections for “Empirical Bayes”, “Predictive Coding”, and “Particle Algorithms”. The middle subsection includes an informal, but firm, definition of “predictive coding” taken from Salvatori et al 2023, which allows us to compare and contrast DCPC/PPC with that definition, thus with the existing literature, in Section 3. We have also endeavored to clarify Section 3 and the construction of the DCPC algorithm in several ways, including discussing how DCPC fits, in some ways, and extends, in others, the definition of predictive coding from Section 2.

> Clarifying the novelty of the algorithm vs microcircuit hypothesis

The reviewers point out that it is difficult from the submitted manuscript to determine which aspects of the PPC method are new, versus coming from published work, and thus whether the PPC paper’s core contribution is the algorithm itself or its mapping onto neuronal circuits. We end our clarifications of Section 2 by discussing the limitations in applying existing particle algorithms, samplers, and variational methods in targeting the joint density of a large-scale, structured graphical model, the better to motivate our novel structured PC algorithm. In the algorithmic details, Kuntz 2023 did not construct or evaluate importance weights, while Lindsten 2017/Kuntz 2024 and Naesseth 2015 did not propose a unique decomposition of a target model into Gibbs kernels as target densities for samplers, nor did they employ gradient-based proposals.

Meta-review: The paper presents a novel algorithm, Population Predictive Coding (PPC), which aims to enhance both the performance and biological plausibility of predictive coding by incorporating  Monte Carlo sampling techniques. The reviewers generally agree that the paper makes a meaningful contribution to the field, offering a fresh approach that integrates ideas from machine learning and neuroscience to address the limitations of existing predictive coding methods. While the reviewers raise concerns about the complexity of some explanations, missing results, and the need for clearer documentation and presentation, the authors have effectively responded to these critiques by clarifying their contributions, updating the manuscript (particularly regarding biological plausibility), and providing additional experimental results. Although the overall connection with neural circuitry is intriguing, the paper would benefit from a more direct comparison with neural data. Nevertheless, given the technical soundness, moderate-to-high impact, and the authors' responsiveness to feedback, the paper represents a valuable advancement in understanding predictive coding and structured Bayesian inference.
Decision: Accept (poster)