Benchmarking Predictive Coding Networks -- Made Simple
Authors: Luca Pinchetti
Chang Qi
Oleh Lokshyn
Cornelius Emde
Amine M'Charrak
Mufeng Tang
Simon Frieder
Bayar Menzat
Gaspard Oliviers
Rafal Bogacz
Thomas Lukasiewicz
Tommaso Salvatori
Venue: ICLR 2025 Spotlight (2025)
Keywords: cognitive science
predictive coding
computational neuroscience
Primary area: applications to neuroscience & cognitive science

Abstract: In this work, we tackle the problems of efficiency and scalability for predictive coding networks (PCNs) in machine learning. To do so, we  propose a library that focuses on performance and simplicity, and use it to implement a large set of standard benchmarks for the community to use for their experiments. As most works in the field propose their own tasks and architectures, do not compare one against each other, and focus on small-scale tasks, a simple and fast open-source library, and a comprehensive set of benchmarks, would address all of these concerns. Then, we perform extensive tests on such benchmarks using both existing algorithms for PCNs, as well as adaptations of other methods popular in the bio-plausible deep learning community. All of this has allowed us to (i) test architectures much larger than commonly used in the literature, on more complex datasets; (ii) reach new state-of-the-art results in all of the tasks and dataset provided; (iii) clearly highlight what the current limitations of PCNs are, allowing us to state important future research directions. With the hope of galvanizing community efforts towards one of the main open problems in the field, scalability, we will release the code, tests, and benchmarks.

Published as a conference paper at ICLR 2025
BENCHMARKING PREDICTIVE CODING NETWORKS
- -MADE SIMPLE
Luca Pinchetti' Chang Qi, Oleh Lokshyn', Gaspard Oliviers?, Cornelius Emde', Mufeng Tang", Amine M'Charrak', Simon Frieder', Rayar Menzat,, Rafal Bogacz', Thomas Lukasiewicz"11, Tommaso Salvatori
'Department of Computer Science, University of Oxford, Oxford, UK
Instituue of Logic and Computation, Vienna University of Technology, Vienna, Austria 3 MRC Brain Network Dynamies Unit, University of Oxford, UK AVERSSS AI Research Lab, Los Angeles, US
ABSTRACT
In this work, we tackle the problems of efficiency and scalability for predictive coding networks (PCNs) in machine learning. To do so, we propose a library, called PCX, that focuses on performance and simplicity, and use set of standard benchmarks for the community to use for their experiments. As most works in the field propose their own tasks and architectures, do not compare one against each other, and focus on small-scale tasks, a simple and fast open- source library and a comprehensive set of benchmarks would address all these concerns. Then, we perform extensive tests such benchmarks using both existing algorithms for PCNs, as well as adaptations of other methods popular in the bio- plausible deep learning community. All this has allowed us (i) test architectures much larger than commonly used in the literature, on more complex datasets; (ii) reach new ofate-of-the-ar results in all of the tasks and datasets provided; (iii) clearly highlight what the current limitations of PCNs are, allowing us to state important future research directions. With the hope of galvanizing community efforts towards one of the main open problems in the field, scalability, we release code, tests, and benchmarks
implement a large
INTRODUCTION
In 1999, Rao Ballard (999) proposed formulation of predictive coding (PC) to model hierarchical information processing in the brain. It was recently realized that this framework could be used to train neural networks using a bio hio-pluusible learning rule (Whittington & Bogacz 2017). This has led to different research directions, whose focus either to explore interesting properties of PC networks (Song et al. 2024 Alonso et al. 2022), or to propose variations that improve the performance on specific tasks "Salvtoor have the tendency of not comparing their results against other works, and to focus on small-scale experiments. The field is hence avoiding what we believe to be the most important open problem: scalability.
al., [024] Ororbia & Kifer 2022). These lines of research, however,
There multiple reasons why the problem of scalability has been overlooked. First, it is hard problem, and it is still unclear why so far PC has been able to perform as well as classical gradient descent with backpropagation (BP) only up to certain scale, which is that of small convolutional models trained to classify the CIFAR10 dataset (Salvatori et al., 2024). Understanding this would allow us to develop regularization techniques that stabilize learning, and hence allow better performance on more complex tasks. Second, the lack of specialized libraries makes PC models extremely slow: a full hyperparameter search on a small convolutional network can take several hours. Third, the lack of common framework makes reproducibility and iterative contributions hard, as implementation details or code are rarely provided. In this work, we make the first steps toward addressing these problems with three contributions, that we call tool, benchmarking, and analysis.
"Corresoonding author: tommaso. sa waw.amarrossaaaaoiiiiiiio vatori ' Link to the library: / www.vvarrrrrree.... @verses.ai iwwidar/iiddaa

---

Published as a conference paper at ICLR 2025
Tool. We release an open-source library for accelerated training for predictive coding called PCX. This library runs in JAX (Bradbury et al.] 2018), and offers user-friendly interface with a minimal learning curve through familiar syntax inspired by Pytorch. We also provide extensive tutorials. It is also fully compatible with Equinox (Kidger & Garcia 2021), a popular deep-learning- oriented extension of JAX, ensuring reliability, extendability, and compatibility with ongoing research developments. It also supports JAX's Ins-In-Timme (JIT) compilation, making it efficient and allowing both easy development and execution of PC networks, gaining efficiency with respect to existing libraries.
Benchmarking. We propose a uniform set of tasks, datasets, metrics, and architectures that should be used as a skeleton to test the performance of future variations of PC. The tasks that we propose are the standard ones in computer vision: image classification and generation. The models that we use, as well as the datasets, are picked according to two criteria: First, to allow researchers to test their algorithm from the easiest task (feedforward network on MNIST) to more complex ones; Second, to favor the comparison against related fields in the literature, such equilibrium and target propagation (Scellier& & Bengio, 2017; Bengio 2014). To this end, we have picked some of the models that consistently used in their research papers. As learning algorithms, we consider standard PC, incremental PC (Salvatori et al.) 2024), PC with Langevin dynamics (Oliviers et al. 2024). and
nudged PC, as done in the Hqprop literature (Scellier& & Bengio, 2017 [Seellier al. 2024). Note
that this is the first time nudging algorithms are applied in PC models.
Analysis. We get ofate-of-the-art (SOTA) results for PC on multiple benchmarks and show for the first time that it is able to perform well on more complex datasets, such as CIFARI00 and Tiny Imagenet, where we get results comparable to those of backprop. In image generation tasks, we present experiments on datasets of colored images, going beyond MNIST and FashionMNIST as performed in previous works. We thoroughly discuss the results and highlight areas of improvement, the main one being generalization to very deep models, and report analysis on the credit assignment of PC in such cases, to better understand the reasons behind some failures. To conclude, in the that
supplementary material we provide detailed explanation of byperppranttexx/ecnnnqgeetrtiiie allowed us to reach SOTA results, to also provide a cookbook for researchers in the field.
2 RELATED WORKS
Rao and Ballard's PC. The most related works are those that explore different properties or optimization algorithms of standard PC in the deep learning regime, using formulations inspired by Rao and Ballard's original work (Rao & Ballard, 1999). Examples are works that study their associative memory capabilities (Salvatori et al. 2021 Yoo & Wood 2022 Tang et al. 2023 2024). their ability to train Bayesian networks (Salvatori et al. [2022] explain, improve, their optimization process (Millidge et al. 2022a b Alonso et al. 2022). Results in this field have allowed either to improve the performance of such models in different tasks, or to study different properties that could benefit from the use of PCNs.
2023b). [002||203bb]
and thcoretical results that
Variations of PC. In the literature, there are multiple variations of PC algorithms. Important examples are biased competition and divisive input modulation (Spratling 2008). the neural generative coding framework (Ororbia & Kifer 2022). The latter is already used in multiple reinforcement learning and control tasks (Ororbia & Mali 2023) Ororbia et al. (023). and has its own JAX-based open source library called NGCLearn. For a review on how different PC algorithms evolved through time, from signal processing to neuroscience, we refer to (Spratling 2017): for a more recent review specific to machine learning applications, to (Salvatori et al.) 2023a). It is also worth mentioning the original literature on PC in the neurosciences has evolved from Rao and Ballard's work into general theory that models information processing in the brain using probability and variational inference, called the free energy principle (Friston 2005 Friston & Kiebel| 2009
Friston 2010).
Neuroscience-inspired deep learning. Another line of related works is that of neuroscience methods applied to machine learning, like equilibrium propagation (Scellier & Bengio (0077), which is the most similar to PC (Laborieux & Zenke, 2022; Millidge et 20222)) Other methods able to
train models of similar sizes target propagation (Bengio) [01uul et al. 2022 Millidge et al.
2

---

Published as a conference paper at ICLR 2025
2022b) and SoftHebb (Moraitis et al.) 2022 Journé al. 2022). The first two communities, that of targetprop and eqprop, consistently use similar architectures in their research papers to test their methods. In our benchmarking effort, some of the architectures proposed are the same ones, to favor a more direct comparison. There are also methods that differ more from PC, such as forward-only 2022), and methods that back- pack-propagate the errors using a designated set of weights (Lillicrap et al. 2014 Launay ct al. 2020).
methods (Kohan et al. 2023 Nokland, 2016; [Hinton
3 BACKGROUND AND NOTATION
Predictive coding networks (PCNs) are hierarchical Gaussian generative models that consist of L levels. Each level models multi-variate distribution, parameterized by the activation of the preceding level, which depends on both the model parameters fb 01, .2.,.... O2 and the model state Let hi h be the realization of the vector of random variables H; of level I, then we have that the likelihood
Po (ho hi....,,y!) ) Po, Ph, (hg Po,, (helhh--))
Where we write Pe, (14) instead of Pe. (Hr hı), that is the likelihood of Hj evaluated at ht. We refer to each of the scalar random variables of Hi as neuron. In PC both the prior on he and the relationships between levels are governed by normal distribution parameterized as follows:
Pe. (ho) = Nho. no: 2o);
μi 10=00 00. Ps. (hy]//-1) = (hi A fi
where Or are the learnable weights parametrizing the transformation fi, and Eg a covariance matrix, that will be fixed to the identity matrix throughout this work. If, for example, 81 = (W,, by) and f( 1 0t) = σy (Wihh-1 + by), then the neurons in level 1- are connected to neurons in level via a linear operation, followed by non-linear map, analogously to fully connected layer. Intuitively, is the set of learnable weights of the model, while = (ho: h1...., h2} cett--pointddeendeen latent state, containing the abstract representations for the given observations.
supervised settings, training consists of learning the relationship between given pairs of input-output observations (I,, v). In PC, this is performed by maximizing the joint likelihood of our generative model with the latent vectors hg and h2 respectively fixed to the input and label of the provided data-point: Po(h) ho==, minimizing the so-called variational free energy F (Friston et al. 2007):
Training.
Pa(hz ...... h1, hog I). This is achieved by
1 F(h.0) 0) = - Po(h)) = - Inn (holll) II (he f((h)
Σ 2 (hx 11) + k. (1)
1=1
1=0
The quantity e (hr ) is often referred to as prediction error of layer I. being the difference between the predicted activation μr and the current state hy. For a full derivation of Eq. (I) we refer to the appendix. To minimize F, the used by iteratively optimizing first the state h, and then the weights according to the equations
(EM) (Dempster et al.) 1977) algorithm
h" argmin, F(h, 0), 6" argmin,
We refer to the first step described by Eq. (2) inference and to the second as learning phase. In practice, we do not train on a single pair (xr,) y) but on dataset split into mini-batches that are subsequently used to train the model parameters. Furthermore, both inference and learning approximated via gradient descent on the variational free energy. the inference phase, firstly is initialized to an initial value (; and then, it is optimized for T iterations. Then, during the learning phase, we use the newly computed values to perform a single update on the weights 0. The gradients of the variational free energy with respect to both and are as follows:
OF Th
de Oly
dett1
OF 1 10e
V
VO, = 00,
2
Thu
2 30,
Then, a new batch of data points is provided to the model and the process is repeated until convergence. As highlighted by Eq. (3) each state and each parameter is updated using local information as the gradients depend exclusively on the pre and post-synaptic errors 6; and c This is the main reason why, in contrast to BP, PC is local algorithm and is considered more biologically plausible. In Appendix A we provide an algorithmic description of the concepts illustrated in these paragraphs, highlighting how each equation is translated to code in PCX.
3

---

Published as a conference paper at ICLR 2025
(a)
Disac Delta
(b)
Learning a dattgoinn (2.9) with PC
In vocter Ry fined to
Spuce
Require: he is diod to x; lixed to y,
target 9 Ak 00p Mo). instad of
N2
for = 1 to (inclided) do
h2
for each laiest waser
Updaze minittize
Uplatc end
minimise
In targee & Alg insicad of and sign weightt updase Inveried:
flxed
end for and for
9, -66
In IPC, cach 6; updazed at every timestep.
Obecrations
000100 y
MCPC, the update of every Intent variable cocrupted via the addilion of Goussian noise g~ A700, y23)
Generative Mode
Diseriminative Mode
Figure I: (a): Generative and discriminative modes; (b): Pseudocode of PC in supervised learning, where both the latent variables by and the weight parameters O2 are updated to minimize the variational free energy F. In the colored boxes, informal description of the different algorithms considered in this work.
Evaluation. Given a test point σ. we fix ha = E. and compute the most likely value of the latent states ht lAg=2 again using the state gradients of Eq. (3). We refer to this as discriminative mode. In practice, for discriminative networks, the values of the latent states computed this way are equivalent to those obtained via a forward pass, that is setting h (0) 11
(0) for every [ + 0, as it corresponds to
the global minimum of F (Frieder & Lukasiewicz 2022
Generative Mode. PCNs can also be used to perform unsupervised learning tasks. Given a data
point x, the goal is to compress the information of into a latent representation, conceptually similar to how variational autoencoders work (Kingma & Welling 2013). Such compression is computed by fixing the state vector H the data point, and running inference that is, we maximize Po (h = via gradient descent on h. The compressed representation will then be the value of hn at convergence (or, in practice, after T steps). we are training the model, we then perform a gradient update on the parameters to minimize the variational free energy of Eq. (I). as we do in supervised learning. A sketch of the discriminative and generative ways of training PCNs is represented in Fig. Ia).
EXPERIMENTS AND BENCHMARKS
The benchmark that we propose is a standardized set of models, datasets, and testing procedures that have been consistently used to evaluate predictive coding, but in non-uniform way. Here, for a comprehensive evaluation, we test models of increasing complexity on multiple computer vision datasets, with both feedforward and www.llutionaadde-ccoooltiiioooii learning algorithms present in the literature. This section is divided into two areas that correspond to discriminative (supervised) and generative (unsupervised) inference tasks. For the former mode, we focus on supervised classification, and unsupervised generation for the latter. A sketch illustrating the two modes is in Fig. [ For every class of experiments, have performed large hyperparameter search, and the details needed to reproduce the experiments, as well as a discussion about 'lessons learned' during such large search, are in the Appendix B and C
layers; and multiple
To provide a comprehensive evaluation, we have tested on multiple computer vision datasets, MNIST (LeCun & Cortes, 2010), FNShionMNIST (Xiao et al., (017)). CIFARIO/100 (Krizhevsky et al. 2009). CelebA (Liu et al. 2018), and Tiny ImageNET (Le & Yangl 2015): on models of increasing complexity, and multiple learning algorithms present the literature. The results, averaged over seeds are reported in Tab. when we used discriminative models, and in Tab. f generative models. Note that, besides a very recent exception on CelebA (Sennesh et al.) (024). this the first time that PCNs with local message passing are tested on datasets such as CelebA, CIFAR100, and Tiny ImageNet.
Algorithms. We consider various learning algorithms present in the literature: (1) Standard PC, already discussed in the background section; (2) Incremental PC (iPC) (Salvatori et al. 2024).
simple and recently proposed modification where the weight parameters are updated alongside the latent variables at every time step: (3) Monte Carlo PC (MCPC) (Oliviers al. (024). obtained by applying unadjusted Langevin dynamics to the inference process; (4) Positive nudging (PN), where the target used is obtained by small perturbation of the output towards the original, 1-hot label; (5)
4

---

Published as a conference paper at ICLR 2025
Table 1: Test accuracies of the different algorithms on different datasets.
& Accuracy
PC-CE
PC-SE
PN
NN
CN
IPC
BP-CE
BP-SE
MI.P
MNIST
11
98.26400.4 89.58*0.15
88.3640000
98, 25..06
98, 2.000
98. 45:+ oo
98.0740.05
98, 28:20
FashionMNIST
89.16+0.08
89.5749.08
89.4640.08
89.56400005
89.90
89.04 #0.00
89.48+
VGG-5
CIFAR-10
86.61
87.9840.11
88.4249.00
88.8320.
89.47=0.1
85.51 tM
88.11 88.1140.11
89.43440.12
CIFAR-100 (Top-1)
60.00
51.08+1.06
61.70 25
65.46+0000
67.19=0.22
50.07 to
60. 60.82#5010
06,28++0.2
CIFAR-100 100 (Top-5)
84.9740011
78.7041.00
84.74499335
85.1540015
86.60=.011
78.91 0191120:2 23
85.84
85.8540007
Tiny ImageNett (Top-1) 1)
41 41.29
30.2849.
34.61 29.2
46.40=001
46.38
29.94 29.9440.47
49.7240.
44.904902
Tiny ImageNett (Top-5)
06.68 06.6840.00
57.31 +0021
30.91#0.24
68.50
69.00 +00.1
54.7340.32
69. 0.0.23=0.23
65.26
VGG-7
CIFAR-10
84.6220.1
81,91
85,9720.5
87.26
88.4422.12
80.15 18
88 60-60.00.
89.91 1:0.1
CIFAR-100 (Top-1)
86.80
37.5242260
56,56449.1
59.9729041
64,76 61.7671017
43, 03,99.00.0
59. 59.96-49-10
65. 36
CIFAR-100 (Top-5)
83.00
81.52#0.2 66.73+22.7
81.65 81.50=0.44
73. 73.23
85, 85.61 85.61=0.10 =0.10
811.1
Tiny ImageNet (Top-
41. 41.15
21.28:t0.66
25.533t9.17 ry
39.4912260
35.39
7.000
19. 19.76:t0. is
45.32.00.1
16:08
Tiny ImageNet (Top-5)
66.2549.11
41.924..77
50.06499.4
6-1.66 .s
59. 19.63366.0 0.000
40.3849 30.4..00 sm
69. 2..0.
66.65:49.20
VGG-9
CIFAR-10 10
78.12#0.14
75.33+
76.90
85.90+0.14
87. 87.19
79.0240.21
89.18
90. 90.02++0.18
CIBAR-100 (Top- 1)
58.25
39.57700.18
43.21 #0.21
60.7430.75
58.92 Lo1
44.76* 40
60.63350.25
65.51 200.2
CIFAR-100 (T)
83.2840006
66.99
71.1340..2
83.19
81.56+
72. 22.88+0.22 20
85, 85.25=00 =011
74.70
Tiny ImageNet (Top-
39.64+0.17
21.78
23.6240.25
41.59:
31.5 40.70
26, 31.34400.00 on
45.66
45.51
Tiny ImageNet (Top-5)
64.60
44.43300.00
46.89100.11
66.15-50.52
54.67
500.4820.06
69.66500.00
65.62
ResNet-18 18
CIFAR-10
43.1940.61
53.74490.4
02.45
62.33440.93
55,2941.66
70.44+0.81
92.83=0.18
93. 21.21 +00.07
CIFAR-100 (Top-1) I)
16.0149.42 12
22.8349.39
25.8649999 96
26.91 29.55
15.45
29.45 30
72.3220026
71.89:b0.16
CIFAR- 100 (Top-5)
40.6740.70
50.1849052
53.80
55.57
39.42428
92.1 92.1440.12 56.70
87.80
Tiny ImageNet (Top-1) 1)
09.5240032
14.19400.2
15.79+1.10
15.95
01. 01.40
06. 19=1001
58 688.00=0.28
56.30+00.6
Tiny ImageeNtt (Top-5)
26.2149050
34.55
37.3641.57
37.7620.52
4..90
16.51
79.94=n00
74.9840.30
Negative nudging (NN), where the target is obtained by small perturbation away from the target, and updating the weights in the opposite direction; (6) Centered nudging (CN), where we alternate epochs of positive and negative nudging (Scellier al. 2024). Among these, PC, iPC, and MCPC will be used for the generative mode, and PC, iPC, PN, NN, and CN for the discriminative mode. See Fig. I and the supplementary material, for a more detailed description.
4.1 DISCRIMINATIVE MODE
We test the performance of PCNs on image classification tasks by comparing PC against BP, using both Squared Error (SE) and Cross Entropy (CE) loss, by adapting the energy function as described in Pinchetti et al. (0022). For the experiments on MNIST and FashionMNIST, we use feedforward models with 3 hidden layers of 128 hidden neurons, while for CIFAR 10/100 and Tiny ImageNET, we compare ResNets and VGG-like models (He et al. 2016 Simonyan & Zisserman 2014).
Results. Table [ shows that the best performing algorithms, at Ieast on the most complex tasks, are the nudging ones (PN, NN, and CN). Among them, CN is almost always the best performing one, result that is in line with previous findings in the Eqprop literature (Scellier et al. 2024). The only case where nudging algorithms are outperformed on Tiny Imagenet on VGG7, where PC-CE performs better than them. However, the results obtained by PC-CE are still worse than the ones obtained by CN on VGG5. The recently proposed iPC, on the other hand, performs well on small architectures, as it is the best performing one on MNIST and FashionMNIST. but its performance worsens when it comes to the training of large architectures. More broadly, the performance of models of depth up to 7 is comparable to those of backprop. while those of deeper models lag behind.
Discussion on depth. An interesting observation is that all the best results for PC have been achieved using VGG5, with the performance trend being VGG5 > VGG7 > VGG9 > ResNet, as shown in Fig 2 Conversely, we observe the opposite for backprop-traines models, with deeper models like VGG9 outperforming VGG5. A similar trend was observed in ResNet18 experiments, where PCNs yielded significantly lower test accuracies, with none of the models coming close to the performance of VGG5. In contrast, backprop-trained ResNet18 models outperformed all previously tested VGG models, further emphasizing the gap in scalability between the two. Future work should investigate the reason of such phenomenon, as scaling up to more complex datasets will require the use of much deeper architectures. In Section we analyze possible causes, as well as comparing the wall-clock time of the different algorithms.
5

---

Published as a conference paper at ICLR 2025
PC-C2
PC-S2
(L.0%)
30%
27 2MG
86-55 25 37.
Alcunccy (h
78
53.7%%
90
WGG5 05677 VO6-4
15565 VOG-T WUG ReOOUT
Wirs V5677 vol Reck
vd6 Wi- Wid Wedie
WiSS Wir7 Voli
Figure 2: Test accuracies of different PC algorithms on the CIFAR1O 10 dataset, using models of different depths.
Table 2: MSE loss for image reconstruction of BP, PC, and iPC on different datascts.
MSE (x10 )
PC
iPC
BP
MSE (x10 ")
PC
iPC
BP
MNIST
9.25=#000
9.09=0.00
9.08=0000
CIFAR-10 10
6.67 2.35+0.12
5.50+0.01
6,17#0.46
MNIST
10.56+0.01
10.11 #0.01
10.04
CELEB-A
1.30
3.30
4.2
GENERATIVE MODE
In this section, test the performance of PCNs image generation tasks. We perform three different kinds of experiments: (1) generation from a posterior distribution; (2) generation via sampling from the learned joint distribution; and (3) associative memory retrieval. In the first case, we provide test image to a trained model, run inference to compute compressed representation こ (stored in the latent vector hg convergence). and produce a reconstructed g hı by performing forward pass with ho F). The models we consider have three layers, and we compare against autoencoders with a three-layer encoder/decoder structure (so, six layers in total). In the case of MNIST and MNSH we use feedforward layers, in the case of CIFAR 10 and CelebA (de-) dde-jconvolutional ones. The results in Tab. and Fig. report comparable performance, with a small advantage for PC compared to BP on the more complex tasks. In this case, iPC is the best performing algorithm, probably due to the small size of the considered models which allows for better stability.
Then, we tested the capability of PCNs to learn, and sample from, complex probability distribution. MCPC extends PC by incorporating Gaussian noise to the activity updates of each neuron. This change enables PCN to learn and generate samples analogous This change shifts the inference of PCNs from a variational approximation to Monte Carlo sampling of the posterior using Langevin dynamics. Data samples can be generated from the learned joint Po (h) by leaving all states hy free and performing noisy inference updates. Figure 4 iillustrates MCPC's ability to learn multimodal distributions using the iris dataset (Pedregosa et al.] (011) and shows generative samples for MNIST. When comparing MCPC to a VAE, both models produced samples of similar quality. MCPC achieved lower FID score (MCPC: 2.5310.17 vs. VAE; 4.198).38) whereas
variational autoencoder (VAE).
the VAE attained a higher inception score (VAE: 7.91 10.03 vs. MCPC: 1..1...
In the associative memory (AM) experiments, we test how well the model is able to reconstruct a training image, after provided with an in- complete or corrupted version of it, as done in a previous work (Salvatori et al. (02T). Fig. 5 show the results obtained by a PCN with 2 hid- den layers of 512 neurons given noise or mask corrupted images. In Tab. B we study the mem- ory capacity as the number of hidden layers in- creases. No visual difference between the recall and original images can be observed for MSE up to 0.005. To evaluate efficiency then trained a PCN with 5 hidden layers of 512 neurons on 500 Tinylmagenet samples, with a batch size of 50 and 50 inference iterations during training. Training takes 0.40 + 0.005 seconds per epoch on an Nvidia V100 GPU.
Mask
Noise
Figure 5: Memory recalled images. Top: Original images. Left: Noisy input (guassian noise, 0.2) and reconstruction. Right: Masked input (bottom half removed) and reconstruction.
Discussion. The results show that PC is able to perform generative tasks, well as associative memory ones using decoder-only architectures. Via inference, PCNs are able to encode complex probability distributions in their latent state which can be used to perform variety of different tasks, as we have shown. While this highlights the flexibility of PCNs when used in the generative mode, this comes at higher computational cost due to the number of inference steps perform.

---

Published as a conference paper at ICLR 2025
Original Images
petal length
9
5
3
5 4
PC
X
7
6
9
iPC
5 9
3
7
-2
2
8
4
6
sepal longth
Figure 4: Generative samples obtained by MCPC. Left:
Figure 3: CIFARIO image reconstruction via autoen- Contour plot of learned generative distribution compared coding convolutional networks. In order: original, PC, to Iris data samples (x). Right: Samples obtained for iPC, and BP.
a PCN. In order: unconditional generation, conditional generation (odd), conditional generation (even). Table 3: MSE (x 10 of associative memory tasks given noisy (left) or masked (right) inputs keys. Columns indicate the number of hidden neurons while rows shows the training images memorize, Results over 5 seeds.
Noise
512
1024
2048
Mask
512
1024
2048
50
6.06
5.91 40.14
5.95-40.06
50
0.06+0.02
0.01 +0.00
0.00=0.000
100
6.99
40.23
6.1670.00
100
1.177
1.01 10.79
0.11 10.03
250
9.95
10.14
4.90
250
39.1
3.74
0.22
5 ANALYSIS AND METRICS
In this section, we report several metrics that we believe are important to understand the current state and challenges of training networks with PC and compare them with standard models trained with gradient descent and backprop when suitable. The first study we perform analyzes how the initialization of the network states h influences the performance of the model. In the literature, they have been either initialized to be equal to zero, randomly initialized via Gaussian prior (Whittington Bogacz 2017). or initialized via forward pass. This last technique has been the preferred option in machine learning papers as it sets the errors Cl = O at every internal layer of the model. This allows the prediction error to concentrated in the output layer only, and hence be equivalent the SE. To provide a comparison among the three methods, we have trained a 3-layer feedforward model on FashionMNIST.. The results, plotted in Fig. ((a, show that forward initialization is indeed the better method, although the gap in performance shrinks the more iterations T are performed.
Energy propagation. Concentrating the total error of the model to the last layer makes it hard for the inference process to then propagate such an energy back the first layers. As reported in Fig. (b). we observe that the energy in the last layer is orders of magnitude larger than the one in the input layer, even after performing several inference steps. An easy way of quickly propagating the energy through the network would be to use learning rates equal to 1.0 for the updates of the states, that do not produce any energy imbalance, as also shown in Fig. d). However, both the
(a)
(b)
(c)
(d)
Test. Accurocy
Energy Norm
Accuracy
Energy Layer Ratio
Layer Hastn
<jte
10 10
Layer
02 04 O1
Opeimizer,
SGD
A
AdanWW
25
20
0.000 001 0,93
0 OJB 0.1 7 (State Learning Rate)
Inference stops
(Inference Step)
7 (State Learning Rane)
Figure 6: (a): Highest test accuracy reported for different initialization methods and iteration steps T used during training; (b): Energies per layer during inference of the best performing model (which has y = 0.003): (c) Decay in accuracy when increasing the learning rate states 7, tested using both SGD and Adam: (d) Imbalance between energies in the layers. Figures are obtained using a three layer model on FaShionMNIST.
7

---

Published as a conference paper at ICLR 2025
Table 4: Comparison of the training times BP against PC on different
Seconds per epoch
architectures and datasets.
Bachaaz
ligens loets (reap)
Epoch time (seconds)
BP
PC (ours)
PC (Song)
MLP FashionMNIST
1.8240.01
1.9410.07
5.94
AlexNet CIFAR-10
1.0
1.04+0.08
3.86+0.06
17.93=0.37
VGG-5 CIFAR 100
1.61 10.04
10.04
5.33+0.00
13.49
VGG-7 Tiny ImageNet
7.59
54.60
137.5844.08
Muttplicative Factor
Figure 8: Training time for different
network configurations.
results reported in Fig. bb), as well as our large experimental analysis of Section 4 show that the best
performance was consistently achieved for state learning rates y significantly smaller than 1.0. This raises the question of whether better initialization or optimization techniques could result in more balanced energy distribution and thus better weight updates.
To better understand how the energy propagation relates to the performance of the model, we have analyzed both the test accuracy and the ratio of the energies subsequent layers as a function the state learning rates 7. The results, reported in Fig c.dd, show that small learning rates lead to better performance,
also to large energy imbalances among layers. On the one hand, the energy the first hidden layer is similar to that of the last layer for 1, and about 6 orders of magnitude lower for y = 0.01. On the other hand, models trained with a learning rate of y = =1 achieve much worse performance. Such results show that the current training setup favors large energy imbalances among different layers, problem that leads to exponentially small gradients when the depth of the model increases. We provide implementation details and results on other datasets in Appendix
Training stability. We have ob- served a link between the weight opti- mizer and the influence of the hidden dimension on the performance of the model. To better study this, we trained feedforward PCNs with different hid- den dimensions, state learning rates and optimizers, and reported the re- sults Fig. 7 The results show that, when using Adam, the width strongly affects the values of the learning rate r for which the training process sta- ble. Interestingly, this phenomenon does appear when using both the SGD optimizer, nor on standard net- works trained with backprop. This behavioral difference with BP is unex- pected and suggests the need for better optimization strategies for PCNs, Adam W was still the best choice in our experiments, but could be a bottleneck for larger architectures.
PC SGD
PC Adan W
BP
100
0.8
Acouracy
0.6
Widh
32
512
64
1024
022
4096
0.
10
10-
10-
10-2
10-1
10*
10-1
10-2
10-2
10-1
(State Learning Rate)
(State Learning Rate)
Figure 7: Updating weights with Adam W hecomes unstable for wide layers as the accuracy plummets to random guessing for pro- gressively smaller state learning rates the network's width in- creases. Contrarely using SGD, the optimal state learning rate depends on the width the layers.
6 LIBRARY, RESOURCES AND IMPLEMENTATIONS DETAILS
In this section, we discuss PCX, the tool that we have used to perform the experiments, and that release open source. PCX developed on top of JAX, focusing on performance and versatility, and built upon the following concepts: compatibility, modularity, and efficiency.
Compatibility. PCX shares the same philosophy of equinox (Kidger Garcia 2021). according to which models just PyTrees. Consequently, it is fully compatible, using complete functional approach, with both libraries and many other tools developed for JAX, such as diffrax (Kidger 202T) and optax (DeepMind et al. 2020). To this end, it will be straightforward to implement
8

---

Published as a conference paper at ICLR 2025
novel development in deep learning into PCX. However. it also offers an imperative object- interface, which allows researchers to build PCNs following a Py Myyorch-like style.
Modularity. Thanks to the object-oriented abstraction, we built the modular primitives that can be combined to create a PCN, mainly: a module class, representing abstract energy- models; the vectorised nodes storing the states h; the optimizers, to perform the inference and learning process in a predictive coding network; and various standard Layers. Each benchmark we showcase in this work can be obtained by combining and configuring different blocks as needed.
Efficiency. PCX extensively relies on just-in-time compilation. From our initial benchmarks, we observed speed-up of up to 50x when compiling PCN. We believe that this stark difference is due to the nature of PC, which relies on multiple smaller operations compared to backpropagation, i.c., the T inference step performed in each layer, and thus more affected by the function calls overhead present in eager execution mode.
PCX offers a unified interface test multiple variations of PC on several tasks. Our modular code base can easily be expanded in the future to support new variations of PC, as we show complete compatibility with existing variations and training techniques. This is different from, for example, the monolithic or low-level approaches used in (Song) 2024) and (Ororbia & Kifer 2022). respectively.
6.1 COMPUTATIONAL RESOURCES AND LIMITATIONS.
We measured the wall-clock time of our PCNs implementation against another existing sour-source library (Song 2024) used in many PC works (Song et al. 2024 Salvatori al. 2021: 2022 Tang et al. 2023), as well as comparing it with equivalent BP-trained networks (developed also with PCX for fair comparison). Tab. 4 reports the measured time per epoch, averaged over 5 trials, using A100 GPU. We also outperform alternative methods such as Eqprop: using the same architecture on CIFARI00, the authors report that one epoch takes v 110 seconds, while we take zs 5.5 on the same hardware (Scellier et al., 2024.. However, this is not an apple-to-apee comparison, as the authors are more concerned with simulations on analog circuits, rather than achieving optimal GPU usage.
Limitations. The efficiency of PCX could be further increased by fully parallelizing all the opera- tions. In fact, in its current state, JIT is unable to parallelize the exccution of the layers; a problem that can be addressed with the JAX primitive vmap, but only in the unpractical case where all the layers have the same dimension. To test how different hyperparameters of the model influence the training speed, we have taken feedforward model, and trained it multiple times, each time increasing a specific hyperparameter by a multiplicative factor. The results, reported in Fig. g show that the two parameters that increase the training time the number of layers L and the number of steps T. Ideally, only T should affect the training time as inference is an inherently sequential process that cannot be parallelized, but this is not the case, as the time scales linearly with the amount of layers. Details are reported in Appendix G
7 DISCUSSION
The main contribution of this work is the introduction and open-source release of PCX, a library that can be used perform deep learning tasks using PCNs. Its efficiency relies on JAX's Just-en-Time compilation and carefully structured primitives built to take advantage of it. A second advantage of our library its intuitive setup, tailored to users already familiar with other deep learning frameworks such as ToTch. This, together with the large number of tutorials we release, will make it easy for new users to train networks using PC. We have then used PCX to perform an extensive comparative study among different models and training algorithms present in the literature, obtained by testing large number of parameter combinations and activation functions.
In terms of results, we have shown that predictive coding networks perform comparably to standard deep learning ones trained with BP, conditioned on the fact that small/medium size architectures are used, such as VGG 7. When this condition is relaxed, the performance of predictive coding fails to match that of BP, able to scale along with model size. In the supplementary material, we add rigorous studies that provide more details about how the energy flows inside PCNs over time, and their training stability, as well as show how PCNs classify iil-of-distrbuttion data, and possible solutions for training extremely deep networks via the use of skip connections.
9

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
We thank the reviewers for their valuable feedback and insightful discussions, which have significantly enhanced this manuscript. Amine M M'Charra gratefully acknowledges support from the Evangelisches Studienwerk e. Villigst through a doctoral fellowship. Tommaso Salvatori gratefully acknowledges funding from VERSES AI. Rafal Bogacz gratefully acknowledged support by Medical Research Council grant MC L___UU__000 1000031/ Cornelius Emde is supported by the EPSRC Centre for Doctoral Training in Health Data Science (EP/S02428X/I), Thomas Lukasiewicz and Comelius Emde are supported by the AXA Research Fund.
REFERENCES
Nick Alonso, Beren Millidge, Jeffrey Krichmar, and Emre O. Neftci. A theoretical framework for inference learning. Advances in Neural Information Processing Systems, 1137335-37345 2022.
Yoshua Bengio. How auto-encoders could provide credit assignment in deep networks via target propagation. arXiv:1407.7906, 2014.
James Bradbury. Roy Frostig. Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL
http: /tt::/gitthu
DeepMind, Igor Babuschkin, Kate Baumli, Alison Bell, Surya Bhupatiraju. Jake Bruce, Peter Buchlovsky, David Budden, Trevor Cai, Aidan Clark, Ivo Danihelka, Antoine Dedieu, Clau- dio Fantacci, Jonathan Godwin, Chris Jones, Ross Hemsley, Tom Hennigan, Matteo Hessel, Shaobo Hou, Steven Kapturowski, Thomas Keck, Iurii Kemaev, Michael King, Markus Kunesch, Lena Martens, Hamza Merzic, Vladimir Mikulik, Tamara Norman, George Papamakarios, John Quan, Roman Ring, Francisco Ruiz, Alvaro Sanchez, Laurent Sartran, Rosalia Schneider, Eren Sezener, Stephen Spencer, Srivatsan Srinivasan, Miloš Stanojević, Wojeiech Stokowiee, Luyu Wang, Guangyao Zhou, and Fabio Viola. The DeepMind JAX Ecosystem, 2020. URL attp://github ccm/ google ditlee-deeinndd
Arthur Dempster, Nan Laird, and Donald B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1-22, 1977.
Maxence M Ernoult, Fabrice Normandin, Abhinav Moudgil, Sean Spinney, Eugene Belilovsky, Irina Rish, Blake Richards, and Yoshua Bengio. Towards scaling difference target propagation by learning backprop targets. In International Conference Machine Learning, pp. 5968-5987. PMLR, 2022.
Simon Frieder and Thomas Lukasiewicz. (Non-) convergence results for predictive coding networks. In International Conference on Machine Learning, pp. 6793-68 10. PMLR, 2022.
Karl Friston. A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(3666)))) 1456). 2005.
Karl Friston. The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11 (2):127-138, 2010.
Karl Friston and Stefan Kiebel. Predictive coding under the free-energy principle. Philosophical transactions of the Royal Society B: Biological sciences, 364(11211:12111-222) 2009.
Karl Friston, Jérémie Mattout, Nelson Trujillo-Bareeto, John Ashburner, and Will Penny. Variational free energy and the Laplace approximation. Neuroimage, 34(1):220-233 2007.
Will Grathwohl, Kuan-Chieh Wang, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In International Conference Learning Representations, 2020.
Jacobsen, David Duvenaud, Mohammad Norouzi,
10

---

Published as a conference paper at ICLR 2025
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations, 2017.
Geoffrey Hinton. The forward-forward algorithm: Some preliminary investigations. arXiv preprint arX1v:2212.13345, 2022.
Adrien Journé, Hector Garcia Rodriguez, Qinghai Guo, and Timoleon Moraitis. Hebbian deep learning without feedback. arXiv preprint arXiv:2209.11888 2022.
Patrick Kidger. On Neural Differential Equations. PhD thesis, University of Oxford, 2021.
Patrick Kidger and Cristian Garcia. Equinox: Neural networks in JAX via callable PyTrees and filtered transformations. Differentiable Programming workshop Neural Information Processing Systems 2021, 2021.
Diederik P. Kingma and Max Welling, 1312.61 14, 2013.
Auto-encoding variational Bayes.
arXiv preprint
Adam Kohan, Edward A. Rietman, and Hava Siegelmann. Signal propagation: The framework for learning and inference in forward pass. IEEE Transuctions on Neural Networks and Learning Systems, 2023.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Axel Laborieux and Friedemann Zenke. Holomorphic equilibrium propagation computes exact gradients through finite size oscillations. Advances in Neural Information Processing Systems, 35: 12950-12963, 2022.
Julien Launay, lacopo Poli, François Boniface, and Florent Krzakala. Direct feedback alignment scales to modern deep learning tasks and architectures. Advances in Neural Information Processing Systems, 33:9346-9366 2020.
Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. The MNIST Database, 2010.
URL http::/yann. wwwp///////m.ee/ecocceoooo/eeeee
Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, and Colin J. Akerman. Random feedback
weights support learning in deep neural networks. arXiv preprint arXiv: vrXiv:141..0247, 2014
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based -iittriiittiiiino detection. Advances in Neural Information Processing Systems, 33:21464-2144 2020.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale celebfaces attributes (celeba)
dataset. Retrieved August, (5:1018):11,
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint 17Xiv:1711.05101, 2017.
Beren Millidge, Yuhang Song, Tommaso Salvatori, Thomas Lukasiewiez, and Rafal Bogacz. Back- propagation at the infinitesimal inference limit of mergy-based models: Unifying predictive coding, equilibrium propagation, and contrastive Hebbian learning. arXiv preprint arXiv:2206.02629 2022a.
Beren Millidge, Yuhang Song, Tommaso Salvatori, Thomas Lukasiewicz, and Rafal Bogacz. A theoretical framework for inference and learning in predictive coding networks. arXiv preprint arXiv:2207.12316, 2022b.
Timoleon Moraitis, Dmitry Toichkin, Adrien Joumé,, Yansong Chua, and Qinghai Guo. SoftHebb: bayesian inference in unsupervised Hebbian soft hinner-take-ul networks. Neuromorphic Com- puting and Fnggneering, 2(4):044017, 2022.
11

---

Published as a conference paper at ICLR 2025
Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. In Advances in Neural Information Processing Systems, 2016.
Gaspard Oliviers, Rafal Bogacz, and Alexander Meulemans. Learning probability distributions of sensory imputs with Monte Carlo predictive coding. bioRxiv, pp. 2024-02, 2024.
Alexander Ororbia and Daniel Kifer. The neural coding framework for learning generative models.
Nature Commuiications, 13(1):2064, 2022.
Alexander Ororbia and Ankur Mali. Active predictive coding: Brain-inspired reinforcement learning for sparse reward robotic control problems. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 3015-3021. IEEE, 2023.
Alexander G. Ororbia, Ankur Mali, Daniel Kifer, and C. Lee Giles. Backpropagation-free deep learning with recursive local representation alignment. In Proceedings of the AAAl Conference Artificial Intelligence, volume 37, pp. 9327-9335, 2023.
Fabian Pedregosa, Gaël Varoquaux. Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg. Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthicu Brucher, Matthieu Perrot, and Edouard Duchesnay, Scikit-leam: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Luca Pinchetti, Tommaso Salvatori, Beren Millidge, Yuhang Song, Yordan Yordanov, and Thomas Lukasiewicz. Predictive coding beyond Gaussian distributions. 36th Conference on Neural Information Processing Systems, 2022.
Rajesh N. Rao and Dana H. Ballard. Predictive coding the visual cortex: A functional in- terpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79-87, 1999.
Tommaso Salvatori, Yuhang Song, Yujian Hong. Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bogacz, and Thomas Lukasiewicz. Associative memories via predictive coding. In Advances in Neural Information Processing Systems, volume 34, 2021.
Tommaso Salvatori, Luca Pinchetti, Beren Millidge, Yuhang Song, Tianyi Bao, Rafal Bogacz, and Thomas Lukasiewicz. Learning arbitrary graph topologies via predictive coding.
22X1v:2201.131800 2022.
Tommaso Salvatori, Ankur Mali, Christopher L. Buckley, Thomas Lukasiewicz, Rajesh P. N. Rao, Karl Friston, and Alexander Ororbia. Brain-inspired computational intelligence via predictive coding. arXiv preprint arXiv:2308.07877, 2023a.
Tommaso Salvatori, Luca Pinchetti, Amine M' Charrak,, Beren Millidge, and Thomas Lukasiewicz. Causal inference via predictive coding. arXiv preprint arXiv:2306. 15479, 2023b.
Tommaso Salvatori, Yuhang Song, Beren Millidge, Zhenghua Xu, Lei Sha, Cornelius Emde, Rafal Bogacz, and Thomas Lukasiewicz. Incremental predictive coding: A parallel and fully automatic learning algorithm. International Conference on Learning Representations 2024, 2024.
Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy- based models and backpropagation. Frontiers in Computational Neuroscience, 11 11:24, 2017.
Benjamin Scellier, Maxence Ernoult, Jack Kendall, and Suhas Kumar. Energy-based learning algorithms for analog computing: comparative study. Advances in Newral Information Processing Systems, 36, 2024.
Eli Sennesh, Hao Wu, and Tommaso Salvatori. Divide-and-comquer predictive coding: structured Bayesian inference algorithm. arXiv preprint arXiv:2408.05834 2024.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.
12

---

Yuhang Song. Prospective-coniiguaatoon Prospective- Cot.iiaattoooiiiaiiiooooooo 2024.
Yuhang Song, Beren Millidge, Tommaso Salvatori, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz, Inferring neural activity before plasticity foundation for learning beyond backpropa- gation. Nature Neuroscience, pp. 1-11, 2024.
Michael W. Spratling. Reconciling predictive coding and biased competition models of cortical function. Frontiers in Computational Neuroscience, 2:4, 2008.
Michael W. Spratling, review of predictive coding algorithms. Brain and Cognition, 112:92-97, 2017.
Mufeng Tang, Tommaso Salvatori, Beren Millidge, Yuhang Song, Thomas Lukasiewicz, and Rafal Bogacz. Recurrent predictive coding models for associative memory employing covariance learning. PLOS Computational Biology, 19(4)ce010779, 2023.
Mufeng Tang, Helen Barron, and Rafal Bogacz. Sequential memory with temporal predictive coding. Advances in Neural Information Processing Systems, 36, 2024.
James C. R. Whittington and Rafal Bogacz. An approximation of the error backpropagation algorithm in predictive coding network with local Hebbian synaptic plasticity. Neural Computation, 29(5), 2017.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: novel image dataset for bench- marking machine learning algorithms. #rXiv::708.00747,, 2017.
Jingkang Yang. Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-distribution detection: A survey, arXiv preprint FrXiv:2110.113322 2021.
Jinsoo Yoo and Frank Wood. BayesPCN: a continually learnable predictive coding associative memory. Advances in Neural Information Processing Systems, 35:29903-29911 2022.
Contents
Introduction
2
Related Works
3
Background and Notation
Experiments and Benchmarks
4.1
Discriminative Mode
Generative Mode
Analysis and metrics)
6
Library, Resources and Implementations Details
6.1 Computational resources and limitations,
Discussion
Supplementary Material
13
Index
13
PCX A Brief Introduction
15
Discriminative experiments
17
Generative experiments
19
Autoencoder
19
MCPC
21
13

---

Published as a conference paper at ICLR 2025
C,3 Associative memories
22
D Energy and Stability
23
D.I Energy propagation
24
D.2 Training Stability
26
Skip Connections into VGG19
27
Properties of predictive coding networks
29
Free energy and out-of- distriibbiti data.
29
Computational Resources
33
SOCIETAL IMPACT
This work adheres to the established ethical standards prevalent in the field Al and machine learning. In the short term, it does introduce specific ethical concerns, the models and technology study are still in early-stage development, and do not perform as well as classic methods. However, we acknowledge the implications and responsibilities that accompany advancements in these technologies. We are committed to ongoing evaluation and responsible stewardship of our contributions to ensure they align with the ethical landscape of this dynamic field.
14

---

Published as a conference paper at ICLR 2025
APPENDIX
Here we provide the details on how experiments were conducted and results obtained. We opt for more descriptive approach to convey the fundamental concepts, and leave all details for reproducibility the provided code, as well as the next sections. There, each section will link to the exact directory corresponding to the described experiments.
A PCX A BRIEF INTRODUCTION
In this section, we illustrate the core ideas of PCX by describing the main building blocks necessary to train and evaluate feedforward classifier in predictive coding. For more detailed and complete explanations, please refer to the tutorial notebooks in the examples folder of the library.
In Section
( 0r. and state h (ho...., h In PCX, we divide a model two main components: layers (i.e., the traditional deep-learning transformations such 'Linear' "Conv2I)") and vodes (i.e., vectorized nodes that store the array of neurons representing state hj). A PCN is defined as follows:
defined PCNs models with parameters
import jax. on as jnn import pcx. aex.predictive proccinnve as pxc import pcx.nn as pxnn
class MLP Brc.x.nergyModulee
def _init_ ir_dim, h_dim, out_dim):
self.
pxnn. Linear (in_dim, h_dim),
pxnn. (h dim, h_dim), Exnn.Linear (h -imm, out_dim)
Pxc.Vode ( ( for dim in (h_dim, h dim, cut_dim)
def _call __all___ee______________________ (self, x, None)
for layer, vode in zip (self layers, vodes) u = jnn. leaky_ (layer (x)
x vode (u)
if y is not None:
self. vodes set].sstt((eeeeseeeee set ("h", y)
return
In the _call_ method, we forward the input through the network. Note that every time we call vode, we are effectively storing in it the activation uz (so that we can later compute the energy of associated to the vode) and return its state by (i.e., vode(u) corresponds vode.sel(") "u", u): vode.get( "h")), "J). During training, the label is provided to the model and fixed to the last vode by overwriting its state (L), Note that, since both during training and evaluation the state of the first vode would be fixed to the input æ, we avoid defining it (i.c., we avoid computing Pe, (ho) since would be constant), and directly forward x to the first layer transformation.
The class pxc. pxc.Energy EnergyModule provides energy() function that computes the variational free energy F as per Eq. (T). We can compute the state and parameters gradients as per Eqs. (3) by calling pxf.valte and grad, wrap around the homonymous JAX function. Having defined two optimizers, optim w and optim h, for parameters and state respectively, we can define training on pair (I, y) as following:
as pxu import pcx. functionall pxf
import
15

---

Published as a conference paper at ICLR 2025
def energy (x, y, *, model)
model (x, y)
return model. model.eners energy ()
grad - pxf. value_and_graa pxu. (pxc. VodeParam, [False, True))
) (energy)
grad_ pxf. value_amd_grad
Pxu.Mast (pxc. LayerParam, [False, True))
) (energy)
def train (T, x, y, *, model, optim h, optim_w): model.train()
Initialization
with pxu. step (model, pxc. INIT, clear_params-pxx. Vodeec.ooooaareee..eo aara model (x)
Inference steps
for range :T)
with pxu.step (model, clear_params-pxx VodeFaram. Cache)
g_h grad_h y, model-modes model)
optim_h. step (model, g ['h["modd (mmode"del",,, True)
# Learning step with pxu. step (model, clear theerr_pramsspaee VodeP aram. Cache)
g_w grad_ (x, y, model =model) opt im__im_ww.. step (model, s_w["modes
few notes on the above code:
JAX (Bradbury et al. 2018) is functional library, PCX is not. Modules in PCX are Py Trees, using the same philosophy as another popular JAX library, equinox (Kidger & Garcia 2021). with which PCX modules so that each parameter transformation automatically tracked. The user can opt in for this behavior by passing arguments keyword arguments (such in the above example). Positional function parameters, instead are ignored by PCX and it is the user's duty to track their state done in JAX equinox.
fully compatible. However, their state is managed by PCX
pxf.value and _rad allows to specify a Mask object to identify which parameters to target with the given transformation. In the case above, we first compute the gradient of with respect of the state Param) and, then, of the weights (LayerParam) of the model.
In the train function, we use pxu.step to set the model status to pxc. ATC.STATUS.NN to perform the state initialization. In PCX, forward initialization is the default method, however other ones can be easily specified. ste is also used to clear the PCN's cache which used to store intermediate values such as the activations Wy.
The actual examples in the library are on mini-batches of data, so all transformations above are vmapped in the actual experiments.
For the evaluation function, being in discriminative mode, we simply perform a forward pass through the PCN which sets 6 = 0 for all layers.
def eval (x, *, model): with pxu. step (model, pxc. STATUS.AUSS. INIT, cleax_paramae VodeParam. Cache) return model (x)
16

---

Published as a conference paper at ICLR 2025
B DISCRIMINATIVE EXPERIMENTS
Model. We conducted experiments three models: MLP, VGG-5, and VGG-7. The detailed
architectures of these models are presented in T
Table 5: Detailed Architectures of base models
MLP
VGG-5
VGG-7
Channel Sizes
[128, 128]
[128, 256, 512, 512]
[128, 128, 256, 256, 512, 512]
Kernel Sizes
[3, 3, 3,
[3, 3, 3, 3,
Strides
[, 1,
1,11,
Paddings
[1. 1,00
[1, 1,0,1,0,1,00
Pool window
2x2
2x2
Pool stride
2
2
For each model, we conducted experiments with the following different algorithms:
1. Standard PC with Cross-Entrope Loss (PC-CE) Mean Squared Error Loss (PC-SE): already discussed in the background section.
2. PC with Positive Nudging (PC-PN):
Unlike standard Predictive Coding with Mean Squared Error Loss (PC-SE), where the output is clamped to the target, we "nudge" the output towards the target in PC with nudging. This is achieved by fixing the representation of last layer HG to μzz + B(y - jrr,), where μr, is the predicted activation of the last layer after forward initialisation, y is the target, and 3 E (0, 1) is scalar parameter that controls the strength of nudging. Note that when 3 = 1, PC with nudging is equivalent to the standard PC.
During training procedure, as the model output gradually approaches to the target, we employ strategy of increasing B. At the end of each epoch, the value of / is incremented by a fixed rate Bir- When 33 becomes greater than or equal to I, we set it to 1. This strategy allows the model more stable learn and explore in the early stages of training, while gradually transitioning to the standard PC in the later stages.
3. PC with Negative Nudging (PC-NN):
In this algorithm, we do the opposite of positive nudging: we push the output away from the target. Therefore, we fix the representation of the last layer to şz - BB ML). We use the same strategy of dynamically increasing 3. When 3 becomes greater than or equal to -1, we set it 1.
In the learning stage, to ensure that the direction of the weight update is consistent with the target (since we fixed h to the opposite direction), we invert the weight update: Or <- 8 - A6; where Ad; defined in the Eq. (3).
4. PC with Center Nudging (PC-CN):
Center Nudging (Scellier et al. 2024) is used in equilibrium propagation to improve and stabilize performance compared to both positive and negative nudging, and is obtained an average of the gradients produced by the two methods. Here, approximate this behavior by randomly alternating between epochs in which we train with either negative or positive nudging. In this way, the training model can benefit from both methods without any extra computational cost.
5. Incremental PC (iPC), simple and recently proposed modification where the weight parameters are updated alongside the latent variables at every time step (Salvatori et al. 2024
6. Standard Backpropagation with Cross-Entrop Loss (BP-CE) Mean Squared Error Loss (BP-SE): the most popular way to do the credit assignment in the neural networks. The model is trained by computing the gradients of the loss function with the weights of the network using the chain rule.
17

---

Published as a conference paper at ICLR 2025
Experiments. The benchmark results of MLP are obtained with MNIST and Fashion-MNISTT the results of VGG-5 are obtained with CIFAR-10, CIFAR- 100 and Tiny ImageNet, the results of VGG-7 obtained with CIFAR-100 and Tiny ImageNet. The data is normalized as in Table
Table 6: Data normalization
Mean (14)
Std (a)
MNIST Aishion-MNIST
0.5
0.5
0.5
0.5
CIFAR-10
[0.4914, 0.4822, 0.4465]
[0.2023, 0.1994, 0.2010]
CIFAR-100
[0.5071, 0.4867, 0.4408]
[0.2675,
0.2565, 0.2761]
Tiny ImageNet
[0.485, 0.456, 0.406]
[0.229, 0.224, 0.225]
For data augmentation on the training sets of CIFAR-10 10, CIFAR-100, and Tiny ImageNet, we apply employ random cropping
random horizontal flipping with a probability of 50%. Additionally, with different settings for each dataset. For CIFAR-I0 10 and CIFAR-100, images are randomly cropped to 32x32 resolution with a padding of 4 pixels on each side. In the case of Tiny ImageNet, random cropping is performed to obtain 56x56 resolution images without any padding. And on the testing set of Tiny ImageNet, we use center cropping extract 56x56 resolution images, also without padding. since the original resolution of Tiny ImageNet is 64x64.
The model hyperparameters are determined using the search space shown in Table The results presented in Table were obtained using 5 seeds with the optimal hyperparameters.
As for the optimizer and scheduler, we use mini-batch gradient descent (SGD) with momentum as the optimizer for the h, and we utilize AdamW Loohhhhilo & Hutter (2017) with weight decay as the optimizer for the 0. Additionally, we apply a warmu--oosine-annallngg scheduler without restart for the learning rates of f.
Table 7: Hyperparameters search configuration
Parameter
PC
iPC
BP
Epoch (MLP)
25
Epoch (VGG and ResNet)
50
Batch Size
128
Activation
[leaky relu, gelu, hard tanh]
[leaky relu, gelu, hard tanh, relu]
B Bir
[0.0, 1.0], 0.05
[0.02, 0.0]
Irk.
(Ie-2, 5e-1)²
(le-2, 1.0)
Tre momentumn, weightdecaye
(le-5, 3e-4)²
Be (le-5, 1e-2)²
[0.0, 1.0], 0.05
(le-5, 1e-2)²
(1e-5, 1e-1)²
T (MLP and VGG-5)
[4.5,6,7.8]
T (VGG-7) T (VGG-9)
18,9,10,11,12)
19,10,12,15,88
T (ResNet-18)
16,10,12,18,24]
1: "[a, b], e" denotes sequence of values from to b with step size of c. 2; "(a, b)" represents log-uniforma distribution between and
Results. All the results presented in this study were obtained using forward initialization, a tech- nique that initializes the model's parameters by performing a forward pass on a zero tensor with the same shape the input data. Besides, in experiments, we limited the range of 7 to ensure a fair comparison with BP in terms of training times. Higher correspond to greater number of optimization rounds of h. which can lead to improved model performance but also increased computational costs and longer training durations. To maintain comparability with BP, we restricted searching space of T that resulted in training times similar those observed in BP-based training.
18

---

Published as a conference paper at ICLR 2025
Momentum helps significantly. In Figure present the accuracy of the VGG-7 model trained on CIFAR- 100 using different momentum values, both without nudging(Figure 9ap and with nudg- ing(Figure 9b). It is evident from Figure g that selecting appropriate momentum value can substantially improve model accuracy. By comparing Figures and 9b
we can observe that dif- ferent training algorithms have different optimal momentum values. The optimal momentum for training with nudging is generally higher than that for training without nudging. Furthermore, the optimal momentum for negative nudging is larger than that for positive nudging. These differences in optimal momentum values highlight the importance of carefully tuning the momentum hyperpa- rameter based on the specific training algorithm and nudging method employed. For reference, the optimal model parameters and momentum values for various tasks and models can be found in the http:errrnnnniieeeeeeeeeeeeeeeee
folder the PCX library. momantum vs. Accuracy (With Nudging)
momentum vs. Acouracy (Without Nudging)
0.s-
algortthe
0.4-
0.4
Accuracy
0.2
Accuracy
0.2
0.1
0.1
alporithm PCL PM PCN An
momentum
momentum
(b)
Figure 9: Comparison of the accuracy of the VGG-7 model trained on CIFAR-100 using different momentum values
Activation function also plays crucial role in improving model accuracy. For models using Cross-Entropy Loss, the "HardTanh" activation function is better choice. the case of models using Mean Squared Error Loss without nudging, the "LeakyReLU" activation function tends to perform better. When using Positive Nudging, the optimal activation function varies depending on the model architecture. For Negative Nudging, the "GeLU" activation function is the most suitable choice.
Nudging improves performance. Fig. 10 illustrates the relationship between the learning rate of and accuracy with or without nudging. From the plot, we can observe that when nudging is not used (red dots), the model achieves better results at lower learning rates. However, when nudging is employed (purple and blue dots), regardless of whether it is positive nudging or negative nudging, the model can attain better accuracy at higher learning rates compared the case without nudging. Additionally, Fig. shows the relationship between momentum and accuracy. We can see that after applying nudging, the model can achieve better results higher momentum values. We believe this is the reason why nudging can improve performance. The ability to use higher learning rates and momentum values without sacrificing accuracy is significant advantage of nudging, as it can lead to faster convergence and improved generalization performance.
C GENERATIVE EXPERIMENTS
C. AUTOENCODER
An Autoencoder is network that learns how to compress high-dimensional input into much smaller dimensional space, called the bottleneck dimension or the hidden dimension, as accurately possible. Thus, backpropagation-brit baased Autoencoder consists of two parts: encoder, that compresses the input from the original high-dimensional space into the bottleneck dimension, and a decoder, that reconstructs the original input from the bottleneck dimension. A mean-squared error
19

---

Published as a conference paper at ICLR 2025
Accuracy
Accuracy
PCM
PCN
PCK
PCR PH
PCN NM
PCN NN
0.5
0.4
Accuracy
0.4-
0.1
0.0
0.0
krh
Ir.h
(a) VGG-7
(b) VGG-5
Figure 10: Comparison the accuracy the VGG-7 and VGG-5 model trained on CIFAR- 100 using different learning rates for h.
Original Input
Activation
Predicted
Encoder Layer
Decoder Layer
Encuuder Layer
Activation
Bicemee
Learne Sttate
Decoder
Decoder Layer
Activation
Reconstructed Input
Decoder
Layer
Reconstructed Input
Original Impac
Figure 11: Left. An Autoencoder implemented with backpropagation consists of both encoder and decoder. The encoder compresses the input data into the bottleneck dimension, and the decoder restores the original image. Right. An Autoencoder implemented with Predictive Coding. The state of the first PC layer is the bottleneck dimension. The state of the last PC layer is the original input, and the predicted state of the last PC layer is the predicted input. Inference steps update the bottleneck dimension to make it good compressed representation.
(MSE) between the original and the reconstructed input used as a loss to train the Autoencoder network in unsupervised manner.
Predictive Coding (PC) alleviates the need in the encoder part of an Autoencoder. Specifically, only the decoder part of an Autoencoder is used, with PC layer acting the bottleneck dimension and as an input to the decoder. Moreover, PC layers are inserted after each layer of the decoder.
PC-based Autoencoder works as follows:
20

---

Published as a conference paper at ICLR 2025
Table 8: Hyperparameters and search spaces for deconvolution-based autoencoders
Parameter
PC
iPC
BP
Number of layers
3
conv layers: 3 deconv layers: 3
Internal state dimension
4x4
Internal state channels
8
Kernel size
[3, 4, 5,
Activation function
[relu, Jeaky_reluy gelu, tanh, hard_tan) tanh] 200
Batch size Epochs
30
T Optim h Irn momentum,, Optim 0 Irg weightdecayo
20 SGD+momentum
(1e-2, 5e-1)² (1e-2, 1.0) [0.0, 0.95]
A
3e-5, le-32
(1e-3, 1e-2)²
(le-5, le-1)²
(1e-5, 1e-2)²
Table 9: Hyperparameters and search spaces for linear-based autoencoders
Parameter
PC
iPC
BP
Number of layers
3
encoder: 3 decoder: 3
Internal state dimension
64
Activation function
[relu, leaky_ relu, gelu, tanh, hard_tanh,
Batch size
200
Epochs T
30
20 SGD+momentum (le-2, 5e-1)² (1e-2, 1.0)
Optim h Irn momentumn Optim lre weightdecayo
[0.0, 0.95]
AdamW
(3e-5, le-3)² (Ie-5, le-1)²
(le-5, 1e-2)²
(Ie-5, le-2)²
1.
The energy function of the last PC layer is set to MSE upon its creation. In PCX, the squared error is the default energy function. The squared error is then summed across all dimensions in the input and averaged over the batch, that approximates the MSE up to a multiplication constant.
2. The current state of the last PC Layer hr, is fixed the original input data, which means that h2 is not changed during inference steps.
3. Since the energy of the last layer now encodes the MSE loss between the predicted image μr. and the original input stored hr., the inference steps will update the current states by of all PC layers but the last one. including the one that represents the bottleneck dimension, to minimize this MSE loss.
4. Once the inference steps are done, the state of the bottleneck dimension PC layer will converge to the compressed representation of the original input.
C.2 MCPC
Model. Monte Carlo predictive coding (MCPC) a version predictive coding that can be used for generative learning. MCPC differs from PC by its noisy neural dynamics. Unlike PC where the
21

---

Published as a conference paper at ICLR 2025
neural activity converges to mode of the free-energy, the neural activity of MCPC performs noisy gradient descent which is used for Monte Carlo sampling. When an input is provided, the noisy neural activity samples the posterior distribution of the generative model given the sensory input. When no input is provided the neural activity samples the generative model encoded in the model parameters. Specifically, the neural dynamics of MCPC leverage the following Langevin dynamies:
A V,,,, (h, V2yN
(4)
where N is Gaussian random variable with variance mopee These neural dynamics can be extended to 2nd-order Langevin dynamics for faster sampling:
b = 7
(5)
Ar VV F(h,0) - -7(1 111-10 -m) + 2 - M
(6)
where m is a momentum constant.
An MCPC model is trained following Monte Carlo expectation maximisation scheme which iterates over the following two steps: (i) MCPC's neural activity samples the model's posterior distribution for the given data, and (ii) the model parameters are updated to increase the model log- likeiiihoood under the samples of the posterior. In practice, we run MCPC inference for a limited number of steps after which we update the model parameters with single sample of the posterior similarly to how model parameters are updated in variational auto encoders.
After training, samples of trained model are generated by leaving all neurons unclamped and recording the activity of input neurons (the neurons clamped to data during training). The activity is recorded after limited number of activity update steps. This process is repeated for each data sample.
MCPC's implementation in PCX utilizes noisy SGD optimizer for the state h. Compared to PC than uses an SGD or Adam optimizer, MCPC incorporates an optimizer that merges the addition of noise to the model's gradients with an SGD optimizer. The variance of the noise added to the gradients needs to be carefully crafted to scale appropriately with the learning rate and the momentum as shown in equations
Experiments. All the MCPC experiments use feedforward models with Squared Error (SE) loss. The SE loss of the state layer hr. is also scaled by variance parameter of This additional parameter is introduced to prevent the Gaussian layer hr. from having variance much larger than the variance of the data which would prevent learning. Moreover, for unconditional learning and generation, the layer ho is left unclamped during both training and generation. In contrast, for the conditional learning task on MNIST, the layer fgg is clamped to labels during training and generation.
For the iris dataset, train model with layer dimensions [2 64 x 2], tanh activation function and default parameter values (state learning rate -=0.01, state moment um = 0.9 noise state variance mapee 1, parameter learning rate Irg. parameter decay 0.0001, Adam parameter optimizer, layer variance o 0.01 and batch size of 150). We use 500 state update steps during learning and 10000 for generation.
For the unconditional learning task on MNIST, we train models with layer dimensions [30 x 256 x 256 x 256 x 784]. The model hyperparameters for MCPC and VAE were determined using the hyperparameter search shown in table 10 to optimize the FID and the inception score separately. Refer to the code for exact optimal parameter values. We use 1000 state update steps during learning and 10000 for generation.
For the conditional learning task on MNIST, we train models with layer dimensions [2 256 256 256 x 784]. The labels used in this task, clamped to hog specify whether an image corresponds to an even or odd number. The model hyperparameters are determined using the search space shown in We use 1000 state update steps during learning and 10000 for generation.
table I0
Results. Figure 12 shows samples generated by the trained models for hyperparameters that maximize the inception score.
C.3 ASSOCIATIVE MEMORIES
This section describes the experimental setup of associative memory tasks.
22

---

Published as a conference paper at ICLR 2025
Table 10: Bayes hyperparameter search configuration for MCPC and VAE (where applicable) on MNIST.
Parameter activation
Value
[ReLU, Silu, Tanh, Leaky-ReLUU Hard- Hard-Tanh}
Nog-uniform(0.000.. 0.05)
momentum 2 mape Irg
(0.0, 0.9)
{1.0, 0.3, 0.01, 0.001]
log-uniform(0.0000 0.1)
parameter decay t batch size
f0.0. 0.1, 0.01, 0.001, 0.0001
1og-uniform(0.03, 1.0)
{ [150, 300, 600, 900)
unconditional
MCPC conditional
even 0 2
MCPC
VAE
6
U
O
L
7 5
2.
6
8
6
3 7
5 8 4
4 0
6
8
8
6
6
2
6
C 3 O 5 3
O 2
a
7 8 4 8 3
5
4
8
0
2
4
6
4
4
9
0
3 6 / 5 3 2
2
2
6
6
4
2 2
2 R J 2 7 7 O
6
L. 6 4 5 2 4 6
7
8 4 0
8
2
& 6 5 5 B E 1
4
4 4 5 / O
odd
3 Z 7 O
6 3
= 5 8 L 4 3
2
9
5
/
7
7
2
7 7 3 9
9 5 ]
9 C R Ø
7
7 6 7 2 5 6 6
6
/
9
3
5
6
7 6 4 R 3 6
3
6
9
5
O
9
5
9
7 3
5
L 9 6
7 3
4
6
6
2
5
6
8
7
9
7
5
O 4 2 O S B 7 2
3
5
9
2
2
6
3
5 7
S
5
6
5
5
Figure 12: Samples generated by trained models that optimize the inception score under the uncondi- tional and conditional learning regimes.
Model. generative PCN first trained on images sampled from the Tiny ImageNet dataset until its parameters have converged. Then, corrupted version of the training images is presented to the sensory layer of the model (hr,) and we run inference Vhy on all layers, including the sensory layer, until convergence. Note that in masked experiments, the intact top half of the images is kept fixed during inference. Intuitively, suppose the model has minimized its free energy with its sensory layer fixed at each of the training examples during training. In that case, it has formed attractors defined by these training examples and would thus tend to "refine" the corrupted images to fall back into the energy attractors.
Experiments. Here, the benchmark results are obtained with Tiny ImageNet, corrupted with either Gaussian noise with 0.2 standard deviation, or a mask on the bottom half of the images (examples shown in Fig. 5). We vary the model size and number of training examples to memorize, to study the capacity of the models. Specifically, we use generative PCN with architecture [512, d, d, 12288) where = [512, 1024, 2048) (12288 being the flattened Tiny ImageNet images) and varied = [50, 100, 250]. We performed hyperparameter search for each d and n the parameter learning rate Ir E {1 x 10 + k x 10 l E Z, 0 < n I 18), the state learning rate y {0.1 + k- 0.05 & Z, 0 < n < 18), training inference steps Tirann E [20, 50, 100) and recall inference steps Trecall E [50000, 100000]. We fix the activation function of the model to Tanh, and the number of training epochs to 500 and batch size of 50. The results in Table are obtained with 5 seeds with the searched optimal hyperparameters.
D ENERGY AND STABILITY
This section describes the experimental setup of Section provides replications on other datasets and ablations.
23

---

Published as a conference paper at ICLR 2025
Biakes 5121 for LonkyRd
Hidden m 1024 for LeakyRe Hilden west you LeskyRalU
Haldan fo LokyRaL
Ascunyy
Optinzzer
Tiiden Warth 912 HanfEanh
Hicden Wints = 1024 y Harditoh
Hliden Wath 2048 HaedToah
Hideen Wath 4006 y Hanffihh
soB
Ada
8
8
8.
Leaming Rani)
Learning
Figure 13: Model accuracies for range of combinations of activation functions and model widths. Adam perfers small learning rates and tends to be less stable than SGD. Obtained on FashionMNIST.
D.1 ENERGY PROPAGATION
We test grid of models multiple datasets to examine the energy propagation in the models. We test on the FashionMNIST, Two Moons, and, Two Circles datasets. The Two Circles dataset is particularly interesting, as poor energy distribution intuitively results linear inductive bias (we primarily learn one-layer network). This linear inductive bias harms the performance on Two Circles (linear model accuracy z 50%) more than FashionMNIST (= 83%) and Two Moons (= 86%).
Experimental Setup. We train a grid of feedforward PCNs with 2 hidden layers. We train on three datasets: FahionMNIST (as reported in the main body) and additionally Two Moons and Two Circles. For all models, we train for 8 epochs with T = inference steps. States are optimized with SGD and forward initialization. The grid is formed over weight learning rate Ira {1 10 1 10-1,... ,.... 1), state learning rate E (1 x 10-3 3 x 10-3 1 x 10-2, 3 10-2, 2, x 10 33 10- activation functions f E (LeakyRel.U, HardTann}} (the former is unbounded the latter is bounded), optimization with AdamW or SGD with momentum m E {0.0, 0.5, 0.9, 0.95} and hidden widths of (512, 1024, 2048, 4096) for FashionMNIST and {128, 256, 512, 1024] for Two Moons and Two Cricles. We replicate all experiments on 3 seeds for MaSHI and 10 seeds for the other datasets.
Results. Fig. left) in the main paper shows the average energy across the last batch at the end of training for the best performing model on the grid. Fig. ((center-leff) compares SGD with momentum 0.9 and Adam W. It obtained for activation function "HardTanh" and width of 1024, We replicate this figure for the other combinations of activation functions and widths below in Fig [ We observe that across all conditions, small to medium state learning rates are generally preferred by SGD, while AdamW has a stronger preference smaller state learning rates. Given the uneven distribution of energies across layers, Adam W, in particular, may not scale to deeper architectures. We further, observe a larger variance in performance for AdamW. W, especially for wider layers, which we discuss in paragraph "Training Instability in Sec. Sec.5land 5nd below. Fig. b[right) is based on all models trained with Adam W. Many models with high state learning rates diverge, we only plot models achieving accuracy > 0.5.
Below we present the results of experiments on the Two Moons and Two Circles datasets. Fig. 14b 14a] and 14c replicateFig 6 for Two Moons, and Fig. 15b 15a and 15c Two Circles. Results are very similar to FashionMNiST: The energy is concentrated in the last layer, even after T' inference steps. However, in the example for Two Circles, we actually observe training effect for earlier layers: While the energy increases first due to eтrоr propagation (still orders magnitude below later layers), the energy is reduced afterwards. Energy ratios consistenly indicating poor energy propagation for state learning rates y, that perform well. As predicted the variance in results is significantly larger for Two Circles, especially for small state learning rates.
24

---

Published as a conference paper at ICLR 2025
Energy Norm
Accuracy
Energy Layer Ratio
70P
D.
10S
Layer Ratio
06
000
101
Layer
04
Optimizer
0000
10-
02
10-2
3
0,001 0.033 0.1
0.03
0.1
03
1.0
0,001 0.003 0.11
0.03
0.1
0.3
1.0
T (Inference Step)
y (State Learning Rate)
(State Learning Rate)
(a)
(b)
(c)
Figure 14: Energy propagation on the Two Moons dataset. shows the imbalance between layers across T steps. 14b shows the model performance across state learning rates and [4c the energy distribution across state learning rates.
Energy Norm
Accuracy
Energy Layer Ratio
1.0
08
T0l
Layer Ratio
11
Layer
06
04
Oplimizer
03
10-3
AdamnW
2
3
0,001 0.003 0.11 0.03
0.1
0.3
1.0
0.001
0.003
0.01
0.03
0.1
0.3
T (Inference Step)
y (State Learning Rate)
(State Learning Rate)
(a)
(b)
(c)
Figure 15: Energy propagation on the Two Circles dataset. 15a shows the imbalance between layers
across T' steps. 15b shows the model performance across state learning rates and 15c the energy distribution across state learning rates.
25

---

Published as a conference paper at ICLR 2025
PC-SGD
3d-maam
BP
Adam
0.1
0.8
Accuracy
Width
0.4
64
128
1024
2048
512
02
1444
1764
0.0
10
10
10
10
10
ViState Leurning Suno)
y: |(State Lourning Rase)
(State Learning Rate)
(State Leariing Rate)
The instability of optimization as result of aedimizerarrciitectuee cnttttcctuur-ntterattton interaction can be (at least partially) be attributed to the absolute size of layers.
Figure 17:
Figure 16: The instability of optimization with Adam given architectural choices can be ob- served for Two Moons.
D.2 TRAINING STABILITY
We test a grid of PCNs to analyze the interaction between model width, state learning rates and weight optimizers.
Experimental Setup. We train models on FashionMNIST (as reported above) and Two Moons. We train feedforward PCNs (2 hidden layers) with "LeakyReLLU" activations over a grid of parameters. All models are trained over 8 epochs. The widths of the hidden layers are {32, 64, 4096). State variables are trained for T 8 steps with SGD and learning rates Y E {1 x 10-5,
5,3 3.3x x 10 ....., 0.3). The weights are updated through SGDor the Adam optimizer with a learning rate of 0.01 for MNSH and 0.03 for Two Moons. Both optimizers uses 0.9 momentum for weights. We further train baseline BP models with the same hyperparameters. For FashionMNIST we replicate each run over random initializations, for Two Moons over 10.
Results. We replicate Fig. 7 (FashionMNISST) here for the Two Moons dataset, see Fig. 16 We observe effects for Two Moons that are analog FashionMNIST as presented above: The stability of optimization strongly depends on the width of the hidden layers for Adam. This effect is not observed for SGD on either dataset. This further supports the our conclusion in Sec. While Adam is the better optimizer, this interaction effect (width x 7) can hinder the scaling of PCNs with Adam. Optimization methods for PCNs require further attention from the research community.
Ablation. We further provide an ablation on FashionMNIST. In the experiments above, the hidden layer width altered, introducing changes in the absolute size of the hidden layers (i.e. number of neurons), but also changing the relative size of the hidden layers in the network, input and output layers remain the same size across all experiments. Hence, we provide another experiment on FashionMNIST,, where the width of all layers equal. All other experimental variables remain as described above. The results are shown in Fig. [7 and follow the trend observed in Fig. and T6 We find that there exists an interaction between the optimization and the width of the network as described above. Hence, accounting for relative changes in layer width does not sufficiently explain the problem and we conclude that the absolute size of the layers plays a role in the stability of optimization with AdamW.
increase the image size and augment the label vector with Os, such that
ResNets Here we discuss the findings on the energy propagation in light of the ResNets18 experi- ments. In this section, we have shown that lower learning rate for the nodes harm energy propagation, and that the AdamW optimizer displays poor performance for larger hidden dimensions. To this end, we have trained ResNets18 using SGD and large learning rates for the nodes, and compared the performance against those in the main body of the paper. The performance are, however, not comparable to the ones reported in 1, as ResNets trained with SGD the CIFAR10 dataset reach accuracies of 39.9% and 43.2% when using PC and iPC, respectively. To better understand the incidence of different hyperparameters show their importance plots. Such quantities are computed by fitting a random forest regressor with hyperparameters as datapoints, accuracies labels, and extracting the feature importance.
the final test accuracy of the models, in Fig. [8
26

---

Published as a conference paper at ICLR 2025
Hyperpoareter Insosmanoe
Hyperprrartte Importncce
Top Accuncy:
Top Accumcy:
55.7%
39.9%
PC-SE, A
PCSE
Heperpuaaattn Imsorcances
Hpperpnurater Imparaacce
6.22
Accuncy
Top Accurag:
43.2%
Hepiega Orgartiio
iPC, Adam
SGD
Figure 18: Importance plots that show the importance each hyperparameter final test accuracy of the model, computed by fitting a random forest regressor with hyperparameters as datapoints, accuracies as labels, and extracting the feature importance.
E SKIP CONNECTIONS INTO VGG19
Skip connections. We investigate the integration of skip connections into the VGG19 architecture to enhance its performance on the CIFARIO 10 image classification task, showing significant increase in test accuracy from 25.32% to 73.95%. The vanishing gradient problem, notable challenge in deep Predictive Coding (PC) models, becomes pronounced with increased network depth, hindering error transmission earlier layers and impacting learning efficacy. To address this, we introduce skip connections that allow gradients to bypass multiple layers, enhancing gradient flow and overall learning performance.
27

---

Published as a conference paper at ICLR 2025
Table 11: Hyperparameter configuration and best accuracy for VGG19 with and without skip connections on CIFAR10
Parameter
Range
Best Value
With Skip Connections
Epochs
30
30
Batch size
128
128
Activation functions
(GELU, Leaky ReLU)
Leaky ReLU
Optimizer for network parameters Learning rate Optimizer for network parameters Momentum Optimizer for weight parameters Learning rate Optimizer for weight parameters Weight decay Number of inference steps (T)
[5e-2, le- 5e-11 {0.0, 0.5, 0.9, 0.99) le-4 (5e-4, le-4, 5e-5) [24, 36)
0.5
0.5
le-4
5e-4
24
Best Accuracy
73.95
Without Skip Connections
Epochs
30
Batch size
128 (GELU, Leaky ReLU) (5e-2, le- 1, 5e-1] [0.0, 0.5, 0.9, 0.99) le-4 [5c-4, lc-4, 5e-5) {24, 36)
128
Activation functions
GELU (default)
Optimizer for network parameters Learning rate Optimizer for network parameters Momentum Optimizer for weight parameters Learning rate Optimizer for weight parameters Weight decay Number of inference steps (T)
0.1
0.99
le-4
le-4
24
Best Accuracy
25.32%
Performance Comparison and without skip connections (Test Accuracy on CIFAR-10)
70-
60
Test Accuracy (56)
50
VGG with skip connections (Mean)
40
VGG without connections (Mean)
30
10
10
20
Epoch
Figure 19: Performance comparison of VGG19 with and without skip connections on the CIFAR-10 10 dataset over 30 epochs. The plot shows the mean test accuracy along with the shaded area representing the variability across three different seeds.
Results Our modified VGG19 model includes skip connection from an early layer within the feature extraction stage, with the output flattened and adjusted using linear layer before being
28

---

Published as a conference paper at ICLR 2025
0.16
0.14
Afriirkreece Afw Pifennza
True Positive Ratee
0.12
0.8
20.00
0.6
-10.0
00.08
0,06
Sohrina PUC 0.21))
0.04
brerp (RUC -08200
ID Bet.
Aft. GOD BetDOD Aft.
0.02
Softmas AUC 5732)
NLL
NLL:
NLL:
NLL:
0.00
0..0
204.1
35.3
1721.4
182.7
0.2
0.3 0.4
0.5
0.6
0.7 0.8
0.9
1.0
0.2
0.4
0.6
0.8
1.0
Data Type
Softmax Scores
False Positive Rate
(a)
(h)
(c)
Figure 20: (a) Energy and NLL of ID/OOD data before and after state optimization. (b) Nonlinearity between energy and softmax post-convergence. (c) ROC curve of OOD detection at the 100th and 25th percentiles of scores. all plots, "ID" refers to MNIST and "OOD" to FashionMNIST.
reintegrated during the classification stage. The model underwent rigorous training and evaluation on the CIFARTO 10 dataset, employing standard preprocessing techniques like normalization and data augmentation (horizontal flips and rotations). Detailed hyperparameter tuning revealed optimal configurations for both models, with and without skip connections, exploring various optimizers, learning rates, momentum values, and weight decay settings, significantly enhancing the model performance with skip connections as summarized in [able progression over 30 epochs for the VGG19 model with and without skip connections on the CIFARIO dataset, using three different seed values and identical hyperparameters for both simulations.
[9 shows the test accuracy
F PROPERTIES OF PREDICTIVE CODING NETWORKS
This section describes the experimental setup of Section F. and displays the utility of using the free energy of a PCN classifier to differentiate between in-distribution (II) and out-of-distribution (OOD) data (Liu et al. 2020). We show how (Grathwohl et al. (020)) under the PCN. We further provide analyses on the relationship between maximum softmax values and energy values before convergence and after convergence at the state optimum. We compare results across multiple datasets to corroborate our results as well as to show how PCNs can be used for OOD detection out of the box based on single trained PCN classifier for which we study the receiver operating characteristic (ROC) curve based on different percentiles of the softmax and energy scores.
compute the negative log- log-likelihood of various datasets
F.1 FREE ENERGY AND OUT-OF-DISTRIBUTION DATA.
With PCX, it is straightforward to inspect and analyze several properties of PCNs. Here, we use F to differentiate between in-distribution (ID) and out-of-distiibution (OOD) due to semantic distribution shift (Liu et al., 2020), as well as to compute the likelihood of a datasets (Grrathwohl et al., 2020). This can occur when samples are drawn from different, unseen classes, such FashionMNIST samples under an MNIST setup (Hendrycks & Gimpel 2017).
Experimental Setup. We train a PCN classifier on MNIST using a feedforward PCNs with hidden layers each of size H 512 with "GELU" activation and entropy loss in the output layer. We train the model until test emor convergence using early stopping at epoch 75. During training the state variables are optimized for T 10 steps with SGD and state learning rate 7 = 0.01 without momentum. The weights are optimized using the SGD optimizer with a momentum of mg 0.9 and the weight learning rate chosen as Iro = 0.01. During test-time inference, we optimize the state variables until convergence for T = 100. To understand the confidence of PCN's predictions, we compare the distribution of energy for ID and OOD samples against the distribution of the softmax scores that the classifier generates. We compute negative log-likelihoods for ID and OOD samples under the PCN classifier via:
F == In 0( p(E, =F
29

---

Published as a conference paper at ICLR 2025
We conduct the experiments on MNIST the in-distribution (ID) dataset and compare it against various out-of-distribution datasets such as notMNIST, KMNIST, EMNIST (letters) as well as FaShionMNIST.
Briefly, the results in Fig. 20a demonstrate that a trained PCN classifier can effectively (1) assess OOD samples out-of the-thxx without requiring specific training for that purpose (Yang et al. 2021), and (2) produce energy scores for ID and OOD samples that initially correlate with softmax values prior the optimization of the states variables, h. However, after optimizing the states for T inference steps, the scores for ID and OOD samples become decorrelated, especially for samples with lower softmax values as shown in Fig. 20b To corroborate this observation, we also present ROC curves for the most challenging samples, including only the lowest 25% of the scores. As shown in 22g the probability (i.e., energy-based) scores provide a more reliable assessment of whether samples are OOD. Experiment details and results on other datasets are provided in in Appendix [F] Additional, and more detailed results for the EMNIST (letters) and KMNIST datasets are provided below.
Results. In the following we briefly interpret the additional results the basis of experiments supported by various figures
(10)
Los Tregerce
Encrgy
Energy
EMNIST (OOO)
ABl Inf ENNIST (O00)
Informnca
Log Freqanoi
08
1.2
08
0.2
3.4
0.5
1.3
Bef. Int. notMNIETT (OOD)
AR. Inf. noMMNIT (OOD)
Log Prequarcs
0.5
02
Energy
Energy
Bul. FMNIST (OCO)
In!. FMNIST
3.
Log Proqure
2
Energy
Energy
Bor, Inf, KMNIST (000)
Att Int. KNINIST (OOD)
Retorn Valvnnce
Log Pracunncy
kng
6.
0.
0.6
1.0
Figure 21: Energy distributions before and after state optimization.
30

---

Published as a conference paper at ICLR 2025
In Fig. 21 we see how the energy is distributed at test-time before and after state optimization. We can see, that all OOD datasets have significantly larger initial energies as well as final energies compared to the ID dataset (MNIST).
Bef. Inf. EMNIST
Aft. Inf. EMNIST
WNSS. (() EMNIST
MNSTT (DD)
EMNIST
Prequcncy
Log Frequerccy
10'
002
0.44
008
2.1
Energy
Energy
Bel. Inf. NOTMNISTT
Alt. Inf. NOTMNIST
141
MN-T (D)
MNIST (D)
NOTMNIST (000)
NOTMMST (OOD)
Log Frequency
tef
10'
6.6.
0/8
606
3.54
0.12
6:14
Energy
Energy
Inf. FMNIST
Aft. FMNIST
WNIS-(())
FMNIST- (000)
FMNIST (OOD)
Frequency
Log Frequency
10¹
Enerryy
Energy
Bef. Inf. KMNIIT
Aft. Inf. KMNIST
WNST (D) KNNIST
MNST (D)
KMNIST (OCO)
Proquency
Log Frequency
10'
00
0.7
0.4
0.5
1.0
0.00
0.09
0.98
5.56
0.00
0.10
Energy
Energy
Figure 22: Energy histograms against ID data before and after state optimization.
In Fig. 22 we then show how each energy distribution for the OOD dataset compares against the energy of the in-distribution dataset by overlaying the histograms of the energies before and after state optimization. We can see that by plotting the histograms, a pattern emerges, namely, that a majority of the OOD data samples do not overlap with ID data samples, which supports the idea that energy can be used for OOD detection.
Next in Fig. 23 we show how this pattern might look like when comparing the softmax scores of ID against OOD datasets. One can see, that the softmax scores are less informative for determining samples are OOD as can be seen by the bigger overlap in the range of softmax values that ID and OOD samples have in common.
In Fig. 24 we further study the relationship between softmax scores and energy values before and after state convergence. The plot shows that while the energy and softmax scores are strongly correlated before inference, non-linear relationship is evident after convergence, especially for smaller values where the model is more uncertain. This indicates, that softmax scores and energy values do not fully agree on which samples we should have less confidence in.
31

---

Published as a conference paper at ICLR 2025
Softmax Scores Distribution EMNIST
10
MNIST (ID)
FMNIST (OGD)
Frequency
10/
Scores
Softmax Scores Distribution NOTMNIST
MNBTT
NOTMAET
10'
Softmax Scores
Softmax Scores Distribution FMNIST
16
MNIST (D)
FMMET
Prequonsy
10
Softmax Scores
Softmax Scores Distribution KMNIST
10"
KMNIST (OCD)
10'
Scores
Figure 23: Softmax histograms overlapped with dataset.
In Fig. 25|we we show how the energy distributions for all datasets look like before and after inference. Each box plot represents a different scenario and a different dataset. In addition, we compute the NLL of each dataset and display the initial and final energy values are significantly higher than the MNIST (ID) dataset. Furthermore, we can see that the variance of the energy scores is smaller for the in nn-ditrribution data can be seen by the fact, that there are outlier samples for MNIST beyond the whiskers of the box plot. Finally, the NI.L values for each scenario confirm this observation, with the likelihood of the MNIST data being significantly higher than that of the OOD distributions.
part of the box plot labels. We observe that across all OOD datasets,
Finally, in Fig. 26 we show how the PCN can be used to classify samples as belonging to the ID or some OOD data. We use the PCN classifier's energy to perform OOD detection and we show that the ROC curves for energy- based detection are superior to ROC curves created via softmax scores. This observation becomes even clearer, when looking at the most challenging samples by picking the 25% percentile of the scores and energies, in effect the samples, that the PCN model is least confident about reflected by small energy softmax values.
32

---

Published as a conference paper at ICLR 2025
SANST (FT)
0.125
MINSST (F))
EXNSST (000)
EXINAST (900)
Energy
6005
0425
0000
0.5
0.5
0.5
Softmax Scores
Softmax Scores
Del. Inf.
Art Inf.
MRAST NOTUNIST
0:15
MNAST (b)
NOTMMBT (000)
Energy
Enegg
0.10
Scores
Bel. Inf.
Alt Inf.
0:15
MNIST 08:
(00K)
FNNNSST (S0O)
Energy
Energ
0.05
02
0.3
0.8
0.5
0.8
0.7
0.8
0.0
1.0
02
0.2
0.4
0.5
0.5
0.7
0.8
0.0
1.0
Softmax Scores
Softmax Scores
Aft.
1.5
SANSTT (ID)
MINIST (D)
KURAST (000)
KAINSSS (000)
Energe
10
Energy
0:10
060
02
03
04
0.5
0-8
0.7
0.8
0.0
Softmax Scores
Softmax Scores
Figure 24: Non-linear relationship between energy and softmax scores.
-25
Log Energy
-7.5
-10.0
-1500
-7.5
MUIST (1)) NLL.
235.8
ANIST (D)
NLL
33.9
EXNIST Bet NLL
1511.3
EMNNSTT NLL
179.
FOMMIST
Bel NLL. 23443
RoMMIIT
All NLL. 8
FMNIST
Bel NLL.
1517.7
FMNIST BLLL 167.3
KUNIST Bal NL.L 21127
KUNIST NLL. 244.5
Data Type
Figure 25: Energy and NLL for various OOD datasets before and after inference.
G COMPUTATIONAL RESOURCES
Fig. was obtained by taking a small feedforward PCN made by layers of 64 neurons each and training on batches of 32 elements (generated random noise avoid any overhead due to loading training data to the GPU) for T = steps. Then, each parameter was scaled independently measure its effect the total training time. Each model obtained this way trained for
33

---

Published as a conference paper at ICLR 2025
EMNIST
NOTMNIST
1.0
1.0
True Positive Rate
True Positive Rate
0.8
0.8
0.6
0.6
0.4
0.4
0.2
Softmax (AUC= = 0.8365)
0.2
Softmax (AUC = 0.9128)
Energy (AUC 0.8408)
Energy (AUC 0.9225)
Sottmax
0.5681)
Softmex perc., AUC 0.5922)
Energy (25th AUC
AUC 0.6188)
Energy (25th perc., AUC = 0.7419)
%0.00
00.00
0.25
0.50
0.75
1.00
0.75 False Positive Rate KMNIST
0.25
0.50
1.00
False Positive Rate FMNIST
1.0
1.0
True Positive Rate
True Positive Rate
0.8
0.8
0.6
0.6
0.4
0.4
Softmax (AUC 0.8255)
Softmax (AUC 0.8967)
0.2
0.2
Energy (AUC = 0.8262)
Energy (AUC 0.9033)
Softmax (25th perc.. AUC 0.5626)
Softmex (25th perc., AUC 0.5877)
Energy perc.. 0.5722)
Energy perc.,
00.00
08.00
0.25
0.50 0.75
1.00
0.50 0.75 False Positive Rate
0.25
1.00
False Positive Rate
Figure 26:
Performing OOD detection with PCN energy and classifier softmax scores.
epochs and the mean time reported. In all timing measurements, skip the first epoch avoid including the JIT compilation time. Results were obtained on a GTX TITAN X, showing that parallelization is potentially achievable also on consumer GPUs.
34


Reviews:
[{"review_id": "63hHkdvMT3", "reviewer": "Reviewer_sAME", "summary": "The authors undertake the very hard task of trying to streamline, standardise and robustify the scientific process in the subfield of Predictive Coding networks. They identify that the subfield suffers from a variety of issues such as poor comparisons between models, too simple baselines or benchmarks, and overall a lack of rigour in how potential proposed techniques are studied, compared, contrasted and analysed. This leads to the issue of making important issues hard to identify, and even harder to get a good critical mass of people working on them for resolution. \n\nThe authors try to improve on this issues by offering: \n\n1.  A unified benchmark for predictive coding network models that is more expansive than others before it, utilising CIFAR100, Tiny ImageNet, FashionMNIST and MNIST. \n2. A library that allows quick and easy application of the benchmark on a target approach. \n3. A thorough evaluation of a number of key architectures and PC approaches on their benchmark, that can serve both a source of insights for the current state of the field and a good starting baseline for future approaches. \n\nOverall the proposal of the authors is an important and valuable one, and while the writing is OK, it is at times sloppy, with fluency and spelling mistakes, as well as inconsistent use of various terms. \n\nHowever overall a very good paper.", "questions": "1. Why only JAX? Why not Pytorch? It seems like this goes against the spirit of the inclusiveness of your work. \n2. On Scalability Solutions: While the paper does an excellent job of identifying scalability issues in PCNs, do the authors have any preliminary ideas or suggestions for future work that could address these challenges?\n\n3. Efficiency vs. Backpropagation: Given that the proposed PCX library is still slower than traditional backpropagation for complex models, are there any plans or ongoing work to bridge this efficiency gap?\n3. Spelling, grammar and clarity points:\n\nHere are the sections that could benefit from improved clarity, fluency, or grammar:\n\n1. Line 110-111:\n\"different directions of research that either explored interesting properties of PC networks, such as their robustness and flexibility (Song et al., 2024; Alonso et al., 2022) , or proposed variations to\"\n- Extra space before comma after parenthesis\n- Somewhat awkward phrasing\n\n2. Lines 180-182: \n\"In practice, we do not train on a single pair (x,y) but on a dataset split in mini-batches that are subsequently used to train the model parameters.\"\n- Should be \"split into mini-batches\"\n\n3. Line 229: \n\"While interesting and important for the progress of the field,\"\n- Awkward start to sentence, could be rephrased for better flow\n\n4. Line 270:\n\"We test the performance of PCNs on image generation tasks.\"\n- Somewhat abrupt transition, could use better connection to previous section\n\n5. Line 394:\n\"We observed a further link between the weight optimizer and the structure of a PCN that might hinder the scalability of PC\"\n- Run-on sentence that could be clearer if split or restructured\n\n6. Line 443:\n\"PCX extensively relies on Just-In-Time compilation.\"\n- Inconsistent capitalization of \"Just-In-Time\" compared to other instances of \"just-in-time\"\n\n7. Line 469:\n\"In fact, in its current state, JIT is unable to parallelize the executions of the layers;\"\n- \"executions\" should be singular \"execution\"\n\n8. Line 476:\n\"we have lied new foundations\"\n- Should be \"laid new foundations\"\n\n9. Line 485:\n\"When this condition is released,\"\n- Should be \"When this condition is relaxed,\" or \"When this limitation is removed,\"\n\n10. Throughout the paper:\n- Inconsistent spacing around mathematical symbols and equations\n- Some inconsistent use of commas in complex sentences\n- Occasional mixing of American and British English spellings\n\nFigure 1: \n\nYou want a figure to stand on its own without any context from the paper ideally. What is F here? Can we improve the clarity and informativeness of the caption and the figure itself?\n\nThese issues are relatively minor and don't significantly impact the paper's readability or scientific contribution, but addressing them would improve the overall polish of the manuscript.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- Addresses Core Issues in PC Research: The paper tackles fundamental shortcomings in the subfield, such as inconsistent comparisons, overly simple benchmarks, and a lack of rigorous, standardized practices.\n\n- Unified Benchmarking Framework: Introduces a comprehensive and standardized benchmark for PCN models, incorporating major datasets like CIFAR-100, Tiny ImageNet, FashionMNIST, and MNIST to ensure consistent and meaningful comparisons.\n\n- Implementation of Open-Source Tool (PCX): Offers a JAX-based library that enhances research efficiency and accessibility, boasting up to 50x speed improvements via JIT compilation and seamless compatibility with modern ML tools.\n\n- Thorough Experimental Evaluation: Conducts extensive tests on key PCN architectures and variants, backed by large-scale hyperparameter studies, establishing a solid baseline for future advancements.\n\n- Deep Analysis of Training Mechanics: Provides valuable insights into energy propagation, initialization strategies, and stability across optimizers and architectures, which can guide targeted future research.\n\n- Community-Centric Contribution: The open-source nature, complete with detailed documentation and tutorials, lowers the barrier to entry and encourages collaboration, setting a strong foundation for collective progress.\n\n- Motivation for Future Research: Clearly identifies current limitations, such as scalability and energy flow issues, laying the groundwork for targeted solutions and deeper exploration.\n\n- Competitive Performance on Benchmarks: Shows that PCNs can achieve performance on par with BP-trained models for small to medium-sized architectures, establishing a base for scaling up and further innovation.\n\nInsightful 'weaknesses' highlighted by the paper that should not be perceived as a weakness of the paper itself imho.\n\nIdentification of Scalability Challenges: The paper highlights scalability issues inherent to PCNs but does not propose new solutions. While this reflects well on the depth of the analysis, it may be seen as a limitation in terms of contribution.\nPerformance Bottlenecks in PCNs: The analysis shows that while the proposed library offers performance improvements over prior implementations, it remains slower than backpropagation for more complex models. This insight underscores the current challenges facing PCNs but could be perceived as a weakness in terms of efficiency.\nSequential Processing Constraints: The authors identify that sequential layer updates are a significant bottleneck, impacting parallelism and efficiency. This is a critical finding for the field, though it might be seen as a shortcoming in practical use.\nTheoretical Insight Depth: The paper provides an analysis of training mechanics, but it could go further in offering deep theoretical explanations for issues like energy propagation and training stability. This gap invites further exploration but could be seen as a weakness by readers looking for comprehensive theoretical coverage.\nImplementation Constraints and Parallelization: The paper highlights that training time scales linearly with the number of layers due to current parallelization limitations. This insight is crucial for understanding the limitations of PCNs but might be perceived as a practical constraint in the work presented.", "weaknesses": "Weaknesses of the Paper:\n- Writing and Terminology: The paper has inconsistencies in terminology and occasional grammar and spelling issues, along with unclear phrasing that affects readability and comprehension.\n- Comparative Analysis Limitations: The paper could be strengthened by including more comparisons with other biologically inspired approaches and better situating its findings within the broader machine learning context.\n- Practical Implications: The paper lacks a thorough exploration of how its findings translate to practical, real-world applications beyond academic benchmarks.\n- Diversity of domains/tasks/modalities while a lot more than anything other out there for PC research, is still very limited compared to much more general benchmarks out there for general deep learning evaluation. Adding more domains, tasks and modalities would be very useful to having rigorous and insightful works done by people using it. \n- Architectures applied do not include any resnet, densenet or transformer architectures -- yes depth is an issue I understand, but using a shallow variant of any of these would be very useful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HgJNl0i6w7", "reviewer": "Reviewer_oRBc", "summary": "The paper contributes a comprehensive library/framework that unifies the developing and testing for predictive coding networks (PCNs). Traditionally, the field suffers from lacking of unified metrics and open-source implementations, the contributed library well addresses the problem. Furthermore, with the contributed library, the paper is able to scale existing works onto larger datasets and achieves new SOTA on all the datasets. The paper also includes the popular algorithms in the field and benchmarked them extensively.", "questions": "1. Can we test some models beyond VGG and MLP as mentioned in weakness? \n2. For the results in table 1, the result of VGG-5 is better VGG-7, does that mean PC works better when the number of layers is small. I think we need more data points here.\n3. Also for table 1, can you include the same datasets for both VGG-5 and VGG-7, currently, the missing data points (CIFAR-10) makes it both hard to read the table and hard to make more in-depth comparison between these two cases.\n4. I think it's better to provide the URLs at the beginning of the paper instead of at the end.\n\nI will consider raising the scores if my questions can be addressed.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The benchmark results included in this work is very extensive which includes most of the popular baselines.\n2. The design and usage of the contributed library are well documented with provided repos (I didn't run the code in this repo).\n3. The analysis provided in the paper is well explained such as showing different results from different methods which makes the paper easy to read\n4. With the contributed library, the paper is able to scale up the field to larger datasets which greatly improved the applicability of the methods in the field.\n5. The work also shared some insights for sub-optimal cases of PCNs which can further guide the development of new methods.", "weaknesses": "1. Some figures can be improved, e.g. for figure 2, the method name can be draw above the images for better representation.\n2. The main models used in the paper is MLP and VGG, since the library has been developed, we should probably test a few more model types such as resnet (shallow is OK if deep ones cannot be test) or vit. This could probably real more insights on the overall states of PCN algorithms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iH2lso5y7Q", "reviewer": "Reviewer_9Atp", "summary": "This paper addresses the trend in predictive coding networks (PCNs) literature of not comparing results against each other. To this end, the authors present a library that facilitates implementing and testing new PCNs, and they also propose several new benchmarks for evaluation. These benchmarks include larger networks and more complex tasks than those in previous works, and span discriminative as well as generative tasks. The paper evaluates several existing methods, including standard backpropagation, on these newly proposed benchmarks and analyzes the results.", "questions": "Can you motivate the use of VGG-5 and VGG-7 models in the benchmarks? In terms of model sizes, they are roughly 3.8M and 4.5M, respectively, so they’re quite close to each other.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "### Originality\nThe main contribution of the paper is the PCX library which can be used to implement predictive coding networks, and the proposed benchmarks, which can be used to evaluate the scalability of new predictive coding networks.\n\n### Quality \nThe PCX library, owing to Just-In-Time compilation, is much faster than the baseline library. The reported performance (training times) of the PCX library shines against the baseline by Song et al.\n\n### Clartity\nThe tutorials are of good quality. The Jupyter Notebook tutorials released with the paper are easy to set up and follow.\n\n### Significance\nThe paper is well motivated.\n1. Providing the community with the tools and benchmarks to compare their methods against others’ is a meaningful endeavour.\n2. The benchmarks are designed to address scalability of PCNs.\n3. The benchmarks are not limited to only discriminative tasks like classification, but also include generative tasks such as autoencoding and sampling from a learned joint distribution.\n\nThe following insights into PCNs were very interesting:\n1. The AdamW optimizer performs poorly as the hidden layer size increases. This was also demonstrated well in the tutorial notebook.\n2. Skip connections have a dramatic effect on the performance of PCNs.\n\nThe evaluation is extensive. A fair number of existing PCN methods were evaluated (standard PC, positive/negative/centered nudging PC, incremental PC, Monte Carlo PC).", "weaknesses": "I gave a score of 3 for Contribution but only 1 for Soundness. This is because I believe the PCX library is a valuable contribution to the community; however, I have reservations about some of the conclusions and claims made in the paper.\n\nConsider the Energy propagation subsection of Section 5.1 (Energy and Stability). L377 says, “Note that the decay in performance as function of increasing γ is stronger for Adam despite being the overall better optimizer in our experiments. This suggests that possible directions for future improvements should aim at reducing the energy imbalance between layers.”. It is unclear to me how this conclusion that energy imbalance must be reduced was arrived at. I understand energy imbalance to be different from energy ratios in that the imbalance is minimized when the ratio is 1, i.e., both the layers have the same energy.\nIn Figure 5 (right), we see that the learning rate which minimizes the energy imbalances (i.e, make the ratio as close to 1 as possible) is 0.3. In Figure 5 (center right), we see that the performance of AdamW optimizer for a learning rate of 0.3 is much worse than 1e-3, 3e-3, and 1e-2, all of which have much higher energy imbalances. The same pattern can be observed in Figures 13 and 14. How do we conclude from this behaviour of AdamW that the energy imbalance must be minimzed? This also affects the claim in the abstract that the paper “clearly highlights what the current limitations of PCNs are”.\n\nNext, in Section 4.1 (Discriminative Mode), L243 says that \"The results show that the best performing algorithms, at least on the most complex tasks, are the ones where the target is nudged towards the real label, that are PN and NN.\". The results for Discriminative mode are reported in Table 1. Assuming, based on L067, that complex tasks refer to CIFAR100 and Tiny ImageNet, I am unable to find data in the table that substantiates this claim. For VGG-5 models, centered nudging (CN) dominates both positive nudging (PN) and negative nudging (NN) for all the tasks except Tiny ImageNet (Top-1).\nFor VGG-7 models, all PC methods are beaten by either of the two variants of Back Propagation (BP-CE and BP-SE). Even considering only the PC methods, PN is consistently beaten by PC-CE, as is NN, with the exception of CIFAR-100 (Top-1).\n\nFinally, in the Discussion subsection of Section 4.2 (Generative Mode), line 339 claims that “Thus, compared to artificial neural networks, PCNs are more flexible and require only half the parameters to achieve similar performance”. The fact that PCNs require only half as many parameters as an autoencoder neural network is evident because they can infer the hidden states of layers that have not been fixed (either to the input image for compression or to the compressed representation for generation). However, the paper demonstrates that this holds true only for small models, as the generative experiments were conducted exclusively on relatively small models (3 layers for autoencoding and generation via sampling, and 2 hidden layers for associative memory experiments). Conducting at least one experiment on a larger model, such as a VGG-7-scale model, could have strengthened this claim.\n\nI am open to reconsidering my scores if these concerns are addressed.\n\nThe following is feedback and minor nitpicks about the presentation of the paper. Addressing these will improve the quality of the paper, but will not affect the scores.\n\n1. The abbreviation “BP” is used in the paper without introducing it. The first reference to it is in the Appendix.\n2. Figure 1 (b). Shouldn’t it be “hL is fixed to y”?\n3. Table 3 can be made more intuitive to read by mentioning in the caption that it is a comparison of the two modes of associative memory tasks - Noise and Mask.\n4. H and theta should be subscripts in L141\n5. Citations are missing for several works (which, admittedly, a lot of us take for granted). Examples are CIFAR-10, MNIST, Fashion-MNIST, Tiny ImageNet, SGD, Adam, AdamW, VGG models.\n6. Citations for each of the methods in the Algorithms subsection of Section 4 would make it easier for the readers to refer to the papers, even though they are all cited before.\n7. Figure 5 would be better if it were indexed as Fig 5a to Fig 5d, instead of Fig 5 left, center-left, center-right and right.\n8. The citation style in L239 is wrong.\n9. L347 “How regularly the energy flows into the model”. Improve phrasing.\n10. All references to Figure 5 right and center-right are inverted. See, e.g., L373 and L377.\n11. Grammatical errors in a few places, such as L267, L355, L467", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)