A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks
Authors: Tommaso Salvatori
Yuhang Song
Yordan Yordanov
Beren Millidge
Lei Sha
Cornelius Emde
Zhenghua Xu
Rafal Bogacz
Thomas Lukasiewicz
Venue: ICLR 2024 poster (2024)
Keywords: Cognitive Science
parallelization
Primary area: applications to neuroscience & cognitive science

Abstract: Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. Training such models, however, is quite inefficient and unstable. In this work, we show how by simply changing the temporal scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one, and has theoretical guarantees in terms of convergence. The proposed algorithm, that we call incremental predictive coding (iPC) is also more biologically plausible than the original one, as it it fully automatic. In an extensive set of experiments, we show that iPC constantly performs better than the original formulation on a large number of benchmarks for image classification, as well as for the training of both conditional and masked language models, in terms of test accuracy, efficiency, and convergence with respect to a large set of hyperparameters.

Published as a conference paper at ICLR 2024
A STABLE, FAST, AND FULLY AUTOMATIC LEARNING ALGORITHM FOR PREDICTIVE CODING NETWORKS
Tommaso Salvatori! Yuhang Song2..,+.1 Yordan Yordanov Beren Millidge?
Cornelius Emde", Zhenghua Xu Lei Sha Rafal Bogacz". Thomas Lukasiewicz
VERSES Al Research Lab, Los Angeles, California, 90016, USA
2 MRC Brain Network Dynamies Unit, University of Oxford, UK
3 Deppatment of Computer Science. University of Oxford, UK
4 School of Health Sciences and Biomedical Engineering, Hebei University of Technology, China
5 Institute of Logic and Computation, Vienna University of Technology, Austria
6 Fractile Ltd, London, UK
tommaso.. waalvaatriieee critterreeeeesseeeeeeeeeeseeeeee .asssaas ai, thomas www.ssee.c.etttie........ ac. at {yuhang song, beren. petren.milll www.miiiied....... hwa.fallbogaccaae @nden. {yordan. yordanov, lei. sha, cornelius emde 0 css wwwa...acoo.oo....o.o..o..ooooo.
oo. ox. ac.ut uk
ABSTRACT
Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. Training such models, however, is quite inefficient and unstable. In this work, we show how by simply changing the tem- poral scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one, and has theoretical guarantees in terms of convergence. The proposed algorithm, which we call in- cremental predictive coding (iPC), is also more biologically plausible than the show
original one, it is fully automatic. In an extensive set of experiments, that iPC constantly performs better than the original formulation on a large num- ber of benchmarks for image classification, conditional and masked language models, in terms of test accuracy, efficiency, and convergence with respect to large set of hyperparameters.
well as for the training of both
INTRODUCTION
In recent years, deep learning has reached and surpassed human-level performance in multitude of tasks, such as game playing (Silver et al. 2017 2016 Bakhtin et af. [2022), image recognition (Krizhevsky et al. [0112] He et 2016)) natural language processing (Chen al. 2020), and image generation (Ramesh et al. 2022 Saharia et al. 2022). These successes are achieved entirely using deep artificial neural networks trained via backpropagation (BP), which is learning algorithm that is often criticized for its biological implausibilities X (Grossberg 1987 1987 1989 Abdelghani et al., 2008 Lillicrap et al., 2016; Roeefssema Holtmaat] [018 W hittington & Bogaez (011)), such as lacking local plasticity and autonomy. In fact, backpropagation requires global control signal to trigger computations, since gradients must be sequentially computed backwards through the compu- tation graph. The biological plausibility of specific algorithm is not niche interest of theoretical neuroscience, but it is of vital importance when comes to implementations on low-energy ana- log/neuromorphic chips: parallelization, locality, and automation els that can be trained end-to-end on non-von- non-von-Neumann Neumann machines, such as analog chips (Kendall et al. 2020). To this end, multiple works are highlighting the need of fueling fundamental research in computational neuroscience to find algorithms and methods that can solve the aforementioned problems (Zador al. 2022 Friston et al.) 2022) A promising learning algorithm in this regard, which has most of the above properties, is predictive coding (PC).
key to building efficient mod-
PC is an influential theory of information processing in the brain (Mumford 1992 Friston 2005). where learning happens by minimizing the prediction error of every neuron. PC can be shown (0
"quual contribution.
Corresponding author.

---

Published as a conference paper at ICLR 2024
approximate BP in layered networks (Whittington& & Bogacz 2017), as well as on any other model
Millidge et al.) 2020a), and can exactly replicate its weight update if some external control is added
Salvatori et al. 2002bb. Also, the differences with BP are interesting, PC allows for much more
flexible training and testing (Salvatori et al. 2022a), has a rich mathematical formulation (Friston
2005 Millidge al., 2022a). and is energy-based model (Bogac7) 2017). Simply put, PC is based on the assumption that brains implement an internal generative model of the world, needed to predict incoming stimuli (or data) (Friston et al. 2006 Friston 2010 Friston et al. 2016). When presented with stimulus that differs from the prediction, learning happens by updating internal neural activities and synapses to minimize the prediction error. In computational models, this is done via minimization of the variational free energy, in this case a function of the total error of the generative model. This minimization happens in two steps: first, internal neural activities are updated in parallel until convergence; then, synaptic weights are updated to further minimize the same energy function. This brings us to the second peculiarity of PC, which is its solid statistical formulation, developed much before its links neuroscience (Elias. 1955p. The message passing scheme of predictive coding is, in fact, an efficient way of inverting a hierarchical Gaussian genera- tive model by approximating evidence lower bound using Laplace and mean field approximations Friston et al. 2008).
Friston 2005
When applying this inversion schema to large-scale neural network training, we encounter three limitations: first, an external control signal is needed to switch from the step that updates the neu- ral activities, and the update of the synaptic weights; second, the update of the neural activities is slow, as it can require dozens of iterations to converge; third, convergence is uncertain and highly dependent on the choice of hyperparameters. Consequently, researchers working the field have always struggled with the slow training of predictive coding models, as well as the extensive hy- perparameter tuning needed reach the optimal performance. Here, we address these problems by considering variant of PC where the update of both the value nodes and the parameters are per- formed in parallel, similarly to how it done in (Ernoult et al. 2020). This algorithm is provably faster, does not require control signal to switch between the two steps, performs empirically much better, has solid convergence guarantees, and is more robust to changes in hyperparameters. We call this training algorithm incremental predictive coding (iPC). Our contributions are briefly as follows:
1. We first present the update rule of iPC, and discuss the implications of this change in terms of autonomy, and its differences and similarities with PC and BP. Then, we show its convergence guarantees by deriving the same equations from the variational free energy of hierarchical generative model using the incremental expectation matiiiiatiiiiiiiiiionn approach (iEM): it has in fact been proven that iEM converges a minimum of the loss function (Neal & Hinton 1998 Karimi et al. 2019), and hence this result naturally extends to iPC.
2. We empirically compare the efficiency of PC and iPC on generation and classification tasks. In both cases, iPC is by far more efficient than the original counterpart, as well as reaching better performance by converging to better local minima. We also compare its efficiency with that of BP in the special case of full batch training.
3. We then test method on image classification benchmarks well conditional and masked language models, showing that iPC performs better than PC, and that the more complex the task, the larger the gap in performance. We then explore metrics that go beyond standard test accura- cies, and show that the best performing models trained with PC have well-calibrated outputs, and that iPC is more parameter-efficient than BP.
2 PRELIMINARIES
In this section, we introduce the original formulation of predictive coding as a generative model proposed by Rao and Ballard (1999). Consider a generative model g: Id x RD - Rº, where Rd is vector of latent variables, called causes, y E Ee is the generated vector, and 0 R is a set of parameters. We are interested in the following inverse problem: given a vector and a generative model g, need the parameters 0 that maximize the marginal likelihood
ply. 0) ply xr, ()p(z, 0)dx.
(1)
2

---

Published as a conference paper at ICLR 2024
Inférence
Number of lireiton
PC
Layer incls
g( 8)
Weights spsbes
T
tepr: layer (ftee))
(BP)
Cuepoi layer (ffod)
Flidknn layer
IPC
inforence)
(8)
Generstive Model
(b)
Differences between PC, Z-II, and IPC
Figure 1: (a) An example of a hierarchical Gaussian generative model with three layers. (b) Com- parison of the temporal training dynamics of PC, Z-IL, and iPC, where Z-IL is variant of PC that is equivalent to BP, originally introduced in (Song et al., 2020). We assume that we train the networks on dataset for supervised learning for a period of time T. Here, f is the time axis during inference, which always starts at t = 0. The squares represent nodes in one layer, and pink rounded rectangles indicate when the connection weights
modified: PC (Ist row) first conducts inference the hidden layers, according to Eq. (6). until convergence, and then it updates the weights via Eq. (7). Z-IL (2nd row) only updates the weights at specific inference moments depending which layer the weights belong to. To conclude, iPC updates the weights at every time step t, while performing inference in parallel.
Here, the first term inside the integral is the likelihood the data given the causes, and the second is prior distribution over the causes. Solving the above problem is intractably expensive. Hence, we need an algorithm that is divided into two phases: inference, where we infer the best causes x, given both 0 and y, and learning, where update the parameters 0 based on the newly computed causes. This algorithm expectation-maxiinee maximization (EM) (Dempster et al.] 1977). The first step, which we call inference or E-step, computes p( 0), which is the posterior distribution of the causes given a generated vector y. Computing the posterior is, however, intractable (Friston 2003). To this end, approximate the intractable posterior with tractable probability distribution g( 0). To make the approximation as good as possible, we want to minimize the KL-divergence between the two probability distributions. Summarizing, to solve our learning problem, we need to (i) minimize Kil-divergence, and (ii) maximize a likelihood. We do it by defining the following energy function, also known variational free energy:
F(z, y, 0) = [L(xx 0)][p(r y, 0)) - [n(p(y, 0))
(2)
where we have used the log-likelihodd. This function is minimized by multiple iterations of the EM algorithm:
Inference (E-step): argmar,F(r, y.
Learning (M-step): 0" = ar argmaze (xx 0).
2.1 PREDICTIVE CODING
So far, have only presented the general problem. To actually derive proper equations for learning causes and update the parameters, and use them to train neural architectures, we need to specify the generative function g(x, 0). Following the general literature (Rao Ballard 1999 Friston 2005). we define the generative model a hierarchical Gaussian generative model, where the causes æ and parameters @ are defined by a concatenation of the causes and weight matrices all the layers, i.c., (z:10) ..... ..())) and 0 (0(0) ...... ((L-1)). Hence, we have multilayer generative model, where layer is the one corresponding to the generated image y. and layer L is the highest in the hierarchy. The marginal probability the causes as follows:
p(
II N'
where (1) is the prediction of layer according the layer above, given by u( 000) 0000 ((((+1)), with being non-linear function and (LL) (L), For simplicity, from on, consider
3

---

Published as a conference paper at ICLR 2024
Algorithm Learning dataset D {84) with iPC.
1: Require: For every i, x is fixed to yı-
2: for t = 0 to 7 do
3: For every i and I, update n, to minimize F via Eq. (6)
4: For every I, update each ( minimize via Eq. (7)
5: end for
Gaussians with identity variance, i.e., E 1l for every layer I. With the above assumptions, the free energy becomes
Σ, - 1,000
(5)
For a detailed formulation on how this energy function is derived from the variational free energy of
Eq. 2). we refer to (Friston| 2005 Bogacz 2017; Buckley et al. 2017 Millidge et al. 2021). Note that this energy function is equivalent to the one proposed in the original formulation of PC (Rao Ballard 1999). key aspect of this model is that both inference and learning are achieved by optimizing the same energy function, which aims to minimize the prediction error of the network. The prediction error of every layer is given by the difference between its real value x(() and its prediction (,()) We denote the prediction error by e(t) = ( - n(), Thus, the problem of learning the parameters that maximize the marginal likelihood given data point y reduces alternation of inference and weight update. During both phases, the values of the last layer are fixed to the data point, i.c. (0)) = y for each < T.
Inference: During this phase, which corresponds to the E-step, the weight parameters (()) are fixed, while the values x!!) are continuously updated via gradient descent:
Ar 7 (-e()) + f'(x(()) ((l-1) T c((-))),
where * denotes element-wise multiplication, and 0. This process either runs until convergence, or for a fixed number of iterations T.
Learning: During this phase, which corresponds to the M-step, the values are fixed, and the weights are updated once via gradient descent according to the following equation:
A -a OF/ (0(0)
((((+)))))
Note that the above algorithm is not limited to generative tasks, but can also be used solve su- pervised learning problems (Whitington & Bogacz, 2017). Assume that data point yin with label your is provided. In this case, we treat the label as the vector y that we need to generate, and the data point as the prior on (1). The inference and learning phases are identical, with the only differ- ence that
we have two vectors fixed during the whole duration of the process: (0()) yout and (T.)) yiu- While this algorithm is able to obtain good results small image image classification tasks, it is much slower than BP due to the large number of inference steps needed to let the causes converge. In what follows, we propose algorithm that addresses this limitation.
We have defined PC on hierarchical Gaussian models, as different probability distributions would result in update rules that do not minimize prediction errors (Salvatori et al.) 2023). However, the applicability of our algorithm can be easily generalized to different probability distributions (Pinchetti et al. 2022), as we also show in Section
3 INCREMENTAL PREDICTIVE CODING
What makes PC much slower than BP is its inference phase, which requires multiple iterations to converge. In this section, we address this limitation by proposing incremental PC, variation of the original algorithm where the inference and learning phases (Eqs. 6 and (7) are simultaneously performed at every time step t. This variation largely improves on the original formulation of PC in terms of both efficiency and performance, is fully automatic, and comes with theoretical guarantees given by the theory of variational inference. The pseudocode of iPC is provided in Algorithm while its dynamics is illustrated in Fig. Ib).
4

---

Published as a conference paper at ICLR 2024
Connections to BP: PC in general shares multiple similarities with BP in supervised learning tasks: when the output error is small, the parameter update of PC is an approximation of that of BP (Mil- lidge et al., 2020a); when controlling which parameters have to be updated at which time step, is possible to define a variation of PC, called zero-divergence inference learning (Z-IL.) in the litera- ture, whose updates are equivalent to those of BP (Song et al.] 2020). In detail, make PC perform exactly the same weight updates of BP, every weight matrix O' must be updated only at t = L, which corresponds to its position in the hierarchy. That is, as soon as the output error reaches a specific layer. This is different from the standard formulation of PC, which updates the parameters only when the energy representing the total error has converged. Unlike PC, iPC updates the parameters at every time step L. Intuitively, it can hence be seen as a "continuous shift" between Z-IL (and hence BP) and PC. A graphical representation of the differences of all three algorithms is given in Fig. (right), with the pseudo-codes provided in the first section of the supplementary material.
Autonomy: Both PC and Z-II. lack full autonomy, as an external control signal always needed to switch between inference and learning: PC waits for the inference to converge (or for I iterations), while Z-IL updates the weights of specific layers at specific inference moments t == I. BP is consid- ered to be less autonomous than PC and Z-IL: a control signal is required to forward signals well as backward errors, and additional places to store the backward errors are required. All these draw- backs are removed in iPC, where the only control signal needed is the switching among different batches. In a full-batch training regime, however, iPC is able to learn a dataset without the control signals required by the other algorithms: given a dataset D, iPC runs inference and weight updates simultancously until the energy F' is minimized, As soon as the energy minimization has converged, training ends.
Incremental EM: iPC can also be derived from the variational free energy of Eq. ② and minimize it using a variation of the EM, precisely developed to address the lack of efficiency of the original algorithm when dealing with multiple data points at the same time, scenario that is almost always present in standard machine learning. This alternative form, which we now present, called incre- mental EM (iEM) (Neal & Hinton 1998). Let DD = {yi)ieN be dataset of cardinality N, and g( 9) be a generative model. Our goal is now to minimize the global marginal likelihood, defined on the whole dataset, i.e.,
(8) The same reasoning also applies to the global variational free energy, which is the sum of the free energies of every single data point. In this case, the iEM algorithm performs the E-step and M-step in parallel, with no external control needed to switch between the two phases. That is, both the values and the parameters 0 are updated simultancously every time step until convergence, on all the points of the dataset. No explicit forward and backward passes are necessary, as each layer is updated in parallel. This also comes with strong theoretical guarantees,
(DD Σ, p(y,, 9).
it has been formally proven that minimizing a free-energy function such as ours (i.c., equivalent to the sum of independent free- energy functions) using iEM, also finds a minimum of the global marginal likelihood of Eq. (8) NNaal & Hinton 1998; Karimi et al. 2019). We actually provide empirical evidence that the model converges to better minima using iPC rather than the original formulation of PC in Fig. 2 and Table I
The pseudocode of iPC is given in Alg.
3.1 EFFICIENCY
In this section, we analyze the efficiency of iPC relative to both the original formulation of PC and BP. We only provide partial evidence of the increased efficiency against BP, as standard deep learning frameworks, such as PyTorch, do not allow to parallelize operations in different layers.
We now show how iPC is more efficient than the original formulation. To do that, we have trained multiple models with iPC and PC on different tasks and datasets. First, we have trained a generative model with 4 layers and 256 hidden neurons Tiny ImageNet and CIFARIO datasets, exactly as in (Salvatori et al. 2021). A plot with the energies as function of the number of iterations is in Fig. (left and centre). In both cases, the network trained with iPC converges much faster than the networks trained with PC with different values of T' Many more plots with different parameterizations are given in the supplementary material.
Comparison with PC:
subset of 100 images of the
To show that the above results hold in different setups as well, we have trained a classifier with 4 layers subset of 250 images of the MaSSionMNIST dataset, following the framework proposed in
5

---

Published as a conference paper at ICLR 2024
Tiny Imagenet
CIFARIO
PashionMNIST
120000
PC, T-8
120000
IPG
125
150
Energy
1000000
PC, Tw16
T=12
Ealegyy
100000
PC, T-12
PC, T=16
Train Lose
100
T=8
T=12
20000
75
60000
40000
1000
2000
3000
1000
2000
2000
Iteraiens
Iteations
Non-parallel Multiplications
Figure 2: Left and centre: Decrease of the energy of generative models as a function of the number of iterations performed from the beginning of the training process. Right: Training loss of different classifiers in a full-batch training regime as a function of the number of non-parallel matrix multi- plications performed from the beginning of the training process.
Table I: Test accuracy of BP, PC, and iPC different architectures trained with different datasets.
*ata augmentation was used here.
BP
PC
iPC
MLP on MNIST
98.26% + 0.12%
98.55% + 0.14%
98.54% = 0.86%
MLP on FashionMNIST
88.54% = 0.64%
1.12%
89.13% = 0.86%
CNN on SVHN
95.35% + 1.53%
+
96.45% -1.04%
CNN on CIFAR-10
69.34% + 0.54%
70.84% + 0.64%
72.54% = -0.93%
AlexNet on CIFAR-10
75.64% = 0.64%
64.63% 1.55%
72.42% + 0..3%%
AlexNet on CIFAR-10*
83.12% + 0.97%
71.99% + 2.43%
80.11% + 0.44%
Whittington & Bogacz 2017), and studied the training loss. As it is possible to train an equivalent model using BP, we have done using the same set-up and learning rate, and included it in the plot. This, however, prevents from using the number of iterations as an efficiency measure, as one iteration of BP is more complex than one iteration of PC, and are hence not comparable. As metric, have hence used the number of non- non-parallel matrix multiplications needed to perform a weight update. This is a fair metric, as matrix multiplications are by far the most expensive operation performed when training neural networks, and the ones with largest impact the training speed. Single iterations of PC and iPC have the same speed, and consist of 2 non-parallel matrix multiplications. One epoch of BP consists of 2L non-parallel matrix multiplications. The results are given in Fig. [2 (right). In all cases, iPC converges much faster than all the other methods. In the supplementary material, we provide other plots obtained with different datasets, models, and parameterizations, as well as a study on how the test error decreases during training.
Comparison with BP: While the main goal of this work is simply to overcome the core limitation of original PC (namely, the slow inference phase), there is one scenario where iPC is potentially more efficient than BP, which is full batch training. Particularly, we first prove this formally using the number of non-parallel matrix multiplications needed to perform a weight update To complete one weight update, iPC requires two sets of non-parallel multiplications: the first uses the values and weight parameters of every layer to compute the prediction of the layer below; the second uses the error and transpose of the weights to propagate the error back to the layer above, needed to update the values. BP, on the other hand, requires 2L sets of non- non-parallel multiplications for a complete update of the parameters: L for a forward pass, and L for backward one. These operations cannot be parallelized. More formally, we prove theorem that holds when training on the whole dataset 'D in a full-batch regime. For details about the proof, an extensive discussion about time complexity of BP, PC, and iPC, we refer to the supplementary material.
metric.
Theorem 3.1. Let M and M' be two equivalent networks with L layers trained on the same dataset. Let M be trained using BP, and M' be trained using iPC. Then, the time complexity needed to perform one full update of the weights is ((1) for iPC and (T.) for BP.

---

Published as a conference paper at ICLR 2024
Table 2: Change of final accuracy when increasing the width.
C
1
2
3
4
5
6
7
8
10
15
20
BP
67.92
71.23
71.65
72.64
73.35
73.71
73.71
74.19
74.51
74.62
75.08
75.51
iPC
70.61
74.12
74.91
75.88
76.61
77.04
77.48
77.41
76.51
76.55
76.12
4
CLASSIFICATION EXPERIMENTS
We now demonstrate that iPC shows similar level of generalization quality compared BP. We test the performance iPC on different benchmarks. Since we focus on generalization quality in this section, all methods are run until convergence, and we have used early stopping to pick the best performing model. These experiments were performed using multi-batch training. In this case, we lose our advantage in efficiency over BP, need to recompute the every time new batch is presented. However, the proposed algorithm is still much faster than the original formulation of PC, and yields better classification performance.
We investigate image classification benchmarks using PC, iPC, and BP. We first trained a fully connected network with 2 hidden layers and 64 hidden neurons per layer on the MNIST dataset (LeCun & Cortes, 2010). Then, we trained a mid-size CNN with three convolutional layers with
Setup of experiments:
128 64 kernels followed by two fully connected layers on FashionMNIST, the Street View House Number (SVHN) dataset (Netzer et al. (011)) and CIFARIO (Krizhevsky et al.) 2012]. Finally, we trained AlexNet (Krizhevsky et al. (012). a large-scale CNN, on CIFARTO. To make sure that our results are not the consequence of specific choice of hyperparameters, we performed a comprehensive grid-search on hyperparameters (more details the supplementary material), and reported the highest test accuracy obtained. We have also carefully checked whether the energy/loss of every model had converged, and this was indeed the case. Hence, the worse performance of PC on AlexNet is probably due to scaling properties of PC, rather than converged network. This is a problem that we have not experienced using iPC, able to well scale to larger architectures.
Convergence: In the experiments on AlexNet, under all the hyperparameter combinations tested, iPC only failed converge when the learning rate of the weights was the largest (0.01). In total, it converged 88 times out of 96 combinations of hyperparameters. This is not the case for PC, which converged in only 26 combinations of hyperparameters out of 96 (We consider model be converged, if the difference between its best test accuracy, and the best test accuracy reached over the whole hyperparameter search, less than 10%).
Results: In Table T iPC constantly outperforms PC, besides in the simplest framework (MNIST on small MLP), where PC has a tiny margin of 0.01%. But PC fails to scale to more complex problems, where it gets outperformed by all the other training methods. The performance of iPC, on the other hand, is stable under changes in size, architecture, and dataset, and comparable to the one of BP.
Change of width: To investigate how iPC behaves when adding max- maxx-pooling layers and increasing the width, we trained CNN with three convolutional layers (8, 16, 8) and maxpools, followed by fully connected layer (128 hidden neurons) on CIFAR10. 100 We have also replicated the experiment by increasing the width of the network by multiplying every hidden dimension by a constant C (e.g., C = 3 means a network with 3 convolutional layers (24, 48, 24), each followed by a maxpool, and fully connected one (384 hidden neurons)). The results in Table / show that iPC (i) outperforms BP under each parametrization, (ii) needs less parameters to obtain good results, but (iii) sees its performance decrease, once has reached a specific parametrization. This is in contrast BP, which is able to generalize well even when extremely overparametrized. This suggests that iPC is more efficient than BP in terms the number of parameters, but that finding the best parameters for iPC may need some extra tuning.
4.1 ROBUSTNESS AND CALIBRATION
Robustness and uncertainty quantification in deep learning have become a topic of increasing interest in recent years. Recently, it has been noted that treating classifiers generative models benefits the
7

---

Published as a conference paper at ICLR 2024
Caltration Einor (MENECE)
06
Tapicos
0.4
2.0 Data Shilt Intensity
Figure 3: Left: Robustness of BP and iPC under distribution shift (AlexNet on CIFARIO under five different intensities of the corruptions rotation, Gaussian blur, Gaussian noise, huc, brightness, and contrast). iPC maintains model calibration significantly better than BP under distribution shift. Right: Dev perplexity during training of the best performing masked language models.
robustness of the model (Grathwohl et al. 2019). The same result is obtained by adding layers, or message passing schemes, that simulate the visual cortex of primates (Dapello et al. 2020 Choksi et al. 2021). Specifically about PC, it has been shown that the training procedure is more stable, as replicates explicit gradient descent schemas (Alonso al., 2022), and that it learns more robust representations (Song et al., 2022) Byiringiro et al., [022). We now empirically show that this also extends to iPC by comparing its robustness and calibration capabilities with the ones of BP. Calibration describes the degree to which predicted logits matches the empirical distribution of observations given the prediction confidence. One may use the output of a calibrated model to quantify the uncertainty in its predictions and interpret it as probability not just model confidence. Let P be our random prediction vector indicating the confidence that the prediction Y is correct. We say that P is well-calibrated, if the model confidence matches the model performance, i.e., F(Ý = P p) = (Guo et al. 2017). We measure the deviation from calibration using the adaptive expected calibration error (AdaECE), which estimates B[II(( = Y p p) pl] (Nguyen & O'Connor 2015). In recent years, it has become well-known that neural networks trained with BP tend to overconfident their predictions (Guo et al. 2017) and that miscalibration increases dramatically under distribution shift (Ovadia et al. 2019).
Our results are shown in Fig. 3] The boxplot indicates the distributions of calibration error over various forms of data corruption with equal levels of intensity, which differ strongly between iPC and BP: The iPC-trained model yields better calibrated outputs and able to much better signal its confidence. This is essential for using the model output as indication of uncertainty. On in- distribution data, we observe that iPC yields an average calibration error of 0.05, whereas BP yields 0.12. Moreover, observe that the increase in calibration error is lot weaker for iPC: The median calibration ситоr of the iPC model is lower across all levels of shift intensitics compared to that of BP for the mildest corruption. Furthermore, iPC displays better calibration up level 3 shifts than BP does on in-distribution data. This has potentially a strong impact of applying either method in safety-critical applications.
Results:
LANGUAGE MODEL EXPERIMENTS
A recent work has shown that it is possible to introduce small modification to the training algorithm of PC improve its performance on small language models (LMs) (Pinchetti al. 2022). Here, we test the performance of PC, iPC, and BP on BERT, popular encoder-only transformer language model architecture (Devlin et al. 2019). This model trained to reconstruct randomly masked tokens from the input. To improve the range of our study, we also train conditional version of BERT, where we add a triangular mask to the attention mechanism, so that the model generates each token only based the previous tokens in the text. This creates a decoder-only language model with a similar architecture to GPT (Radford et al., 2018).
Setup: The training and dev datasets are generated by randomly sampling 200, 000 and 10, 000 instances, respectively, from the One Billion Word Benchmark (Chelba et al. 2013). The test dataset is the original test dataset of the IB Word Benchmark. For both models, blocks with one head and a hidden size of 128. The vocabulary is obtained via byte-pairenncodng with 8001 tokens, generated via the SentencePiece tokenizer (Kudo & Richardson 2018). After
two transformer
8

---

Published as a conference paper at ICLR 2024
the best hyperparameters are selected, we rum each method 9 additional seeds for total of 10 seeds. This allows us to compare the expected perplexity (ppl) performance and see the performance convergence threshold to discard those models that do not
variation across seeds. We also converge. For iPC and BP, we define the convergence threshold at 200 test perplexity, and for PC, define it 800. For complete per-seed results, well as all the details needed to reproduce the results, we refer to the supplementary material.
Table 3: Perplexity of the three compared methods on 10 random seeds, for both the masked and conditional language models, with the number of converged models.
BP
PC
iPC
Model
Perplexity
#Conv
Perplexity
#Conv
Perplexity
#Cony
Masked LM
120.02 + 13.19
7
523.08 + 12.3
3
106.19 + 10.54
10
Cond. LM
113.32 0.36
10
206.34 + 6.46
10
142.54 + 4.23
7
Our experiments show that iPC significantly outperforms PC in both masked and condi- tional language models. For masked LMs, iPC also exhibits a much better convergence, with all 10 seeds converging, whereas PC has only 3 converging seeds. The poor performance of PC is due to its poor training stability, as evident by Fig. (right), where we can also see that the training curves of iPC and BP are similar. In fact, iPC performs similarly BP in terms of test perplexity, with iPC performing better than BP on masked LMs with 106 vs. 120 ppl (where all 10 runs converged), and worse on conditional LMs with 113 vs. 143 ppl (where runs have not converged). The results are reported in Table 3 We can then conclude that the experiments performed on language models showed that iPC is significantly better than PC in terms of both performance and stability, obtaining results that are comparable to those of BP.
Results:
6 RELATED WORKS
perrooccience-inspiie algorithms have recently gained the attention of the machine learning com- munity, and multiple works have used PC tackle machine learning problems, from generation tasks (Ororbia & Kifer, 2020). to image classification on complex datasets such as ImageNet (He) al. 2016), associative memories (Salvatori et al.] |021 Tang al., 2023), continual learning OOroriia et al. (020), and NLP (Pinchetti et al.) 2022). In terms of potential implementations on neuromorphic chips, there are multiple lines of work that are parallel to PC, such as local represen- tation alignment (Ororbia II et al. / Ororbia & Mali 2019). equilibrium propagation (Scellier Bengio, 2017), feedback alignment (Lillicrap et al., 2016). and SoftHebb (Journé et al. 2022). Theoretical works, on the other hand, have studied the similarities between PC, backpropagation, and the aforementioned algorithms (Millidge et al. (022b]c).
7 DISCUSSION
Researchers working in the field of predictive coding have certainly experienced the slow and un- stable training of predictive coding networks. In this paper, we have proposed variation of PC that enables all the computations to be executed simultaneously, locally, and autonomously, and has theoretical convergence guarantees in asymptoti time (Karimi al., 2019). This allows a solid gain in efficiency compared to the original formulation of PC, as shown with extensive experiments, as well as improved performance and robustness in all the considered tasks. Many other works that speed up training algorithms and converge to better minima, such as ADAM optimization (Kingmal & Ba 20114), have had huge impact in the communities that they were proposed in, while also being simple on the surface. Similarly, we forsee that many researchers working in PC can now use the proposed update rule, which comes with no apparent drawbacks with respect to the original one. The fact that it empirically converges to better minima also allows PC to reach performance comparable to those of BP on complex tasks, such as image classification in convolutional models, or language generation in transformer models.

---

Published as a conference paper at ICLR 2024
8 AKNOWLEDGEMENTS
Beren Millidge and Rafal Bogacz were supported by BBSRC grant BB/S006338//1 Rafal Bogacz was supported by MRC grant V.C.0U00000//I. This work was also supported by the AXA Re- search Fund, by the Alan Turing Institute under the EPSRC grant EP/N510129/1 and by the EP- SRC grant EP/RO1 3667/1. We also acknowledge the use of the EPSRC-funded Tier 2 facility JADE (EP/P020275/1) and GPU computing support by Scan Computers International Ltd. C. Emde is supported by the EPSRC Centre for Doctoral Training in Health Data Science (EP/S02428X/1) and Cancer Research UK (CRUK).
REFERENCES
Mohammed. Abdelghani, Timothy. Lillicrap, and Douglas Tweed. Sensitivity derivatives for flexible sensorimotor learning. Neural Computation, 20(8):2085-2111 2008.
Nick Alonso, Beren Millidge, Jeffrey Krichmar, and Emre O. Neftci. A theoretical framework for inference learning. Advances in Neural Information Processing Systems, 35:37335-37348, 2022.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. arXiv: 2016.
Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al. Human-level play in the game of diplomacy by com- bining language models with strategic reasoning. Science, 378(6624):1067-10000
2022.
Rafal Bogacz. A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematicul Psychology, 76: 198-211, 2017.
Chris Buckley, Chang Kim, Simon McGregor, and Anil Seth. The free energy principle for action and perception: A mathematical review. Journal of Mathematical Psychology, 2017.
Billy Byiringiro, Tommaso Salvatori, and Thomas Lukasiewicz. Robust graph representation learn- 2022.
ing via predictive coding, arXiv preprint
Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. One billion word benchmark for measuring progress in statistical language modeling. 2013.
arXiv preprint
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self- supervised models strong semi-supervised learners. 34th Conference Neural Information Processing Systems, NeurIPS, 2020.
Bhavin Choksi, Milad Mozafari, Callum Biggs O'May, Benjamin Ador, Andrea Alamia, and Rufin VanRullen. Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics. Advances in Neural Information Processing Systems, 144:1466--14088 2021.
Francis Crick. The recent excitement about neural networks. Nature, 337(6203):129-111 1989.
Joel Dapello. Tiago Marques, Martin Schrimpf, Franziska Geiger, David Cox, and James J. Di- Carlo. Simulating primary visual cortex at the front of CNNs improves robustness to image perturbations. Advances in Neural Information Processing Systems, 33: 13:1333--33887 2020.
Arthur Dempster, Nan Laird, and Donald B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1): 1-22, 1977.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 2019.
Peter Elias. Predictive coding-I. IRE Transactions Information Theory, 1(1):16-24, 1955.
10

---

Published as a conference paper at ICLR 2024
Maxence Emouut, Julie Grollier, Damien Querlioz, Yoshua Bengio, and Benjamin Scellier. Equilib- rium propagation with continual weight updates. arXiv preprint arXiv:2005.04166, 2020.
Karl Friston. Learning and inference in the brain. Neural Networks, 16(9):1325-1333) 2003.
Karl Friston. A theory of cortical responses. Philosophical Transactions of the Royal Society B:
Biological Sciences, 360(1456), 2005.
Karl Friston. The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11 (2):127-138, 2010.
Karl Friston, James Kilner, and Lee Harrison. A free energy principle for the brain. Journal of Physiology, 2006.
Karl Friston, Thomas FitzGerall, Francesco Rigoli, Philipp Schwartenbeck, Giovanni Pezzulo, et al.
Active inference and learning. Neuroscience & Biohehavioral Reviews, 188662-879 2016.
Karl J. Friston. N. Mijillo-BBaretto and Jean Daunizeau. DEM: A variational treatment of dynamic systems. Neuroimage, 41(3):849-885, 2008.
Karl J. Friston, Maxwell D. Ramstead, Alex B. Kiefer, Alexander Tschantz, Christopher L. Buck- ley, Mahault Albarracin, Riddhi J. Pitliya, Conor Heins, Brennan Klein, Beren Millidge, et al. Designing ecosystems of intelligence from first principles. arXiv preprint arXiv:2212.01335 2022.
Will Grathwohl, Kuan-Chich Wang, Jörn Henn-He Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly energy-based model and you should treat it like one. arXiv preprint arXiv:19/2.03263. 2019.
Stephen Grossberg. Competitive learning: From interactive activation to adaptive resonance. Cog- nitive Science, 11(1):23-63 63, 1987.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. ICMI. 2017, 3:2130-2143, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Adrien Journé, Hector Garcia Rodriguez, Qinghai Guo, and Timoleon Moraitis. Hebbian deep learning without feedback. arXiv preprint wMXiv:2209.11883 2022.
Belhal Karimi, Hoi-To Wai, Eric Moulines, and Mare Lavielle. On the global convergence of (fast) incremental expectation maximization methods. Advances Neural Information Processing Sys- fems, 32, 2019.
Jack Kendall, Ross Pantone, Kalpana Manickavasagam, Yoshua Bengio, and Benjamin Scel- lier. Training end-to-end analog neural networks with equilibrium propagation. arXiv preprint arXiv:2006.01981, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
Alex Krizhevsky, llya Sutskever, and Geoffrey E. Hinton. ImageNet classification with deep convo- lutional neural networks. 26th Annual Conference on Neural Information Processing Systems (NIPS) 20/2, 2012.
Taku Kudo and John Richardson. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods Natural Language Processing, System Demonstrations, pp. 66-7666-66 2018.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. The MNIST Database, 2010.
URL _ http:: //aann. lecun, wtt..//wwwww..ccceee

---

Published as a conference paper at ICLR 2024
Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed, and Colin J. Akerman, Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, (1):1-10, 2016.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2017.
Beren Millidge, Alexander Tschantz, and Christopher L. Buckley. Predictive coding approximates backprop along arbitrary computation graphs. arXiv:2006.04 04102, 2020a.
Beren Millidge, Alexander Tschantz, Anil Seth, and Christopher L. Buckley. Relaxing the straints on predictive coding models. arXiv:2010.01047, 2020b.
Beren Millidge, Anil Seth, and Christopher L. Buckley, Predictive coding: A theoretical and exper- imental review. arXiv:2107.12979 2021.
Beren Millidge, Tommaso Salvatori, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Pre- dictive coding: Towards a future of deep learning beyond backpropagation? arXiv preprint arXiv:2202.09467, 2022a.
Beren Millidge, Yuhang Song, Tommaso Salvatori, Thomas Lukasiewicz, and Rafal Bogacz. Back- propagation at the infinitesimal inference limit of energy-based models: unifying predictive cod- ing, equilibrium propagation, and contrastive Hebbian learning. arXiv preprint arXiv:2206.02669, 2022b.
Beren Millidge, Yuhang Song, Tommaso Salvatori, Thomas Lukasiewicz, and Rafal Bogacz. A theoretical framework for inference and learning in predictive coding networks. arXiv preprini arXiv:2207.12311 12316, 2022c.
David Mumford. On the computational architecture of the neocortex. Biological Cybernetics, 66 (3):241-251, 1992.
Radford M. Neal and Geoffrey E. Hinton. A view of the EM algorithm that justifies incremental, sparse, and other variants. In Learning in Graphical Models, pp. 355-368. Springer, 1998.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Khanh Nguyen and Brendan T. O'Connor. Posterior calibration and exploratory analysis for natural language processing models. In EMNLP, 2015.
Alex Ororbia and Daniel Kifer. The neural coding framework for learning generative models. arXiv:2012.03305. 2020.
Alex Ororbia, Ankur Mali, C. Lee Giles, and Daniel Kifer. Lifelong neural predictive coding: Learning cumulatively online without forgetting. Advances in Neural Information Processing Systems, 35:5867-5881, 2022.
Alexander Ororbia and Daniel Kifer. The neural coding framework for learning generative models. Nature Communications, 13(1) 13(1):2064, 2022.
Alexander Ororbia, Ankur Mali, C. Lee Giles, and Daniel Kifer. Continual learning of recurrent neural networks by locally aligning distributed representations. IEEE Transactions Neural Networks and Learning Systems, 3 31(10):4267-4275 2020.
Alexander G. Ororbia and Ankur Mali. Biologically motivated algorithms for propagating local target representations. In Proc. AAAI, volume 33, pp.
2019.
Alexander G. Ororbia II, Patrick Haffner, David Reitter, and C. Lee Giles. Learning to adapt by minimizing discrepancy, 1rXiv:1711.115422 2017.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan. and Jasper Snock. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances Neural Information Process- ing Systems, volume 32, 2019.
12

---

Published as a conference paper at ICLR 2024
Luca Pinchetti, Tommaso Salvatori, Beren Millidge. Yuhang Song. Yordan Yordanov, and Thomas Lukasiewicz. Predictive coding beyond Gaussian distributions. 36th Conference on Neural Infor- mation Processing Systems, 2022.
A. Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language under- standing by generative vre-training, 2018. URL hotthttp://// /api. www.steeastttstteee www.sstcccccccccaccooooo Corpus ID: :9:3113244
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.0612 06125, 2022.
Rajesh P. N. Rao and Dana H. Ballard. Predictive coding the visual cortex: functional interpre- tation of some extra-classical receptive-fiele field effects. Nature Neuroscience, 2(1):79 87, 1999.
Picter R. Roelfismma and Anthony Holtmaat. Control of synaptic plasticity in deep cortical networks
Nature Reviews Neuroscience, 19(3):666, 2018.
João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcir- cuits approximate the backpropagation algorithm. In Advances in Neural Information Processing Systems, pp. 8721-8732, 2018.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Scyed Kam- yar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205. 1487, 2022.
Tommaso Salvatori. Yuhang Song, Yujian Hong, Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bo- gacz, and Thomas Lukasiewicz. Associative memories via predictive coding. In Advances in Neural Information Processing Systems, volume 34, 2021.
Tommaso
Salvatori, Luca Pinchetti, Beren Millidge, Yuhang Song, Tianyi Bao, Rafal Bogacz, and Thomas Lukasiewicz. Learning on arbitrary graph topologies via predictive coding. arXiv:2201. 13180, 2022a.
Tommaso Salvatori, Yuhang Song, Zhenghua Xu, Thomas Lukasiewiez, and Rafal Bogacz. Reverse differentiation via predictive coding. In Proceedings of the 36th AAAI Conference on Artificial Intelligence. AAAI Press, 2022b.
Tommaso Salvatori, Ankur Mali, Christopher L. Buckley, Thomas Lukasiewicz, Rajesh P. N. Rao, Karl Friston, and Alexander Ororbia. Brain-inspired computational intelligence via predictive coding. arXiv preprint arXiv:2308 07870, 2023.
Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy- based models and backpropagation. Frontiers Computational Neuroscience, 11 11:24, 2017.
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittieserr Joannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukeuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529, 2016.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui. Laurent Sifre, George van den Driessche. Thore Graepel, and Demis Hassabis. Mastering the game Go without human knowledge. Nature, 550, 2017.
Yuhang Song, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Can the brain do backpropa- gation? Exact implementation backpropagation in predictive coding networks. In Advances in Neural Information Processing Systems, volume 33, 2020.
Yuhang Song, Beren Gray Millidge, Tommaso Salvatori, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Inferring neural activity before plasticity: A foundation for learning beyond back- propagation. bioRxiv, 2022.
13

---

Published as a conference paper at ICLR 2024
Mufeng Tang, Tommaso Salvatori, Beren Millidge, Yuhang Song, Thomas Lukasiewicz. and Rafal Bogacz. Recurrent predictive coding models for associative memory employing covariance learn- ing. PLOS Computational Biology, 1010719 2023.
James C. Whittington and Rafal Bogacz. An approximation of the error backpropagation algo- rithm in predictive coding network with local Hebbian synaptic plasticity. Neural Computation, 29(5), 2017.
James R. Whittington and Rafal Bogacz. Theories of error back propp-aapoggtionn in the brain. Trends in Cognitive Sciences, 2019.
Will Xiao, Honglin Chen, Qianli Liao, and Tomaso Poggio. Biologically-plausin learning algo- rithms can scale to large datasets. orXiv:/81.03500 2018.
Anthony Zador, Blake Richards, Bence Ōlveczky, Sean Escola, Yoshua Bengio, Kwabena Boa- hen, Matthew Botvinick, Dmitri Chklovskii, Anne Churchland, Claudia Clopath, al. To- ward next-generation artificial intelligence: catalyzing the neuroai revolution. arXiv preprint arXiv:2210.08334. 2022.
14

---

Published as a conference paper at ICLR 2024
W
X1
X2
8
W
X3
L2
W
X1
82
X2
W
83
X3
Dynamics:
Inhibitory
8, Error Nodes
Excitatory
6 X1 1-1-
Interneurons
x 8, + WT 81+1
Synaptic Weights
Figure 4: Standard and dendritic neural implementation of predictive coding. The dendritic im- plementation makes use of interneurons i Wizi (according to the notation used in the figure). Both implementations have the same equations for all the updates, and are thus equivalent; however, dendrites allow a neural implementation that does not take error nodes into account, improving the biological plausibility of the model. Figure taken and adapted from (Whittington Bogacz 2019).
A A DISCUSSION ON BIOLOGICAL PLAUSIBILITY
In this section, we discuss the biological plausibility of the proposed algorithm. the literature, there is often a disagreement on whether a specific algorithm is biologically plausible or not. Gen- erally, it is assumed that an algorithm biologically plausible when satisfies list of properties that are also satisfied in the brain. Different works consider different properties. In our case, we consider list of minimal properties that include local computations and a lack of global control signals to trigger the operations. Normally, predictive coding networks take error nodes into ac- count, often considered implausible from the biological perspective (Sacramento et al. 2018). Even so, the biological plausibility of our model is not affected by this: it is in fact possible to map PC on a different neural architecture, in which errors are encoded in apical dendrites rather than sepa- rate neurons (Sacramento al. 2018 Whittington the differences hetween the two implementations can be found in Fig. 4 taken (and adapted) from (Whittington & Bogacz 2019). Furthermore, formulation is more plausible than the original formulation of PC, as it is able to learn without the need of external control signals that trigger the weight update.
Bogacz, 2019). Graphical representations of
Weight Transport: While predictive coding is designed to model information processing in vari- ous brain regions, the current formulation still exhibits certain biological implausibilities. A notable example is the weight transport problem, which posits that the synaptic weight responsible for for- warding information is identical to that which conveys error information backward, as highlighted by] Lillicrap al. (2016). This issue has garnered substantial attention, contributing the evolution of deep learning models that achieve performance levels on par with backpropagation large-scale image classification tasks, as discussed in (Xiao et al. 2018). Within predictive coding research, has been shown that the removal of these features does not markedly impair classification per- formance (Millidge et al.) (020b). While important emphasize that our proposed algorithm remains functional in the weight transport framework described in the aforementionedd study, testing it goes beyond the scope of the paper. To conclude, other works have introduced predictive coding-
like algorithms that do not rely on weight transport (Ororbia et al. 2022 Ororbia & Kifer 2022 Ororbia al. 2020).
15

---

Published as a conference paper at ICLR 2024
B PSEUDOCODES OF Z-IL AND PC
Algorithm Learning a dataset y/ with PC.
1: Require: For every i, is fixed to yi-
2: for t = 0 to I' do
3: For every i and 4, update 20 to minimize F' via Eq. (6)
if then
5: For every / update each g()) to minimize F via Eq. (7),
6: end if
7:
end for
Algorithm 3 Learning one training pair (s"" sout) with Z-IL
1: Require: z6 is fixed to s"h, x6 is fixed to s!m
Require: (²()
(0()) for (I...., L-1,, and
3: for (o 0 to T do
4: for cach level do
5: Update (²()) to minimize F via Eq. (6)
end for
then
8: Update ((4) to minimize F via Eq. (7).
9:
end if
10:
end for
16

---

Published as a conference paper at ICLR 2024
Table 4: Theoretical Efficieney of PC, 7-II,, BP, and iPC.
One inference step
PC
Z-IL
BP
iPC
Numher of MMs per weight update Number of SMMs per weight update
(2 1)
(21. 2T
1)7'
(21, 1)(L-))
(2l
(21.
2
2(L-1)
(2L 1)
2
C ON THE EFFICIENCY OF PC, BP, AND IPC
In this section, we discuss the time complexity and efficiency of PC, BP, Z-II,, and iPC. We now start with the first three, and introduce metric that we use to compute such complexity. This metric the number of simultaneous matrix multiplications (SMMs), i.e., the number of non-parallelizable matrix multiplications needed to perform single weight update. It is reasonable approximation of running time, as multiplications are by far the most complex operation (=e O(N3) performed by the algorithm.
C.1 COMPLEXITY OF PC, BP, AND Z-IL
Serial Complexity: To complete a single update of all weights, PC and Z-IL. run for T and (L - 1) inference steps, respectively. To study the complexity of the inference steps, we consider the number of matrix multiplications (MMs) required for each algorithm: One inference step requires (2L -2L-1 1)
MMs: L for updating all the errors, and (L 1) for updating all the value nodes (Eq. (6)) Thus, to complete one weight update, PC and Z-IL require (2L-1)7 and (2L-1)(L-1) MMs, respectively. Note also that BP requires (2L- 1) MMs to complete a single weight update: L for the forward, and (L - 1) for the backward pass. These numbers are summarized in the first row of Table 4 According to this measure, BP the most efficient algorithm, Z-II ranks second, and PC third, as in practice, T is much larger than L. However, this measure only considers the total number of matrix multiplications needed, without considering whether some of them can be performed in parallel, which could significantly reduce the time complexity. We now address this problem.
The MMs performed during inference can be parallelized across layers. In fact, computations in Eq. (6) are layer-wise independent, thus L MMs that update all the error nodes take the time of only one MM if properly parallelized. Similarly, in Eq. (6. (L-1) MMs that update all the value nodes take the time of only one MM if properly parallclized. As result, one inference step only takes the time of 2 MMs properly parallelized (since, as stated, consists of updating all errors and values via Eq. O Thus, one inference step takes 2 SMMs; one weight update with PC and Z-IL takes 27' and 2 L - 1) SMMs, respectively. Since no MM can be parallelized in BP (the forward pass in the network and the backward pass of error are both layer-dependent), before performing single weight update, (2L- - 1) SMMs are required. These numbers are summarized in the second row of Table Overall, measured over SMMs, BP and Z-IL are equally efficient (up to constant factor), and faster than PC.
Parallel complexity:
C.2 COMPLEXITY OF IPC
To complete one weight update, iPC requires one inference step, thus (2L- 1) MMs or 2 SMMs, as also demonstrated in the last column of Table Compared to BP, iPC takes around L times less SMMs per weight update, and should hence be significantly faster in deep networks. Intuitively, this is because matrix multiplications in BP have to be done sequentially along layers, while the ones in iPC can all be done in parallel across layers (Fig. 5). More formally, we have the following theorem, which holds when performing full-batch training:
Theorem 3.1 Let M and M' be two equivalent networks with layers trained on the same dataset. Let M (resp., M') be trained using BP (resp., iPC). Then, the time complexity measured by SMMs needed to perform one full update of the weights is ((1) and (LL for iPC and BP, respectively.
Proof. Consider training on MLP with L layers, and update weights for multiple times single data point. Generalizations to multiple data points and multiple mini-bater batches are similar and will be provided after. We first write the equations needed to be computed for iPC produce weight
17

---

Published as a conference paper at ICLR 2024
update:
4,6 and (0)
in
n
40 = Σ
0(+1)
for E (1,...,hh ,T]
(9)
3
==1
112 ii Zit
(1) (8) (1)
for I E [I....,
= + (-o92 + f( (
T 1,141
Σ
Ret
(1+1)
(
for E {1,.... [,....,].
(10)
0
1.++1
a.
11+1)
for l E (1.......) L.).
(11)
We then write the three equations needed to be computed for BP to produce one weight update:
1-1
T. 111 1=1
(1)
Σ
f(
for le (l....,I.
(12)
(L)
1,6
squt
I i,t
(++
E.e = (110)
Σ
k Le for {I,...,
(1+1)
(13)
(++1) 1. i.j.t - a iit ((x for I E (I,..., LJ.
0 (0))
First, we notice that the matrix multiplication (MM) is the most complex operation. Specifically, for two adjacent layers with the size of n' and +11 the complexity of MM is O but the maximal complexity of the other operations is Omax(n'' n )). In the above equations, only equations with MM numbered, which are the equations that investigate in our complexity analysis.
in par- allel for different This is further because the variables on the right side of Eq. 9 are immedi- ately available. Differently, Eq. (12) for iPC takes L MMs, and also SMMs, since the for-loop for E (I....,4) has to be executed one after another, following the specified order ( L}. This is further because the values the right side of Eq. are immediately available, but require
Eq. (9) for iPC takes L. MMs, but SMM, since the for-loop for l E (1,..., I.)
to solve Eq. 12) again for another layer. That is, to get x Eq. 12 has to be solved recursively from 1 to
Similar sense applies to the comparison between Eqs. 10 and 13
Eq. 10 for iPC takes L
MMs but 1 SMMs; Eq. (13) for BP takes L 1 MMs and also L
SMMSs.
Overall, Eqs. and (10) for iPC take 2L MMs but 2 SMMs: Eqs. 12 and 13 for BP take 2L MMs and also 2L SMMs. Then, the time complexity measured by SMMs needed to perform one full update of the weights is (()) and O(L) for iPC and BP, respectively.
18

---

Published as a conference paper at ICLR 2024
m == 0
11
==
m
==
m=
m
n == M 6
771 Bockward SMM
1 0
Inpull
1= 1 B
Hidden neuron (error not updated)
I 2
I 3
Hidôen neuron (error updated)
[
Output neuron
1-1
Weights (not updated)
1 2 [ 1 3
Weights (updated)
Figure 5: Graphical illustration of the efficiency over backward SMMs of BP and iPC on 3-layer network. iPC never clears the eттor (red neurons), while BP clears it after every update. This allows iPC to perform 5 full and 2 partial updates the weights in the first 6 SMMs. frame, BP only performs 3 full updates. Note that the SMMs of forward passes are excluded for simplicity, w.l.o.g.. g. as the insight from this example generalizes to the SMMs of the forward pass.
the same time
C.3 EFFICIENCY ON ONE DATA POINT
To make the difference more visible and provide more insights, we explain this in detail with a sketch of this process small network in Fig. where the horizontal axis of m is the time step measured by simultancous matrix multiplications (SMMs), i.e., within a single m, one can perform one matrix multiplication multiple ones in parallel; if matrix multiplications have be executed in order (e.g., the second needs results from the first), they will need to be put into two steps of m. Note that we only consider the matrix multiplications for the backward pass, i.c.. the matrix multiplications that backpropagate the error of layer from adjacent layer for BP and the inference of Eq. (6) for iPC, thus the horizontal axis is strictly speaking "Backward SMM" The insight for the forward pass is similar as that of the backward pass. As mentioned above, for BP, backpropagating the error from one layer to an adjacent layer requires one matrix multiplication; for iPC, one step of inference on one layer via Eq. (6) requires one matrix multiplication. BP and iPC are presented in the first and second rows, respectively. Before both methods are able to update weights in all layers, they need two matrix multiplications for spreading the error through the network, i.c., weights update of all layers occurs for the first time at m for both methods. After m 2, BP cleared all errors BP backpropagates the error from 1 =
all neurons, so backpropagates the error from = 1 to = 2 after which it can make an update of weights at all layers again for the second time. Note that the matrix multiplication that backpropagates errors from I = to | 2 at = 4 cannot be put at = 3, as it requires the results of the matrix multiplication m 3, i.e., it requires the eгror be backpropagated to
1 and m 4, BP
from 1 0 at m However, this is different for iPC. After m = 2, iPC does not reset x;_, to Mi,, i.e., the error signals are still held in Ei,, At m 3, iPC performs two matrix multiplications in parallel, corresponding to two inferences steps at two layers, 7 and = 2, updating x and hence the error signals are held in E,, of these two layers. Note that the above two matrix multiplications of two inference steps run in parallel and be put into a single m, inference requires only locally and immediately this way, a weight update in iPC is able to be performed at every m ever since the very first few steps of m.
available information.
19

---

Published as a conference paper at ICLR 2024
C.4
CPU IMPLEMENTATION
10
Hidden Dim.
iPC TB
109
125
156
512
10-1
1024
2
8 16 32
Figure 6: Ratio of the actual running time needed to perform a single weight update between BP and iPC on a CPU. Every dot represents a model; if the model lies below the horizontal line with label 100, its weight update performed using iPC is faster than one performed using BP.
To further provide evidence of the efficiency of iPC with respect BP, have implemented the parallelization of iPC on a CPU, and compared it to BP, also implemented on CPU. We compute the time in milliseconds (ms) needed to perform data point. In Fig. 6 we have plotted the ratio
weight update of both randomly generated
ms of iPC / ms of BP
for architectures with different depths and widths. The results show that our naive implementation adds computational overhead given by communication and synchronization across threads that makes iPC slower than BP small architectures (hidden dimension < 64). However, this difference inverted in large networks: the most extreme case, one weight update on a network with 32 hidden layers and 1024 parameters per layer using iPC is 10 times faster than that using BP. This is still below the result of Theorem 3.1 due to the large overhead introduced in our implementation.
D TRAINING DETAILS
We now list some additional details reproduce our results.
D.1 EXPERIMENTS OF EFFICIENCY
The experiments for the efficiency of generative models were run on fully connected networks with 128, 256, or 512 hidden neurons, and L E (4, 5}. Every network was trained on CIFAR10 or Tiny ImageNet with learning rates a = 0.00005 and 7 0.5, and T {8, 12, 16). The experiments on discriminative models are performed using networks with 64 hidden neurons, depth L {3, 4, 6), and learning rates a = 0.0001 and 7 = 0.5. The networks trained with BP have the same learning rate All the plots for every combination of hyperparameters can be found in Figs. and
D.2 EXPERIMENTS OF GENERALIZATION QUALITY
As already stated in the paper body, make sure that our results are not the consequence of specific choice of hyperparameters, we performed a comprehensive grid search on hyperparameters, and reported the highest accuracy obtained, and the search further made robust by averaging over 5 seeds. Particularly, we tested over 8 learning rates (from 0.000001 to 0.01), 4 values of weight decay (0.0001, 0.001, 0.01, 0.1), and 3 values the integration step 7 (0.1, 0.5, 1.0). We additionally verified that the optimized value of each hyperparameter lies within the searched range of that hyperparameter. As for additional details, we used standard Py Torch initialization for the parameters. For the hardware, cluster. Despite the large search, most of of the best results were obtained using the following hyperparameters: 0.5 (7 for AlexNet), 0.00005.
used single Nvidia GeForce RTX 2080 GPU
internal
20

---

Published as a conference paper at ICLR 2024
In the experiments AlexNet, under all the hyperparameter combinations tested, iPC only failed to converge in some cases where the learning rate of the weights was the largest (0.01). In total, converged 88 times out of 96 combinations of hyperparameters. This is not the case for PC, which converged in only 26 combinations of hyperparameters out of 96.
D.3 LANGUAGE MODELS: IMPLEMENTATION DETAILS
For PC and iPC, we use the FKL modification (Pinchetti et al. 2022). which replaces the Gaussian distribution for the layers with softmax activation with a non Gaussia distribution. Initial experi- ments confirm that this helps both PC and iPC for both language models. Initial experiments with the masked language models also suggest that PC and iPC benefit from disabling the layer normal- ization (Ba et al. 2016) found BERT, but BP does not. Therefore, for the masked language model, assume layer normalization for BP only. For the conditional language models, we observe that iPC and BP benefit from layer normalization, but PC does not.
D.3.1 HYPERPARAMETERS
We train each model on 16 epochs of the training dataset. We do early stopping every 100 128 /batch size batches. For iPC, and PC, we have observed that a batch size of 128 works best, so we assume it throughout the experiments.
Based on early experiments, we assume the parameter (weight) optimizer to be Adam W (Loshchilov Hutter 2017), and the inference optimizer to stochastic gradient descent (SGD). We denote the parameter (weight) learning rate as lr, and the inference learning rate z/.r.
For PC, we also use an inference learning rate discount (between 0 and 1), which we use multiply the learning rate by if the energy has not improved. We have observed that this can increase the performance and decrease the optimal T (the number of updates per batch), hence speeding up the training. We use 0.9 for the discount value, which we have found works best. For iPC, we assume the f. value to be 0.5, as suggested by our initial experiments. We do grid search over the following hyperparameter ranges:
Masked language model:
BP: batch_size {32, 64, 128, 256, 512). Ir (0.0002, 0.0004, 0.0008, 0.0016, 0.0032, 0.0064, 0.0128). Best combination: batch_size = 256, Ir = 0.0032.
PC: (5, 6, 7,8, 9, 10, 12).
xilr e (0.015625, 0.03125, 0.0625).
Ir E (0.0001, 0.0002, 0.0004, 0.0008).
Best combination: T = 8, æ Ir = 0.03125, Ir = 0.0004.
iPC: T (4, 5, 6, 7.8).
Ir E {0.0004, 0.0008, 0.0016, 0.0032].
Best combination: T = 6, Ir = 0.0008.
Conditional language model:
BP: batch_size e {32, 64, 128, 256, 512).
Ir E (0.0002, 0.0004, 0.0008, 0.0016, 0.0032, 0.0064, 0.0128).
Best combination: batch_size = 64, Ir = 0.0016.
PC: {7,8,9 9, 10, 11, 12).
xilr e {0.00390625, 0.0078125, 0.015625, 0.03125).
Ir E (0.0001, 0.0002, 0.0004, 0.0008).
Best combination: T = 10, x Ir 0.0078125, Ir 0.0004.
iPC: 7 {3, 4, 6).
Ir {0.0008, 0.0016, 0.0032, 0.0064, 0.0128).
Best combination: T = 4, Ir 0.0032.
21

---

Published as a conference paper at ICLR 2024
D.3.2 PER-SEED RESULTS
Conditional LM results:
PC: [201.096, 206.322, 198.798, 187.0, 214.385, 213.66, 218.577, 221.076, 219.609, 182.898]
iPC: [137.086, 138.435, 235.153, 160.08, 139.284, 140.68, 149.148, 237.234, 281.977, 133.098]
BP: [112.655, 113.454, 113.454, 113.235, 113.726, 115.153, 112.391, 112.877, 113.359, 112.921]
Masked LM results:
PC: [503.548, 1042.125, 1158.864, 909.209, 557.766, 1155.093, 1176.144, 901.295, 507.916, 1162.502]
iPC: [93.998, 93.899, 93.449, 137.027, 94.491, 108.241, 92,411, 97.724, 156.112, 94.504)
BP: [83.619, 147.775, 787.334, 133.073, 358.486, 455.535, 92.951, 95.322, 152.135, 135.3]
22

---

Published as a conference paper at ICLR 2024
CIFARIO
HD 128
HD 256
HD 512
BPC
IPC
IPC PC. T--B
INC T-ll
PC) Two
PC
T--22
Energy
Energy
Tw
Energyy
PC, Tw36
L=4
1000 Iterations
Iterations
lierations
T= PC; Tw
Energy
PCC
Energy
PV, This
Energy
7-16
L=5
1000
2000 lterations
3000
1000
2000 lirrations
3000
1000
2000 3000
llerations
Tiny ImageNet
HD 128
HD 256
HD 512
IPC PC. TwB
IPC
PC, TwB
PC. T-12
PC: Dw12
PC Tw12
Energy
PC, Tw16
Emneree
PC, Tw1d
Energy
PC, Tw16
L=4
1000 Iterations
Iterations
Iterations
Tw122
Energy
INC, Twint
Energy
PNC
Energy
Tw16
L-5
LOC0
2000 Iterations
3000
1000
2000 Iterations
3000
1500
2000 Iterations
3000
Figure 7: Decrease of the energy of generative models as function the number of iterations performed from the beginning of the training process. These experiments follow the same procedure of the ones in Fig. 2 (left), and serve the purpose of showing that the results that we provided are solid and robust under changes hyperparameters.
23

---

Published as a conference paper at ICLR 2024
MNIST
L=3
L=4
L= 6
150
BP
IPC
125
T-8
100
103
T-12
Training Loss
75
T-16
90
1000
2000
3000
SMMs
SMMs
SMMs
0.8
BP
7
7-12
Test Error
1-16
0.5
1-15
0.4
0.2
1200
2000
3009
1000
2000
3000
1000
2000
3000
SMMs
SMMs
SMMs
FashionMNIST
HD 128
HD 256
HD 512
150
125
100
T-12
T-
100
T-12
Training Loss
T-16
75
T-16
23
SMMs
SMMs
SMMs
INC
0.8
T-B
Tw12
T-12
Tw12
Test Error
T-16
7-16
0.6
L-16
04
1000
2000
3000
0.2
1000
2000
3000
0.2
1000
2000
3000
SMMs
SMMs
SMMs
Training loss and test errors of different classifiers in a full-batch training regime as a function of the number of non-parallel matrix multiplications performed from the beginning of the training process. These experiments follow the same procedure of the ones in Fig. 2 (right), and serve the purpose of showing that the results that we provided solid and robust under changes of hyperparameters.
Figure 8:
24

---

Published as a conference paper at ICLR 2024
E ROBUSTNESS
BP
Calibration (AdaECE)
0.8
BP
0.5
0.4
0.4
01
Intensity
Intensity
Figure 9: Larger image of the one provided in the main body of this work. Robustness of BP and iPC under distribution shift (AlexNet CIFARIO under five different intensities of the corruptions rotation, Gaussian blur, Gaussian noise, hue, brightness, and contrast). Left: Comparable decline of model accuracy between BP and iPC. Right: iPC maintains model calibration significantly better than BP under distribution shift.
25


Reviews:
[{"review_id": "34gXpvti7A", "reviewer": "Reviewer_Ctxj", "summary": "Predictive coding presumably uses an EM-like algorithm for learning representation and for making inferences. The authors of this paper proposed a method called incremental predictive coding that allows all computations involving learning and inference to take place locally, simultaneously, and autonomously. The results speak for themselves. iPC achieves accuracy comparable to BP and even lower calibration error than BP.", "questions": "The intuition of WHY the algorithm works is not clearly explained.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "It is rather impressive that iPC can achieve accuracy comparable to BP, and lower calibration error than BP.  If this unsupervised learning algorithm can run as fast as BP, it could be fairly significant.", "weaknesses": "Even though the work could be important and the paper is well-written, the authors have archived the paper in public,  revealing their identities. Hence, this review is no longer double-blinded. This is perhaps an unfortunate oversight.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bq4mEdqzg1", "reviewer": "Reviewer_Tect", "summary": "This paper proposes iterative predictive coding (iPC) a predictive coding algorithm aims to be faster and more scalable than traditional predictive coding approaches. The key algorithmic innovation is that both intermediate layer activations and weights are simultaneously minimized (well, alternating training every step) rather than allowing hidden state activations to equilibrate as in previous models. In a series of experiments using Gaussian models, iPC outperforms competing approaches.", "questions": "- The analysis in the paper is focused on feedforward Gaussian networks, but can these ideas be extended to networks with substantial recurrence? In the original PC formulation, it's these recurrent inputs that carry error signals, but I'm curious as to whether the \"out of equilibrium\" method proposed here would also be able to dynamically balance multiple kinds of feedback. \n- Put another way: how much biological plausibility is lost if the Gaussian and feedforward assumptions are relaxed to mirror more cortical-like networks.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- Advances the state of the art for predictive coding, which remains a highly promising idea without a commensurately performant implementation.\n- Extends PC results beyond the small-scale experiments in previous work.\n- Brings PC algorithms closer to biological reality, where computations run simultaneously in real time. \n- Strong set of experiments against comparison models.", "weaknesses": "- I came away with the impression that the key algorithmic innovation is simply to alternate weight and hidden layer activations every step (without running either to convergence). It's quite hard to believe this hasn't been tried before. Is there something new here that makes this work?\n- The analysis is limited to Gaussian networks, which limits the range of applicability of the learning rules in (6) and (7). \n- While the experiments are good, there is not much insight provided as to *why* this new approach works better.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JzV0AH4qCu", "reviewer": "Reviewer_sVBr", "summary": "The paper introduces the incremental Predictive Coding (iPC) framework, a more biologically-plausible version of PC networks. It addresses the update locking problem in standard PC algorithms by parallelizing neural dynamics and learning steps using Neal \\& Hinton's incremental expectation maximization. This advancement eliminates the need for a global control signal to switch between inference and learning steps. The paper evaluates the proposed method on image classification tasks and language models, comparing it to standard PC and backpropagation (BP) algorithms.", "questions": "* It is not really intuitive  to me why PC with smaller T values converges faster (in terms of Energy in Figure 2). Could you give an insight about it, or elaborate it? \n\n* What is the meaning of the notation $\\mathcal{O}(\\text{max} n^l, n^l)$ on page 17?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The paper addresses an important problem in the field of biologically-plausible learning and is generally well-written.\n* The efficiency gained by iPC is a notable contribution.\n* The numerical experiments showcase improved results compared to standard PC and comparable results to BP.\n* The experimental descriptions in the appendix and code availability enhance reproducibility.", "weaknesses": "The paper has some clarity issues in my opinion. Please see the following items and the questions section.\n\n* Regarding the following sentence in the page 3: \"For a detailed formulation on how this energy function is derived from the variational free energy of Eq. 2, we refer to ..., or to **the supplementary**\", which supplementary section includes this derivation? I do not see it. \n\n* The caption of Table 1 should specify whether these are train or test accuracies.\n\n* The sentence preceding the \"Comparison with BP\" section in page 6 seems redundant (seems to repeat the previous sentence); consider removing it if not necessary.\n\n* Could you clarify which dataset the statement \"reported the highest accuracy obtained\" in Figure 3 refers to (train or test set)?\n\n* The statement about iPC performing better on standard CNN than on AlexNet contradicts Table 1. According to Table 1, CNN accuracy is around 72\\% whereas AlexNet accuracy is around 80\\%. Therefore, iPC does not perform better on standard CNN than on AlexNet.\n\n* The proposed method has one aspect that is not bio-plausible: it suffers from the weight transport problem, i.e. Eq. 6 requires the transpose of the forward mapping. I think this limitation can be discussed in Appendix section A (A Discussion on Biological Plausibility).\n\n* There is an artificial empty space in the top right of page 19 caused by the caption of Figure 6.\n\n* I think the plots in Figure 7 should include y-axis labels. Are they accuracy? energy? loss?\n\n* I think the captions of Figure 7 and 8 should include more explanations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4Pi5bJU9jT", "reviewer": "Reviewer_seiB", "summary": "The paper proposes incremental predictive coding. At a high level, predictive coding is the idea that one way to see brain function is that there is a top down generative model whose parameters are updated based on discrepancy between model output and stimulus received from the environment. More specifically, this paper talks about a specific generative model: a hierarchical Gaussian generative model where the means are passed thru non-linear functions from layer to layer of a generative neural network, with the covariance held fixed at identity. The optimal solution to the generative model (given data) is of course intractable and the next best is the standard expectation maximization update algorithm. The authors point out that this is generally very slow since the expectation step takes a very long time to converge when you have multiple layers in the neural network/hierarchical model. The authors propose to overcome the speed issue with incremental expectation maximization ala Neal and Hinton where gradient steps are taken with the expectation and maximization step either interleaved or in parallel.", "questions": "None", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper does an excellent job of introducing the topic of predictive coding. For the particular case of a hierarchical Gaussian generative model, (which is the generative model this paper is concerned with), the paper meticulously works out the details of the EM objective function. And finally, the paper has comprehensive experiments demonstrating that incremental EM does better than EM timewise, when implemented for the neural network modeling the h-Gaussian generative model.", "weaknesses": "Unfortunately, all of the results are derivative of Neal and Hinton's incremental EM where they showed that both the expectation and the maximization step can be done at the same time and is a theoretically sound way of doing decent for model parameters. There is nothing that this paper contributes conceptually beyond this well known result. The hierarchical Gaussian generative model is also very well known. All in all, the paper lacks sorely in novelty of contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)