Incremental Predictive Coding: A Parallel and Fully Automatic Learning Algorithm
Authors: Tommaso Salvatori
Yuhang Song
Beren Millidge
Zhenghua Xu
Lei Sha
Cornelius Emde
Rafal Bogacz
Thomas Lukasiewicz
Venue: Submitted to ICLR 2023 (2023)
Keywords: Cognitive Science
deep learning
predictive coding

Abstract: Neuroscience-inspired models, such as predictive coding, have the potential to play an important role in the future of machine intelligence. However, they are not yet used in industrial applications due to some limitations, such as efficiency. In this work, we propose incremental predictive coding (iPC), a variation of the original model derived from the incremental expectation maximization algorithm,  where every operation can be performed in parallel without external control. We show both theoretically and empirically that iPC is more efficient than the original algorithm by Rao and Ballard, with performances comparable to those of backpropagation in image classification tasks. This work impacts several areas, as it has general applications in computational neuroscience and machine learning, and specific applications in scenarios where automatization and parallelization are important, such as distributed computing and implementations of deep learning models on analog and neuromorphic chips. 

Under review as a conference paper at ICLR 2023
INCREMENTAL PREDICTIVE CODING: A PARALLEL AND FULLY AUTOMATIC LEARNING ALGORITHM
Anonymous authors Paper under double-blind review
ABSTRACT
Neuroscience-inspine models, such as predictive coding, have the potential to play an important role in the future of machine intelligence. However, they are not yet used in industrial applications due to some limitations, one of them be- ing the lack of efficiency. In this work, we address this by proposing incremental predictive coding (iPC), variation of the original framework derived from the incremental expectation maximization algorithm, where every operation can be performed in parallel without external control. We show both theoretically and empirically that iPC is more efficient than the original algorithm by Rao and Bal- lard 1999 while maintaining performance comparable backpropagation in im- age classification tasks. This work impacts several areas, has general applications in computational neuroscience and machine learning, and specific applications in scenarios where automatization and parallelization are important, such as dis- tributed computing and implementations of deep learning models on analog and neuromorphic chips.
INTRODUCTION
In recent years, deep learning has reached and surpassed human-level performance in a multitude of tasks, such as game playing (Silver et 2017 2016), image recognition (Krizhevsky et al. 2012 He et al. 2016). natural language processing (Chen et al. (020), and image generation (Ramesh al. (022). These successes are achieved entirely using deep artificial neural networks trained via backpropagation (BP), which is learning algorithm that is often criticized for its biological implau-
sibilities (Grossherg) 1987, Crick 1989 Abdelghani et al. 2008 Lillicrap et al. 2016 Roelfsema
& Holtmaat 2018: W Whittington & Bogacz, 2019). such as lacking local plasticity and autonomy. In fact, backpropagation requires a global control signal required to trigger computations, since gradi- ents must be sequentially computed backwards through the computation graph. These properties are not only important for biological plausibility: parallelization, locality, and automation are key to build efficient models that be trained end-to-end on non Von-Neumann machines, such analog chips (Kendall et al. 2020). A learning algorithm with most of the above properties is predictive coding (PC).
PC is an influential theory of information processing in the brain (Mumford 1992 Friston 2005). where learning happens by minimizing the prediction error of every neuron. PC can be shown to approximate backpropagation in layered networks (Whittingto & Bogacz 2017). as well as on any other model (Millidge et al.] 2020), and can exactly replicate its weight update if some external control is added (Salvatori et al. 2022b). Also the differences with BP are interesting, as PC allows for much more flexible training and testing (Salvatori al. 2022a). has rich mathematical formulation (Friston 2005 Millidge et al. (2022). and is an energy-based model (Bogac7 (017)- This makes PC unique, as is the only model that jointly allows training on neuromorphic chips, is an implementation of influential models of cortical functioning in the brain, and can match the performance of backpropagation in different tasks. Its main drawback, however, is the efficiency, as it is slower than BP. In this work, address this problem by proposing variation of PC that is much more efficient than the original one.
Simply put, PC is based on the assumption that brains implement an internal generative model of the world, needed to predict incoming stimuli (or data) (Friston et al. (006): Friston 2010 |Fistonn et al. 2016). When presented with stimulus that differs from the prediction, learning happens by

---

Under review as a conference paper at ICLR 2023
updating internal neural activities and synapses to minimize the prediction error. In computational models, this is done via multiple expectation-maxiin maximization (EM) (Dempster et al. 1977) steps on the variational free energy, in this case function of the total error of the generative model. During the E- step, internal neural activities are updated in parallel until convergence; during the M-step, a weight update further minimize the same energy function is performed. This approach results in two limitations: first, the E-step is slow, as it can require dozens of iterations before convergence; second, an external control signal is needed switch from the E to M step. In this paper, we show how to address both of these problems by considering variation of the EM algorithm, called incremental expectation-mantt maximization (iEM), which performs both E and M steps in parallel (Neal Hinton 1998). This algorithm is provably faster, does not require a control signal to switch between the two steps, and has solid convergence guarantees (Neal & Hinton 1998; Karimi et al. 2019). What results is training algorithm that we call incremental predictive coding (iPC) that is simple variation of PC that addresses the main drawback of PC (namely, efficiency), with no drawbacks from the learning perspective, as it has been formally proven to have equivalent convergence properties to standard PC. Furthermore, we provide initial evidence that iPC is also potentially more efficient than BP in the specific case of full-batch training. In fact, theoretically show that, machine, to complete one update of all weights on a network with L layers, the time complexity of iPC is O(1). while that of BP is (L). However, additional engineering efforts are needed reach this goal, which are beyond the focus of this work: our experiments are performed using PyTorch (Paszke et al.] 2017). which is not designed parallelize computations across layers on GPUs. We partially address this limitation by performing some experiments on CPUs, which empirically confirm our claims about efficiency, as shown in Fig. Our contributions are briefly as follows:
ideal parallel
1. We first develop the update rule of iPC from the variational free energy of hierarchical generative model using the incremental EM approach. We then discuss the implications of this change in terms of autonomy and convergence guarantees; it has in fact been proven that iEM converges to a minimum of the loss function (Neal & Hinton 1998 Karimi et al.) 2019), and hence this result naturally extends iPC. We conclude by analyzing similarities and differences between iPC, standard PC, and BP.
2. We empirically compare the efficiency of PC and iPC on generation tasks, by replicating some experiments performed in (Salvatori et al. (021), and classification tasks, by replicat- ing experiments similar to those presented in ( (Whittinttn iPC is by far more efficient than the original counterpart. Furthermore, we present initial evidence that iPC can decrease the training loss faster than BP, assuming that a proper parallelization is done.
Bogaez, (017). In both cases,
3.
We then test our model on a large number of image classification benchmarks, showing that that iPC performs better than PC, on average, and similarly to BP. Then, we show that iPC requires less parameters than BP to perform well on convolutional neural networks (CNNs). Finally, we show that iPC follows the trends of energy-based models on training robust classifiers (Grathwohl et al. 2019), and yields better calibrated outputs than BP on the best performing models.
PRELIMINARIES
In this section, we introduce the original formulation of predictive coding (PC) a generative model proposed by Rao and Ballard 1999 Let us consider a generative model g RR x R - Ro, where Rd is vector of latent variables called causes, y IR is the generated vector, and 0 R is set of parameters. We are interested in the following inverse problem: given a vector and a generative model g. we need the parameters 0 that maximize the marginal likelihood
ply. 0) -f. ply xr, 0)p(2, 0)dx.
Here, the first term inside the integral is the likelihood of the data given the causes, and the second is a prior distribution over the causes. Solving the above problem is intractably expensive. Hence, we need an algorithm that is divided in two phases: inference, where we infer the best causes x given both e and y, and learning, where update the parameters 0 based on the newly computed causes. This algorithm is expectation- maximmzatoniiiiiiinn (EM) (Dempster et al.] 1977). The first step. which we call inference or E-step, computes p(x y, 0), that is the posterior distribution of the causes given
2

---

Under review as a conference paper at ICLR 2023
Inference
Numiba of lientioon
Layar index
g(z, 0)
Weights update
||
Iepui layer (fxed)
(BP)
Outpor leyer (fixed)
Hüdden layer
(rmning inference)
Generative Model
Differences between PC, 2-I1, and iPC
Figure I: (a) An example of an hierarchical Gaussian generative model with three layers. (b) Com- parison of the temporal training dynamics of PC, Z-IL, and iPC, where Z-IL variation of predic- tive coding that is equivalent to BP, originally introduced in [Song et al. (2020). We assume that train the networks on a dataset for supervised learning for period of time T. Here, t is the time axis during inference, which always starts at O. The squares represent nodes in one layer, and pink rounded rectangles indicate when the connection weights are modified: PC (1st row) first conducts inference the hidden layers, according Eq. equation 6 until convergence, and then it updates the weights via Eq. 7 Z-IL (2nd row) only updates the weights at specific inference moments de- pending on which layer the weights belong to. To conclude, iPC updates the weights every time step i, while performing inference in parallel.
a generated vector y. Computing the posterior is, however, intractable (Friston 2003). To this end, approximate the intractable posterior with tractable probability distribution q(ær, 0). To make the approximation as good as possible, we want to minimize the KL-divergence between the two probability distributions. Summarizing, solve our learning problem, we need (i) minimize KL-divergence, and (ii) maximize a likelihood. We do it by defining the following energy function, also known as variational free-energy:
F(x. y. 0) = KL(g(x, 8) || p( y. 0)) In(p(v. 9)).
(2)
where we have used the log- ligeliik. This function is minimized by multiple iterations of the EM algorithm as follows:
Inference (E-step): argmax,FFa; 0)
(3)
Learning (M-step): e" = argmazeF(z.g.
2.1 PREDICTIVE CODING
So far, have only presented the general problem. To actually derive proper equations for learning causes and update the parameters, and use them to train neural architectures, we need to specify the generative function g(x, 0). Following the general literature, (Rao & Ballard 1999. Friston 2005). we define the generative model hierarchical Gaussian generative model, where the causes æ and parameters @ are defined by a concatenation of the causes and weight matrices all the layers, i.c., (z:(0) ,..., .. (L)),, and 0 (0(0) .g.o..... ((L-1)). Hence, we have multilayer generative model, where layer 0 is the one corresponding to the generated image y, and layer L the highest in the hierarchy, The marginal probability the causes as follows:
L-1
....,, (11) p(a,.0)...,,111)))
p())
II p(x((-1)) ( N (,))
(4)
where (1)) is the prediction of layer according the layer above, given by 4 g(0) ((((+1)),
with ff being non-linear function and (() (T.) For simplicity, from now on, we consider
Gaussians with identity variance, i.e., )) Il for every layer I. With the above assumptions, the
3

---

Under review as a conference paper at ICLR 2023
Algorithm Learning dataset D {84) with iPC.
1: Require: For every i, x is fixed to y2-
2: for t = 0 to 7 do
3: For every i and I, update n, to minimize F via (q.(
4: For every I, update each ( minimize via Eq.(7)
5: end for
free-energy becomes
F Σ (z=(0) 11
(S)
For detailed formulation on how this energy function is derived from the variational free-energy of Eq. 2 we refer to (Friston 2005 Bogacz, 2017] Buckley et al. 2017), or to the supplementary material. Note that this energy function is equivalent to the one proposed in the original formulation of predictive coding (Rao & Ballard 1999). A key aspect of this model that both inference and learning achieved by optimizing the same energy function, which aims to minimize the prediction error of the network. The prediction error of every layer is given by the difference between its real value (1) and its prediction () We denote the prediction error el!) = x -,( Thus, the problem of learning the parameters that maximize the marginal likelihood given data point y reduces to an
alternation of inference and weight update. During both phases, the values of the last layer are fixed to the data point, i.e., (0) = y for every S T.
Inference: During this phase, that corresponds to the E-step, the weight parameters ((0) are fixed, while the values (!)) are continuously updated via gradient descent.
OF
Ar
Ox = (-e0) + f'(x())= (.--))T_(--1))
where denotes clement-wise multiplication, and 1 > 0. This process either runs until convergence, or for a fixed number of iterations T.
Learning: During this phase, which corresponds to the M-step, the values are fixed, and the weights are updated once via gradient descent according to the following equation:
0F A aa ae(1)
a x: (1++))
Note that the above algorithm is not limited to generative tasks, but can also be used to solve super- vised learning problems (Whittington & Bogacz (017). Assume that we provided with data point yin with label yout- In this case, we treat the label as the vector we need to generate, and the data point as the prior on z (L)) The inference and learning phases are identical, with the only differ- ence that now we have two vectors fixed during the whole duration of the process: (0) Youts and (T())) You- While this algorithm is able to obtain good results on small image image classification tasks, it much slower than BP due to the large number of inference steps I needed to let the causes r converge.
INCREMENTAL PREDICTIVE CODING
One of the main drawbacks of energy based models such as PC and equilibrium propagation (Scellier & Bengio 2017). is their efficiency. In fact, these algorithms are much slower than BP due to the inference phase, which requires multiple iterations to converge. The goal of this paper address this problem for predictive coding, by developing variation based from the incremental EM (iEM) algorithm (Neal & Hinton 1998). which was developed to address the lack of efficiency of the original EM. This algorithm excels when dealing with multiple data points at the same time (Neal & Hinton 1998), a scenario that is almost always present in standard machine learning.
Let D = fus be a dataset of cardinality N, and 9(x, 0) be generative model. Our goal is now minimize the global marginal likelihood, defined on the whole dataset, i.c..
PDD,0) 0) Σ p(ys. 0).
(8)
4

---

Under review as a conference paper at ICLR 2023
Tiny Imagenet
CIFARIO
PashionMNIST
120000
PC, T-8
120000
IPG
T-a
125
150
Energy
1000000
PC, Tw
T=12
Eaergy
100000
PC, T-12
PC, Tw
Train Looe
100
T-8
T=12
20000
75
60000
60000
40000
1000
2000 3000
1000
3000
1000
2000
2000
Iteratins
Iteraions
Non-parallel Multiplications
Figure 2: Left and centre: Decrease of the energy of generative models as a function of the number of iterations performed from the beginning of the training process. Right: Training loss of different classifiers trained using iPC, BP, and multiple parameterizations of PC as a function of the number of non-parallel matrix multiplications performed from the beginning of the training process.
The same reasoning also applies to the global variational free energy, which will be the sum of the free energies of every single data point. In this case, the iEM algorithm performs the E-step and M-step in parallel, with no external control needed to switch between the two phases. In detail, both the values x and the parameters 0 are updated simultaneously at every time step I, until convergence (or for a fixed number of iterations 7'), according to the same update rule defined in Eqs. Eqs. 7 on all the points of the dataset. No explicit forward and backward passes are necessary as each layer is updated in parallel. To our knowledge, this is the first learning algorithm for deep neural networks where every single operation is performed in parallel. Note that this increased speed does not harm it has been formally proven that minimizing free-energy function such
the final performance, as ours (i.e., equivalent to the sum of independent free-energy functions) using iEM, also finds a minimum of the global marginal likelihood of Eq 8 (Neal & Hinton 1998: Karimi et al. 2019). We actually provide empirical evidence that the model converges to better minima using iPC rather than the original formulation of PC in Fig. 2 and Table The pseudocode of iPC is given in Alg. DI
Connections to BP: PC in general shares multiple similarities with BP in supervised learning tasks: when the output error is small, the parameter update of PC is an approximation of that of BP (Mil- lidge et al. 2020): : when controlling which parameters have to be updated at which time step, the two updates can even be made equivalent (Salvatori et al., 2022b). To make PC perform exactly the same weight update of BP, every weight matrix O' must be updated only at I = 7, which corresponds to its position in the hierarchy (Song al. 2020). That is, as soon as the output error reaches spe- cific layer. This is different from the standard formulation of PC, which updates the parameters only when the energy representing the total error has converged. Unlike PC, iPC updates the parameters at every time step f. Intuitively, it can hence be seen where Z.II. is a variation of PC that is equivalent to BP, originally introduced in Song et al. (2020).. A graphical representation of the differences of all three algorithms is given in Fig. (right), with the pseudo-codes provided in the first section of the supplementary material.
"continuous shift" between Z.-II and PC,
Autonomy: Both PC and Z-IL lack full autonomy, as external control signal is always needed to switch between inference and learning: PC waits for the inference to converge (or, for iterations), while Z-IL updates the weights of specific layers specific inference moments t L. BP is con- sidered to be less autonomous than PC and Z-IL: a control signal is required to forward signals as well as backward errors, and additional places to store the backward errors are required. All of those drawbacks are removed in iPC, which is able to learn a dataset without the control signals required by the other algorithms: given a dataset D, iPC runs inference and weight updates simultaneously until the energy F is minimized. As soon as the energy minimization has converged, training ends.
3.1 EFFICIENCY
In this section, we analyze the efficiency of iPC with respect to both the original formulation of PC and BP. We only provide partial evidence of the increased efficiency against BP, as standard deep learning frameworks, such as Pytorch, do not allow to parallelize operations in different layers. While we leave the development of framework able to perform every operation in parallel to future work, we provide evidence that the speed up against BP in full batch training is theoretically possible using iPC.
5

---

Under review as a conference paper at ICLR 2023
Comparison with PC: We now show how iPC is more efficient than the original formulation. To do that, we have trained multiple models with iPC and PC on different tasks and datasets. First, we have trained a generative model with 4 layers and 256 hidden neurons of the Tiny ImageNet and CIFARIO datasets, as done in (Salvatori et al. 2021). ). A plot with the energies function of the number of iterations is presented in Fig. (lett and centre). In both
subset of 100 images
cases, the network trained with iPC converges much faster than the networks trained with PC with
different values of T', Many more plots with different parameterizations are given in Fig. 7 in the supplementary material.
To show that the above results hold in different set-ups as well, we have trained a classifier with layers on subset of 250 images of the FashionMNIST dataset, following the framework proposed in (Whittington & Bogacz 2017), and studied the training loss. As it is possible train an equivalent model using BP, we have done it using the same set-up and learning rate, and included it in the plot. This, however, prevents us from using the number of iterations as an efficiency measure, as one iteration of BP is more complex than one iteration of PC, and are hence not comparable. As a metric, we have hence used the number of non-parallel matrix multiplications needed to perform weight update. This is fair metric, matrix multiplications operation performed when training neural networks, and the ones with largest impact on the training speed. One iteration of PC and iPC have the same speed, and consist of 2 non-parallel matrix multiplications. One epoch of BP, consists of 2T. non-parallel matrix multiplications. The results are given in Fig. 2 (right). In all cases, iPC converges much faster than all the other methods. In the supplementary material, we provide other plots obtained with different datasets, models, and parameterizations, as well study on how the test error decreases during training. Again, many more plots with different parameterizations are given in Fig. ge in the supplementary material.
by far the most expensive
Comparison with BP: While the main goal of this work is simply to overcome the core limitation of original PC - the slow inference phase - there is one scenario where iPC is potentially more efficient than BP, which is full batch training. Particularly, we first prove this formally using the number of non- con-parallel matrix multiplications needed to perform a weight update metric. To complete one weight update, iPC requires two sets of non-parallel multiplications: the first uses the values and weight parameters of every layer to compute the prediction of the layer below; the second uses the error and transpose of the weights to propagate the error back to the layer above, needed to update the values. BP, on the other hand, requires 2L sets of non-paralled multiplications for a complete update of the parameters: L for a forward pass, and L for backward one. These operations cannot be parallelized. More formally, we prove theorem that holds when training on the whole dataset 'D in a full-batch regime. For details about the proof, and an extensive discussion about time complexity of BP. PC, and iPC, we refer to the supplementary material.
Let M and M' be two equivalent networks with L layers trained on the same dataset. Let M be trained using BP, and M' be trained using iPC. Then, the time complexity needed to perform one full update of the weights is (()) for iPC and OLL) for BP.
Theorem
3.2 CPU IMPLEMENTATION
10a
Hidden Dim.
To further provide evidence of the efficiency of iPC with respect to BP, we have implmented the parallelization of iPC on a CPU, and compared it BP, also implemented pute the time in milliseconds (ms) needed to perform one weight update of both on a ran-
16
iPC/ IPP
32
10"
CPU. We com-
512
domly generated datapoint. In Fig. 3 we have plotted the ratio
10-1
16
32
ms of iPC ms of BP
Figure 3: Ratio of the actual running time needed to perform a single weight update between BP and iPC on CPU. Every dot represents model, if the model lies below the horizontal line with la- bel 10", its weight update performed using iPC is faster than one performed using BP.
for architectures with different depths and widths. The results show that our naive imple- mentation adds computational overhead given by communication and synchronization across threads that makes iPC slower than BP on small

---

Under review as a conference paper at ICLR 2023
Table 1: Final accuracy of BP, PC, and iPC on different architectures trained with different datasets.
BP/Z-IL
PC
iPC
MLP on MNIST
98.26% + 0.12%
98.55% + 0.14%
98.54% = 0.86%
MLP on FashionMNIST
88.54% + 0.64%
85.12% + 0.75%
89.13% 00.8%%
CNN on SVHN
95.35% + 1.53%
94.53% + 1.54%
96.45% = -1.4%
CNN on CIFAR-10
69.34% + 0.54%
70.84% + 0.64%
72.54%
0.93%
AlexNet on CIFAR-10
75.64% + 0.64%
64.63% + 1.55%
72.42%
+ 0.53%
architectures (hidden dimension 64). How-
ever, this difference is inverted in large networks: in the most extreme case, one weight update on network with 32 hidden layers and 1024 parameters per layer using iPC is 10 times faster than that using BP. This still below the result of Theorem due to the large overhead introduced in our implementation.
CLASSIFICATION EXPERIMENTS
We now demonstrate that iPC shows similar level of generalization quality compared BP. We test the performance iPC on different benchmarks. Since we focus on generalization quality in this section, all methods run until convergence, and we have used early stopping to pick the best performing model. These experiments were performed using multi-batch training. In this case, we lose our advantage in efficiency over BP, as need to recompute the error every time new batch is presented. However, the proposed algorithm is still much faster than the original formulation of PC, and yields better classification performance.
We investigate image classification benchmarks using PC, iPC, and BP. We first trained a fully connected network with 2 hidden layers and 64 hidden neurons per layer on the MNIST dataset (LeCun & Cortes) 2010). Then, we trained a mid-size CNN with three convolu- tional layers with 64 128 64 kernels followed by two fully connected layers on MNSST. the Street View House Number (SVHN) dataset (Netzer et al. 2011), and CIFAR10 10 (Krizhevsky jet al. 2012) with no data augmentation. Finally, we trained AlexNet (Krizhevsky et al.] 2012), a lcrge-scale CNN, CIFARIO. To make sure that our results are not the consequence of spe- cific choice of hyperparameters, we performed a comprehensive grid-search on hyperparameters, and reported the highest accuracy obtained. The search is further made robust by averaging over 5 seeds. Particularly, we tested over 8 learning rates (from 0.000001 to 0.01). 4 values of weight decay (0.0001, 0.001, 0.01, 0.1), and 3 values of the integration step y 10.1, 0.5, 1.0), and each com- bination of hyperparameters are evaluated with 5 seeds with mean and standard error reported. To conclude, we have used no data augmentation in the experiments.
Setup of experiments:
Results: In Table I iPC outperforms BP in all the small- and medium-size architectures. For the simplest framework (MNIST on small MLP), PC outperforms all the other training methods, with iPC following by a tiny margin (0.01%). However, PC fails to scale to more complex problems, where it gets outperformed by all the other training methods. The performance of iPC, on the other hand, is stable under changes in size, architecture, and dataset. In fact, iPC reaches a slightly better accuracy than BP on most of the considered tasks.
Table 2: Change of final accuracy when increasing the width.
C'
1
2
3
5
6
7
8
10
15
20
BP
67.92
71.23
71.65
72.64
73.35
73.71
74.19
74.51
74.62
75.08
75.51
iPC
70.61
74.12
74.91
75.88
76.61
77.04
77.48
77.41
76.51
76.55
76.12
Change of width: Table 1 shows that iPC performs better on standard CNN than on AlexNet, which has many more parameters and maxpooling layers. To investigate how iPC behaves when adding max- mox-pooling layers and increasing the width, we trained CNN with three convolutional layers (8, 16, 8) and maxpools, followed by a fully connected layer (128 hidden neurons) on CI- FAR10. We have also replicated the experiment by increasing the width of the network by multi-

---

Under review as a conference paper at ICLR 2023
NETHOD
METHOD
Calibration Error (AdaEGE)
0.6
06
Accuracy
0.4
0.3
0.
0.2
011
Data Shift Intensity
Data Shift Intensity
Figure 4: Robustness of BP and iPC under distribution shift (AlexNet on CIFARIO under five dif- ferent intensities of the corruptions rotation, Gaussian blur, Gaussian noise, hue, brightness, and contrast), Left: Comparable decline of model accuracy between BP and iPC. Right: iPC maintains model calibration significantly better than BP under distribution shift.
plying every hidden dimension by a constant C, (e.g., C' means a network with convolutional layers (24, 48, 24), each followed by maxpool, and fully connected one (384 hidden neurons)), The results in Table show that iPC (i) outperforms BP under each parametrization, (ii) needs less parameters to obtain good results, but (iii) sees its performance decrease, once it has reached specific parametrization. This in contrast to BP. which is able to generalize well even when ex- tremely overparametrized. This suggests that iPC is more efficient than BP in terms of the number of parameters, but that finding the best parameters for iPC may need some extra tuning.
4.1 ROBUSTNESS AND CALIBRATION
Robustness and uncertainty quantification deep learning have become topic of increasing interest in recent years. While neural networks trained via BP reach a strong model performance, their lack of explainability and robustness has been widely studied (Abdar al. [021]] Ovadia et al.) 2019). Recently, it has been noted that treating classifiers generative energy-basee models benefits the robustness the model (Grathwohl et al.) 2019). As PC is precisely an energy-based classifier, originally developed for generation tasks, postulate that iPC possesses better robustness and cal- ibration characteristics than BP. Calibration describes the degree to which predicted logits matches the empirical distribution of observations given the prediction confidence. One may calibrated model's s output to quantify the uncertainty in its predictions and interpret it as probability-no just model confidence. Let P be our random prediction vector indicating the model's confidence that the prediction Ý is correct. We say P is well-calibrated, if the model confidence matches the model per- formance, i.e., P(ý P = p) = p (Guo et al. 2017). We measure the deviation from calibration using the adaptive expected calibration error (AdaECE), which estimates E[II((Y Y|P p) pl] (Nguyen & O' Connor] 2015). In recent years, has become well-known that neural networks trained with BP tend to be overconfident in their predictions (Guo al. (017) and that miscali- bration increases dramatically under distribution shift (Ovadia et al. 2019). More details on the experiments are in the supplementary material.
Our results are shown in Fig. The boxplots indicate the distributions of accuracy (left) and calibration error (right) find that the discriminative performance of the BP and iPC models are comparable under distribution shift. Both models keep a reasonable classification performance for mild corruptions, but show accuracies going down to chance performance under extreme corruptions. The calibration of model output, however, differs strongly: The iPC-trained model yields better calibrated outputs and is able to signal its confidence lot better. This is essential for using the model output as indication of uncertainty. in-distribution data, we observe that iPC yields an average calibration error of 0.05, whereas BP yields 0.12. Moreover, we observe that the increase in calibration error is lot weaker for iPC: The median calibration error of the iPC model is lower across all levels shift intensities compared to that of BP for the mildest corruption. Furthermore, iPC displays better calibration up to level 3 shifts than BP does on in-distribution data. This has potentially strong impact of applying either method in safety-critical applications.
Results:
various forms of data corruption with equal levels of intensity. We
8

---

Under review as a conference paper at ICLR 2023
5 RELATED WORKS
Several previous research efforts aim to achieve supervised learning in biologically plausible way. One is to explore how the error can be encoded differently than in BP where the error is not encoded locally. One of the earliest works was to use second of "error" neurons that can act as the feed- back variables (encoding error in BP) (Stork] |989| [Schwartz]] 1993). Another promising assumption that the error can be represented in neurons' dendrites (Körding & König 2001; 2000; Richards
& Lillicrap 2019 Sacramento et al.
2018). Such efforts are unified in (Lillicrap et al. 2020)) with a
broad range of works (Pineda 1987 1988 O O'Reilly 1996 Ackley et al. [985 Hinton et al. 1995
Bengio 2014 Lee et al. 2015) encoding the error term in activity differences.
prrroscience-inspiree algorithms have recently gained the attention of the machine learning com- munity, due to interesting properties such as locality, autonomy and their energy-based formulation. To this end, multiple works have used PC tackle machine learning problems, from generation tasks (Ororbia & Kiferj (020), to image classification on complex datasets such as ImageNet (He)
(et al. 2016). associative memories (Salvatori et al. 2021 continual learning (Ororbia et al.) 2020),
and NLP (Pinchetti et al. 2022). There more theoretical line of work that is relatet to the free energy principle and active inference (Friston 2008 2010: Friston et al.) 2006 2016). which aims to model learning, perception, and behavior an imperative to minimize free energy. While being initially a theoretical framework, it has been used in multiple applications in fields such as control theory (Baltieri & Buckley| 2019; Friston 2011 and reinforcement learning (Friston et al. 2009). To conclude, it is important to note that iEM is not the only formulation that improves the efficiency of the original EM. some other variations have been proposed, such as an online version (Cappé
& Moulines) 2009), a stochastic one (Chen et al. 2018), or a newer incremental version (Karimi 2019) inspired by the SAGA algorithm (Defazio al. 2014).
ct al.
DISCUSSION
In this paper, we have proposed a biologically inspired learning rule, called incremental predictive coding (iPC) motivated by the incremental EM algorithm. iPC enables all the computations to be executed simultaneously, locally, and autonomously, and has theoretical convergence guarantees in non-asymptotic time (Karimi al.] 2019). This allows solid gain in efficiency compared to the original formulation of PC as well as BP in the full-batch case, as shown with extensive experiments, with no drawbacks in the converging to minimum of the loss. This is confirmed by the good experimental results in terms of accuracy and robustness in classification tasks.
An interesting aspect worth discussing, the time step that triggers the weight update in the three variations of PC: the original formulation, Z-IL., and, now, iPC. The first method updates the pa- rameters only in the last step of the inference, when the neural activities have converged. This has interesting theoretical properties, as it has been shown to simulate how learning is performed in multiple models of cortical circuits, as its credit assignment converges to an equilibrium called prospective configuration (Song et al.) 2022). The second, Z-IL, shows that it suffices time the updates at different levels of the hierarchy in different moments of the inference, to exactly replicate the update given by BP on any possible neural network (Song et al., 2020 Salvatori al. 20226). This is interesting, as it connects PC, a theory developed to model credit assignment in the brain, to BP, method developed train deep learning models. Our newly proposed iPC, on the other hand, updates the parameters continuously, resulting in great gains in terms of efficiency, and no apparent loss in terms of performance. Future work will investigate whether there are better variations of iPC, or whether the optimal update rule can be learned with respect to specific tasks and datasets. Again, the answer may lie in some variations of the EM algorithm. such as dynamical EM (Anil Meera & Wisse| 2021 Friston et al. 2008), or in an implementation of precision-weighted prediction errors, as in Jiang& & | 2022).
On broader level, this work shrinks the gap between computational neuroscience and machine intelligence by tackling the problem of the computational efficiency of neuroscience-inspired train- ing algorithms. Advances in this direction are also interesting from the perspective of hardware implementations of deep learning on energy-based based chips, such as analog and quantum computers. In this case, iPC is an interesting improvement, as it is still not known how external control can be implemented on these chips, and hence algorithms able to train neural networks in a fully automatic fashion may play an important role in the future.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadch, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. A review of uncertainty quantification in deep learning: Tech- niques, applications and challenges. Information Fusion, 76:243-297, 2021.
Mohammed. Abdelghani, Timothy, Lillicrap, and Douglas Tweed. Sensitivity derivatives for flexible sensorimotor learning. Neural Computation, 20(8):2085-2111 2008.
David Ackley, Geoffrey E. Hinton, and Terrence J. Sejnowski. A learning algorithm for boltzmann machines. Cognitive Science, 91):147-16,, 1985.
Ajith Anil Meera and Martijn Wisse. Dynamic expectation maximization algorithm for estimation of linear systems with colored noise. Entropy, 23(10), 2021.
Manuel Baltieri and Chris Buckley. PID control as process of active inference with linear genera- tive models. Entropy, 2019.
Yoshua Bengio. How auto-encoders could provide credit assignment in deep networks via target propagation. arXiv: 7906, 2014.
Rafal Bogacz. A tutorial on the free-energy framework for modelling perception and learning. Journal of Mathematical Psychology, 76: 198-211, 2017.
Chris Buckley, Chang Kim, Simon McGregor, and Anil Seth. The free energy principle for action and perception: A mathematical review. Journal of Mathematical Psychology, 2017.
Olivier Cappé and Eric Moulines. On-line appecaaioonmmximivzaion algorithm for latent data mod- els. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(3)5993-613, 2009.
Jianfei Chen, Jun Zhu, Yee Whye Teh, and Tong Zhang. Stochastic expectation maximization with variance reduction. Advances in Neural Information Processing Systems, 31, 2018.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self- supervised models are strong semi-supervised learners. 34th Conference on Neural Information Processing Systems, NeurlPS, 2020.
Francis Crick. The recent excitement about neural networks. Nature, 337(6203) 1989.
Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives. Advances in Neural Informa- tion Processing Systems, 27, 2014.
Arthur Dempster, Nan Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1): 1-22, 1977.
Karl Friston. Learning and inference in the brain. Neural Networks, 16(9): 2003.
Karl Friston. A theory of cortical responses. Philosophical Transactions of the Royal Society B: Biological Sciences, 360(1456). 2005.
Karl. Friston. Hierarchical models in the brain. PLoS Computational Biology, 2008
Karl Friston. The free-energy principle: a unified brain theory? Nature Reviews Neuroscience, 11 (2):177-13,, 2010.
Karl Friston. What is optimal about motor control? Neuron, 2011.
Karl Friston, James Kilner, and Lee Harrison. A free energy principle for the brain. Journal of Physiology, 2006.
Karl Friston, Jean. Daunizeau, and Stephan. Kiebel. Reinforcement learning or active inference? PloS One, 2009.
10

---

Under review as a conference paper at ICLR 2023
Karl Friston, Thomas FitzGerald, Francesco Rigoli, Philipp Schwartenbeck, Giovanni Pezzulo, al. Active inference and learning. Neuroscience & Biobehavioral Reviews, 085862-879, 2016.
Karl J. Friston, N. Irujill--Barrtt,, and Jean Daunizeau. DEM: A variational treatment of dynamic systems. Neuroimage, 41(3):849-885, 2008
Will Grathwohl, Kuan-Chich Wang, Jörn-eerii Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03263 2019.
Stephen Grossberg. Competitive learning: From interactive activation adaptive resonance. Cog- nilive Science, 11(1):23-63, 1987.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. JCML 2017, 3:2130-2143, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Geoffrey E. Hinton, Peter Dayan, Brendan J. Frey, and Radford M. Neal. The" wake-sleep" algo- rithm for unsupervised neural networks. Science, 268(5214): 158-1161, 1995.
Linxing Preston Jiang and Rajesh P. N. Rao. Dynamic predictive coding: A new model of hierar- chical sequence learning and prediction in the cortex. bioRxiv, 2022.
Belhal Karimi, Hoi-To Wai, Eric Moulines, and Mare Lavielle. On the global convergence of (fast) incremental expectation maximization methods. Advances Neural Information Processing Sys- tems, 32, 2019.
Jack Kendall, Ross Pantone, Kalpana Manickavasagam, Yoshua Bengio, and Benjamin Scel- lier. Training end-to-end analog neural networks with equilibrium propagation. arXiv preprint (Xiv:2006.0198/1 2020.
Konrad P. Körding and Peter König. Learning with two sites of synaptic integration. Network: Computation Neural Systems, 11, 2000.
Konrad P. Körding and Peter König. Supervised and unsupervised learning with two sites of synaptic integration. Journal of Computational Neuroscience, 11(3):07-215,, 2001.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet classification with deep convo- lutional neural networks. 26th Annual Conference on Neural Information Processing Systems (NIPS) 2012, 2012.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. The MNIST Database, 2010.
URL http:://yannnn..... lecun. cwtt://////c/eececcccccc
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propagation. In Proc. ECMLPKDD. 2015.
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, (1):1-10, 2016.
Timothy Lillicrap, Adam Santoro, Luke Marris, Colin Akerman, and Geoffrey Hinton. Back- propagation and the brain. Nature Reviews Neuroscience, 2020.
Beren Millidge, Alexander Tschantz, and Christopher L Buckley. Predictive coding approximates backprop along arbitrary computation graphs. arXiv:2006.04182, 2020.
Beren Millidge, Tommaso Salvatori, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Pre- dictive coding: Towards a future deep learning beyond backpropagation? arXiv preprint arXiv:2202.09467 2022.
11

---

Under review as a conference paper at ICLR 2023
David Mumford. On the computational architecture of the neocortex. Biological Cybernetics, 66 (3):241-251, 1992.
Radford M. Neal and Geoffrey E. Hinton, A view of the EM algorithm that justifies incremental, sparse, and other variants. In Learning in graphical models, pp. 355-368. Springer, 1998.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Khanh Nguyen and Brendan T. O'Connor. Posterior calibration and exploratory analysis for natural language processing models. In EMNLP, 2015.
Randall C O' O'Reilly. Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural Computation, 8(5):895-993) 938, 1996.
Alex Ororbia and Daniel Kifer. The neural coding framework for learning generative models. arXiv:2012.03405, 2020.
Alexander Ororbia, Ankur Mali, C. Lee Giles, and Daniel Kifer. Continual learning of recurrent neural networks by locally aligning distributed representations. IEEE Transactions on Neural Networks and Learning Systems, (110044267-4278, 2020.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, Sculley, Sebastian Nowozin, Joshua V. Dil- lon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? eval- uating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, volume 32, 2019.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Luca Pinchetti, Tommaso Salvatori, Beren Millidge, Yuhang Song, Yordan Yordanov, and Thomas Lukasiewicz. Predictive coding beyond gaussian assumptions. 36th Conference on Neural Infor- mation Processing Systems, 2022.
Fernando Pineda. Generalization back-propagation to recurrent neural networks. Physical Review Letters, 59(19):2229, 1987.
Fernando Pineda. Dynamics and architecture for neural computation. Journal of Complexity, (3):216-245, 1988.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.0612. 2022.
Rajesh P. N. Rao and Dana H. Ballard. Predictive coding in the visual cortex: functional interpre- tation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79-8) 87, 1999.
Blake A. Richards and Timothy P. Lillicrap. Dendritic solutions to the credit assignment problem. Current Opinion in Neurobiology, 54:28-36, 2019.
Pieter R. Roelfsema and Anthony Holtmaat. Control of synaptic plasticity in deep cortical networks. Nature Reviews Neuroscience, 19(3):166, 2018
João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcir- cuits approximate the backpropagation algorithm. In Advances in Neural Information Processing Systems, pp. 8721-8732, 2018.
Tommaso Salvatori, Yuhang Song, Yujian Hong, Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bo- gac7, and Thomas Lukasiewiez. Associative memories via predictive coding. In Advances in Neural Information Processing Systems, volume 34, 2021.
Tommaso Salvatori, Luca Pinchetti, Beren Millidge, Yuhang Song, Tianyi Bao, Rafal Bogacz, and Thomas Lukasiewicz. Learning on arbitrary graph topologies via predictive coding. arXiv:2201.131880 2022a.
12

---

Under review as a conference paper at ICLR 2023
Tommaso Salvatori, Yuhang Song, Zhenghua Xu, Thomas Lukasiewicz, and Rafal Bogacz. Reverse differentiation via predictive coding. Proceedings of the 36th AAAI Conference on Artificial Intelligence. AAAI Press, 2022b.
Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy- based models and backpropagation. Frontiers in Computational Neuroscience, 11:24, 2017.
Eric L. Schwartz. Computational Neuroscience. Mit Press, 1993.
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre. George den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panncershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of Go with deep neural networks and tree search. Nature, 529, 2016.
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. Mastering the game of Go without human knowledge. Nature, 550, 2017.
Yuhang Song. Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Can the brain do backpropa- gation? - Exact implementation backpropagation in predictive coding networks. In Advances in Neural Information Processing Systems, volume 33, 2020.
Yuhang Song, Beren Gray Millidge, Tommaso Salvatori, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Inferring neural activity before plasticity: A foundation for learning beyond back- propagation. bioRxiv, 2022.
David G. Stork. backpropagation biologically plausible. International Joint Conference on Neural Networks, volume 2, pp. 241-246. IEFE Washington, DC, 1989.
James C. R. Whittington and Rafal Bogacz. An approximation of the error backpropagation algo- rithm in a predictive coding network with local Hebbian synaptic plasticity. Neural Computation, 29(5), 2017.
James R. Whittington and Rafal Bogacz, Theories of error poc-ppropagttion in the brain. Trends in Cognitive Sciences, 2019.
13

---

Under review as a conference paper at ICLR 2023
W
X1
X2
82
W
X3
Reodiing
W
i2
W1
X1
82
X2
W
83
X3
Dynamics:
Inhibitory
81 Error Nodes
Excitatory
81 X1 1W1-1x1-1
i Interneurons
x1 = - + W 81+1
W Synaptic Weights
Figure 5: Standard and dendritic neural implementation of predictive coding. The dendritic im- plementation makes use of interneurons i Wiz1 (according to the notation used in the figure). Both implementations have the same equations for all the updates, and are thus equivalent; however, dendrites allow a neural implementation that does not take error nodes into account, improving the biological plausibility of the model. Figure taken and adapted from (Whittington Bogaez 2019).
A A DISCUSSION ON BIOLOGICAL PLAUSIBILITY
In this section, we discuss the biological plausibility of the proposed algorithm, a topic overlooked in the main body of this paper. In the literature, there is often disagreement whether a specific algorithm is biologically plausible or not. Generally, is assumed that an algorithm is biologically plausible when it satisfies list of properties that are also satisfied in the brain. Different works consider different properties.
our case, we consider as list of minimal properties that include local computations and lack of global control signals to trigger the operations. Normally, predictive coding networks take error nodes into account, often considered implausible from the biological perspective (Sacramento fected by this: is in fact possible to map PC on a different neural architecture, which errors encoded in apical dendrites rather than separate neurons (Sacramento al.) 2018 Whittington & Boggaz 2019). Graphical representations of the differences between the two implementations can be found Fig. taken (and adapted) from (Whitlington & Bogacz 2019). Furthermore, our formulation is more plausible than the original formulation of PC, as it able to learn without the need of external control signals that trigger the weight update.
al. 2018). Even so, the biological plausibility of model is not af-
14

---

Under review as a conference paper at ICLR 2023
B PSEUDOCODES OF Z-IL AND PC
Algorithm 2 Learning a dataset D = {vi} with PC.
1: Require: For every i, x. is fixed to yi.
2: for to T do
For every i and update (1 to minimize F Eq.(7)
4: if T then
For every / update each ( to minimize F via Eq. (8)
5: end if
6: end for
Algorithm Learning one training pair (s" s with Z-IL
1: Require: w1 is fixed th .g fixed to SM
2: Require: (d() = (x for E (L, L., and = 0.
3: for <= 0 7' do
4: for each level do
Update (0()) minimize F Eq.(7)
if = | then
7:
Update ( to minimize F via (8).
8: end if
end for
end for
15

---

Under review as a conference paper at ICLR 2023
Table 3: Theoretical Efficiency of PC, 7-II., BP, and iPC.
One inference step
PC
Z-IL
BP
iPC
Number of MMs per weight update Number of SMMs per weight update
(2L 1)
(2L 1)7
(2L 1)(L - 1)
(2L (2L 1)
(2L 1)
2(L 1)
C ON THE EFFICIENCY OF PC, BP, AND IPC
In this section, we discuss the time complexity and efficiency of PC, BP, Z-IL,, and iPC. We start with the first three, and introduce metric that we use to compute such complexity. This metric the number of simultaneous matrix multiplications (SMMs), i.e., the number of non-parallelizable matrix multiplications needed to perform single weight update. It is a reasonable approximation of running time, as multiplications are by far the most complex operation (== O(N")) performed by the algorithm.
C.I COMPLEXITY OF PC, BP, AND Z-IL
Serial Complexity: To complete single update of all weights, PC and Z-IL run for T and (L 1) inference steps, respectively. To study the complexity of the inference steps we consider the number of matrix multiplicatiins (MMs) required for each algorithm: One inference step requires (2L
MMs: L for updating all the errors, and (L 1) for updating all the value nodes (Eq, equation 6 Thus, to complete one weight update, PC and Z-IL require (2L 1)T and (2L 1)(L - 1) MMs, respectively. Note also that BP requires (2L - 1) MMs to complete a single weight update: for the forward, and (L - 1) for the backward pass. These numbers are summarized in the first row of Table 3 According to this measure, BP is the most efficient algorithm, Z-IL ranks second, and PC third, as in practice T is much larger than L. However, this measure only considers the total number of matrix multiplications needed, without considering whether some of them can be performed in parallel, which could significantly reduce the time complexity. We now address this problem.
The MMs performed during inference can be parallelized across layers. In fact, computations in Eq. equation 6 are layer-wise independent, thus MMs that update all the error nodes take the time of only one MM if properly parallelized. Similarly, in Eq. equation 6 (L (L-1) 1) MMs that update all the value nodes take the time only one MM if properly parallelized. As result, one inference step only takes the time of 2 MMs if properly parallelized (since, as stated, consists of updating all errors and values via Eq. equation 6). Thus, one inference step takes
Parallel complexity:
SMMs; one weight update with PC and Z-IL takes 2T and 2 - 1) SMMs, respectively. Since no MM can be parallelized in BP (the forward pass in the network and the backward pass of error both layer-dependent), before performing single weight update, (2I - 1) SMMs are required. These numbers are summarized in the second row of Table 3 Overall, measured over SMMs, BP and Z-IL are equally efficient (up a constant factor), and faster than PC.
C.2 COMPLEXITY OF IPC
To complete one weight update, iPC requires one inference step, thus (2L- MMs or 2 SMMs, as also demonstrated in the last column of Table Compared to BP, iPC takes around L times less SMMs per weight update, and should hence be significantly faster in deep networks. Intuitively, this is because matrix multiplications in BP have to be done sequentially along layers, while the ones in iPC can all be done in parallel across layers (Fig. 6). More formally, we have the following theorem, which holds when performing full-batch training:
Theorem D. Let M and M' be two equivalent networks with L layers trained on the same dataset. Let M (resp., M') be trained using BP (resp., iPC). Then, the time complexity measured by SMMs needed to perform one full update of the weights is ((1) and (L) for iPC and BP, respectively.
Proof. Consider training on an MLP with L layers, and update weights for multiple times single datapoint. Generalizations to multiple datapoints and multiple mini-batches are similar and will be provided after. We first write the equations needed to be computed for iPC produce weight
16

---

Under review as a conference paper at ICLR 2023
update:
1,1 s and (0)
2,1
5;
n
20 = Σ
0
((++1)
for E (1,..., [,]
(9)
1,
1=1
(1) (1)
Zi
for I E (I...., L}
n
0++3
3 1,411
( (1)
-it +
- + x
!1)
(1)
Rit
(1+1)
@ kid
for (1...., L.
(10)
00)
++++1
00) j.tt
-
i,
(1+1)
((m
for I E (l....,L}
(11)
We then write the three equations needed be computed for BP produce one weight update:
0 i,t
= Σ
g
f(x)(t))))))) for {I....,L}
(12)
1,t 1=1
(L)
spet Hitt
(L)
n (413
5 = f
(1)
Σ
o k,t
e forl {L,...,
(13)
"
- = 0
- -
11+1) f(a))) for I LJ.
First, we notice that the matrix multiplication (MM) is the most complex operation. Specifically, for two adjacent layers with the size of nı' and n', the complexity of MM is O(n'n'), but the maximal complexity of the other operations O Ommax n²). the above equations, only equations with MM are numbered, which the equations that investigate in complexity analysis.
Eq. equation for iPC takes L MMs, but one SMM, since the the for-loop for E {1,.... ,..,,, L} can run in parallel for different f. This is further because the variables the right side of Eq. equation are immediately available. Differently, Eq. equation [2] for iPC takes L MMs, and also L SMMs, since the for-loop for E (1,...,,,] L} has be executed order (2...., ,....,,,,, LJ. This is further because the qualities on the right side of Eq. equation 12 are immediately available, but require to solve Eq. equation 12 again for another layer. That is, get
after another, following the specified
2 Eq. equation 12 has to be solved recursively from / to L.
Similar sense applies the comparison between Eqs. equation and equation 13 Eq. equation 10 for iPC takes MMs but SMMs; Eq. equation T3 for BP takes T. MMs and also
SMMs.
Overall, Eqs. equation 9 and equation 10 for iPC take 27. MMs but SMMs; Eqs. equation 12 and equation [3 for BP take 2L MMs and also 2L SMMs. Then, the time complexity measured by SMMs needed to perform one full update of the weights is O(1) and ((L) for iPC and BP, respectively.
C.3 EFFICIENCY ON ONE DATA POINT
To make the difference more visible and provide more insights, explain this in detail with sketch of this process on a small network in Fig. 6 where the horizontal axis of m is the time step measured by simultancous matrix multiplications (SMMs), i.e., within single m, perform one matrix multiplication or multiple ones in parallel; if two matrix multiplications have to be executed in order (e.g., the second needs results from the first), they will need to be put into
17

---

Under review as a conference paper at ICLR 2023
m == 0
11
==
m
==
m=
m
m= == M 6
771 Bockward SMM
1 0
Inpull
1= 1 E
Hidden neuron (error not updated)
I 2
I 3
Hidden neuron (error updated)
[
Output neuron
1-1
Weights (not updated)
2 2 [ 1 3
Weights (updated)
Figure 6: Graphical PClustration of the efficiency over backward SMMs of BP and iPC on a 3-layer network. iPC never clears the eтror (red neurons), while BP clears it after every update. This allows iPC to perform 5 full and 2 partial updates of the weights in the first 6 SMMs. frame, BP only performs 3 full updates. Note that the SMMs of forward passes are excluded for simplicity, w.l.o.g.. as the insight from this example generalizes to the SMMs of the forward pass.
the same time
two steps of m. Note that only consider the matrix multiplicattins for the backward pass, i.e., the matrix multiplications that backpropagate the error of a layer from an adjacent layer for BP and the inference of Eq. equation 6 for iPC, thus the horizontal axis m is strictly speaking "Backward SMM". The insight for the forward pass is similar as that of the backward pass. As it has been said, for BP, backpropagating the eтror from one layer to an adjacent layer requires one matrix multiplication; for iPC, one step of inference on one layer via Eq. equation 6 requires one matrix multiplication. BP and iPC are presented the first and second rows, respectively. Before both methods are able to update weights in all layers, they need two matrix multiplications for spreading the error through the network, i.c., a weights update of all layers occurs for the first time at m 2 for both methods. After m 2, BP cleared all errors all neurons, at = 3, BP backpropagates the error from = 0 to I = 1, and at m = 4, BP backpropagates the error from = 1 after which it can make update of weights all layers again for the second time. Note that the matrix multiplication that backpropagates errors from = to = 2 at m = 4 cannot be put at 3, i.e., it requires the error be
m 3, as it requires the results of the matrix multiplication at backpropagated to = 1 from = 0 at m = 3. However, this is different for iPC. After m = 2, iPC does not reset o to Mi,,. i.c.. the error signals are still held in ei, At matrix multiplications in parallel, corresponding to two inferences steps at two layers, 1= updating x/,,, and hence the error signals are held in ci,, of these two layers. Note that the above two matrix multiplications of two inference steps can run in parallel and be put into a single m, as inference requires only locally and immediately available information. update in iPC is able to be performed at every ever since the very first few steps of m.
3, iPC performs two
1 and
this way, a weight
D TRAINING DETAILS
We now list some additional details to reproduce our results.
D.1 EXPERIMENTS OF EFFICIENCY
The experiments for the efficiency of generative models were run on fully connected networks with 128, 256 or 512 hidden neurons, and L E 54. Every network was trained on CIFARIO or Tiny Imagenet with learning rates a = 0.00005 and y = 0.5, and T E {8, 12, 16). The experiments on discriminative models are performed using networks with 64 hidden neurons, depth L {3, 4, 6), and learning rates a = 0.0001 and ? = 0.5. The networks trained with BP have the same learning rate All the plots for every combination of hyperparameters can be found in Figures 8 and 7
18

---

Under review as a conference paper at ICLR 2023
D.2 EXPERIMENTS OF GENERALIZATION QUALITY
As already stated in the paper body, to make sure that our results are not the consequence of specific choice of hyperparameters, and reported the highest accuracy obtained, and the search is further made robust by averaging over 5 seeds. Particularly, we tested over 8 learning rates (from 0.000001 0.01). values of weight decay (0.0001, 0.001, 0.01, 0.1), and 3 values of the integration step y (0.1, 0.5, 1.0). We additionally verified that the optimized value of each hyperparameter lies within the searched range of that hyperparameter. As for additional details, we used standard Pytorch initialization for the parameters. For the hardware, we used single Nvidia GeForce RTX 2080 GPU on an internal cluster. Despite the large search, most of of the best results were obtained using the following hyperparameters: = 0.5 ( = for Alexnet), a = 0.00005.
performed comprehensive grid search on hyperparameters.
19

---

Under review as a conference paper at ICLR 2023
CIFARIO
HD 128
HD 256
HD 512
PC:
PS,
PC, 1w
L=4
1000
2000
3000
1000
2010
3000
1000
2000
Itcrations
Iterations
Iterations
IPC
PC: TwB
PCC, 7-8
PC. Tw12
P1) 2-12
PC, T-10
T-16
PC; T-16
L-5
1000
2000
2000
3000
Iterations
Iterations
Itcrations
Tiny Imagenet
HD 128
HD 256
HD 512
BPC
IPC
IPC
PS, T-dl
PR; 2-B
PC, Tw8
PC T-12
Tw12
T-12
T-16
L-4
1000
2000
2000
1000
2000
3000
1000
2000
3000
lterations
Iterations
Irerations
IPC
PCC
JPC
PC T-B
PC T-0
PCC Twd
PC Tw12
PC 7-
PC T-12
PC, 1w20
7-16
PC: Tw16
L=5
Itcrations
Iterations
Iterations
Figure 7: Efficiency of multiple generative networks trained with PC.
20

---

Under review as a conference paper at ICLR 2023
MNIST
L=3
L=
IPC
125
IPC
28
525
IPC
Tw3
190
T-12
Tw12
T-12
[w10
75
Tw11
Train Loss
25
1020
2000
3000
0
1000
2000
3000
SMMs
SMMs
SMMs
IPC
0.8
T-8
3P
0.8
Tw12
T-12
Test
T-B
[w10
T-16
12
Accuracy
0.4
Tw
0.2
0.2
SMMs
SMMs
SMMs
FashionMNIST
HD 128
HD 256
HD 512
150
150
125
175
12
1.0
100
T-12
Train Loss
16
T-16
50
0
0
1000
2000
3000
1000
2000
3000
1000
2000
3000
SMMs
SMMs
SMMs
IPC
UPC
IPC
0.5
0.8
T-B
Tw12
Test Accuracy
0.6
0.6
0.4
0.4
0.2
SMMs
SMMs
SMMs
Figure 8: Efficiency of multiple discriminative networks trained with PC and BP.
21


Reviews:
[{"review_id": "uB6i9jaDc9", "reviewer": "Reviewer_dzfS", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper describes a variant of predictive coding, named incremental predictive coding (iPC), based on incremental EM, which it is argued should be considered a biologically plausible approach to learning in the brain.  The complexity of iPC is considered in relation to back-propagation (BP), and a CPU implementation is provided.  Further, the generalization performance is investigated on a number of datasets, and the algorithm is shown to perform well in comparison to BP and PC.", "summary_of_the_review": "An interesting investigation of an algorithm that may have relevance in neuroscience, and deserves further attention.  Potentially, the paper may be of interest to those working in neuroscience and optimization.", "strength_and_weaknesses": "Strengths:\n\n-  The biological plausibility argument is interesting, and in general the argument is convincing that some form of 'localized EM' algorithm is more plausible than BP or PC alternatives, while retaining convergence and generalization properties.\n-  The experimentation convincingly demonstrates that iPC should be considered a viable alternative to BP generally, at least for simple architectures and specialized hardware.\n\nWeaknesses:\n\n- I'm mainly concerned about the paper's novelty - essentially, iPC is equivalent to iEM applied to a hierarchical Gaussian model.  The theoretical properties are described elsewhere (e.g. Karimi 2019) and the biological plausibility argument is hard to evaluate, although likely to be worth pursuing further.\n- There is little theoretical novelty, since the time-complexity analysis (Theorem 1) essentially follows simply by definition.  As discussed by the authors, the comparison of training-loss convergence rates in terms of 'non-parallel matrix multiplications' is an interesting result, but this is investigated solely empirically (Fig. 2 right).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "sNHBcUVHeV", "reviewer": "Reviewer_fv42", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper points out that the predictive coding implementation from Rao and Ballard is essentially expectation maximization, and therefore can be sped up by using the incremental (aka partial) variant of EM instead of the full version. The paper also argues that this is more efficient than backpropagation, at least for large networks and full-batch learning, and has better uncertainty calibration than models trained with backprop. \n", "summary_of_the_review": "I enjoyed reading the paper and I think the topic is important, and a great fit for ICLR. I pointed out a variety of claims that appear questionable to me, which maybe stem from thinking about predictive coding as a model formulation (as I do, which I think is standard) vs. thinking about predictive coding as an optimization technique (as the paper does). ", "strength_and_weaknesses": "I have some issues with the claims and descriptions made in this paper. I want to acknowledge early that some of my issues may be misunderstandings, because I am still learning about predictive coding here. \n\nThe high-level issue for me is: the background seems to be \"predictive coding vs backpropagation\", where predictive coding is essentially equivalent to expectation-maximization, but this is not the way I think about predictive coding. This sentence matches my understanding better: \"PC is based on the assumption that brains implement an internal generative model of the world, needed to predict incoming stimuli (or data).\" Expectation-maximization is one way to optimize a predictive coding model, and backpropagation is another, and there are more still, like Hebbian learning -- these are all optimization techniques, and choices here are orthogonal to the issue of predictive inference vs non-predictive inference. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "cCsz7LlU5f", "reviewer": "Reviewer_szhT", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is basically an implementation of the prospective configuration algoithm (Yuhang Song, et al, Inferring neural activity before plasticity: A foundation for learning beyond backpropagation) where each layer of a heirarchical Gaussian generative model (with the further assumption that the covariance matrix of the gaussians is identity), is updated in parallel. Convergence guarantees to some local optima come from Neal and Hinton's incremental EM paper. ", "summary_of_the_review": "The paper has very limited novelty. It is just an implementation of ideas found in other papers.", "strength_and_weaknesses": "Strength: Empirical results show faster convergence\nWeekness: There is nothing novel in this paper. Neither does this paper claim to introduce the prospective configuration algo (I am guessing it is done in another submission), nor does it need to show convergence (which follows from Neal and Hinton). It is therefore just an implementation paper. Had the experiments been large and comprehensive, there would be reason to consider this paper.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "XT8uTdid6jv", "reviewer": "Reviewer_kkJ5", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an incremental predictive coding method that performs faster on larger models than BP.", "summary_of_the_review": "Overall, this is an interesting work. Some clarification on test results would make it more convincing. ", "strength_and_weaknesses": "I do like the idea and the demonstrated performance of the proposed method. That being said, there are a few weaknesses I wish that the author could address:\n* 3.2 on CPU implementation, I do not fully understand what is the overhead for iPC. Besides, the time complexity is related to L, then why does the improvement over BP also scales with hidden dimension (is this the amount of neurons in hidden layers)?\n* For the experimental results reported in table 1, is the PC trained with the same amount of epochs as iPC, so that it behaves worse on AlexNet as it is not yet converges? The comparison with PC states that the iPC converges faster, but my understanding is that he iPC is a more efficient way to approximate PC, I do not understand why it failed to scale with model complexity. Besides, is it possible to test on some more complex model or image data set to back the idea that iPC preforms better on complex image related tasks?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject