Learning on Arbitrary Graph Topologies via Predictive Coding
Authors: Tommaso Salvatori
Luca Pinchetti
Beren Millidge
Yuhang Song
Tianyi Bao
Rafal Bogacz
Thomas Lukasiewicz
Venue: NeurIPS 2022 Accept (2022)
Keywords: Cognitive Science

Abstract: Training with backpropagation (BP) in standard deep learning consists of two main steps: a forward pass that maps a data point to its prediction, and a backward pass that propagates the error of this prediction back through the network. This process is highly effective when the goal is to minimize a specific objective function. However, it does not allow training on networks with cyclic or backward connections. This is an obstacle to reaching brain-like capabilities, as the highly complex heterarchical structure of the neural connections in the neocortex are potentially fundamental for its effectiveness. In this paper, we show how predictive coding (PC), a theory of information processing in the cortex, can be used to perform inference and learning on arbitrary graph topologies. We experimentally show how this formulation, called PC graphs, can be used to flexibly perform different tasks with the same network by simply stimulating specific neurons. This enables the model to be queried on stimuli with different structures, such as partial images, images with labels, or images without labels. We conclude by investigating how the topology of the graph influences the final performance, and comparing against simple baselines trained with BP.

Learning on Arbitrary Graph Topologies via Predictive Coding
Tommaso Salvatori!** Luca Pinchetti Beren Millidge Yuhang Song 122
Tianyi Bao! Rafal Bogacz² Thomas Lukasiewicz
Department of Computer Science, University of Oxford, UK
MRC Brain Network Dynamies Unit, University of Oxford, UK
Institute of Logic and Computation, TU Wien, Austria luc..pinche...........ee beren.millidgehdden ac uk, yuhang. . song@some pinchetti@cs ax. uk rafal. bogaczondcn. ox ac. uk, thomas. lukasiewicz@cs ox. ac uk
tommaso salvatori@cs ox ac. uk, luca.
uk, tianyi. baoêcs. ac uk
Abstract
Training with backpropagation (BP) in standard deep learning consists of two main steps: a forward pass that maps data point to its prediction, and backward pass that propagates the error of this prediction back through the network. This process is highly effective when the goal is minimize specific objective function. However, it does not allow training on networks with cyclic or backward connections. This is obstacle reaching brain-like capabilities, the highly complex heterarchical structure of the neural connections in the neocortex are potentially fundamental for its effectiveness. In this paper, show how predictive coding (PC), theory of information processing in the cortex, can be used to perform inference and learning on arbitrary graph topologies. We experimentally show how this formulation, called PC graphs, can be used to flexibly perform different tasks with the same network by simply stimulating specific neurons. This enables the model be queried on stimuli with different structures, such as partial images, images with labels, or images without labels. We conclude by investigating how the topology of the graph influences the final performance, and comparing against simple baselines trained with BP.
1 Introduction
Classical deep learning has achieved remarkable results by training deep neural networks to minimize an objective function. Here, every weight parameter gets updated minimize this function using reverse differentiation [1] 22 However, in the brain, every synaptic connection is independently updated to correct the behaviour of its post-synaptic neuron ( using local information, and is unknown whether this process minimizes global objective function. The brain maintains an internal model of the world, which constantly generates predictions of external stimuli. When the predictions differ from reality, the brain immediately corrects this error (difference between reality and prediction) by updating the strengths of the synaptic connections 4177 This theory of information processing, called predictive coding (PC), is highly influential, despite experimental evidence in the cortex being mixed 8-II, and it is at the centre large amount of research in computational neuroscience I22 [6]. From the machine learning perspective, PC has promising properties: it is able to achieve excellent results in classification [17]]19]]] and memorization (20] 21], and is able to process information in both bottom up and top down direction. This last property is fundamental
Corresponding author. Equal contribution.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Artificial Neural Network
Biological Neural Network
Figure I: Difference in topology between an artificial neural network (left), and sketch of network of structural connections that link distinct neural elements in a brain (right) 26
for the functioning of different brain areas, such as the hippocampus generalization capabilities of standard deep learning, as it is able to approximate backpropagation (BP) on any neural structure ( and a variation of PC is able to exactly replicate the weight update of BP on any computational graph [24] 25 Moreover, PC only uses local information update synapses, allowing the network to be fully parallelized, and to train on networks with any topology.
PC also shares the
Training on networks of any structure is not possible in standard deep learning, where information only flows in one direction via the feedforward pass, and then BP is performed in sequential steps backwards. If a cycle is present inside the computational graph of an artificial neural network (ANN), BP becomes stuck in an infinite loop. More generally, the computational graph of any function F: R - R is poset, and hence acyelic. While the problem of training some specific cyclic structures has been partially addressed using BP through time (2 on sequential data, the restriction to hierarchical architectures may present limitation to reaching brain-like intelligence, since the human brain has an extremely complex and entangled neural structure that is heterarchically organized with small-world connections 126- - topology that is likely highly optimized by evolution. This shape of structural brain networks, shown in Fig. [ generates unique communication dynamics that is fundamental for information processing in the brain, as different aspects of network topology imply different communication mechanisms, and hence perform different tasks [26). The heterarchical
topology of brain networks has motivated research that aims to develop learning methods graphs of any topology. A popular example is the assembly calculus 23 29 a Hcbbian learning method that can perform different operations implicated in cognitive phenomena.
In this work, we address this problem by proposing PC graphs, a structure that allows to train on any directed graph using the original (error-driven) framework by Rao and Ballard 77]. We then demonstrate the flexibility of such networks by testing the same network on different tasks, which can be interpreted as conditional expectations framework enables the model to queried on stimuli with different structures, such as partial images, images with labels, images without labels. This is significantly more flexible than the strict input-outpus structure of standard ANNs, which are limited to scenarios when they are always presented with data and labels in the same format.
different neurons of the network. Our PC graphs
Note that the main goal of this work is not to propose a specific architecture that achieves state-of-the. art (SOTA) results on a particular task, but to present PC graphs as a new flexible and biologically plausible model, which can achieve good results on many tasks simultaneously. In this work, we study the simultaneous generation, classification, and associative memory capabilities of PC graphs, highlighting their flexibility and theoretical advantages over standard baselines. Our contributions are briefly summarized as follows:
We present PC graphs, which generalize PC to arbitrary graph topologies, and show how a single model can be queried in multiple ways to solve different tasks by simply altering the values of specific nodes, without the need for retraining when switching between tasks. Particularly, we define two different techniques, which we call query by conditioning and query by initialization.
We then experimentally show this in the most general case, i.e., for fully connected PC graphs. Here, we train different models on MNIST and FashionMNIST, and show how the two queries can be used to perform different generation tasks. Then, we test the model on classification tasks, and explore its capabilities an associative memory model.

---

(8)
(b)
error, which lies not Aos 10
fiz
12,1
E1,3
eag
e3,1
2,3
Krrs
3
14
f
t=
Graph Virtex
Node
Exoitatory Connection
8
Training Pattem
Conszalned Sensory
Directed Connection
Vilus Nods
intiblory Connection
Sensory Nodiss
Internal Vilue Nods
Figure 2: (a) An example of fully connected PC graph with three vertices. Zoomed is the neural implementation of PC, where learning is made local via the demonstrated inhibitory and excitatory connections. (b) A sketch of the training process, where the value nodes of the sensory vertices are fixed to the pixels of the image. nodes is fixed to the top half of image, and the bottom half is recovered via inference.
(c) A sketch of query by conditioning, where a fraction of the value
We next investigate how different graph topologies influence the performance of PC graphs on generation tasks, reproducing common network architectures such as feedforward, recurrent, and residual networks as special cases of PC graphs, and investigate how the chosen structure influences the performance on generative tasks. Finally, also show how PC graphs be used derive the popular assembly calculus (28]
2 PC Graphs
Let G = (V, E) be a directed graph, where V a of n vertices {1, .1.2.......) .n). and E S X V is a set of directed edges between them, where every edge (i,j) E E has weight parameter 0z.j. The set of vertices is partitioned into two subsets, the sensory and internal vertices. External stimuli are always presented to the network via sensory vertices, which we consider to be the first d vertices of the graph, with d < n. The internal vertices, on the other hand, are used to represent the internal structure of the dataset. Each vertex encodes several quantities. The main quantity is given by the values of its activity, which change over time, and we refer to as a value node Ti, We call the value nodes of the sensory vertices sensory nodes. Additionally, each vertex computes the prediction Us,t of its activity based on its input from value nodes other vertices:
He, E,0zs((3z))
where the summation is over all the vertices connected to via outgoing edges, and is non- linearity. Equivalently, it is possible to consider the summation on every j. and have 0,, = (i,5) F. The error of every vertex every time step is then given by the difference between its value node and its prediction, i.e., Ei = Fi,t Hi,s- This local definition only in the output, but in every vertex of the network, is what allows PC graphs learn using only local information. The value nodes Wi,t and the weight parameters 0i, are updated to minimize the following energy function defined locally on every vertex:
& = 1
(2)
fully connected PC graph with 3 vertices is sketched Fig. 2.. along with the operations that describe the dynamics of the information flow, showing also how every operation can be represented via inhibitory and excitatory connections.
When presented with a training point taken from training set, the value nodes of the sensory vertices are fixed to be equal the entries of for the whole duration of the training process, i.c., for every t. A sketch of this is shown in Fig. Pp. Then, the total energy of Eq. 2 is minimized in two phases: inference and weight update. During the inference phase, the weights are fixed, and the value nodes are continuously updated via gradient descent for I iterations, where T is hyperparameter of the model. The update rule is the following (inference):
Learning:
Arit = -----000 00/001. 7 (-Ei,
+ -
(x..))) (4,,00
(3)
3

---

where y is the learning rate of the value nodes. This process of iteratively updating the value nodes distributes the output error throughout the PC graph. When the inference phase is completed, the value nodes get fixed, and single weight update is performed as follows (weight update):
A0., -a 080/00.,- a- - ((,,))
(4)
where a is the learning rate of the weight update. We now describe two different ways to query the internal representation of trained model, where the values of some sensory vertices undefined, and have to he predicted. In both cases, the weight parameters 0i, are now fixed, and the total energy E is continuously minimized using gradient descent on the re-initialized value nodes via Eq. (
Query by conditioning: While each value node is randomly re-initialized, the value nodes of specific vertices are fixed to some desired value, and hence not allowed to change during the energy minimization process. The unconstrained sensory vertices will then converge to the minimum of the energy given the fixed vertices, thus computing the conditional expectation of the latent vertices given the observed stimulus. Formally, let I = {fi.....&y } C {1,2,.... ........., n) be a strict subset of vertices. Assume now that know that subset of the value nodes corresponding to the vertices / is equal to a stimulus & E IRV. Then, running inference until convergence allows to estimate the conditional expectation
E(Er Vt: (Is,, Rs,,e) = q)
(5)
where Ër is the vector of value nodes at convergence. Examples of tasks performed this way are (i) classification, where internal nodes are fixed to the pixels of image, and the sensory nodes fixed to 1-hot vector with the labels, (ii) generation, where the single value node encoding the class information is fixed, and the value nodes of the sensory nodes converge to an image of that class, and (iii) reconstruction, such as image completion, where a fraction of the sensory nodes are fixed to the available pixels of an image, and the remaining ones converge to a reasonable completion of it. A sketch of this process is shown in Fig. 2
Query by initialization: Again, every value node is randomly initialized, but the value nodes of specific nodes are initialized (for t= 0 only), but not fixed (for all 1), to some desired value. This differs from the previous query, as here every value node is unconstrained, and hence free to change during inference. The sensory vertices will then converge to the minimum found by gradient descent, when provided with that specific initialization. Again, let / = {ii..... 3g C 21, ... n be strict subset of vertices, and assume that we have an initial stimulus q E R'. Then, we can estimate the conditional expectation
E(ir (I "..,, -
(6)
Examples of tasks performed this way are (i) denoising, such as image denoising, where the sensory neurons are initialized with noisy version of an image, which is cleared during the energy minimiza- tion process, and (ii) reconstruction, such as image completion, where the fraction of missing pixels is now not known priori.
3 Proof-of-concept: Experiments on Fully Connected PC Graphs
In this section, we perform experiments on a fully connected PC graph G = (V, E), i.e. where E = V x V. Such PC graphs fully general and encode implicit priors the structure of the dataset. It is possible to obtain any possible graph topology by simply pruning specific weights of G.
Given dataset of m datapoints D = (Si)iem. with 8; E Rd, we train the PC graph as described in Section 2: The first neurons are fixed to the entries of a training point, and the energy function &, is minimized via inference and weight updates, via Eqs. 4 and 4 When the training is complete, we show the different tasks that can be performed, without the need of retraining the model. We use MNIST and FashionMNIST [30], fixing the first d nodes to the data point, and show how to perform the tasks of generation, denoising, reconstruction (without and with labels), and classification by querying the PC graph described in Section
Setup: For every dataset, we have trained models: one for generation and classification tasks, one for denoising and reconstructions, and one for associative memories. The first two models consist of a fully connected graph with 2000 vertices, trained with 794 sensory vertices for classification and generation tasks (784 pixels plus 1-hot vector for the 10 labels), and 784 sensory vertices

---

2
4
5
6
8 1 7 4
0
8
1
4
0
(a)
Generation
Denoising
2
1
4
O 5
9
O
2
7
10
A
0
19 29 6
3
(b)
Reconstruction
(d) Reconstruction Label
(c)
Associative Memories
Figure Generation experiments using the first 6 classes of the MNIST and FashionMNIST datasets from the labels {0, 1, 2, 3, 4, 5, 6) and (t-shirt, trouser, pullover, dress, coat, sandal, shirt), respectively; (b) reconstruction of incomplete images using query by conditioning, when only the top half is available; (c) reconstruction of corrupted images using query by initialization; (d) reconstruction of incomplete images using query by conditioning when also providing the correct label of the test image; and (e) associative memory experiments when presented with half of a training image (left) or a corrupted version (right) that it has already seen and memorized; from top to bottom row: image provided to the network, retrieved image, and original image.
for reconstruction and denoising. Further details about other hyperparameters are given in the supplementary material.
Generation: To check the generation capabilities of a trained PC graph, we queried the model by conditioning on the labels: Here, the value nodes dedicated the 10 labels were fixed to each 1-hot value, and the energy of the model (Eq. ) was minimized using Eq. S until convergence. The generated images are then taken to be the value nodes of the unconstrained sensory nodes, which were originally fixed to the pixels of the images during training. An example of the images generated for each label is given in Fig.
Reconstruction: We provide the PC graph with half of a test image, and ask it to reconstruct the second half. This can be done using both queries: when querying by conditioning, half of the pixels of a test image are fixed to the corresponding sensory nodes; when querying by initialization, the value nodes simply initialized to the same values. At convergence, we consider the value nodes of the unconstrained nodes, which should reconstruct the missing part of the image based on the information learned during training. The results are given in Fig. 3b. We have also replicated the same experiment using a network trained with the labels, and provided the label during the reconstruction. This computes the distribution of the missing pixels knowing the available ones and the label. The results in this case are visibly better and are given in Fig. Bi.
We provide the PC graph with corrupted image, obtained by adding zero-mean Gaussian noise with variance 0.5. This is done by querying by initialization: before running inference, the value nodes of the sensory nodes are initialized to be equal to the pixels of the corrupted image. At convergence, we consider the value nodes of the unconstrained nodes, which should reconstruct the original image. The results are given in Fig.
Denoising:
Results: As stated above, we picked a fully connected PC graph due to its generality, and not to obtain the best performance. However, the results show that this framework is able to learn an internal representation of dataset, and that it can be queried to solve multiple tasks with reasonable accuracy. The PC graph was in fact able to always generate the correct digit, and almost always able to generate the correct clothing item in generation tasks, and always able to provide a noisy but reasonable reconstruction of incomplete test points. The same happened with denoising experiments, as a cleaner (plausible) image was always produced. In Section 4, we show how to improve all these performances by using different PC graph topologies.
Classification: We consider the same PC graph trained for the generation experiments. To check its generalization capabilities, we query by conditioning the pixels of every test image to the first 784 sensory nodes, and run inference to reconstruct the 1-hot label vector. We do not expect to obtain results directly comparable with standard multilayer perceptrons for two reasons: firstly, the
5

---

Table 1: Test accuracy of different models MNIST, FashionMNIST, and SVHN.
Model
Ours
Hopfield Nets
Boltzmann Machine
Almeida Pineda
MNIST
91.76 = 0.02 %
65.23 + 2.21 %
79.23 + 0.15 %
76,36 + 0.14 %
FaShionMNIST
83.72 = 0.33 %
51.74 - 3.94 %
61.31 0.17 %
69.63 + 1.64
SVHN
84.51 = 0.11 %
48.92 + 3.11 %
55.74 + 1.23 %
59.14 + 2.64 %
M
M
0
0
0
j
j
(n)
Feedforward Network
(b)
Generative Network
M
1001
M
1010
0100
00000 0
(c) Recurrent Generative Network
Assemblies of Neurons
Figure 4: Examples of PC graphs that can be built by masking part of the weights of fully connected PC graph. (a) Masking required build a standard multilayer architecture, such as the one in [17. (b) Masking required to build multilayer architecture, where the weights go in the opposite direction. Here, the sensory nodes are at the end of the hierarchical structure. This model is equivalent to the generative networks in 20 (c) Examples of masking needed to implement popular architectures with lateral connections, similar to the model in 35. (d) This the model in [28] which consists of set of Erdös-Reny. Renyi graphs that simulate brain regions (dark squares on the diagonal) and connections between them (dark squares off the diagonal).
model does not contain any implicit hierarchy, which empirically appears crucial to obtaining good classification results. Secondly, the PC graph is also simultancously learning to generate the pixels, which are much more numerous than labels. However, to check whether the obtained results were acceptable, we tested against different learning algorithms that train on similar or equivalent fully connected architectures, such as Hopfield networks, unconstrained Boltzmann machines, and local variation of BP introduced in the late '80, called Almeida-Pineda named after the two scientists As for Hopfield networks, we used the implementation provided in [33] The results, given in Table I| show that our model outperforms every other learning algorithm that can be trained on fully connected architectures. Despite this, the results also show that the obtained test accuracy is not nearly comparable to the results obtained by multilayer perceptrons, as they are only slightly better than a linear classifier (obtaining 88% accuracy on MNIST). However, this is not due the learning rule of PC, which is well-known to be able to reach competitive performance when provided with a hierarchical multilayer structure 177 For the SVHN 3E experiment, used models with 5000 vertices.
who independently invented it [Il
Associative memory: We now test whether PC graphs are able to memorize training images and retrieve them given corrupted or incomplete version of it. Particularly, we show that fully connected PC graph is able to store complex data points, such as colored images, and retrieve them via running inference. To do that, we trained novel fully connected PC graph on 100 data points of the MNIST, FashionMNIST, CIFAR10, and SVHN datasets. We have used model with 1000 vertices for MNIST and FashionMNIST,, and 3500 for SVHN and CIFAR10, and asked it to retrieve

---

the original memories by presenting it either only half of the original pixels, or a corrupted version with Gaussian noise variance 0.2. This task is similar image reconstruction and denoising, with the non-trivial difference that here we only use already seen data points, and hence no generalization is involved. The results of these experiments are given in Fig. F.. and show that our method is able to successfully store and retrieve data points via energy minimization. More details about the capacity of fully connected PC graphs are given in the supplementary material.
4 Extension to Different PC Graph Topologies
As well-known in deep learning, the performance of the trained model strongly depends its architecture: the number of vertices, layers, and their intrinsic structure. In Section 3, we studied the general architecture of fully connected PC graphs. Here, we show how to reduce fully connected PC graph to lighter and even more powerful PC graphs. Particularly, we show how to generate different neural architectures by simply pruning specific edges of fully connected PC graph G = (V. E). In this case, the pruning is performed by applying a sparse mask M. However, there multiple equivalent ways of implementing it. Consider now the weight matrix O R"x", where every entry Oz,j represents the weight parameter connecting vertex i to vertex j. To generate a neural architecture that consists of subset of the original connections, it suffices to mask the matrix 0 via entry-wise multiplication with binary matrix M, where M, = 1 if the edge (f. exists in E, and Ma = 0 otherwise. This allows the creation of hierarchical discriminative architectures such as a PC equivalent of the multilayer perceptron (MLP) in Fig. 2. or hierarchical generative networks in Fig. A c. More generally, it creates a framework to generate and study architectures with any topology, such as small-world networks inspired by brain regions [3 shown in Fig. ft.. Guidance on which topology should be used depends on the tasks and dataset, and it is hence hard to propose a general theory (as it is with BP). In what follows, however, we provide multiple examples.
Here, we study how the network topology influences the final performance, performing the same experiments shown on the fully connected PC graph. We expect the generated images to be visibly better due to the enforced hierarchical structure of the PC graph.
Experiments:
Setup: We trained generative PC graphs, recurrent generative PC graphs, assemblies of neurons PC graphs, and standard BP autoencoders with different numbers of hidden layers and hidden dimension, and report the best results. For the generation results, we used the same setup, but added an input layer with 10 vertices, whose value nodes during training were initialized with the 1-hot label vector. We performed search across learning rates y and and on the number of iterations per batch T. More details are given in the supplementary material, as well as a long discussion on how different parameters influence the final performance of the architecture
The results are given in Fig. 5h and b. As expected, the hierarchical structure of the considered PC graphs improves over the fully connected PC graph, despite being comparable in the number of parameters. Compared against autoencoders (Fig. 5P)). the standard ANN baseline trained with BP, the PC graph results are similar in image denoising, and better in image reconstruction. FID scores on denoising tasks for different levels of noise are given in Table 7
Results:
Conditioning on Labels
Assume that we need to reconstruct test image from an incomplete version of it, with the further assumption that that this time we are also provided with the label of the corrupted image. would be useful to be able use this extra information to obtain better reconstruction. In PC graphs, this is straightforward: it suffices to simultancously fix the value nodes representing the labels to the 1hht vector of the provided label, and the sensory nodes to the pixels of the corrupted image. This method can be applied when it is difficult to infer to which class an incomplete image belongs, and providing the label during the reconstruction allows the preferred label to influence the reconstruction. Hence, we perform the following task: we provide images of digits that look similar when incomplete, and ask the model to reconstruct the missing half when giving the label information, i.e., use the additional label information to correctly resolve the inherent ambiguity in the reconstruction task.
Experiments: We used the same PC graphs from above for generation tasks. We provided the PC graph the bottom 33% of random images representing 7s or 9s. Note that it is hard to distinguish between these two numbers when only this small portion of the image is available. Then, we generated
7

---

7
1
1
/
Generativa Natwork
Recument Generative Natwork
Neurons
Autoencoder
Figure 5: Query by initialization (top) and query by conditioning (bottom) on three different PC graph architectures and different datasets. Particularly, tested these PC graphs against ANN autoencoders trained with BP (d), which perform comparably to the PC graphs on denoising tasks, but less well image reconstruction.
0
7 9
t 1
1
1
1
9 9
9
7
9
9
9
3 3 3
3
3
3
3
O
/ 2
3
4
5 6
7 7
7
7
7 7
7
5 5
5
5
5
5
s
(a)
Generation
(b)
Conditioned Reconstruction
Figure 6: Left: Generated images given the labels using feedforward (top) and recurrent (bottom) graphs. Right: conditional inference the labels.
the missing 67% the pixels by first giving 7 as a label, and then giving 9. We have repeated the same task using 3s and 5s. The results, available in Fig 6b. show that our model is able to perform conditional inference, as the reconstructed digits always agree with the provided labels.
Assembly of Neurons
Figure 7: FID Score on MNIST on images corrupted with Gaussian noise of different variance.
Recently, model made by assemblies of neurons that are sparsely connected with each other has been proposed to em- ulate brain regions [28] This model consists of ordered
Method
PC
Autoencoder
clusters of neurons (Ci...., Cm), and any two ordered neu- rons of the same cluster connected by synapse with probability creating an Erdös-Renyi graph Gm.p- Depend- 0.7
0.2
25.61
43.93
0.5
44.53
53.79
51.38
57.56
ing the desired task, two clusters can be connected via sparse connections following the same rule: cluster C. is connected to cluster Co, then, given connection connecting v to U with probability Note that this structure is highly general, and allows to build networks such the represented in Fig. D.. To conclude, at each time step, only the l neurons of every cluster with the highest neural activity fire. In the original work, the authors propose Hobbian-like learning algorithm, however, graphs. A graphical representation on how to encode as a PC graph network made by assemblies of neurons is given in Fig. 4d. In this case, each dark block on the diagonal represents connections between neurons of the same region. Unlike the other networks in the same figure, these are sparse matrices where every entry is either zero, or one with probability p. As in the brain, not every region is connected with the other, and whether two regions are directly connected has to be decided priori when designing the architecture. Again, two neurons between connected regions are directly connected with probability p. In Fig. fi. dark blocks off the diagonal represent the presence of directed connections between two regions C2 and C,. situated below the diagonal, the connections go from C. Co, with < b; if situated above the diagonal, they go from C2 to C..
U; E C, and vertex vy Co. there exists a synaptic
show that can also be trained using PC

---

Experiments: We replicated this structure, using 4 clusters with 3000 vertices each, connected in feedforward way: the first cluster is connected with the second, which is connected with the third, and so on. As sparsity and top-k constants, we used generative experiments. The results are given in Fig. D. While the results look cleaner than the other methods, note that they are specific to MNIST and FashionMNIST, as the top-k activation on the last cluster well cleans the noise surrounding the reconstructions.
0.1 and k = 0.2, and performed the same
7
Related Work
Our work shares similarities and the final goal with a whole field of research that aims to improve current neural networks by using techniques from computational neuroscience. In fact, the biological implausibility and limitations of BP highlighted in
have fueled research finding new learning algorithm to train ANNs. with the most promising candidates being energy- baaegy models such as equilibrium propagation 39 40 machines | and Hopfield networks 44 45. These differ from PC, as they do not encode the concept of error, but learn in a pure Hebbian fashion. Furthermore, they have undirected synaptic connections, and make predictions by minimizing physical system initialized with specific input. This is different from PC, that has directed synaptic connections and is tested by fixing specific nodes to an input, while letting the latent ones converge. The PC literature ranges from psychology to neuroscience and machine learning. Particularly, it offers a single mechanism that accounts for diverse perceptual phenomena observed in the brain, examples of which are end- stopping 77. repetition-suppresiii even attentional modulation of neural activity [51 52. and has even been used to describe the
37 38
Other interesting energy- based methods are Boltzmann
[4.. illusory motions 47 48,, bistable perception 40 500 and
retrieval and storage of memories in the human memory system [2
Although inspired by neuroscience models of the cortex, the computational model introduced by Rao and Ballard 77 still presents some implausibilities, with the main one being the presence of symmetric connections. An implementation of PC with symmetric connections that is able to successfully learn image classification tasks has been presented in [53], and in the neural generative coding models, used for continual learning, generative models, and reinforcement learning [54]
8 Discussion
In this work, we have shown that PC is able to perform machine learning tasks on graphs of any topology, called PC graphs. Particularly, we have highlighted two main differences between our framework and standard deep learning: flexibility in structure and query. On the one hand, flexible structure allows for learning on any graph topology, hence including both classical deep learning models, and small-world networks that resemble sparse brain regions. On the other hand, flexible querying allows the model to be trained and tested on data points that carry different kinds of information: supervised signals, unsupervised, and incomplete. On much broader level, this work strengthens the connection between the machine learning and the neuroscience communities, as it underlines the importance of PC in both areas, both as highly plausible algorithm to train brain- inspired architectures, and as an approach to solve corresponding problems in machine intelligence.
The research of this paper (and current PC literature in general) is also of great importance from another perspective: training modern neural networks with BP has become computationally extremely expensive, making modern technologies inaccessible. Biological neural networks, on the other hand, do not have these drawbacks thanks to their biological hardware. Recent breakthroughs in the development of neuromorphic and analog computing, such as the finding of the missing memristor (50). could allow the training of deep neural models using only a tiny fraction of energy and time that modern GPUs need. To do this, however, we need to train neural networks end-to-end on the same chip, something that is not possible using BP (or BP through time), due to the need of a control signal that passes information between different layers. The energy formulation of neuroscience- inspired models allows to overcome this limitation, making them perfect candidates to train deep neural models and-t--end on the same chip 57 This strongly motivates research in PC and other neuroscience-inspiee algorithm, with potentially huge long-term impact.

---

Acknowledgments
This work was supported by the Alan Turing Institute under the EPSRC grant EP/N5101299 by the AXA Research Fund, the EPSRC grant EP/RO136677 3667/1, the MRC grant MC_ MC_UU_ LUU0000001, the BBSRC grant BB/S006338/1, and by the EU TAILOR grant. We also acknowledge the use of the EPSRC-funded Tier 2 facility JADE (EP/P020275/1) and GPU computing support by Scan Computers International Ltd. Yuhang Song was supported by the China Scholarship Council under the State Scholarship Fund and by a J.P. Morgan Al Research Fellowship.
References
[1] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back- propagating errors," Nature, vol. 323, no. 6088, pp. 533-536, 1986.
[2] S. Linnainmaa, "The representation of the cumulative rounding error of an algorithm as Taylor expansion of the local rounding errors," Master's Thesis (in Finnish), Univ. Helsinki, pp. 6-7, 1970.
[3] D. Hebb, The Organization of Behavior. Wiley, New York, 1949.
[4] M. V. Srinivasan, S. B. Laughlin, and A. Dubs, "Predictive coding; A fresh view of inhibition in the retina," Proceedings of the Royal Society of London. Series B. Biological Sciences, vol. 216, no. 1205, pp. 427-459, 1982.
[5] D. Mumford, "On the computational architecture of the neocortex," Biological Cybernetics, vol. 66, no. 3, pp. 241-251, 1992.
[6] K. Friston, "Learning and inference in the brain," Neural Networks, vol. 16, pp. 1325- 1352, 2003.
[7] R. P. Rao and D. H. Ballard, "Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects," Nature Neuroscience, vol. 2, no. 1, pp. 79-87, 1999.
[8] K. S. Walsh, P. MeGovern, Clark, and R. G. O'Connell, "Evaluating the neurophysio- logical evidence for predictive processing as a model of perception," Annals of the New York Academy of Sciences, vol. 1464, no. p. 242, 2020.
[9] A. J. Kell, D. L. Yamins, E. N. Shook, S. V. Norman-Haignere, and J. H. McDermott, "A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals cortical processing hierarchy," Neuron, vol. 98, 2018.
[10] B. Millidge, A. Seth, and C. L. Buckley, "Predictive coding: A theoretical and experimental review," arXiv:2107. 2021.
[11] A. M. Bastos, W. M. Usrey, R. A. Adams, G. R. Mangun, P. Fries, and K. J. Friston, "Canonical microcireuits for predictive coding," Neuron, vol. 76, no. 4, pp. 695-711, 2012.
[12] K. J. Friston, T. Parr, and B. de Vries, "The graphical brain: Belief propagation and active inference," Network Neuroscience, vol. 1, no. 4, pp. 381-414, 2017.
[13] K. Friston, "A theory of cortical responses." Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 360, no. 1456, 2005.
[14] M. W. Spratling, "A review of predictive coding algorithms," Brain and Cognition, vol. 112, pp. 92-97, 2017.
[15] Y. Huang and R. P. Rao, "Predictive codiing Wiley Intendicciplinarry Reviews: Cognitive Science, vol. 2, no. 5, pp. 580-593, 2011.
[16] K. J. Friston, N. Trujillo-Baree Barreto, and Daunizeau, "DEM: A variational treatment of dynamic systems," Neuroimage, vol. 41, no. 3, pp. 849-885, 2008.
10

---

[17] J. C. Whittington and R. Bogacz, "An approximation of the error backpropagation algorithm in predictive coding network with local Hebbian synaptic plasticity," " Neural Computation. vol. 29, no. 5, 2017.
[18] B. Byiringiro, T. Salvatori, and T. Lukasiewicz, "Robust graph representation learning via predictive coding," " arXiv preprint arXiv:2212.04665, 2022.
[19] T. Salvatori, Y. Song, B. Millidge, Z. Xu, L. Sha, C. Emde, R. Bogacz, and T. Lukasiewicz. "Incremental predictive coding: A parallel and fully automatic learning algorithm," arXiv preprint arXiv:2212.000720 2022.
[20] T. Salvatori, Y. Song, Y. Hong. L. Sha, S. Frieder, Z. Xu, R. Bogacz, and T. Lukasicwicz. "Associative memories via predictive coding," in Advances in Neural Information Processing Systems, vol. 34, 2021.
[21] M. Tang, T. Salvatori, B. Millidge, Y. Song, T. Lukasiewicz, and R. Bogacz, "Recurrent predictive coding models for associative memory employing covariance learning," " bioRxiv, 2022.
[22] H. C. Barron, R. Auksztulewicz, and K. Friston, "Prediction and memory: A predictive coding account," Progress in Neurobiology, vol. 192, p. 101821, 2020.
[23] B. Millidge, A. Tschantz, and C. L. Buckley, "Predictive coding approximates backprop along arbitrary computation graphs," arXiv:2006.04182, 2020.
[24] Y. Song, T. Lukasiewicz, Z. Xu, and R. Bogacz, "Can the brain do backpropagation? - Exact implementation of backpropagation in predictive coding networks," in Advances in Neural Information Processing Systems, vol. 33, 2020.
[25] T. Salvatori, Y. Song, T. Lukasiewicz, R. Bogacz, and Z. Xu, "Reverse differentiation via predictive coding," " in Proc. AAAI, 2022.
[26] A. Koengs-Knen Misic, and O. Sporns, "Communication dynamics complex brain networks," Nature Reviews Neuroscience, vol. 19, no. I, pp. 17-33, 2018
[27] S. Hochreiter and J. Schmidhuber, "Long short-term memory," " Neural Computation, vol. 9, no. 8, 1997.
[28] C. H. Papadimitriou, S. S. Vempala, D. Mitropolsky, M. Collins, and W. Maass, "Brain compu- tation by assemblies of neurons," Proceedings of the National Academy of Sciences, 2020.
[29] M. Dabagia, C. H. Papadimitriou, and S. S. Vempala, "Assemblies of neurons can learn to classify sell-separated distributions," arXiv:2110.03171, 2021.
[30] H. Xiao, K. Rasul, and R. Vollgraf, "Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms," arXiv:1708.07747, 2017.
[31] L. B. Almeida, "A learning rule for asynchronous perceptrons with feedback in combinatorial environment," pp. 102-111, 1990.
[32] J. Pineda, "Generalization of back-propagation to recurrent neural networks," " Physical Review Letters, vol. 59, 1987.
[33] M. A. Belyaev and A. Velichko, "Classification of handwritten digits using the Hopfield net- work." in IOP Conference Series: Materials Science and Engineering, vol. 862, IOP Publishing. 2020.
[34] Y. Netzer, Wang, Coates, A. Bissacco, B. Wu, and A. Y. Ng, "Reading digits in natural images with unsupervised feature 2011
[35] A. Ororbia and D. Kifer, "The neural coding framework for learning generative models," arXivv2012.03405, 2020.
[36] Q. K. Telesford, K. E. Joyce, Hayasaka, J. H. Burdette, and P. Laurienti, "The ubiquity of small-world networks." Brain Connectivity, vol. I, no. 5, pp. 367-375, 2011.
11

---

[37] T. Lillicrap, A. Santoro, L. Marris, C. Akerman, and G. Hinton, "Backpropagation and the brain," Nature Reviews Neuroscience, vol. 21, 04 2020.
[38] J. C. Whittington and R. Bogacz, "Theories of error back propaaaroottnonn in the brain," Trends in Cognitive Sciences, 2019.
[39] B. Scellier and Y. Bengio, "Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in Computational Neuroscience, vol. 11, p. 24, 2017.
[40] B. Scellier, A. Goyal, J. Binas, T. Mesnard, and Y. Bengio, "Generalization of equilibrium propagation vector field dynamies," arXiv: 04873, 2018.
[41] R. Salakhutdinov, A. Mnih, and G. Hinton, "Restricted Boltzmann machines for collaborative filtering." in Proceedings of the 24th International Conference on Machine Learning, 2007.
[42] R. Salakhutdinov and G. Hinton, "Deep Boltzmann machines," in Artificial Intelligence and Statistics, pp. 448 455, PMLR, 2009.
[43] G. E. Hinton, "Deep belief networks," Scholarpedia, vol. 4, no. 5, p. 5947, 2009.
[44] J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy of Sciences, vol. 79, 1982.
[45] J. J. Hopfield, "Neurons with graded response have collective computational properties like those two-state neurons," Proceedings of the National Academy of Sciences, vol. 81, 1984,
[46] R. Auksztulewicz and K. Friston, "Repetition suppression and its contextual determinants in predictive coding," Cortex, vol. 80, 2016.
[47] W. Lotter, G. Kreiman, and D. Cox, "Deep predictive coding networks for video prediction and unsupervised learning," /605500005.08/ 2016
[48] E. Watanabe, A. Kitaoka, K. Sakamoto, M. Yasugi, and K. Tanaka, "Illusory motion reproduced by deep neural networks trained for prediction," Frontiers Psychology, vol. 9, p. 345, 2018.
[49] J. Hohwy, A. Roepstorff, and K. Friston, "Predictive coding explains binocular rivalry: An epistemological review." Cognition, vol. 108, no. 2008
[50] V. Weilnhammer, H. Stuke, G. Hesselmann, P. Sterzer, and K. Schmack, "A predictive coding account of bistable perception-a model-based fMRI study," PLoS Computational Biology, vol. 13, no. 5, 2017.
[51] H. Feldman and K. Friston, "Attention, uncertainty, and free-energy," Frontiers in Human Neuroscience, vol. 4, 2010.
[52] R. Kanai, Y. Komura, S. Shipp, and K. Friston, "Cerebral hierarchies: Predictive processing, precision and the pulvinar," Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 370, 2015.
[53] B. Millidge, A. Tschantz, A. Seth, and C. L. Buckley, "Relaxing the constraints on predictive coding models," arXiv:2010.01047, 2020.
[54] A. G. Ororbia and A. Mali, "Biologically motivated algorithms for propagating local target representations," in Proc. AAAI, vol. 33, pp. 4651 4651- 2019.
[55] A. Ororbia and A. Mali, "Active predicting coding: Brain- irain-inspired reinforcement learning for sparse reward robotic control problems," arXiv preprint arXiv:2209.09174, 2022.
[56] D. B. Strukov, G. Snider, D. R. Stewart, and R. S. Williams, "The missing memristor found." Nature, vol. 453, no. 7191, pp. 80-83, 2008.
[57] Kendall, R. Pantone, Manickavasagam, Bengio, and Scellier, "Training end-to-end analog neural networks with equilibrium propagation," arXiv:2006.011 01981, 2020.
12

---

[58] J. Sacramento, R. P. Costa, Y. Bengio, and W. Senn, "Dendritic cortical microcircuits approxi- mate the backpropagation algorithm," in Advances in Neural Information Processing Systems, pp. 8721-8732, 2018.
[59] D. Krotov and J. J. Hopfield, "Dense associative memory for pattern recognition," in Advances in Neural Information Processing Systems, 2016.
[60] J. Kendall, R. Pantone, K. Manickavasagam, Y. Bengio, and B. Scellier, "Training end-to-end analog neural networks with equilibrium propagation." arXiv preprint #rXiv:2006.0/98),, 2020.
[61] L. Wright, T. Onodera, M. M. Stein, I. Wang. D. T. Schachter, Z. Hu, and P. L. McMahon, "Deep physical neural networks trained with backpropagation." Nature, vol. 601, no. 7894, pp. 549-555, 2022.
13


Reviews:
[{"review_id": "zJ8LHibjjY1", "reviewer": "Reviewer_5yBi", "summary": "This paper proposes a new framework, PC graphs, which can train on arbitrary network structures without the limitation of backpropagation.\nIt is inspired by neuroscience and implemented by predictive coding. The authors show how to use this framework to solve different tasks, e.g., classification, generation, reconstruction and denoising. ", "questions": "1. From the accuracy of MNIST, the performance of PC graphs is still a issue, which needs to be improved. The authors should show the audience its potential ability to surpass artificial neural networks.\n2. The training time of PC graphs, or convergence situation of different graphs, is not mentioned in the paper.\n3. Could author use some quantitative evaluation metrics to show the performance of PC graphs in generation, reconstruction and denoising tasks?", "limitations": "not available ", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. It is very interesting to leverage biological insights to design a new training framework.\n2. The authors analyze how this framework is applied to solve different tasks, and the relationships between classical neural networks, like the feedforward network.\n3. The paper is well-written and easy to follow.\n\n\nWeaknesses:\n1. The theoretical analysis of PC graphs is missing, such as time complexity, speed of convergence, and so on.\n2. The experiments are limited in some datasets, or qualitative. The accuracy of MNIST is ~92%, which is much lower than simple CNNs.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ei2Z_7yLK6H", "reviewer": "Reviewer_RWvW", "summary": "The paper describes an approach to training arbitrary neural topologies based on predictive coding (PC). The nodes of the graph corresponding to the inputs are clamped to the observed values, and the remaining nodes are trained locally to minimise the discrepancy between their value and their prediction of the value itself (which is a weighted function of the values of its neighbors). The authors describe the general principle, and then perform a few experiments on MNIST and FashionMNIST showcasing how the model can be used for different applications depending on which nodes are initialised or updated.", "questions": "Note: see further comments to the authors for the updated evaluation.\n\nUpon reading several of the papers mentioned in the bibliography, I am unsure about the novelty which is claimed in the paper. In particular, the authors state that they propose an extension to “PC graphs” allowing PC to be run on arbitrary graph topologies, but this was already done in the majority of cited papers, including:\n\nhttps://proceedings.neurips.cc/paper/2021/hash/1fb36c4ccf88f7e67ead155496f02338-Abstract.html\n\nThe above paper also provides examples about using the network as an associate memory, which encompasses half of the experiments done in this paper. The algorithm which is used is also equivalent (gradient descent for T steps on values and weights after fixing a single input example). Since this is my major concern, I am currently rating this as a borderline paper, as it appears to be a collection of previous known results.\n\nA few additional questions I have:\n\n•\tThe authors should be more comprehensive in describing the comparisons of their method with Boltzmann machines and Hopfield networks, since they share a number of common characteristics.\n•\tThe convergence guarantees of the algorithms wrt the loss (sum of local errors squared) is not discussed.\n•\tIn general, how efficient is the training compared to a standard backpropagation?", "limitations": "NA", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Note: see further comments to the authors for the updated evaluation.\n\nLet me premise that I am not familiar with the literature on PC and its applications to deep learning, and I have learned about most papers by following the references listed in the paper itself. The paper is well written, and the authors makes a good case for the strengths of the approach (locality) and in particular its neuroscience background. The experiments are interesting, and all the diagrams in the paper are very clear (e.g., how different masks over the connectivity can results in different known formulations).+\n\nExperiments are done on very simple CV datasets with no comparisons. However, I think this is expected of papers that are trying to modify the way neural networks are trained at a fundamental level. Still, the authors are not even mentioning (or comparing) to the literature they are citing on PC, which is strange.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "aTf8HIwTibq", "reviewer": "Reviewer_v3jK", "summary": "This paper presents the use of predictive coding (PC) to perform various learning and inference tasks. In particular, the paper proposes PC graphs that generalize PC to arbitrary network structures. The flexibility of PC graphs is then demonstrated experimentally using a variety of network structures and learning tasks.", "questions": "- Corruption noise variance values of 0.2 and 0.5 are quite small. How would a larger choice affect the analysis? Would the method still work in that case?\n- P. 6, line 211: Missing SVHN dataset", "limitations": "The authors do not discuss the limitations and societal impact of their work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Originality: While the idea of PC graphs in the paper is not new, this paper however focuses on showing how a single PC graph model can be extensively used in various ways to learn different tasks.\n\n- Quality: The PC graph model is technically valid though calling it a biologically plausible model seems slightly misleading since the existence of error nodes is biologically questionable. The claim for the flexibility of the model is well supported by the experiments on various learning settings.\n\n- Clarity: The paper is clearly written and well-organized.\n\n- Significance: While the paper does not advance state-of-the-art results, the paper clearly presents the flexibility of PC graphs in various learning tasks through experiments. However, the overall practicality/utility of the model is still hard for readers to assess since 1) the potential computational overhead of the method is only briefly illustrated in the supplementary material, which is not convincing and 2) experimental results are mostly qualitative with some quantitative comparison against a few simple baselines. Additional knowledge of its computational efficiency and demonstrated effectiveness would instill more trust into the method.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept