Learning on Arbitrary Graph Topologies via Predictive Coding
Abstract
Training with backpropagation (BP) in standard deep learning consists of two mainsteps: a forward pass that maps a data point to its prediction, and a backward passthat propagates the error of this prediction back through the network.
This process ishighly effective when the goal is to minimize a specific objective function.
However,it does not allow training on networks with cyclic or backward connections.
This isan obstacle to reaching brain-like capabilities, as the highly complex heterarchicalstructure of the neural connections in the neocortex are potentially fundamental forits effectiveness.
In this paper, we show how predictive coding (PC), a theory ofinformation processing in the cortex, can be used to perform inference and learningon arbitrary graph topologies.
We experimentally show how this formulation,called PC graphs , can be used to flexibly perform different tasks with the samenetwork by simply stimulating specific neurons.
This enables the model to bequeried on stimuli with different structures, such as partial images, images withlabels, or images without labels.
We conclude by investigating how the topology ofthe graph influences the final performance, and comparing against simple baselinestrained with BP.
Introduction
Classical deep learning has achieved remarkable results by training deep neural networks to minimizean objective function.
Here, every weight parameter gets updated to minimize this function usingreverse differentiation [1, 2].
However, in the brain, every synaptic connection is independentlyupdated to correct the behaviour of its post-synaptic neuron [3] using local information and itis unknown whether this process minimizes a global objective function.
The brain maintains aninternal model of the world, which constantly generates predictions of external stimuli.
When thepredictions differ from reality, the brain immediately corrects this error (difference between realityand prediction) by updating the strengths of the synaptic connections [4, 5, 6, 7].
This theory ofinformation processing, called predictive coding (PC) , is highly influential, despite experimentalevidence in the cortex being mixed [8, 9, 10, 11] and it is at the centre of a large amount of researchin computational neuroscience [12, 13, 14, 15, 16].
From the machine learning perspective, PC haspromising properties: it is able to achieve excellent results in classification [17] and memorization[18], and is able to process information in both a bottom up and a top down direction.
This lastproperty is fundamental for the functioning of different brain areas, such as the hippocampus [19, 18].
PC also shares the generalization capabilities of standard deep learning, as it is able to approximate
BP on any neural structure [20], and a variation of PC is able to exactly replicate the weight update of BP on any computational graph [21, 22].
Moreover, PC only uses local information to updatesynapses, allowing the network to be fully parallelized, and to train on networks with any topology.
[Figure]
Training on networks of any structure is notpossible in standard deep learning, where information only flows in one direction via the feedforward pass and then backpropagation (BP) isperformed in sequential steps backwards.
If acycle is present inside the computational graphof an artificial neural network (ANN), BP becomes stuck in an infinite loop.
More generally, the computational graph of any function
F : R d → R k is a poset, and hence acyclic.
While the problem of training on some specificcyclic structures has been partially addressedusing BP through time [24] on sequential data,the restriction to hierarchical architectures may present a limitation to reaching brain-like intelligence,since the human brain has an extremely complex and entangled neural structure that is heterarchicallyorganized with small-world connections [23]—a topology that is likely highly optimized by evolution.
This shape of structural brain networks, shown in Fig.
1, generates a unique communication dynamicsthat is fundamental for information processing in the brain, as different aspects of network topologyimply different communication mechanisms, and hence perform different tasks [23].
The heterarchical topology of brain networks has motivated research that aims to develop learning methods ongraphs of any topology.
A popular example is the assembly calculus [25, 26], a Hebbian learningmethod that can perform different operations implicated in cognitive phenomena.
In this work, we address this problem by proposing PC graphs , a structure that allows to train onany directed graph using the original (error-driven) framework by Rao and Ballard [7].
We thendemonstrate the flexibility of such networks by testing the same network on different tasks, whichcan be interpreted as conditional expectations on different neurons of the network.
Our PC graphsframework enables the model to be queried on stimuli with different structures, such as partialimages, images with labels, or images without labels.
This is significantly more flexible than thestrict input-output structure of standard ANNs, which are limited to scenarios when they are alwayspresented with data and labels in the same format.
Note that the main goal of this work is not to propose a specific architecture that achieves state-of-theart (SOTA) results on a particular task, but to present PC graphs as a new flexible and biologicallyplausible model, which can achieve good results on many tasks simultaneously.
In this work, westudy the simultaneous generation, classification, and associative memory capabilities of PC graphs,highlighting their flexibility and theoretical advantages over standard baselines.
Our contributions arebriefly as follows
• We present PC graphs, which generalize PC to arbitrary graph topologies, and show how a singlemodel can be queried in multiple ways to solve different tasks by simply altering the values ofspecific nodes, without the need for retraining when switching between tasks.
Particularly, wedefine two different techniques, which we call query by conditioning and query by initialization .•
We then experimentally show this in the most general case,
i
e., for fully connected PC graphs.
Here, we train different models on MNIST and FashionMNIST, and show how the two queries can
be used to perform different generation tasks.
Then, we test the model on classification tasks, andexplore its capabilities as an associative memory model.•
We next investigate how different graph topologies influence the performance of PC graphs ongeneration tasks, reproducing common network architectures such as feedforward, recurrent, andresidual networks as special cases of PC graphs, and investigate how the chosen structure influencesthe performance on generative tasks.
Finally, we also show how PC graphs can be used to derivethe popular assembly calculus [25].
[Figure]
Excitatory
[Figure]
PC Graphs
Let G =
( V, E ) be a directed graph, where V is a set of n vertices { 1 , 2 , . .
. , n } , and E
⊆ V × V isa set of directed edges between them, where every edge ( i, j ) ∈
E has a weight parameter θ i,j .
Theset of vertices V is partitioned into two subsets, the sensory and internal vertices .
External stimuliare always presented to the network via sensory vertices, which we consider to be the first d verticesof the graph, with d < n .
The internal vertices, on the other hand, are used to represent the internalstructure of the dataset.
Each vertex i encodes several quantities.
The main quantity is the values ofits activity, which change over time, and we refer to it as a value node x i,t .
We call the value nodesof the sensory vertices sensory nodes .
Additionally, each vertex computes the prediction µ i,t of itsactivity based on its input from value nodes of other vertices:
[Equation]
(1) where the summation is over all the vertices j connected to i via outgoing edges, and f is a nonlinearity.
Equivalently, it is possible to consider the summation on every j , and have θ i,j =
if( i, j ) ̸∈ E .
The error of every vertex at every time step t is then given by the difference between itsvalue node and its prediction, i.e., ε i,t = x i,t − µ i,t .
This local definition of error, which lies notonly in the output, but in every vertex of the network, is what allows PC graphs to learn using onlylocal information.
The value nodes x i,t and the weight parameters θ i,j are updated to minimize thefollowing energy function defined locally on every vertex:
[Equation]
A fully connected PC graph with 3 vertices is sketched in Fig.
2a, along with the operations thatdescribe the dynamics of the information flow, showing also how every operation can be representedvia inhibitory and excitatory connections.
Learning: When presented with a training point ¯ s taken from a training set, the value nodes ofthe sensory vertices are fixed to be equal to the entries of ¯ s for the whole duration of the trainingprocess, i.e., for every t .
A sketch of this is shown in Fig. 2b.
Then, the total energy of Eq.
(2) isminimized in two phases: inference and weight update .
During the inference phase, the weights arefixed, and the value nodes are continuously updated via gradient descent for T iterations, where T isa hyperparameter of the model.
The update rule is the following ( inference ):
[Equation]
where γ is the learning rate of the value nodes.
This process of iteratively updating the value nodesdistributes the output error throughout the PC graph.
When the inference phase is completed, thevalue nodes get fixed, and a single weight update is performed as follows ( weight update ):
[Equation]
where α is the learning rate of the weight update.
We now describe two different ways to query theinternal representation of a trained model, where the values of some sensory vertices are undefined,and have to be predicted.
In both cases, the weight parameters θ i,j are now fixed, and the total energy
E is continuously minimized using gradient descent on the re-initialized value nodes via Eq.
Query by conditioning: While each value node is randomly re-initialized, the value nodes ofspecific vertices are fixed to some desired value, and hence not allowed to change during the energyminimization process.
The unconstrained sensory vertices will then converge to the minimum of theenergy given the fixed vertices, thus computing the conditional expectation of the latent vertices giventhe observed stimulus.
Formally, let I =
{ i 1 , . . .
, i q } ⊂ { 1 , 2 , . . . , n } be a strict subset of vertices.
Assume now that we know that a subset of the value nodes corresponding to the vertices I is equalto a stimulus ¯ q ∈ R q .
Then, running inference until convergence allows to estimate the conditionalexpectation
[Equation]
where ¯ x T is the vector of value nodes at convergence.
Examples of tasks performed this way are (i)classification, where internal nodes are fixed to the pixels of an image, and the sensory nodes arefixed to a 1 -hot vector with the labels, (ii) generation, where the single value node encoding the classinformation is fixed, and the value nodes of the sensory nodes converge to an image of that class,and (iii) reconstruction, such as image completion, where a fraction of the sensory nodes are fixed tothe available pixels of an image, and the remaining ones converge to a reasonable completion of it.
A sketch of this process is shown in Fig. 2c.
Query by initialization: Again, every value node is randomly initialized, but the value nodes ofspecific nodes are initialized (for t = 0 only), but not fixed (for all t ), to some desired value.
Thisdiffers from the previous query, as here every value node is unconstrained, and hence free to changeduring inference.
The sensory vertices will then converge to the minimum found by gradient descent,when provided with that specific initialization.
Again, let I =
{ i 1 , . . .
, i q } ⊂ { 1 , 2 , . . . , n } be a strictsubset of vertices, and assume that we have an initial stimulus ¯ q ∈ R q .
Then, we can estimate theconditional expectation
[Equation]
Examples of tasks performed this way are (i) denoising, such as image denoising, where the sensoryneurons are initialized with a noisy version of an image, which is cleared during the energy minimization process, and (ii) reconstruction, such as image completion, where the fraction of missing pixelsis now not known a priori.
Experiments on Fully Connected PC Graphs
In this section, we perform experiments on a fully connected PC graph G =
( V, E ) , i.e., where
E = V × V .
Such PC graphs are fully general and encode no implicit priors on the structure of thedataset.
It is possible to obtain any possible graph topology by simply pruning specific weights of G .
Given a dataset D
= { ¯ s i } i<m , with ¯ s i ∈ R d , we train the PC graph as described in Section 2: Thefirst d neurons are fixed to the entries of a training point, and the energy function E t is minimizedvia inference and weight updates, via Eqs.
(3) and (4).
When the training is complete, we show thedifferent tasks that can be performed, without the need of retraining the model.
We use MNIST and
FashionMNIST [27], fixing the first d nodes to the data point, and show how to perform the tasks ofgeneration, denoising, reconstruction (without and with labels), and classification by querying the PCgraph as described in Section 2.
Setup:
For every dataset, we have trained 3 models: one for generation and classification tasks, onefor denoising and reconstructions, and one for associative memories.
The first two models consistof a fully connected graph with 2000 vertices, trained with 794 sensory vertices for classificationand generation tasks ( 784 pixels plus a 1-hot vector for the 10 labels), and 784 sensory verticesfor reconstruction and denoising.
Further details about other hyperparameters are given in thesupplementary material.
[Figure]
Generation:
To check the generation capabilities of a trained PC graph, we queried the model byconditioning on the labels:
Here, the value nodes dedicated to the 10 labels were fixed to each 1-hotvalue, and the energy of the model (Eq.
(2)) was minimized using Eq.
(3) until convergence.
Thegenerated images are then taken to be the value nodes of the unconstrained sensory nodes, whichwere originally fixed to the pixels of the images during training.
An example of the images generatedfor each label is given in Fig. 3a.
Reconstruction: We provide the PC graph with half of a test image, and ask it to reconstruct thesecond half.
This can be done using both queries: when querying by conditioning, half of the pixelsof a test image are fixed to the corresponding sensory nodes; when querying by initialization, thevalue nodes are simply initialized to the same values.
At convergence, we consider the value nodesof the unconstrained nodes, which should reconstruct the missing part of the image based on theinformation learned during training.
The results are given in Fig.
3b.
We have also replicated the sameexperiment using a network trained with the labels, and provided the label during the reconstruction.
This computes the distribution of the missing pixels knowing the available ones and the label.
Theresults in this case are visibly better and are given in Fig.
3d.
Denoising: We provide the PC graph with a corrupted image, obtained by adding zero-mean Gaussiannoise with variance 0 .
This is done by querying by initialization: before running inference, thevalue nodes of the sensory nodes are initialized to be equal to the pixels of the corrupted image.
Atconvergence, we consider the value nodes of the unconstrained nodes, which should reconstruct theoriginal image.
The results are given in Fig.
3c.
Results: As stated above, we picked a fully connected PC graph due to its generality, and not toobtain the best performance.
However, the results show that this framework is able to learn aninternal representation of a dataset, and that it can be queried to solve multiple tasks with a reasonableaccuracy.
The PC graph was in fact able to always generate the correct digit, and almost alwaysable to generate the correct clothing item in generation tasks, and always able to provide a noisy butreasonable reconstruction of incomplete test points.
The same happened with denoising experiments,as a cleaner (plausible) image was always produced.
In Section 4, we show how to improve all theseperformances by using different PC graph topologies.
Classification: We consider the same PC graph trained for the generation experiments.
To checkits generalization capabilities, we query by conditioning the pixels of every test image to the first784 sensory nodes, and run inference to reconstruct the 1 -hot label vector.
We do not expect toobtain results directly comparable with standard multilayer perceptrons for two reasons: firstly, the
[Table]
model does not contain any implicit hierarchy, which empirically appears crucial to obtaining goodclassification results.
Secondly, the PC graph is also simultaneously learning to generate the pixels,which are much more numerous than labels.
However, to check whether the obtained results wereacceptable, we tested against different learning algorithms that train on similar or equivalent fullyconnected architectures, such as Hopfield networks, unconstrained Boltzmann machines, and a localvariation of BP introduced in the late ′ 80 , called Almeida-Pineda, named after the two scientistswho independently invented it [28, 29].
As for Hopfield networks, we used the implementationprovided in [30].
The results, given in Table 1, show that our model outperforms every other learningalgorithm that can be trained on fully connected architectures.
Despite this, the results also show thatthe obtained test accuracy is not nearly comparable to the results obtained by multilayer perceptrons,as they are only slightly better than a linear classifier (obtaining 88% accuracy on MNIST).
However,this is not due to the learning rule of PC, which is well-known to be able to reach a competitiveperformance when provided with a hierarchical multilayer structure [17].
For the SVHN [31]experiment, we used models with 5000 vertices.
Associative memory: We now test whether PC graphs are able to memorize training images andretrieve them given a corrupted or incomplete version of it.
Particularly, we show that a fullyconnected PC graph is able to store complex data points, such as colored images, and retrieve themvia running inference.
To do that, we trained a novel fully connected PC graph on 100 data pointsof the MNIST, FashionMNIST, CIFAR10, and SVHN datasets.
We have used a model with 1000vertices for MNIST and FashionMNIST, and 3500 for SVHN and CIFAR10, and asked it to retrievethe original memories by presenting it either only half of the original pixels, or a corrupted versionwith Gaussian noise variance 0 .
This task is similar to image reconstruction and denoising, with thenon-trivial difference that here we only use already seen data points, and hence no generalization isinvolved.
The results of these experiments are given in Fig.
3e, and show that our method is able tosuccessfully store and retrieve data points via energy minimization.
More details about the capacityof fully connected PC graphs are given in the supplementary material.
Extension to Different PC Graph Topologies
As well-known in deep learning, the performance of the trained model strongly depends on itsarchitecture: the number of vertices, layers, and their intrinsic structure.
In Section 3, we studied thegeneral architecture of fully connected PC graphs.
Here, we show how to reduce a fully connected
PC graph to lighter and even more powerful PC graphs.
Particularly, we show how to generatedifferent neural architectures by simply pruning specific edges of a fully connected PC graph
G =
( V, E ) .
In this case, the pruning is performed by applying a sparse mask M .
However, thereare multiple equivalent ways of implementing it.
Consider now the weight matrix ¯ θ ∈ R n × n , whereevery entry θ i,j represents the weight parameter connecting vertex i to vertex j .
To generate a neuralarchitecture that consists of a subset of the original connections, it suffices to mask the matrix ¯ θ viaentry-wise multiplication with a binary matrix M , where M i,j = 1 if the edge ( i, j ) exists in E , and
M i,j = 0 otherwise.
This allows the creation of hierarchical discriminative architectures such as a
PC equivalent of the multilayer perceptron (MLP) in Fig.
4a, or hierarchical generative networksin Fig.
4b, c. More generally, it creates a framework to generate and study architectures with anytopology, such as small-world networks inspired by brain regions [33], as shown in Fig.
4d.
Guidanceon which topology should be used depends on the tasks and dataset, and it is hence hard to propose ageneral theory (as it is with BP).
In what follows, however, we provide multiple examples.
[Figure]
Experiments: Here we study how the network topology influences the final performance, performingthe same experiments shown on the fully connected PC graph.
We expect the generated images to bevisibly better due to the enforced hierarchical structure of the PC graph.
Setup: We trained generative PC graphs, recurrent generative PC graphs, assemblies of neurons PCgraphs, and standard BP autoencoders with different numbers of hidden layers and hidden dimension,and report the best results.
For the generation results, we used the same setup, but added an inputlayer with 10 vertices, whose value nodes during training were initialized with the 1-hot label vector.
We performed a search across learning rates η and α , and on the number of iterations per batch T .
More details are given in the supplementary material, as well as a long discussion on how differentparameters influence the final performance of the architecture.
Results: The results are given in Fig.
5a and b. As expected, the hierarchical structure of the considered PC graphs improves over the fully connected PC graph, despite being comparable in the numberof parameters.
Compared against autoencoders (Fig. 5c), the standard ANN baseline trained with BP,the PC graph results are similar in image denoising, and better in image reconstruction.
Conditioning on Labels
[Figure]
Assume that we need to reconstruct a test imagefrom an incomplete version of it.
As seen above,this can be done by querying the model by conditioning.
However, assume now that this time weare also provided with the label of the corruptedimage.
It would be useful to be able use this extra information to obtain a better reconstruction.
In PC graphs, this is straightforward: it suffices
[Figure]
to simultaneously fix the value nodes representing the labels to the 1-hot vector of the providedlabel, and the sensory nodes to the pixels of the corrupted image.
We have already shown how thisimproves the reconstruction quality in Fig. 2d.
This method, however, has a second, more significant,application.
Sometimes, it is difficult to infer to which class an incomplete image belongs, andproviding the label during the reconstruction allows the preferred label to influence the reconstruction.
Hence, we perform the following task: we provide images of digits that look similar when incomplete,and ask the model to reconstruct the missing half when giving the label information, i.e., use theadditional label information to correctly resolve the inherent ambiguity in the reconstruction task.
Experiments: We used the same PC graphs from above for generation tasks.
We provided the PCgraph the bottom 33% of random images representing 7 s or 9 s. Note that it is hard to distinguishbetween these two numbers when only this small portion of the image is available.
Then, we generatedthe missing 67% of the pixels by first giving 7 as a label, and then giving 9 .
We have repeated thesame task using 3 s and 5 s.
The results, available in Fig 6b, show that our model is able to performconditional inference, as the reconstructed digits always agree with the provided labels.
Assembly of Neurons
Recently, a model made by assemblies of neurons that are sparsely connected with each other hasbeen proposed to emulate brain regions [25].
This model consists of m ordered clusters of neurons( C 1 , . . .
, C m ) , and any two ordered neurons of the same cluster are connected by a synapse withprobability p , creating an Erdõs–Renyi graph G m,p .
Depending on the desired task, two clusters canbe connected via sparse connections following the same rule: if cluster C a is connected to cluster C b ,then, given a vertex v i ∈ C a and a vertex v j ∈ C b , there exists a synaptic connection connecting v ito v j with probability p .
Note that this structure is highly general, and allows to build networks suchas the one represented in Fig.
1b.
To conclude, at each time step, only the k neurons of every clusterwith the highest neural activity fire.
In the original work, the authors propose a Hebbian-like learningalgorithm, however, we show that it can also be trained using PC graphs.
A graphical representationon how to encode as a PC graph a network made by assemblies of neurons is given in Fig.
4d.
In thiscase, each dark block on the diagonal represents connections between neurons of the same region.
Unlike the other networks in the same figure, these are sparse matrices where every entry is eitherzero, or one with probability p .
As in the brain, not every region is connected with the other, andwhether two regions are directly connected has to be decided a priori when designing the architecture.
Again, two neurons between connected regions are directly connected with probability p .
In Fig.
4d,dark blocks off the diagonal represent the presence of directed connections between two regions C aand C b .
If situated below the diagonal, the connections go from C a to C b , with a < b ; if situatedabove the diagonal, they go from C b to C a .
Experiments: We replicated this structure, using 4 clusters with 3000 vertices each, connected in afeedforward way: the first cluster is connected with the second, which is connected with the third,and so on.
As sparsity and top-k constants, we used p = 0 .
1 and k = 0 .
2 , and performed the samegenerative experiments.
The results are given in Fig.
5c.
While the results look cleaner than the othermethods, note that they are specific to MNIST and FashionMNIST, as the top-k activation on the lastcluster well cleans the noise surrounding the reconstructions.
Related Work
Our work shares similarities and the final goal with a whole field of research that aims to improvecurrent neural networks by using techniques from computational neuroscience.
In fact, the biologicalimplausibility and limitations of BP highlighted in [34, 35] have fueled research in finding a newlearning algorithm to train ANNs, with the most promising candidates being energy-based modelssuch as equilibrium propagation [36, 37]. Other interesting energy-based methods are Boltzmannmachines [38, 39, 40], and Hopfield networks [41, 42]. These differ from PC, as they do notencode the concept of error , but learn in a pure Hebbian fashion.
Furthermore, they have undirectedsynaptic connections, and make predictions by minimizing a physical system initialized with aspecific input. This is different from PC, that has directed synaptic connections and is tested byfixing specific nodes to an input, while letting the latent ones converge.
The PC literature rangesfrom psychology to neuroscience and machine learning.
Particularly, it offers a single mechanismthat accounts for diverse perceptual phenomena observed in the brain, examples of which are endstopping [7], repetition-suppression [43], illusory motions [44, 45], bistable perception [46, 47], andeven attentional modulation of neural activity [48, 49], and it has even been used to describe theretrieval and storage of memories in the human memory system [19].
Although inspired by neuroscience models of the cortex, the computational model introduced by
Rao and Ballard [7] still presents some implausibilities, with the main one being the presence ofsymmetric connections.
An implementation of PC with no symmetric connections that is able tosuccessfully learn image classification tasks has been presented in [50].
Discussion
In this work, we have shown that PC is able to perform machine learning tasks on graphs of anytopology, called PC graphs.
Particularly, we have highlighted two main differences between ourframework and standard deep learning: flexibility in structure and query.
On the one hand, flexiblestructure allows for learning on any graph topology, hence including both classical deep learningmodels, and small-world networks that resemble sparse brain regions.
On the other hand, flexiblequery allows the model to be trained and tested on data points that carry different kind of information:supervised signals, unsupervised, and incomplete.
On a much broader level, this work strengthensthe connection between the machine learning and the neuroscience communities, as it underlinesthe importance of PC in both areas, both as a highly plausible algorithm to train brain-inspiredarchitectures, and as an approach to solve corresponding problems in machine intelligence.
The research of this paper (and current PC literature in general) is also of great importance fromanother perspective: training modern nets with BP has become computationally extremely expensive,making modern technologies inaccessible.
Biological neural networks, on the other hand, do nothave these drawbacks thanks to their biological hardware.
Recent breakthroughs in the developmentof neuromorphic and analog computing, such as the finding of the missing memristor [51], couldallow the training of deep models using only a tiny fraction of energy and time that modern GPUsneed.
To do this, however, we need to train networks end-to-end on the same chip, somethingthat is not possible using BP (or BP through time), due to the need of a control signal that passesinformation between different layers.
The energy formulation of neuroscience-inspired models allowsto overcome this limitation, making them perfect candidates to train deep models end-to-end on thesame chip [52].
This strongly motivates research in PC and other neuroscience-inspired algorithm,with a potentially huge long-term impact.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] The code and the data are proprietary
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Supplementary material
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Require: ( x 1 ,t , . . . , x d,t ) is fixed to ( s 1 , . . . , s d ) . 1: for t = 0 to T do 2: for each vertex i do 3: update x i,t to minimize E t via Eq.
(3) 4: end for 5: if t = T then 6: update every θ i,j to minimize E t via Eq. 7: end if 8: end for ; Artificial Neural Network
Biological Neural Network (b)
[Figure]
A A Discussion on Biological Plausibility
In the literature, there is often a disagreement on when a specific algorithm can be consideredbiologically plausible.
This follows, as every computer simulation fails to be completely equivalentto every aspect on how the brain works, as there will always be some details that make the simulationimplausible.
Hence, it is normally assumed that an algorithm is biologically plausible when it satisfiesa list of properties that are also satisfied in the brain.
Different works consider different properties.
In our case, we consider as list of minimal properties that a learning rule should satisfy, the onesthat allow to have a possible neural implementation, such as local computations and lack of a globalcontrol signal to trigger the operations.
However, the neural implementation proposed in Fig.
2 takeserror nodes into account, often considered implausible from the biological perspective [53].
Even so,the biological plausibility of our model is not affected by this: it is in fact possible to map PCNs ona different neural architecture, in which errors are encoded in apical dendrites rather than separateneurons [53, 35].
Graphical representations of the differences between the two implementations canbe found in Fig.
7, taken (and adapted) from [35].
B Methodology and Further Experiments
Compared to backpropagation (BP), predictive coding (PC) allows for more flexibility in the definition, training, and evaluation of the model.
The experiments reported in this paper show thebest results achieved on each specific task and, as a consequence, only the effects of a specific setof hyperparameters.
Therefore, the complete range of possibilities that exist in PC has not beendisplayed, however, those alternative configurations may be helpful in other scenarios.
A pseudocodethat describes the learning process of PC graphs is given in Algorithm 1.
B.1 Architectures and Hyperparameters
In this section, we provide a detailed description of the models and parameters used to obtain theresults in the various generation tasks presented in this work, to guarantee their reproducibility.
Notethat our goal was to compare the performance of different models, hence we compare networks thathave a similar number of parameters.
We now briefly summarize the PC graphs used in this work:
• Fully connected networks: The experiments in the
paper body
are obtained by using a fully
connected graph with 2000 vertices, trained with 794 sensory vertices for classification and
generation tasks ( 784 pixels plus a 1-hot vector for the 10 labels), and 784 sensory verticesfor reconstruction and denoising.
For colored images, we used a network with 5000 vertices.
We trained every model for 20 epochs, and reported the best results using early stopping.
As learning rates, we used α ∈ { 1 , 0 . 5 } for the value nodes, and η ∈ { 0 . 0001 , 0 . 00005 } forthe weights, and a weight decay λ
= { 0 . 01 , 0 .
To conclude, we computed
each query using T = 2000 , making sure that the energy had converged before reaching
that value.
• Feedforward network: A network composed by a sequence of L fully connected layers of
dimension H .
The best results were achieved with L ∈ { 3 , 4 } and H = 512 for MNIST andH = 1024 on FashionMNIST.
We did not experience any benefits in adding extra layers,as it only resulted in higher convergence times.
The width, instead, directly determines the
quality of the images produced: as expected, very narrow networks fail to store enough
information to accurately reconstruct (or denoise) the input images.
However, wide networks
manifest sub-optimal performance as well.
This follows, as having more parameters allows
the network to easily overfit.
As a consequence, the generation process is less stable, and
the images can appear more noisy and composed by strokes belonging to different classes.
Using a strong weight decay alleviates these problems as we will later discuss
• Recurrent network: A recurrent layer consists of a layer whose output is transformed by
a non-linear transformation and fed in input to the layer.
The recurrent networks used in
this paper consist of two recurrent layers (for a total of four non-linear transformations)
with hidden dimension H = 512 when trained on MNIST, and H = 1024 when trained
on FashionMNIST.
The behaviour, given the choice of width and depth, seems similar to
feedforward networks.
The performance, however, seems to be less impacted by the usage
of wide layers.
This is due to the recurrent connections that establish more constraints, and
thus stability.
• Assembly of neurons: As stated in the paper body, we used models with 4 clusters with3000 vertices each, connected in a feedforward way.
As sparsity and top-k constants, we
used p = 0 .
1 and k = 0 .
2 , and performed the same generative experiments.
Again, we
trained each model for 20 epochs, and reported the best results using early stopping.
As
learning rates, we used α ∈ { 1 , 0 . 5 } for the value nodes, and η ∈ { 0 . 0001 , 0 . 00005 } for
the weights.
To conclude, we computed each query using T = 2000 , making sure that
the
energy had converged before reaching that value.
• Autoencoders: The autoencoder was defined using the same shape as the feedforward
networks: it is as a fully connected network with L ∈ { 3 , 4 } hidden
layers of width
H ∈ { 256 , 512 , 1024 } .
In this way, the structure and the number of parameters directly correspond to the feedforward network trained using predictive coding.
It was trainedthrough BP using the Adam optimizer, with learning rate α = 1 e − 4 and weight decay ofparameter λ ∈ { 1 e − 2 , 1 e − 4 , 1 e − 6 , 0 } (the best results were achieved with the lowest value).
As predictive coding requires two sets of updatable parameters, the value nodes x i,t and the weightsθ i,j , we defined two separate optimizers.
The learning rate for the weights was set to α = 1 e − 4 , andthe optimizer algorithm chosen was Adam (as for the autoencoder).
We experimented with differentvalues of weight decays, noticing how the final performance is highly affected by this value.
For thegiven tasks, the best results were achieved with weight decay = 1 e − 2 .
Instead, the learning rate forthe value nodes was set to γ = 1 . 0 , and optimized using SGD .
To conclude, we have tested differentactivation functions; the most promising seems to be HardTanh .
B.2 Feedforward vs. Recursive Networks
In this work, we highlighted how in different situations, one may prefer to query by conditioning orby initialization.
As a rule of thumb, conditioning means that we expect the partial data given to thenetwork to be correct and be recognized as a memory , by being reconstructed by the network withoutmodifications.
Therefore, it makes sense to use it in the reconstruction generative task.
Instead, whenperforming image denoising, we do not want the network to recall the noisy image from its memory,instead, we are asking it to retrieve the memory (or to generate a realistic sample), representing aplausible image, that is the closest to the noisy input.
It makes therefore sense to only initialize theoutput layer, giving the network a direction to follow and let it evolve unconstrained.
However, itmay not always be clear which querying technique is most preferable.
A desirable behavior may beusing the network to identify which querying data is realistic (i.e., similar to the training samples)and which not.
Ideally, we would like the network to perfectly fit previously seen data points, whilestruggling to reconstruct unfamiliar shapes.
We tested both the feedforward and recursive networksby training them on the MNIST dataset and querying them by conditioning the output layer with afull size image composed by half uniform noise and half digit.
The results are reported in Fig. 8.
Wecan see how feedforward networks easily fit the noise, reconstructing the two halves independently.
On the other hand, employing recurrent connections (and thus imposing stricter constraints) forcesthe network to reconstruct the image as a whole.
We can see a similar behavior in Fig.
9, wherenetworks trained on MNIST are use to denoise FashionMNIST images.
Feedforward networks easilyoverfit the input samples.
Recurrent networks, instead, correctly do not recognize the given imagesand reconstruct an unrelated and confused blob.
In this last case, it would therefore be possible todistinguish between familiar and unfamiliar images by computing the distance between the inputand output images.
B.3 Importance of Weight Decay
As previously mentioned, weight decay plays a fundamental role in determining the properties ofthe reconstructed images.
Compared to other tasks (e.g., classification) or models (e.g., autoencodertrained by BP), a higher value of weight decay seems to be necessary when training with PC.
Fromour experiments, weight decay prevents the networks from overlearning the task they are trainedon (i.e., reproduce any image that they are given in input), and instead allows them to “understand”the several concept classes of each dataset.
This behaviour makes it possible to generalize their knowledge to new and unseen tasks, such as the denoising and reconstructing tasks seen in this work.
It is worth noticing how, when optimizing for a single specific problem (e.g., image recognition),lower values of weight decay seem to be more effective.
To show this, we trained a recurrent network to reconstruct images by conditioning the bottom half ofthe output layer and giving the target class label in input.
The result is that, with low weight decay,the network treats each half of the image independently, reconstructing the bottom part by fitting theconditioning data and the top half using the given label.
It can be observed that there is no relationbetween the two halves.
With higher weight decay, instead, we can see that the image is reconstructedas a whole, incorporating both the information provided via the label and the conditioning data(Fig. 10).
C Associative Memory Experiments
In the paper body, we claimed that a fully connected PC graph is able to perform AM experiments.
To show this, we trained fully connected PC graphs with { 3500 , 7000 } vertices on different subsetsof cardinality { 50 , 100 , 200 } of CIFAR10 and FashionMNIST.
Then, we used query by initializationand conditioning to retrieve the original memories.
In this setting, we considered a memory to beretrieved if the mean squared error between the original training point and its reconstruction is lessthan 0 .
As corruption, we either removed the top half of the image, or corrupted it with Gaussiannoise of mean zero and variance 0 .
The results are shown in Fig.
Results: The experiments show that our model is able to well store and retrieve memories, evenwhen tested on colored images.
The reconstruction quality, as expected, decreases when adding morememories, and improves when adding more parameters to the model.
As hyperparameters, we usedη = 0 . 0001 , α = 0 .
5 , T = 5 . [Figure] [Table] [Table]
D Classification Results
In the paper body, we stated that multilayer PCNs are known to perform similarly to BP on classification.
Here, we tested this, and compared against popular models in the literature, such as restricted
Boltzmann machines (RBMs) [38] and dense associative memories (DAMs) [54].
Overall, PCNsare the only models able to perform similarly to BP on the test set.
We performed experiments on 4datasets: MNIST, FashionMNIST, SVHN
[Table]
and CIFAR10, and the results are in Table 2.
Setup:
The networks trained using PC and BP have L = { 2 , 3 } and 256 hidden neurons each.
Theyare trained using Adam optimization, a weight decay λ ∈ { 0 . 001 , 0 . 0001 , 0 } , and the learning ratefor the weights α ∈ { 0 . 001 , 0 . 0001 } .
We report the best average results in Table 2.
For the RBM,we used a model with 512 hidden nodes, and for the DAM, we copied the official implementationprovided by the authors, with the same hyperparameters.
[Figure]
E Restricted Boltzmann Machines
To provide a full comparison between the generation capabilities of our model and existing ones inthe literature, we trained a different RBM, and performed both reconstructions and denoising tasks.
The results are in Fig.
Particularly, they show that RBMs sometimes fail to retrieve the correctimage, returning a blurry cloud of points in denoising, and tend to often return the same image evenwhen presented with different inputs in reconstruction ones.
This problem was consistent in differentbatches and parametrizations of RBMs, and had never happened in any of the models that we haveproposed.
Setup: We trained several RBMs with h ∈ { 256 , 512 , 1024 } hidden nodes, and performed{ 1 , 2 , 5 , 10 } Gibbs samplings .
We always picked the best result.
F High Levels of Noise
Here, we push the limits of the model in denoising tasks, where the variance of the Gaussian noiseis high enough such that it is often hard for a human evaluator to distinguish different numbers.
Particularly, we use a 3 layer PCN with 256 hidden neurons, and we test it against an autoencoder withthe same parametrization.
The results, provided in Fig.
13 show that both model fail to reconstructsome examples, and the reconstructed ones are noisy.
However, we note that PCNs are able todistinguish more numbers than autoencoders, and hence have a better overall performance in thistask.
G Efficiency of the Model
Training a deep PC network is almost as fast as training deep neural networks with backpropagation.
This is despite the fact every hardware and library is highly optimized for the latter.
However, whilenot faster today, efficiency is an interesting property of PC graphs, and many other neuroscienceinspired learning methods, such as equilibrium and target propagation: all these algorithms are slowerthan backpropagation; however, they are extremely promising with respect to future developments onthe hardware side.
In fact, they would allow to train deep neural networks in an end-to-end fashionon physical chips, such as analog circuits [55].
This is something that is not possible to do withbackpropagation: in [56], the authors implement exact backpropagation on physical chips.
However, the process is quite slow, as there is the need of a digital control signal at every layer of the network.
This is due to the sequential structure of deep models, where every operation of a layer has to (1) waitfor the information of all the previous (following during the backward pass) layers, and (2) be savedin memory via a Von-Neumann digital device.
The situation would be completely different if usingmethods that would allow to train neural networks end-to-end, i.e., without any digital component,on the same chip: in this case, the learning process would be much faster, and would not need anyexternal control to be performed.
This is possible by using predictive coding.
However, despitepotential applications on physical chips, PC is also fast on current GPUs, and hence this is not anobstacle towards applications.
We now show multiple plots that shot the training and inference timesof multiple PC models.
Note that these results are obtained by using an implementation that doesnot make use of the full parallelization capabilities of PC, as this is not supported by standard deeplearning frameworks (in our case, Pytorch).
Hence, the proposed plots largely overestimate the actualefficiency of predictive coding networks, that can be obtained via a correct implementation.
Experiments : Here, we provide multiple plots that show that PC graphs quickly converge to astationary point.
Particularly, we show that the provided experiments are fast: training a recurrent 3layer PCN takes about 1 minute on an RTX Titan, as shown in the plots in Fig.
Same for testing:reconstructing/denoising an image takes 0.1/0.3 secs, as shown by the plots provided in Fig. 15.
Hence,the proposed models are robust to hyperparameter changes and converge rapidly.
All the proposedplots are generated via training and testing on a multilayer generative PCN with 3 layers and 512hidden neurons per layer.
We also provide the convergence plot of 48 different PC graphs, of differentparametrizations ( N ∈ { 1500 , 2000 , 2500 , 3000 } ), learning rates ( α ∈ { 0 . 0001 , 0 .
00005 , 0 . 00001 } )and integration steps ( γ ∈ { 1 . 0 , 0 . 5 } ), on both MNIST and FashionMNIST.
As shown in Fig.
16, PCgraphs always and quickly converge.
[Figure]
Corrupted Key
Reconstruction
Original Image
Incomplete Key
Reconstruction
Original Image
MNIST
FashionMNIST