Bidirectional Predictive Coding
Authors: Gaspard Oliviers
Mufeng Tang
Rafal Bogacz
Venue: ICLR 2026 Poster (2026)
Keywords: predictive coding
sensory processing
discriminative and generative tasks
Primary area: applications to neuroscience & cognitive science
TL;DR: We introduce bidirectional predictive coding, a biologically plausible model combining generative and discriminative inference, demonstrating improved performance on biologically relevant tasks and better alignment with the brain's visual processing.

Abstract: Predictive coding (PC) is an influential computational model of visual learning and inference in the brain. Classical PC was proposed as a top-down generative model, where the brain actively predicts upcoming visual inputs, and inference minimises the prediction errors. Recent studies have also shown that PC can be formulated as a discriminative model, where sensory inputs predict neural activities in a feedforward manner. However, experimental evidence suggests that the brain employs both generative and discriminative inference, while unidirectional PC models show degraded performance in tasks requiring bidirectional processing. In this work, we propose bidirectional PC (bPC), a PC model that incorporates both generative and discriminative inference while maintaining a biologically plausible circuit implementation. We show that bPC matches or outperforms unidirectional models in their specialised generative or discriminative tasks, by developing an energy landscape that simultaneously suits both tasks. We also demonstrate bPC's superior performance in two biologically relevant tasks including multimodal learning and inference with missing information, suggesting that bPC resembles biological visual inference more closely.

BIDIRECTIONAL PREDICTIVE CODING
Gaspard Oliviers Mufeng Tang" 12, & Rafal Bogacz!?
1. Nuffield Department of Clinical Neurosciences, University of Oxford, United Kingdom 2. MRC Centre Research Excellence in Restorative Neural Dynamics, United Kingdom Correspondence to: rafal rafal.bogaca..... .owgacz@ddcn......ooooomoooooccm ox. ackuk
ABSTRACT
Predictive coding (PC) is an influential computational model of visual learning and inference in the brain. Classical PC was proposed as top-down genera- tive model, where the brain actively predicts upcoming visual inputs, and infer- ence minimises the prediction errors. Recent studies have also shown that PC can be formulated as discriminative model, where sensory inputs predict neu- activities in a feedforward manner. However, experimental evidence suggests that the brain employs both generative and discriminative inference, while unidi- rectional PC models show degraded performance in tasks requiring bidirectional processing. In this work, propose bidirectional PC (bPC), PC model that incorporates both generative and discriminative inference while maintaining a bi- ologically plausible circuit implementation. We show that bPC matches outper- forms unidirectional models in their specialised generative or discriminative tasks, by developing an energy landscape that simultaneously suits both tasks. We also demonstrate bPC's superior performance in two biologically relevant tasks includ- ing multimodal learning and inference with missing information, suggesting that bPC resembles biological visual inference more closely.
INTRODUCTION
Visual inference plays critical role in the brain, providing information processing for interpret- ing and interacting with the environment. Two frameworks have emerged to explain how the brain could implement visual inference. The first describes vision as a bottom-up discriminative pro- cess, where sensory stimuli are progressively filtered through layered neural architectures, predict- ing behaviourally relevant outputs (Hubel & Wiesel 1962). This framework resembles inference in feedforward neural networks commonly employed machine learning for image classification. The other framework formalises vision generative process, where the brain constructs prob- abilistic model of sensory inputs (Knill & Pouget 2004). From this perspective, the brain learns a top-down generative model with priors over incoming sensory activity, and neural responses arise from the Bayesian inversion of this model, estimating posterior probabilities of brain states given sensory information. Various neural implementations of this inversion have been proposed, includ- ing variational methods (Friston) 2005 2010) or sampling approaches (Fiser et al. 2010) Orbán eet al. 2016 Haefner al.) 2016). Experimental evidence suggests that visual perception may be a combination of both frameworks (Teufel & Fletcher) 2020 Peters et al., 2024). Discriminative models explain the rapid initial responses to visual stimuli through efficient bottom-up processing of the brain (Peters et al. (024). whereas generative models capture the probabilistic computations critical for optimally integrating noisy sensory inputs with prior knowledge. as displayed in percep-
tion and behaviour (Ernst & Banks, 2002. Wolpert et al. 1995 Knill & Richards 1996. van Beers)
al. 1999).
One computational model capable of capturing both generative and discriminative processing modes in the brain is predictive coding (PC). In its generative formulation, PC accounts for broad range of biological data from the visual system (Srinivasan et al. 1982 Rao & Ballard, 1999: Hosoya jet al. 2005) and is successful in tasks such as associative memory (Salvatori et al. 2021) and image generation OOliviers al. 2024). In its discriminative formulation, PC matches backpropagation- trained neural networks in image classification Whittington & Bogacz 2017)] Pinchetti et al. 2025) Pinchett
while also explaining neural data (Song et al. 2024). Crucially, PC relies on local computations

---

and Hebbian learning rules consistent with biological constraints (Hebb 1949 Posner et al. 1988
Lisman 2017) and implementable in realistic neural circuits (Bogacz 2017). making it strong candidate for modelling both generative and discriminative learning the brain. However, current PC formulations remain restricted to a single inference mode. Hybrid approaches that combine generative and discriminative versions exist (T'cssantz et al. (023) but sacrifice performance in at
least one domain, leaving open how a single PC model could flexibly support both inference modes as required for biological vision.
In this work, we propose bidirectional predictive coding (bPC), a novel model of biological vision. bPC provides a biologically grounded neural mechanism that explains how the brain can simulta- neously perform generative and discriminative inference based on PC. We focus on evaluating and understanding bPC's performance in range of computational tasks, where PC models with either generative or discriminative inference have been individually successful. Our key contributions can be summarised as follows:
We propose bidirectional predictive coding, a biologically plausible model of visual per- ception employing both generative and discriminative inference that naturally arises from minimising a single energy function;
We show that bPC performs well as its purely discriminative or generative counterparts on both supervised classification tasks and unsupervised representation learning tasks, and outperforms precedent hybrid models;
We provide an explanation for the superior performance of bPC in both tasks, by showing that it learns an energy landscape that better captures the training data distribution than its unidirectional counterparts;
We further show that bPC outperforms other PC models in two biologically relevant tasks, including learning in bimodal model architecture and inference with occluded visual scenes, indicating its potential as a more faithful model for visual inference in the brain.
BACKGROUND AND RELATED WORK
Discriminative predictive coding. Recent research has explored PC models that employ bottom- up predictions from sensory inputs latent neural activities (Whittington & Bogacz, [017]] Song jt al. 2024), analogous to traditional feedforward neural networks. These models are structured as hierarchical Gaussian models consisting of layers, each characterized by neural activity æ7, where T corresponds to the sensory input and Iy represents a label. PC learns this hierarchical Gaussian structure by minimizing an energy function corresponding to the negative joint log- likelihood of the model:
1 Edisc (x, V) v) -
where denotes the set of neural states from 21 to LL, the parameters V include the bottom-up weights Vi of each layer and f represents an activation function. We refer to this model as discrim- inative predictive coding (discPC) and illustrate it in the top panel of FiA. In discPC, inference is performed by a forward pass from 21 to 22 's and the learning rule of this model approximates BP, using only local computations and Hebbian plasticity (Whittington & Bogacz 2017). Recent work showed that discPC performs comparably to BP in classifying MNIST, Fashion-MMa MNIST and CIFAR- 10 (Pinchetti et al. 2025) and outperforms BP learning scenarios encountered by the brain, such as online and continual learning (Song et al. 2024). However, discPC lacks unsupervised learning capabilities due to the non-uniqueness of the solution its generative dynamies (Sun & Orchard. 2020).
Generative predictive coding. Classically, PC uses top-down predictions from neural activities to sensory data, based on the hypothesis that the brain learns by minimizing the error between its predicted sensory inputs and the actual sensory input (Rao & Ballard 1999. Friston 2005). With neurons arranged hierarchically, the negative joint log- likelihoood can be written as:
Egen (z, W) = E'-1
2 || (144
(2)
2

---

A
discriminative
B
generative
C bidirectional
discPC
genPC
hybridPC
bPC ours
local predictive coding
O0
O0
O
O0
00
discBP
genBP
hybridBP
AE
non local backpropagation
-00
O
discriminative prediction
generative prediction
initialisation
error propagation
Figure 1: Model architectures used in our experiments. Arrows indicate the direction of pre- diction and errоr propagation. Dashed arrows represent initialisation. Discriminative models (red) are parametrised by bottom-up mappings from sensory inputs to brain states, generative models (blue) are parametrised by top-down mappings from brain states to sensory inputs, and bidirectional models incorporate both directions. Models in the top row employ local computations and error propagation that are considered biologically plausible, whereas those the bottom row utilise non- local, backpropagation- based error computations that lack biological plausibility.
where is set to sensory data and Iz can be clamped to label for supervised learning or remain free for unsupervised learning. The parameters Wi are top-down weights. We refer to this formu- lation of predictive coding generative predictive coding (genPC). In genPC, inference of latent activities 2ı is performed by clamping Ij to sensory inputs and iteratively updating 2i via gradient descent on the energy Egen. genPC's top-down predictions are illustrated in the top-left panel of Figure BB. genPC has explained various visual phenomena, such as extra-classical receptive field effects and repetition suppression (Rao & Ballard 1999; Hohwy
al.) 2008 Auksztulewicz & Friston 2016). More recently, genPC has been employed to model associative memory (Salvatori al., 2021) and unsupervised image generation (Oliviers al.] 2024 Zahid al., 2024). The computational framework of genPC can be implemented within neural networks that utilize local computations and Hebbian plasticity (Bogacz (017)). However, genPC. has poor performance in supervised learning tasks (Tscshantz et al. 2023).
Predictive coding models with mixed inference modes. In this work, we benchmark our bPC model primarily against hybrid predictive coding (hybridPC) (Tseshantz al. 2023). In hybridPC, iterative inference proceeds in the same way as in genPC. However, an additional bottom-up network introduced provide feedforward initialisation of neural activities. This network only sets the initial states and does not influence the subsequent dynamics, as illustrated in the top-right panel of Figure LB. Although hybridPC performs both supervised and unsupervised learning, its supervised performance falls short of discPC. In this work, we show that bPC performs supervised learning on par with discPC, and we provide an explanation of hybridPC's inferior performance through the lens of bPC (see SM Sun & Orchard (2020) noted the energy-minimising nature of PC models could discPC (Eq, D. by clamping Ft. to a label and iteratively updating æ1. However, the generated images appear nonsensical due to the non-unique inferential dynamics solutions. Salvatori et al. (2022) generalised the idea of Sun & Orchard (2020) to a PC model where all neurons are interconnected with each other. However, it only slightly outperforms the classification performance of a linear classifier. Finally, Qiu et al. (2023) proposed a bidirectional PC model in which the connections between separate layers share the same weights, e.g. the bottom- up weights from layer / - to I equal the top-down weights from layer to + 1. is unlikely that the brain shares synaptic connections between separate layers of processing.
theoretically allow image generation
Discriminative and generative models of the brain. Cortical processing models are often di-
vided into discriminative, which use feedforward networks to filter inputs (Yamins et al. 2014
Fukushimal 1980), and generative, grounded in the Bayesian Brain hypothesis (Helmholtz 1866
Knill & Pouget (004) and theories such as predictive coding (Rao & Ballard 1999] Friston 2005
or adaptive resonance (Groossberg 2012). Growing evidence suggests the brain combines these ap-
3

---

proaches (Lamme& & Roelisema 2000 Teufel & Fletcher 2020 Peters al. 2024): for example, it might employ a discriminative approach to rapidly extract important features, and perform Bayesian inference on these features to form high-level perceptions such object categories (ildirim al. 2024). Few models explicitly integrate both. The wake-sleep algorithm used to train the Helmholuz machine (Dayan et al. 1995) is analogous to a pnmneratvvediicriminne
process. The Symmetric Predictive Estimator (Xu et al., 2017) models bidirectional processing, though only on toy tasks. A related bidirectional model based on information maximization (Bozkurt et al.) (023) improves biological plausibility but requires two inference phases per update. Interactive activation mod- els (Rumelhart & McClelland 1982) also perform bidirectional inference, but do not incorporate a learning mechanism.
Discriminative and generative models in machine learning. Research in machine learning has likewise sought to integrate discriminative and generative pathways. Encoder- architec- tures such as U-Nets (Ronneberger et al., 2015) combine bottom-up and top-down processing via skip connections, while models including VAVAE (Yang & Wang. (225)) and VAR (Tian et al. 2024) unify both pathways within a shared latent representation. Other approaches, exemplified by PredNet (Lotter et al., 2016), take more explicit inspiration from cortical circuitry implement bidirectional information flow. Despite their strong empirical performance, these models depend on non-local learning signals and therefore do not provide biologically plausible account of bidirec- tional learning in the brain.
3 BIDIRECTIONAL PREDICTIVE CODING
In contrast genPC and discPC, bPC neurons perform both top-down and bottom-up predictions, as shown in the top panel of Figure TC. bPC achieves this bidirectional inference by unifying the energy functions of genPC and disePC into a single formulation, enabling both generative and dis- criminative prediction within the same circuit. Using the notations previously introduced, the energy function of bPC given by:
E(x, W. V) = - Ogen || - Wi+ + Odisee |||| - V1-1((11-111 (3)
2
1=2 2
where Wi are the top-down weights and Vi are the bottom-up weights. agen and Odisc are scalar weighting constants, which are needed to account for magnitude differences in the errors of bottom- up and top-down predictions. The weighting constants can be viewed as learnable precision parame- ters (Friston| 2005); however, they are tuned and kept constant in our implementation for simplicity.
In each trial of learning, we first initialise the layers of neural activity using a feedforward sweep from layer Ii to Fy, along the bottom-up predictions, similar to discPC (Whittington & Bo- gacz 2017). For instance, the second and third layers are initialised as 22 Vi)(x) and T3
(V (xx)) respectively. This initialisation strategy can interpreted as a mechanism for fast amortised inference when sensory input is initially encountered, which is also observed in the brain (Thorpe et al. [996] Lamme& & Roelfsemal 2000). All bPC experiments used this activ- ity initialisation scheme. After, update the neural activities minimise E via several gradient descent steps (neural dynamics) following:
dx dl
x
V o
El
disc + f' @ (WT e VT ediec) + +N((0,021,,
(4)
where
eeen = Agen(x) - ( qiie; (ziso azi Vi-1f(II-11)
(5)
denote the top-down and bottom-up prediction enrors of neurons in layer i respectively, f' denotes the derivative of the function f, and @ is the element-wise product. The normally distributed noise A is zero-mean, temporally uncorrelated, and independent across neurons. By default, set o² 0, yielding deterministic dynamics that converge to the maximum a posteriori estimate. Unless stated otherwise, all experiments in this paper use these inference dynamics. Setting d² 1 induces stochastic dynamics that sample from the model's posterior (Oliviers et al. 2024). These dynamics enable learning the distributions of sensory inputs.
After updating neural activities, the weights are updated to minimise E via single gradient descent step:
AW
F = F= { AV - c [ (m)
Vv,
diec
4

---

3.1 NEURAL IMPLEMENTATION
Xua
The computations described in Eqs 4 and can be imple-
mented in neural network with fully local computation and
W.n
layer I+1
plasticity, as illustrated in Figure 2 This network contains value neurons, which encode 21, eπrоr neurons, which repre- sent prediction errors, and synaptic connections, which en- code the model parameters. All computations are local, rely- ing solely on pre- and post- synaptic activity. Value neuron dynamics depend on local error signals, their own activity, and incoming synaptic weights. Similarly, error neuron ac- tivity depends only on adjacent value neurons and synaptic weights. The plasticity of the weights is also Hebbian, as is a product pre- and post-synaptic activity. The local imple- mentation of bPC inherits that of genPC discPC (shown in Figure 2 right), although has two distinet error neurons per value neuron for bottom-up and top-down prediction errors.
layer
V
excitatrry
X
W
layer
inhibitory
generative
V₂s
discriminative
Figure 2: Neural implementation of bPC (left) and disePC (right).
3.2 FLEXIBLE LEARNING
bPC can be trained both in supervised and unsupervised settings. In all cases, neurons in inter- mediate layers (layer I. 1) are un-clamped and evolve according to the neural dynamics in equation 4 In supervised settings, the first layer 2 is clamped to the input, while the top layer az is clamped the target labels. unsupervised settings, Iy is left unclamped, allowing bPC to learn compressed representations of the input. A mixed setting is also possible, where only subset of neurons in #L are clamped to label information, while others remain free. In this setting, the model can jointly infer labels and learn compressed representation.
EXPERIMENTS
4.1 BPC PERFORMS SIMULTANEOUSLI WELL CLASSIFICATION AND GENERATION
In this experiment, we assessed bPC's capacity for both clas- sification (discriminative) and class-conditional image generation (generative). We compared bPC with disePC, genPC, hybridPC, and their backpropagation equivalents on MNIST and Fashion-MNIST, using identical architectures with two hidden layers 256 neurons each. Additional baselines, including bPC with shared. bottom-up and top-down weight proposed by Qui et al (2023), are provided in the supplementary material (SM)
A training
00010
B
classificlion accuracy (%6
100
95
fixed
90
85
80
75
70
MNIST
Fashlon-MNIST
C
conditional generation
conditionlggnerration
conditional generation RMSE
0
bPC
bPC
discPC
0.3
genPC
hybridPC
hybridPC
0.2
discSP
disPC
genBP
genPC
During training, the input layer was clamped images and the out- put layer Iz = 24 to their corre- sponding one-hot labels (Figure (3)) After training, discriminative perfor- mance was assessed by fixing I to an input image and inferring the label at z over 100 inference steps. Gen- erative performance clamping v, to class label and mea- suring the root mean squared error
gen@P
MNIST
Fashion-MNIST
Figure 3: bPC accurately classifies and generates class average images on MNIST and Fashion-MNIST. A: Training set-up. The models are trained with x; fixed to images and xy fixed to labels. B: Classification accuracy of models. C: Examples of the generated images conditional on class labels (left) and RMSEs between generated images and mean images of each class (right). Error bars denote the standard error of the mean (s.c.m.) across 5 seeds.
evaluated by
5

---

A
training
B
reconstrlntto tintttnn
RMSS
0.3
genPC
BPC
AE
hybrdd8
base
2
hybraPC
fixed
reconstructio
genPC
free
0.15
0.1
hyoridPC
0.05
AE
0.0
hybridBP
MNIST Feshorr-NISST
CIFAR-10
CIFAR-100
Dacoding Accuracy
100
BPC
рanPC
Frechht inception disiance
10
MNIST
genPI
LPC
bpC
hybriPCC
hybriPC
80
WALL
genPC
4709 6709007
(96
hybricBP
60
hybridPC
#60/700
5 559237
2
AE
170008 2+6
40
MNIST
Feshior-MNIST
CIFAR-10
CIFAR-100
T=50
T
Figure 4: bPC matches the performance of hybridPC at unsupervised learning. A: Training set-up, where only iı is clamped to input images. B: Reconstruction RMSE from representations. C: Example reconstructions. D: Linear decoding accuracy from representations. E: Fréchet Incep- tion Distance of samples generated by models with stochastic dynamics, with examples shown. denotes the number of inference steps used during training. Error bars show s.c.m. across 5 seeds.
(RMSE) between the inferred image at æ1 (after 100 inference steps) and the average image for that class.
The results are shown Figure 3.. On both datasets, bPC achieved classification accuracy compa- rable to discPC and discBP, while genBP and hybridPC performed less well. For generation, bPC obtained RMSE scores similar to genPC, hybridPC, and genBP, whereas discPC exhibited much higher errors. Visualizations in Figure 3C further illustrate these differences: discPC generated im- ages with little class-relevant structure, while bPC and other bidirectional models produced clear and representative samples.
These findings align with previous reports that unidirectional PC models excel only their special- ized domain (Sun & Orchard 2020 Tseshantz al., (023), and demonstrate that bpC can integrate both discriminative and generative capabilities within a single framework. Notably, this integration uses the same number of error neurons but only half the value neurons required for maintaining separate unidirectional pathways for the two tasks. This indicates that bPC is more energy -efeggeefficient,, especially as errоr signals can handled in the dendrites of neurons rather than by separate neu- ron population (Mikulasch et al.) (023). We observed that the discriminative weighting parameter (addise) must be set higher than the generative weighting (Aigen) due to the larger magnitude of top- down prediction errors. An exploration of the trade-off between Odise and Qgen is provided in SM
4.2 BPC PERFORMS UNSUPERVISED REPRESENTATION AND DISTRIBUTION LEARNING
Next, show that bPC learns compressed representations and data distributions in the absence of supervision. We compared bPC, genPC, hybridPC, and their backpropagation (BP) equivalents on MNIST, Fashion-MNIST, CIFAR-IO, and CIFAR-100, excluding discPC since cannot perform unsupervised learning. We clamped only æ1 to input images, leaving all other layers free (see Figure 4A). For MNIST and Mushion-MNISS models used two hidden layers (256 neurons each) and 30- neuron representation layer Ft, trained with only 8 inference steps per update to test fast inference. CIFAR models had five convolutional layers and 256-neuron representation layer, trained with 32 inference steps. An activity decay term at zy stabilized lcarning and regularized representations. After training, representations were obtained from Iy using 100 inference steps. We evaluated these representations in ways. First, reconstruction quality measured by reinitializing layers d1-#1-1, clamping L. and running 100 inference steps, with RMSE computed against the input. Second, linear readout classification accuracy measured from Fr.
To assess distribution learning. we compared bPC, genPC, hybridPC, and VAE (Kingma et al. 2013) MNIST, extending all PC models with stochastic dynamics infer full posteriors (a²

---

SM A.5 for more details). Architectures matched those above, trained with 50 or 250 inference steps. Models were evaluated using Fréchet Inception Distance (FID). Samples were generated ancestrally by following top-down predictions starting from sampling zy, from its Gaussian prior and propagating downward through conditional Gaussians.
in equation 4
As shown in Figure BB-E, bPC consistently outperformed genPC. yielding lower reconstruction RMSE, higher decoding accuracy, and better FID. It matched hybridPCC across most tasks, and on CIFAR datasets significantly surpassed it in reconstruction error, approaching BP-based baselines. Reconstructions and generated samples (Figures Cc. E) illustrate these differences.
These results highlight that bPC outperforms genPC. when inference steps limited, and matches hybridPC despite the latter's amortized pathway. This suggests that bPC's bottom-up pathway also serves as amortized inference, rapidly initializing neural activities toward the optimal state. The slight edge of bPC over hybridPC likely arises from the active involvement of bottom-up weights Vi during iterative inference, which continuously deliver sensory information latent neurons. This effect becomes more pronounced with complex inputs such as CIFAR.
4.3 BPC PERFORMS COMBINED SUPERVISED AND UNSUPERVISED LEARNING
In this experiment, we combined the supervised and unsupervised settings described above to test whether bPC can simultancously develop discriminative capabilities and compact representations. This setting is motivated both computationally and biologically: real-world learning rarely occurs in isolation, and cortical circuits appear to integrate categorical supervision (e.g., from higher-order areas) with unsupervised structure learning from sensory input.
We trained bPC, hybridPC. and their BP equivalents on MNIST, Fashion-MNIST, and CIFAR- -1. During training, the input layer was clamped to images, while the top layer was partially clamped to one-hot labels, leaving the remaining neurons free to learn complementary representa- tions (iguureSAA For MNIST and Fashion- each, and x 1 comprised 40 neurons (10 for labels, 30 for representations). For CIFAR-10 10, we used a convolutional architecture with four hidden convolutional layers and 266-neuron latent layer (æ with 10 label and 256 representation neurons). Activity decay was applied to the representational subpopulation in vy.. regularize learning. Classification accuracy was evaluated in Section 4. by presenting only images and inferring labels at xT, Generative quality was assessed as in Section 4.2 but with reconstructions conditioned jointly on the inferred representation and the label.
models had two hidden layers of 256 neurons
Figure 5B reveals that bPC achieves classification accuracy on par with BP models across all datasets. Furthermore, significantly surpasses hybridPC, particularly on CIFAR-10, where hybridPC exhibits more than 45% lower accuracy than bPC. In terms of generative quality, Figure 5D shows that bPC achieves reconstruction errors similar hybridPC across all datasets. The reconstructed examples in Figure illustrate that bPC goes beyond generating class-average images, capturing features such as shape, color, and spatial locations (e.g., it generates '4's in different styles). However, fine details are absent in bPC's CIFAR-10 reconstructions due to artefacts introduced by max-pooling op- erations in the discriminative bottom-up pathway, affecting the
B
classficaiion acouracy (%6
00OD
training
GOOOOO
80
fixed
20
Fashior-MNiST
RMSE
0.3
reconstruction
0.25
base
reconstruciion
ВPC
hybridPC
0.1
0.05
AE
hybridBp
MNIST CIFAR-10
bPC
hybridPC
AE
hybridBP
Figure 5: bPC is the only PC model that can jointly learn low-dimensional representations of images and accurately classify them. A: Training set-up. where the latent layer is only partially clamped class labels. B: Classification aceu- racy. C: Example reconstructions MNIST and CIFARIO. D: Reconstruction RMSEs. Error bars show s.c.m. across 5 seeds.

---

A two unidirectional models
B
XOR energy conditioned on class
C
high confidence samples
Ersc
Egt
Es..
Egr
training
X
high
PONSO dunstty
MANST
discPC
gennC
X- 11
zauron
X
mi.ron 1etx
non-specfiic average biased
bidirecttionalmodel
Esu:
training
01
x-11
fixed
free
K-100 1]
X1
discPC* gennC
Figure 6: bPC develops energy landscape suitable for both generation and classification. A: We compare bPC (bottom) to training a discPC and a genPC separately (top). B: Visualisation of the models' energy landscape after training on XOR. Shown are the discriminative, generative, and summed energies. C: MNIST samples considered highly likely (bottom 10% energy) by the models.
generative top-down reconstruction process. Consequently, bPC shows stronger grid-like artefacts than hybridPC, which uses only max-pooling for initialization, and than BP models with separate generative and discriminative layers. While removing max-pooling removes artefacts, reduces classification accuracy (see SM E. Nonetheless, even without max-pooling, bPC's classification accuracy significantly outperforms hybridPC, while generative performance remains comparable. Future research could refine bpC's architecture balance classification accuracy with artefact-free generation.
4.4 BPC's SHARED LATENT LAYERS PREVENT BIASED OR OVERCONFIDENT ENERGY LANDSCAPE
In this experiment, we investigated why bPC can perform both discriminative and generative tasks effectively, focusing on the role of its shared latent layers in shaping the energy landscape (Eq. 3 As toy example, we considered the XOR problem, where the landscape can directly visualized. We trained bPC. discPC, and genPC (each with two hidden layers of 16 neurons) by clamping æ1 to 2D imputs XOR and æ4 to scalar outputs (Figure (A)). bPC used the same number of parameters discPC and genPC combined but half the neurons.
An ideal XOR landscape has well-localized minima only at valid input-label pairs ([0,1) and [1,0] for one class; [0,0] and [1,1] for the other). Figure 6B top shows that discPC instead develops broad low-energy regions, indicating overconfidence even for implausible or out-of-distribation (OOD) inputs. genPC collapses each class into a single mean, failing to capture the true structure. A combined discPC+genPC model inherits both flaws. In contrast, bPC learns sharp, class-specific minima centred on valid inputs (Figure@ß bottom).
To test generality, we evaluated the models from Section 4.1 by sampling MNIST digits. We clamped æ to label, initialized T2 randomly, and iterated interence until reaching energies within the lowest 10% of those observed on test images. For the combined discPC+genPC model, we re- quired the generated samples satisfy this low-energy criterion for both components. This set-up serves sampling process, allowing us to inspect images considered highly likely by the mod- els. As shown in Figure bPC produced realistic digits, while the combined model yielded poorly formed shapes. Quantitatively, bPC achieved higher Inception Scores (6.05 + 0.17 3.62 + 0.03) and lower FID (44.4 2.2 vs. 140.5 + 2.1). Similar trends held for BP-based baselines (see SM
discPC fails at generation because it produces broad, overconfident minima that admit many nonsensical and OOD inputs, leading to the noisy conditional generations in Figure 3 and in Sun & Orchard (2020). genPC, by contrast, learns narrow minima around class means, which rejects OOD inputs but identifies deviations from training-so mean high-energy states, ultimately sacrificing discriminative precision. bPC integrates the two:
These results clarify the observations in Section 4.1
8

---

learning in bimodal architectur
bPC
Riod
dossification accuracy
100
85
geneeration RMSE
0.2
genPC
generation from label
0.15
multimodal genPC
85
0.1
genPC
multimodalbPC
75
Fashion-MNIST
Fashion-MNIST
Inferenceewthhmissinggnuuuuuuu
inference
classification accuracy (SS)
100
classification alcuracc (%)
100
MNIST
imput
S
M
Toed
LG
LO
6G
G .
bPC
G
U
U
G
genPC
disc/NC
40
99
disc3P
S
10 30 50 70 90 missing pixels (%)
10:20 30 40 50 60 70 80 90 missing pixets (56)
20
40
60
80
missing pizels (%)
Figure 7: bPC performance in bimodal architecture and robustness against missing infor- mation. A: Bimodal bPC and genPC. Note that bimodal bPC is fully equivalent to unimodal bPC in Section 4.I it is bent here to serve visual purposes. B: Classification accuracy of bimodal PC models. C & D: Generation RMSE and examples. E: Model evaluation with partially occluded sensory inputs. F & G: Classification accuracy against percentage of missing pixels with MNIST and Fashion-MNIST respectively. H: Activity of xı after inference for genPC and bPC. Error bars and shaded region show s.c.m. across seeds.
bidirectional predictions effectively regularize each other, sharpening discriminative minima while anchoring them to specific data points through the generative process. Shared latent layers thus prevent solutions overly biased training mean or those that are overconfident, yielding superior performance in both classification and generation.
4.5 LEARNING BIOLOGICALLY RELEVANT TASKS
In this section, we demonstrate that bPC outperforms other predictive coding models in two biolog- ically relevant scenarios: (1) learning in a model architecture with two input streams analogous to two sensory modalities, and (2) classifying images from partially occluded inputs, similar to how human vision includes regions with missing sensory information, such as the retinal blind spot.
Bimodal model architecture. The brain often develops neural representations through associa- tions across modalities, such as linking spoken names visual objects (Rosen al. 2018). To test whether bPC can form such associations, we trained bPC and a bimodal variant of genPC on MNIST and PasHion-MNIST, with one latent layer connected two inputs: image and its hot label (Figure [A; note that bPC is naturally bimodal and thus requires no restructuing). bPC incorporated both bottom-up and top-down connections for each modality, whereas genPC relied only on top-down pathways. After training, we evaluated cross-modal transfer by providing input one modality and measuring inference quality the other. classification accuracy for labels and RMSE for reconstructed images. As shown in Figure BBD. bPC significantly outperformed bimodal genPC on both tasks. This result consistent with Sections 4.1 bPC's bidirectional pathways nat- urally support associative coding, while genPC must be restructured to handle multimodal inputs, where one pathway predicts the image (similar to genPC) and the other predicts the label (similar to discPC) and inherits the weaknesses of both genPC and discPC.
Robustness to missing information. The cortex can recognize objects even when sensory input incomplete or occluded (Komatsu 2006). We therefore tested the classification performance of the trained models from Section 4.1 (bPC, genPC, discPC, and discBP) under progressive occlu- sion of MNIST and Fashion-Man -NIST inputs, masking up to 90% of pixels (Figure - Observed pixels were clamped in .1. while missing ones were left free and initialized to zero. We then run inference on the missing pixels and latent layers, with extended iterations to accommodate slower convergence. Results (Figure F- show that bPC maintained high classification accuracy

---

with 80% missing pixels. genPC also retained some robustness but at consistently lower accuracy. In contrast, discPC and discBP collapsed beyond 50% occlusion. Visualizations in Figure 7H reveal that bPC actively reconstructed missing inputs via top-down priors, integrating them with observed evidence. genPC. shared this generative capacity but lacked strong discriminative accuracy, while purely discriminative models could not compensate for missing information at all.
4.6 GENERATIVE AND DISCRIMINATIVE PROCESSING IN DEEP MODELS
To investigate whether bPC scales effectively to deeper models and more complex datasets, we trained bPC, discPC and discBP using the following architecture-dataset combinations: VGG-5 on CIFAR-10, VGG-9 on CIFAR- 100, and VGG-16 on Tiny-ImageNet. After training, we evaluated discriminative performance by measuring the model's classification accuracy. Additionally, we as- sessed generative capabilities by evaluating classification accuracy when 30% and 50% of the input pixels were missing, following the methodology outlined Section |4. To efficiently simulate the predictive coding models in this experiment, we employed error optimisation (Goemaere et al.) 2025). This approach prevents energy decay in predictive coding models and enables the training of larger architectures. Refer to SMi]for more implementation details.
Figure S confirms that bpC successfully combines discriminative and generative processing in deep networks. bPC achieves classification accuracy comparable to discPC and discBP when full im- ages are presented and significantly outperforms discPC and discBP on images with missing input information. For example, when 50% of pixels are missing, bPC classifies more than 60% more accurately than discPC and discBP on CIFAR-I0 (Figure This improvement stems from bPC's generative processing, which fills in missing information. Figure RD illustrates this effect: after inference, the activity of the input layer I2 reveals that neurons lacking inputs have been updated to predict the missing values. Once the missing information is inferred, bPC classifies images ac- curately. Overall, even in larger models, bPC effectively balances discriminative and generative performance by jointly minimising bottom-up and top-down prediction errors.
A
VGGS/CIFAR-10
B
VGG9/CIFAR- C IiGge//inn D bPCinfers missing inputs
Classiiicalion accuracy (%)
100
100
Classfication sccuracy (%)
100
bPC
30% missing inputs
80
Classfiiation accuracy
80
80
discBP
disePo
original
input
inferred
40
40
40
50% missing inputs
20
20
original
input
Missing imputs
Missing inputs (56)
Missing inputs (5)
30
inferred
Figure 8: bPC's discriminative and generative properties scale to deep networks. A & B & C: Classification accuracy against the percentage of missing pixels for bpC, discPC and discBP on the architecture- tatttes combinations: VGG-5 on CIFAR-10, 10, VGG-9 on CIFAR- 100, and VGG-16 on Imay-lm D: Activity of input neurons æ after inference for bpC trained on Imag- Error bars show s.e.m. across 5 seeds.
5 CONCLUSION
Inspired by empirical and theoretical insights into visual processing, we propose bidirectional pre- dictive coding, a biologically plausible computational model of visual inference that integrates gen- erative and discriminative processing. We demonstrate that bPC performs effectively across both supervised classification and unsupervised representation learning tasks, consistently outperform- ing or matching traditional predictive coding models. Our experiments reveal that the performance of bPC emerges from its ability to develop an energy landscape optimized simultaneously for both discriminative and generative tasks, thereby improving its robustness to out-of-distribution data. Furthermore, we show bPC's effectiveness in biologically relevant scenarios such as multimodal integration and inference with partially missing inputs. Overall, bPC offers hypothesis for how flexible inference could emerge the brain, while also providing a method to enhance the robust- ness of discriminative models in machine learning applications.
10

---

REPRODUCIBILITY STATEMENT
detailed description of all experiments is provided in the first section of the supplementary materi- als. The complete anonymised codebase is also attached to the supplementary materials to facilitate the reproducibility our results.
LCKNOWIEDGEMEENT
This work has been supported by the Medical Research Council UK grant MC.UU_00003/1 and by the Wellcome Trust grant 31.955///24/Z awarded to R.B.
REFERENCES
Ryszard Auksztulewicz and Karl Friston. Repetition suppression and its contextual determinants in predictive coding. cortex, 80:125-140, 2016.
Rafal Bogacz. A tutorial on the free-energy framework for modelling perception and learning. Journal of mathematical psychology. 76: 16:198-211, 2017.
Bariscan Bozkurt, Cengiz Pehlevan, and Alper Erdogan. Correlative information maximization: A biologically plausible approach to supervised deep neural networks without weight symmetry. Advances in Neural Information Processing Systems, 36:34928-34999 2023.
Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel. The helmholtz machine. Neurul computation, 7(5):889-904, 1995.
Mare O Ernst and Martin Banks. Humans integrate visual and haptic information statistically optimal fashion. Nature, 415(6870):429 433, 2002.
József Fiser, Pietro Berkes, Gergö Orbán, and Máté Lengyel. Statistically optimal perception and learning: from behavior to neural representations. Trends in Cognitive Sciences, 14(3):11--130, 2010. doi: 10. 10.1016).iess.211001..000
Karl Friston. A theory of cortical responses. Philosophical transactions of the Royal Society B: Biological sciences, 360(1456):815 836, 2005.
Karl Friston. The free-energy principle: unified brain theory? Nature Reviews Neuroscience, 11 (2):127-138, 2010. doi: 10.1038/mm2287
Kunihiko Fukushima. Neocognitron: A self-organine neural network model for mechanism of pattern recognition unaffected by shift in position. Biological cybernetics, 3634)2193-20,, 1980.
Cédric Goemaere, Gaspard Oliviers, Rafal Bogacz, and Thomas Demeester. Error optimization:
Overcoming exponential signal decay in deep predictive coding networks, 2025. URL https:
/ //axxiv..rr//bb//25555 20137
Stephen Grossberg. Adaptive resonance theory: How a brain learns consciously attend, learn, and recognize a changing world. neural. Neural networks: the official journal of the International doi: 10.10166
Neural Network Society, 37, 10 2012.
Ralf M. Haefner, Pietro Berkes, and József Fiser. Perceptual decision-making probabilis- tic inference by neural sampling. Neuron, 90(3):649 660, 2016. ISSN 0896-6273. doi:
Donald Hebb. The Organization of Behavior: Wiley, New York, 1949.
von Helmholtz. Concerning the perceptions in general. Treatise on physiological optics,, 1866.
Jakob Hohwy, Andreas Roepstorff, and Karl Friston. Predictive coding explains binocular rivalry:
An epistemological review. Cognition, 108(3):687-701, 2008.
Takahiro Hosoya, Stephen A. Baccus, and Markus Meister. Dynamic predictive coding by the retina. Nature, 436(7047):16000 July 2005. doi: 10. 0091

---

David H Hubel and Torsten N Wiesel. Receptive fields, binocular interaction and functional archi- tecture in the cat's visual cortex. The Journal of physiology. 160(1):106, 1962.
Diederik Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
David Knill and Whilman Richards, Perception as Bayesian Inference. Cambridge University Press, New York, 1996.
David C Knill and Alexandre Pouget. The bayesian brain: the role of uncertainty in neural coding and computation. TRENDS in Neurosciences, 27(12):722-79,, 2004.
Hidehiko Komatsu. The neural mechanisms of perceptual filling-in. Nature Reviews Neuroscience, 7(3):220-231, 2006. doi: 1031038/mm188 1869.
Victor AF Lamme and Pieter R Roelfsema. The distinct modes of vision offered by feedforward and recurrent processing. Trends in neurosciences, 23(1 (5:15):77-579 2000.
John Lisman. Glutamatergic synapses are structurally and biochemically complex because of multiple plasticity processes: long-term potentiation, long-term depression, short-term potenti- ation and scaling. Philos. Trans. R. Soc. Lond. B Biol. Sci., 372(1715):211602) 2017. doi:
10.1098/rssb2211..266)
William Lotter, Gabriel Kreiman, and David Cox. Deep predictive coding networks for video pre- diction and unsupervised learning. 05 2016. doi: 1005550/rXii.1605.08888
Fabian A. Mikulasch, Lucas Rudelt, Michael Wibral, and Viola Priesemann. Where is the error? hierarchical predictive coding through dendritic error computation. Trends in Neurosciences, 46 (1):45-59, 2023. ISSN 0166-2236. doi: 10. 10.10066..inns.202220.00000 URL https://doi.
org/10. 6/3. tin.s.i..s... 2022.022200 09.007
Gaspard Oliviers, Rafal Bogacz, and Alexander Meulemans. Learning probability distributions of sensory inputs with monte carlo predictive coding. PLOS Computational Biology, 20(10): 1-34, 10 2024. doi: 10.137 /jouunal..pcbi.1112555.. journal acbi..01253.
URL https:///ci..rgg//
Gergö Orbán, Pietro Berkes, József Fiser, and Máté Lengyel. Neural variability and sampling-bard probabilistic representations in the visual cortex. Neuron, 92(2):530-543, 2016. doi: 10.1016/; neuron.2016.099 20.6.09.0.8.
Benjamin Peters, James DiCarlo, Todd Gureckis, Ralf Haefner, Leyla Isik, Joshua Tenenbaum, Talia Konkle, Thomas Naselaris, Kimberly Stachenfeld, Zenna Tavares, et al. How does the primate brain combine generative and discriminative computations in vision? ArXiv, pp. arxiv 2401, 2024.
Luca Pinchetti, Chang Qi, Oleh Lokshyn, Cornelius Emde, Amine M' Charrak,, Mufeng Tang. Simon Frieder, Bayar Menzat, Gaspard Oliviers, Rafal Bogacz, Thomas Lukasiewicz, and Tommaso Salvatori. Benchmarking predictive coding networks made simple. In Proceedings of the 13th International Conference on Learning Representations, ICLR 2025, Singapore, 24-28 April 2025, 2025.
Michael Posner, Steven E. Petersen, Peter T. Fox, and Mareus E. Raichle. Localization of cognitive operations in the human brain. Science, 240: 140:16677163111 1988. doi: 10. 1e.116//secence.228999 16.
Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: functional interpreta- tion of some extra-classical receptive-field effects. Nature neuroscience, 2(1):79-87, 1999.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medicul Image Computing and Computer-Assisted Intervention MICCAI 2015, volume 9351 of Lecture Notes in Computer Science, pp. 234-241. Springer, Cham, 2015. doi: 10.10077978331124555
28.
Maya L Rosen, Margaret A Sheridan, Kelly A Sambrook. Matthew R Peverill, Andrew N Meltzoff, and Katie A McLaughlin. The role of visual association cortex in associative memory formation across development. Journal of cognitive neuroscience, 30(3):365-380, 2018.
12

---

David E. Rumelhart and James L. McClelland. An interactive activation model of context effects in letter perception: Part 2. the contextual enhancement effect and some tests and extensions of the model. Psychological Review, 89(1):60-99) 040600094444 1982. doi: 10.1037/0033-29950 2000000333-29988....60
Tommaso Salvatori, Yuhang Song, Yujian Hong, Lei Sha, Simon Frieder, Zhenghua Xu, Rafal Bo- gacz, and Thomas Lukasiewicz. Associative memories via predictive coding. Advances in neural information processing systems, 34:3874-3888 2021.
Tommaso Salvatori, Luca Pinchetti, Beren Millidge, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Learning on arbitrary graph topologies via predictive coding. CoRR, Abs/2201.13188 2022.
13180
Maximilian Seitzer. pytorch-fid -irdch-fid: FID Score for PyTorch. httpas///gittuu..coomm/messs pytorch-fic. fid August 2020. Version 0.3.0.
Yuhang Song, Beren Millidge, Tommaso Salvatori, Thomas Lukasiewicz, Zhenghua Xu, and Rafal Bogacz. Inferring neural activity before plasticity foundation for learning beyond back- propagation. Nature Neuroscience, 27(2) 358, 2024. ISSN 1546-1726 doi: 10. 1038/
+41593-023-01514-1 URLpttts::////// doi httptttpss//doi..rg////e
038
M. V. Srinivasan, S. B. Laughlin, and A. Dubs. Predictive coding: a fresh view of inhibition in the retina. Proceedings of the Royal Society of London. Series B. Biological Sciences, 216(1205):42 216(12200ee22216055 459, 1982 doi: 10. 10.1009//spb..o 1et11099/pb..12...00.0.. 1985.0085. URL hattps:/rooplllaciiityyy.b....a
10 1098 rtpp 1982. 0085
Wei Sun and Jeff Orchard. A predictive-coding network that is both discriminative and generative. Neural Computation, 32:1836- 1862, 2020. UtLhttps: www.appssssmmannscchocsssssss org/
CorpusID:22111171
Christoph Teufel and Paul C Fletcher. Forms of prediction in the nervous system. Nature Reviews Neuroscience, 21 2144)2331225 2020.
Simon Thorpe, Denis Fize, and Catherine Marlot. Speed of processing in the human visual system. nature, 381 (6582))))5520 1996.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng. and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. 2024.
Alexander Tscshantz, Beren Millidge, Anil K. Seth, and Christopher L. Buckley, Hybrid pre- dictive coding: Inferring, fast and slow. PLOS Computational Biology, 19(8):1-31, 08 2023. doi: 10. (0..33//jurraa..chh..00 1280. http::/ http:::://oo..ogg/101111///oo1 pebi 1011280
Robert Beers, Anne Sittig, and Jan Gon. Integration of proprioceptive and visual position- information: An experimentally supported model. Journal of Neurophysiology. 81(3):1355-1366 1999.
James CR Whittington and Rafal Bogacz. An approximation of the error backpropagation algorithm in predictive coding network with local hebbian synaptic plasticity. Neural Computation. 29(5):
1229-1262, 2017. doi: 10.1162/NECO) a .0949. URL /ttp:::://ci..rr/10.11111
NECO_ 00949
Daniel M Wolpert, Zoubin Ghahramani, and Michael Jordan. An internal model for sensorimotor integration. Science, 269(5232):188), 1995.
David Xu, Andrew Clappison, Cameron Seth, and Jeff Orchard. Symmetric predictive estimator for biologically plausible neural learning. IEEE Transactions Neural Networks and Learning Systems, 29(9):4 29(9):4140-411 4151-4151, 2017.
Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. ieftommance-opiimiae hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy sciences, (11(2))88199866) 2014.
13

---

Bin Yang and Xinggang Wang. Reconstruction generation: Taming optimization dilemma in latent diffusion models. pp. 15703-15712, 06 2025. doi: 10.110/CCVP5523322000.044444
llker Yildirim, Max H Siegel, Amir A Soltani, Shraman Ray Chaudhuri, and Joshua B Tenenbaum. Perception of 3d shape integrates intuitive physics and analysis-by-synthesss. Nature Human Behaviour, 8(2):320-335, 2024.
Umais Zahid, Qinghai Guo, and Zafeirios Fountas. Sample as you infer: Predictive coding with Langevin dynamics. Ruslan Salakhutdinov. Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 58105-58121. PMLR, 21-27 Jul 2024. URL v235/ / zahid2 a html
mlr. htt.pst.....ss./.// press/
14

---

SUPPLEMENTARY MATERIAL
The supplementary material is organised as follows:
We begin with a detailed explanation of the methods underlying the experiments presented in main paper. Next, provide summary tables containing the exact values plotted in the figures. We then explore the learning performance of three additional predictive coding models. Following that, examine the effect of scaling the discriminative and generative energies in bPC on both super- vised and unsupervised learning performance. We also analyse the impact of removing max-pooling on generative artefacts. We present samples generated by combined discBP and genBP model. Af- ter, we perform parameter count matched experiments to validate our results. We then discuss the relationship between hybridPC and bPC. Finally, evaluate whether bPC learns effectively in deep models.
A DETAILS OF EXPERIMENTS
Here, we detail the training setup and the evaluation procedure of the experiments in the paper. The code with all models and experiments will be available upon publication Our implementation of the predictive coding models was adapted from Pinchetti et al. (2025a).
A.1 DATASETS
We evaluate the models on four standard image classification benchmarks: MeCST al. 2010), Fashion-MNIST(Xiine et al. 2017). CIFAR-10, and CIFAR- (RRR1000(Krizeess/ (009a). Below, we summarise their properties and how they were subdivided in our experiments for training. vali- dation and testing. The images of all datasets were normalised so that pixel values varied between and 1.
MNIST: dataset grayscale handwritten digits (0-9).
Fashion-MNIST:: A dataset of grayscale images of clothing items.
CIFAR-10: A dataset of colour images categorised into 10 classes of everyday objects.
CIFAR-100: Similar to CIFAR-10, but with 100 fine-grained object categories.
Table 1: Summary of dataset characteristics. The evaluation set of each dataset split 50/50 into validation and test subsets.
Dataset
Image Size
Channels
Classes
Train
Validation
Test
MNIST
28 28
10
60,000
5,000
5,000
Fashion-MNIST
28 x 28
10
60,000
5,000
5,000
CIFAR-10
32 32
10
50,000
5,000
5,000
CIFAR-100
32 x 32
100
50,000
5,000
5,000
In addition to these benchmarks, we also consider the XOR learning task. XOR is simple, non- linearly separable binary classification problem, often used to test the capacity of neural networks to learn non-linear functions. The input-output relationship of XOR is given in Table 2
Table 2: XOR truth table with scaled inputs.
Input
Input 2
Output (XOR)
-1
-1
I
I
-1
1
1
-1
15

---

A.2 MODEL ARCHITECTURES
We consider five different neural architectures in our experiments, selected based on the dataset and the learning setting.
MLP for MNIST and Fashion-MNIST For MNIST and Pashion-MNIST in Sections 4.1 and 4.3
We use network with neural layers of dimension [784, 256, 256, latent_size]. The latent layer size is set to 10 for supervised experiments, 30 for unsupervised experiments, and 40 for experiments with partial clamping of the top-layer activity (xz)) Both generative and discriminative predictions between layers follow the form f(Wxx + 6), where f is a nonlinearity, W is a weight matrix, b is a bias vector, and x is the activation of the previous layer. This is the transformation of a layer in multilayered perceptron (MLP). Consequently, unidirectional models (discPC, genPC, discBP, genBP) have the architecture of an MLP. hybridPC and bPC have the same number of parameters as two MLPs (one generative and one discriminative).
CNN for unsupervised learning with CIFAR-10 and CIFAR-100 the unsupervised representa- tional learning experiments (Section 4.2) on CIFAR-I0 and CIFAR- 100, we use model architecture with convolutions and a representational layer æy = 2g of 256 neurons. In this model, discrimi- native predictions consist of 5 strided convolution layers, with a stride of two, and one MLP layer. Generative predictions mirror the discriminative layers with 5 transposed convolution layers, with stride of two, and one MLP layer. The convolutional- baase layers perform the transformation I' (c(x)), where e is a (transposed) convolution layer and f is a nonlinearity. A full description of the nonvouuiionbbaee layers can be found in Table
VGG-5 for combined supervised and unsupervised learning with CIFAR-10 For the CIFAR-10 10 experiments that combine supervised and unsupervised representational learning in Section 4.3 we use a VGG-5-style network with a 266-dimensional latent layer ry zg (10 neurons for class labels and 256 for representations). layers with max pooling and one MLP layer. The generative predictions consist of 4 transposed convolution layers and one MLP layer. The discriminative convolution layers have stride of one, and the transpose convolutions have stride of two to compensate for the dimensionality reduction of the max-pooling. A full description of the convolution-bant
this model, discriminative predictions consist of 4 convolutional
layers can be found in Table
For the above three models, the activation function used in the discriminative prediction from TL-1 to Z is an identity layer (no activation). The generative prediction from layer 22 to I uses a tan/(()) activation to constrain outputs to the range [-1, 1], matching normalised pixel values. The other activation functions are the same for the whole network and are determined using a hyperparameter search.
MLP for XOR Task For the XOR task in Section we use a network with neural layers of dimension [2, 16, 16, 1). The generative and discriminative predictions equivalent to the ones used for MNIST and Fashion-MNIST, with the exception that: a sigmoid activation is used for the prediction from 23 to I4 and an identity activation is used for the prediction from 22 to Tj. These changes stabilise training and help avoid bias because the tanh activation used for (Fashion-JMNIST saturates near +1, but never reaches these values, causing the model to systematically under-predict XOR inputs.
MLP for Bimodal Learning Task In the bimodal generation task of Section 4.5 the network con- sists of two input layers, with 784 (image) and 10 (label) neurons, and one latent layer with 256 neurons. The predictions between the latent layer and the input layers are MI. layers. The bimodal bPC model is equivalent to the bPC model trained on (rashion-MMNISS only has one hidden layer. In contrast, the bimodal genPC is different from the models considered unidirectional model with one generative MLP layer
in Section 4.1 except it
in Section 1 and it can be re-interpreted and one discriminative MLP layer. For fair comparison, we include an additional nonlinearity in the prediction layers of the genPC variant. The MLP layers become fi fz(x) + b). For the image modality, fi = tanh(-); for the label modality, fi is the identity. The additional nonlinearity, f2. ensures parity with bPC models, which include nonlinear discriminative projections.
16

---

Table 3: Detailed architectures of convolution-base models. Convolution and transposed convolu- tion have the same kernel sizes and paddings. The values in the brackets give the parameter value for each convolution-base layer the model, starting from the layer closest the input image.
CNN
VGG-5
Channel Sizes
[32, 64, 128, 256, 512]
[128, 256, 512, 512]
Kernel Sizes
[3, 3,
[3, 3, 3, 3]
Paddings
1,
1]
Strides conv.
[2, 2, 2, 2,2]
[1, 1,
Strides transposed conv.
2, 2, 2.2]
[2 2,
Output Paddings transposed conv.
[1. 1,
[0. 0,
Pooling Window
2
Pooling Stride
2
Algorithm 1: Training procedure of PC models.
Require: Model with neural activities x, parameters 0, energy and initialisation function
init(-). Dataset {up) with P mini-batches of B elements. Number of epochs N. Activity optimiser o-t number of activity updates K, and parameter optimiser o-tima(-)
for 1 to N do
for p = 1 to do
// Independent inference for each sample in batch 15b5
26 < (y, for 1 = to K do
2b - optimx (an
// Sum of parameter updates for batch
0 + optimol B3 Σb 8E
00
A.3 TRAINING PROCEDURES
We followed the procedures outlined in Algorithm train predictive coding models and hybridBP, and the procedure outlined in Algorithm 2 for the remaining backpropagation models. For a given task, both algorithms trained the same dataset for the same number of epochs.
Optimisers
The optimisation procedures differ between activity updates and parameter updates. In Algorithm I neural activities are updated using stochastic gradient descent with momentum. This optimiser outperforms Adam in PC models (Pinchetti et al. 2025a). In contrast, Algorithm 2 not include activity updates and only optimises parameters. For parameter updates in both algorithms, we use the
Algorithm 2: Training procedure of BP models.
Require: Model with parameters 0 and forward pass fp(., f). Energy function E. Dataset {2p PP with P mini-batches of B elements. Number of epochs N. Parameter optimiser o-time(-)
for n 1 to N do
for p = 1 to P do
// Independent forward pass for each sample in batch in <- fp(un,, 0), 1 B
// Sum of parameter updates for batch
0 + oplimo B
ΣB (E(r..v.a) 80
17

---

AdamW optimiser (Loshchllov and Hutter 2017). The optimiser hyperparameters were determined via a hyperparameter search.
Initialisation functions
The initialisation procedure in A generally consists of (1) clamping the activity x1 and æt to images and labels depending the learning task and (2) performing feedforward sweep. This sweep initialises the activity of each layer by propagating inputs through the model in the direc- tion of prediction. For example, bottom-up feedforward sweep for three-layer model initialises [(x]] and æa VV2 [(x) This approach, originally
the second and third layers as: x2 proposed by WWittington and Bogacz (217), significantly reduces the time required for inference and improves learning. The following models use bottom-up feedforward initialisation: discPC, hybridPC. bpC, and hybridBP. The genPC model. by contrast, uses top-down feedforward initial- isation. The difference between hybridBP and the other models initialised using bottom-up sweep that hybridBP contains a single layer of neural activities, IL, that can be initialised using a full bottom-up forward pass.
The bimodal genPC model from Section 4.3 differs from the other PC models in its initialisation procedure. Because there are direct prediction paths from the two mopelit input layers to the shared latent layer, standard feedforward sweep cannot be used to initialise the latent layer. Instead, we tested two alternatives for initialising the latent layer: vero-initialisation and Xavier uniform initialisation. The best initialisation depended on the task and was determined with a hy- perparameter search. For the bimodal bPC model, we retained the feedforward sweep used for other bPC models, using a forward sweep from the image modality to the latent layer.
Energy functions
The energy functions for all PC and BP models are listed in Table For PC models, the energy functions follow the formulations deseribed in the main text, except that use generalised nota- tion: fi denotes the transformation applied at layer 7, allowing for models beyond standard MLPs. BP model energy functions use squared errors to remain consistent with PC formulations.
The hybridBP model includes a stop-gradient operation in its energy function. This ensures that its second loss term is only applied during parameter updates, not iterative inference. This loss term objective is to improve the discriminative initialisation of the model's latent layer. In the broader machine learning literature, hybridBP can be viewed as a generative model that performs Bayesian inference by combining amortised inference with iterative refinement of the latent variables (Marino ct al. 2018).
In the case of the autoencoder (AE), the energy function departs from standard autoencoders to accommodate experiments described in Section 4.3 where part of the latent state Ty fixed fp.dise (x), V)ll where F1.ace = 1 (1r, k>] that combines & fixed neurons and discriminative predic- tions. This term penalises the error between the fixed components of LL,oe and their prediction from the discriminative part of the model. Notably, this term is only non-zero for the fixed neurons. The additional energy term is also equal to zero in the representational learning experiments of Section 4.2 because none of the neurons of LLae are fixed. For reference, the discriminative part of our AE usually referred to as the encoder and the generative part the decoder.
to class labels. The AE energy includes an additional term: ||xr,,0
In the combined supervised and unsupervised learning task (Section 4.3), we apply separate energy scaling to different parts of the models. Unsupervised learning benefits from equal weighting of discriminative and generative energies, while supervised tasks perform better when the generative energy is down-scaled. To balance these, we set Adise of the free (unclamped) neurons in 2y to a'gen This allows for the generative energy to be down-scaled in the whole network except for the free neurons of xy that specialise in unsupervised representational learning. This adjustment was used for bPC, hybridBP, and AE, but not for hybridPC. In hybridPC, the discriminative weights predicting free and fixed neurons are independent due to the use of local learning rules. Moreover, these discriminative weights have minimal influence on the generative weights, as inference is fully driven by the generative energy. As a result, scaling the discriminative and generative energies in different parts of the model merely scales the corresponding gradients. This effect has impact learning because the Adam optimiser normalises gradicnts. In contrast, in hybridBP, some parameters are shared in predicting both free and fixed neurons. This introduces dependency between the scaling
18

---

of energies associated with free and fixed neurons in LL. Therefore, introduced explicit scaling factors Adisc and Ogen in hybridBP's energy function for this task.
Model
Loss Function
disePC
Edise 2 V) = El=z I|l - f1( Vi-1)
discBP
1 V) 21101 Sp.disc (21, V)
genPC
1 1|10= Egen (2r, W) = =
- f1 11+1)ll
genBP
(21, W) ||ا fp.gen (IZ, W)lB
hybridPC
Enybria (Ir, W, V) = -
L-1 2 |25 (1 W4+1)ll Ei= & (s((s)) [((g((1)11), V-11
hybridBP
Lnybrial (21, R1, W,V) || p. (xr., W)1 + 1 llsg((r)) Sp,dis V)
bPC
E(x, W, V)= = E1=1 1 3pm [1| f W )l + 2 aiu |,| f(x1-11 1
2
AE
L(8,, LL, W, V)= = | - Sp.gen LL
W) + ass 2 | - Sp.disc V)
LL,0 = [0L,0:1-1, fp.dio (x1, V) R:
Table Energy functions for the predictive coding and backpropagation models considered in the paper. In the PC models. (txx 0) is the transformation between layers with input v and parameters 0. In the BP models, Sp denotes forward pass and FL,ae is an activity vector composed of l: fixed neurons and fills the remaining outputs of the discriminative forward pass of the AE.
A.4 TRAINING HYPERPARAMETER
In this Section, we report the hyperparameter search space for the experiments in Sections 4.1 4.2
and 4.5 as well as the training parameters for the models in Section 4.4 All hyperparameter tuning was performed using Bayesian optimisation from Weights and Biases (Biewald 2020).
Tables 6 and list the hyperparameter search spaces for the models trained in Sections 4. and respectively. The hyperparameter search was conducted using the validation sets of each dataset. In Section [. discPC and genPC were tuned for classification accuracy and generation RMSE separately, while bPC and hybridPC were jointly tuned on both metrics. This ensures fair comparison, as bPC and hybridPC have twice as many parameters as discPC and genPC. For joint tuning, we combined the two metries using the objective 2 (1 accuracy /0npaaayyy100/aaaa + RMSE. In Section 4.2 the models were tuned for the image reconstruction MSE, linear decoding accuracy and gener- ation FID separately. For the reconstruction, the RMSE was reported instead of the MSE for consis- tency with the generation RMSE. In Section 4.3 all models were jointly optimised for classification accuracy and reconstruction MSE. We combined the two metries using 2- accuracy / 1100 MSE for MNIST and MNSITOn-MNIST. and (1 accuracy (ouuracy/100)- + 4. MSE for CIFAR-10 10. The RMSE was also reported after tuning instead of the MSE. The weightings in the combined objectives compen- sate for the scale difference between the metrics.
Table 8 reports the training parameters for the models used to generate MNIST samples Section The leaky ReLU activation function was used because it generated the best samples across model types, and the other parameters were set to default values that ensured stable learning across models.
4.4
Table 9 lists the hyperparameter search space for the bimodal bPC and genPC models trained in Section 4.5 The genPC model was tuned separately for classification accuracy and reconstruction RMSE, while the bPC model was tuned jointly on both metrics. The joint tuning followed the same combined metric as described above for MNIST. During training and evaluation, more inference steps were used for genPC than bPC, as genPC lacks an effective foedforward initialisation scheme, which slows inference convergence.
19

---

Table 5: Hyperparameter search configuration for experiments in Section 4.1 for both MNIST and Fashion-MNIST datasets.
Parameter
bPC
hybridPC
discPC
genPC
genBP
discBP
Epoch
25
Batch Size
256
Activation Irx momentum Irg
[leaky relu, tanh, gelu]
(Ie-3, 5e-1)²
(Ie-4, 5e-2)²
[0.0, 0.5, 0.9]
(1e-5, le-3)²
(1e-6, le-4)a
(le-5, 1e-3)²
weight decay,
(le-5, le-2)²
T
T eval
100
Ogen
1e[0,-4:-1]
Odisc
:: "[a, b; cJ" denotes sequence of values from with step size of 2; "(a. b)" represents a log-uniform distribution between a
Table 6: Hyperparameter search configuration for experiments in Section 4.2 for MNIST, Fashion- MNIST, CIFAR-10, and CIFAR-100 datasets. Parameters unique the models trained MNIST and FasSion-MNIST are indicated with MLP. Parameters unique to the CIFAR datasets are indicated with CNN.
Parameter
bPC
genPC
hybridPC
hybridBP
AE
Epoch (MLP)
25
Epoch (CNN)
50
Batch Size Activation Ira momentum, Iro (MLP)
256
[leaky relu, tanh, gelu]
(Ie-3, 5e
[0.0, 0.5, 0.9]
(le-5, le-3)1
Tro (CNN)
(le-5, le-3)1.2 (le-5, le-2)
weight decay,
(MLP)
T (CNN)
32
eval
100
Ofgen
1
Oldiec
1; "(a, b)" represents log log-uniform distribution between and
Learning rates of were scaled with varmup-cosine-anneell scheduler without restart.
A.5 EVALUATION
In our experiments, we consider six different evaluation procedures discussed below.
Classification accuracy We evaluate classification performance using accuracy, defined as the per- centage of input images correctly classified by each model.
For the PC models, classification is performed by first setting x1 to the input image. We initialise the remaining layers using a bottom-up feedforward sweep, followed by 100 steps of iterative inference. The predicted class determined by identifying neuron the output layer æy,. (corresponding to class encoding units) with the highest activity. The bottom-up initialisation significantly reduces the time required to reach steady state during inference. However, bottom-up initialisation does not apply genPC and bimodal genPC due the absence of bottom-up predictive pathways. In genPC, we initialise E to zero and perform a top-down feedforward sweep instead. For bimodal genPC, the latent layer is initialised identically to the procedure used during training.
For the BP models, classification is performed via a standard discriminative forward pass from the input image, with the predicted class taken as the one corresponding the highest output activation.
20

---

Table 7: Hyperparameter search configuration for experiments in Section Parameters unique to the models trained on MNIST and Fashion-MNIST are indicated with MLP. Parameters unique to the CIFAR-10 dataset are indicated with VGG. Two activity optimisers and two parameter optimisers are used for the VGG models. This allows the models to have different learning rates for different task-specific parts of the networks. One activity optimiser was used for the free neurons of æ with learning rate free, and another for all other neurons with learning rate Ir.. Both activity optimisers use the same momentum parameter. One parameter optimiser with learning rate tre.gen was used for the generative parameters and the discriminative parameters predicting the free neurons of zy. The other parameter optimiser was used for the remaining discriminative parameters. Both parameter optimisers the same weight decay.
Parameter
bPC
hybridPC
hybridBP
AE
Epoch (MLP)
25
Epoch (VGG)
50
Batch Size
256
Activation
[leaky relu, tanh, gelu]
lr
(Ie-3, 5e-1)²
Ir free (VGG)
(Ie-3, 5e-1)²
momentum
[0.0, 0.5, 0.9]
Iro (MIP)
(le-5, 1e-3)1
trsee (VGG)
(le-5, le-3)
ggen (VGG)
(le-5, 1e-2)
weight decay,
(le-5, 1e-2)²
T (MLP)
8
T (VGG)
32
eval
100
agen (MLP)
lc[0,-4:-1] le 1e[-4,-7:-1]
le[0,-4:-1] le(0,-8:-1]
le[0,-4:-1]] le(0,-8:-1]'
gen (VGG)
Oddisc
"(a, b)" represents a log-uniform distribution between a and b. 2. "[a, b; c]" denotes sequence of values from with step size of Learning rates of were scaled with warmup-cosiin www.-ooannnnneiiiinneeee scheduler without restart.
Table Hyperparameters for models used to sample MNIST images in Section
Parameter
bPC
genPC
discPC
Epoch
25
Batch Size
256
Activation
leaky relu
Ir. momentum (re
0.01
0.0
le-4
weight_decay, T
5e-3
T eval
100
Agen
le-4
Odisse
1
Conditional generation of mean class images We evaluate the generative performance of the mod- els by computing the root mean squared error (RMSE) between images generated for each class and the corresponding class-average image. We obtain the class-average image by computing the average image across all images belonging to that class in the evaluation set. To compute the RMSE, calculate the squared error for each pixel between the generated image and the average image for the corresponding class. These errors are averaged across all pixels and classes, and the square root of this mean is reported as the final RMSE.
21

---

Table 9: Hyperparameter search configuration for bimodal models in Section 4.5 for both MNIST
and Fashion-MNIST datasets.
Parameter
bPC
genPC
Epoch Batch Size
25
256
Activation
[leaky relu, tanh, gelu]
lrz momentum
(Ie-3, 1)
[0.0, 0.5, 0.9]
Irg
(Ie-5, 1e-3)²
weight _dcauy
(le-5, 1e-2)²
T
20
T eval
100
1000
le(0,-4:-1]
Ogen
Odisc
1: "la, b; el" denotes sequence of values from with step size of 2; "(a, b)" represents a log-uniform distribution between a
To generate an image from a PC model conditioned on a class label, we set the top-layer activity Iz to the one-hot encoding of the target label. The remaining layers are then initialised using a top- down feedforward sweep. One exception is the discPC model, which does not support top-down initialisation. For discPC, we initialise the input layer 21 to zero activity and then perform a bottom- up feedforward sweep. Following initialisation, we run 100 steps of iterative inference and extract the final activity of the input layer xy as the generated image.
For the BP models, we generate an image by performing a generative forward pass starting from one-hot label vector.
Image reconstruction from representations We assess the quality of learned representations by measuring the RMSE between reconstructed images and their original inputs. RMSE computed in the same manner for conditional image generation.
For PC models and hybridBP, the reconstruction procedure is as follows: (1) clamp æ to the input image, (2) initialise the remaining layers via a bottom-up feedforward sweep, (3) perform 100 steps of iterative inference, (4) clamp IL to its activity after the inference, (5) re-initialise the other layers using top-down feedforward sweep, (6) run 100 additional inference steps. and (8) record the final activity in 21 as the reconstructed image.
For the autoencoder, image is reconstructed by performing discriminative forward pass using the input image to obtain a representation, followed by generative forward pass from the representation get the reconstructed image.
In Section 4.3 where labels are also provided during reconstruction, the above procedure is slightly modified. For PC models and hybridBP, certain neurons of xı. additionally clamped to the ground truth label during the first step. During the fourth step, the label components and the re- maining latent representation (recorded after inference) are clamped æg. For the AE, we retain the standard discriminative forward pass, but replace the predicted label portion of its representa- tion with the true label before passing it through the generative model to produce the reconstructed image.
Linear readout/decoding accuracy We further evaluate the quality of the learned representations by measuring their linear readout accuracy. The representations are obtained in the same way for image reconstruction. A linear classifier head then trained with backpropagation for each model to classify images based on their representations. This evaluation tests whether representations of different classes are linearly separable. Higher decoding accuracy reflects better representations.
FID of generated image samples. We compare the ability of bPC, genPC, and hybridPC models to learn probability distributions using stochastic extensions of predictive coding. Predictive coding can be equipped with stochastic neural dynamics model distributions of sensory inputs in an unsupervised manner by injecting noise into the inference process. This idea was originally proposed in (Oliviers et al. 2024) for the genPC model, and here we apply it genPC, hybridPC, and bPC.
22

---

When stochastic dynamics are applied, the noisy inference process can be shown to generate samples from the posterior distribution of the probabilistic model defined by predictive coding, conditioned on the sensory input. These posterior samples can then be used for parameter learning through Monte Carlo Expectation Maximisation.
The training procedure mirrors that of the deterministic experiments: each iteration consists of sev- eral (now noisy) activity updates followed by a single parameter update. This is equivalent to using a single posterior sample per parameter update, a common practice in related generative models such as variational autoencoders.
To accelerate inference, we further incorporate momentum into the neural dynamics, introduced for predictive coding in (Pinchetti et al. 2025b). The resulting dynamics correspond to discrete- time second-order Langevin process. In practice, this is implemented by adding Gaussian noise to the gradients of the energy function and updating activities with stochastic gradient descent with momentum:
Ar.- Ir
An = Ir Ei
m)ni+ 2(1 m)br N.
(8)
After training, the models can be used to generate input samples. For a top-down predictive coding model, this is done by ancestral sampling: first sampling the top latent layer Vr. (O, 1), and then recursively sampling each lower layer from its conditional Gaussian distribu- tion (1 (1 W1+1); I). The prior distribution on Fr, emerges from the additional activity decay applied to æ for the unsupervised learning experiments.
To evaluate generative performance, we compute the Fréchet Inception Distance between generated and real samples. We use the open-source library pytorch-fid (Seitzer 2020a), adapted to MNIST by replacing the standard Inception network with a ResNet- 18 trained on MNIST as the feature extractor.
As a bascline, we include a variational autoencoder (VAE) trained on the same learning task. The VAE architecture and parameter count are matched to those of hybridPC and bPC to enable fair comparison. Training follows the standard VAE procedure: the encoder network approximates the posterior distribution over latent variables, while the decoder parametrises the generative model. The model is optimised end-to-end via backpropagation using the standard VAE objective.
Visualisation of Energy Landscapes on XOR We visualise the energy landscapes of trained genPC, discPC, and bPC models on the XOR task. To generate the landscape, we clamp the in- put layer neurons to 2D coordinate within the range [-3,3], sampled at intervals of 0.25 along both axes. Simultancously, we clamp the output layer æy to the one-hot encoding of one of the two classes. For each coordinate-lad label combination, we 10,000 steps of iterative inference to ensure convergence to equilibrium and record the final energy of the model. This procedure is repeated over the grid of 2D inputs and both class labels, allowing to plot the full energy landscape. For fair comparison, we also include the combined energy landscape of discPC and genPC. Together, the combined parameter count of discPC and genPC matches the total parameter count of bpC. Equal weighting is applied to the energy values of the discPC and genPC models when adding their en- engy, as we use equal weighting for the discriminative and generative components in the bPC model (Oidise agen 1).
MNIST image generation with PC models We evaluate the generative capabilities of the bPC model and combined genPC and discPC model on the MNIST dataset. The goal is to assess how likely these models are to assign low energy to implausible, label-inconsistent samples.
As a baseline, we first estimate the energy distribution over the test set for each model. For each test image, we clamp 2 to the image and Fy, to its associated label. The model is then initialised as during training, and we run 50,000 steps of iterative inference to reach steady state. We compute the 10th percentile from the resulting energy values as an energy threshold for high-quality, in- distribution samples. This process was repeated independently for bPC, genPC and discPC.
To generate images conditioned on a label, we use the following procedure: (1) randomly initialise the input layer 21 by sampling each neuron's ''s activity uniformly from the interval [-1,1], (2) clamp Fr. to the target label, (3) initialise all hidden layers to zero, (4) run iterative inference until the median energy of the batch falls below the previously determined 10th percentile threshold, (5)
23

---

retain the 50% of samples within the batch that fall below equal to the energy threshold and discard the rest. Using this method, we generate 256 samples per model.
In the combined genPC and discPC model, the input layer 21 is shared by both models, and its up- dates during inference are influenced by both energy functions. However, genPC typically exhibits energy magnitudes significantly larger than those of discPC. To prevent nating the updates to 2i, we scale the generative energy by a factor equal to the ratio of the discPC energy threshold to the genPC energy threshold. Additionally, inference is only terminated once the generative and the discriminative energies fall below their respective thresholds.
component from domi-
The models used for sample generation are trained separately from those used in the supervised experiments from Section[4.1 During preliminary testing, we found that generative quality strongly depends on the choice of activation function. In particular, leaky-ReLU yielded superior image samples. As a result, we train all models using leaky-ReL.U activations and shared set of training hyperparameters. These parameters are selected for their stability and effectiveness across all model types for classification and generation tasks.
We assess the quality and diversity the generated samples using two standard metrics:
Fréchet Inception Distance (FID): The FID measures the distance between the distributions of real and generated images (Heusel et al. 2017). We use the public implementation of ResNet classifier trained on MNIST,
FID from [Seitzer (2020b), but modify to instead of the original Inception model trained on ImageNet. This ensures that the FID score better reflects visual quality and diversity on the MNIST domain.
Inception Score (IS): The Inception Score evaluates how easily classifier can identify the class of generated image (Salimans al. 2016). We used publicly available implemen- tation for the MNIST dataset (Chen 2020
Classification accuracy with partially missing inputs
To assess model robustness to missing input data, we evaluate classification accuracy under varying levels of input occlusion. This evaluation follows the same procedure as standard classification (described above), but for images missing random subset of pixels. The proportion of missing pixels ranges from 10% to 90%, in increments of 10%. Missing pixels are selected uniformly at random, independently of their spatial location within the image. For all models, missing pixels in 21 are initialised to zero activity and are left unclamped, allowing the model to update their values during inference. We repeat the same classification procedure before, but we increased the number of inference steps to 600,000 to ensure convergence. This is necessary because predictive coding models converge more slowly when input information incomplete (Tang et al., (023). In addition to measuring classification accuracy, we also record the post-inference activity of neurons in Ti to qualitatively assess the model's ability to fill in the missing input.
A.6 COMPUTE RESOURCES
All experiments were conducted on NVIDIA RTX A6000 GPUs Training an MLP model for MNIST and Mushion-MNISS experiments across tasks took less than one minute. Training unsuper- vised learning models on CIFAR-10 and CIFAR-100 of Section 4.2 approximately 15 minutes. Training the combined supervised and unsupervised models of Section 4.3 took approximately one hour and 15 minutes. The majority of the compute was spent on hyperparameter tuning. The total training time for hyperparameter tuning of the models of Scction|4.1p +50h. The total training time for hyperparameter tuning of the models of Section 4.2 is +300 for MNIST and Fashion-MNIST and +:00h for CIFAR-10 and CIFAR-100. The total training time for hyperparameter tuning of the models of Section 4.3 is 25h for MNIST and Fashion-MNIS" and +750h for CIFAR-10. The total
training time for the tuning of the bimodal models of Section 4.5 is 22hhh.
A.7 LLM USE
ChatGPT Edu was used to polish writing.
24

---

B RESULTS
These results are obtained on Tables 10 to 15 report the result illustrated in Figures 4 the test set of the datasets for five different weight initialisations.
5 and 7
Table 10: Classification accuracy and class average image generation for models considered in Section 4.1 Higher accuracy and lower RMSE are better. We report the mean +/- sem over five seeds. Results indicated with are significantly worse in performance than bPC determined using an independent-samples I-test (n=5, p < 0.05).
Acc %
RMSE
Model
MNIST
FaSHion-MNIST
MNIST
Fashion-MNIST
bPC
98. 98.0055
89. 0.
0.0581 00000
0.0415=+0.0005
hybridPC
86.22 86.22=0.156
80.34+0.111
0.0612=0.0003
0.0480+000011
genPC
83. 83.48:t0.21+
77.00
0.0198=00.000
0.0140 0.0140 40.0001
disePC
98.43=0.01
89. 74
1.3133=0.02244
0.3326=0000022
BP
98.48=0.10
89.66+0011
0.0198=0.0001
0.0128
Table 11: Image reconstruction RMSE from latent representations for models considered in Section Lower RMSE is better. We report the mean +/- sem over five seeds. Results indicated with are
significantly worse in performance than bPC determined using independent--aamples t-test (n=5,
p < 0.05).
Model
MNIST
Fashion-MNIST
CIFAR-10
CIFAR-100
bPC
0.2320+0.0010
0.2497=0.000x
0.1311=0.0000
0.1366+0.0000
genPC
0.2473=000020+
0.2068=0000013
0.1837=0.0009.
0.2077=0.0003
hybridPC
0.2401 =0.0012+
0. 0.2508=0.00m
0.166440.0011
0.2089=0.0071
AE
0.1565+000000
0.1 1.18888=0000 0.0001 0001
0.1135+0.0050
0.1171 0000042
BP
0.1969=000000
0.20844+0.00
0.0964+0.0004
0.0983+0.0002
Table 12: Linear decoding accuracy (%) across datasets for different models considered in Section
Higher better. We report the mean sem over five seeds. Results indicated with are
significantly worse in performance than bPC determined using independent-samples t-test (n=5,
p < 0.05).
Model
MNIST
Fashion-MNIST
CIFAR-10
CIFAR-100
bPC
89.99=0000
81.82
50.52=0.01
00.68=0.01
genPC
86.93=0.01
80.44
48. 48.23-10.02.
50.45-10.02+
hybridPC
89.0740.01.
81.59
49.61 10.01
59.31 1.0011
AE
90.76 00.761
83.31 10.01
47.29 00002
63.78-0.04
hybridBP
93.44+0.01
78.69+0.018.
50.63=0.01
85.14=0.01
Table 13: FID scores for different models trained with 50 and 250 activity updates before each
weight update considered in Section[4.2 Lower better. We report the mean + sem over five seeds. The VAE only has one value because it does not have iterative inference.
Model
FID @ 50 updates
FID 250 updates
bPC
5.21 #0.26
+.34
genPC
7.80
4..30
hybridPC
5.01 +0.30
+0.32
VAE
5.79
25

---

Table 14: Classification accuracy and image reconstruction RMSE from latent representations for models considered in Section 4.33 Higher accuracy and lower RMSE are better. We report the mean +/- sem over five seeds. Results indicated with are significantly worse in performance than bPC determined using an independent-samples t-test (n=5, p < 0.05).
Acc %
RMSE
Model
MNIST
Fashion-MNIST
CIFAR-10
MNIST
Fashion-MNIST
CIFAR-10
bPC
97. 97.06+#0.08
88. 28.00
85.06#0.11
0.2394=000000
0.2531 000002
0.3036#000014
hybridPC
16.336
80.11 =0.11
37. 07.40
0.2418#00.000
0.2527=20.000
0.300g=000055
AE
98.32=#0.06
89.49=00110
89.97#0.12
0.1576
0.2164400.0000
0.1599#+0000
hybridBP
98.48=0003
89.61 +0.13
90. 90.11#0.11
0.1249=000000
0.1707=#0.000
0.1048+00000
Table 15: Classification accuracy and class average image generation for bimodal genPC and bPC considered in Section [4.]] Higher accuracy and lower RMSE are better. We report the mean +/- sem over five seeds. Results indicated with are significantly worse performance than bPC determined using mudependent-samplle t-test (n=5, p < 0.05).
Acc %
RMSE
Model
MNIST
Aishion-MNIST
MNIST
MaSSiO
bPC
97. 77.80-0..0
89.05 10.10
0.0506 10.0007
0.0431 1.00004
genPC
88.2 88.28 1..100
80.53 1..14
0.1561 10.0004.
0.2027
C OTHER TYPES OF PREDICTIVE CODING MODELS
In this experiment, we train three additional types of predictive coding models on the supervised
task described in Section 4.1 (I) PC along arbitrary Cruphs(Salvator al. 2022). (2) bPC with
shared weights for discriminative and generative predictions Qiu et al. 2023 and (3) discPC with
activity decay during generation(Sum and Orchard 2020).
PC along arbitrary graphs (agPC) differs from bPC in that each neuron has a single energy func- tion. with predictions computed from all incoming connections. In contrast, bPC uses separate energy terms for bottom-up (discriminative) and top-down (generative) predictions. this formula- tion, the energy associated with agPC layer xq given by:
E | S(Wx++ + VxI- + b)1l
where W and are top-down and bottom-up weights, respectively. We use the same training algorithm, hyperparameter tuning, and evaluation procedure as bpC, modifying only the energy function. Initialisation is done using bottom-up sweeps (for training and classification evaluation) and top-down forward sweeps (for generation evaluation), matching bpC. We also tested zero and Xavier initialisation. However, this resulted worse learning performance.
bPC with shared weights (shared bPC) uses single set of weights for both discriminative and generative predictions. Its energy function associated with a layer oq given by:
E
Orgen
|x] f(W1+13)-1)13
Oldise
x - J(W x-1)
2
2
where W,T is reused for both directions. This model omits bias terms and relies on non-local com- putations. We follow the same training, tuning, and evaluation protocol as bPC.
discPC with activity decay (decay discPC) extends standard discPC by adding an activity decay term during generation. We use the same training and hyperparameter tuning protocol as discPC. Additionally, we tune an activity decay rate from a log- uniform distribution over the range [10-5, 1]. During generative evaluation, we increase the number of inference steps to 10,000 due slower convergence and we initialise neurons to zero activity before generation for consistency with Sun and Orchard (2020).
26

---

agPC
3
9
8
2
shared bPC
S
decay discPC
Figure 9: MNIST (left) and Fashion-MNIST (right) class average image generation for agPC., shared bPC and decay disePC.
reports classification accuracy and reconstruction RMSE for these models. Figure Iilus- trates the images generated by each model for each MNIST and Fashion-MNIST class. None of the models match bPC in both classification and generative performance.
Table 16
Table 16: Classification accuracy and class average image generation for agPC, shared bPC and decay disePC on the supervised training task of Section Higher accuracy and lower RMSE are better. We report the mean +/- sem over five seeds.
Acc %
RMSE
Model
MNIST
MaShiS
MNIST
MaSSion-MNIST
arbitrary graph PC
71.47:0.27
6..00
0. 1685510.0299
0.2426-40.0077
shared-bPC
97 10.008
87 97.9910.00
0. 13.100
0.256011.000 10.0003
decay discPC
0.4047:10.0005 0.0000
0.3433 0.3433-20.0000 10.00022
D BALANCING DISCRIMINATIVE AND GENERATIVE ENERGIES IN BPC
In this experiment, we investigate how the relative weighting of the discriminative and generative energy terms (adisee/ee /age)))//g)))))))) /agen) affects bPC's learning in the supervised and unsupervised settings de- scribed in Sections 4.1 and 4.2 For each setting, we perform hyperparameter tuning the bPC model while constraining the ratio Gddss//Uggns and report the test performance of the model with the optimal parameters. We use a grid search over the following hyperparameters: the activity learn- ing rate [0.01, 0.003, 0.001]. the parameter learning rate [0.001, 0.0003, 0.0001], and the weight decay [O., 0.0001, 0.0003, 0.001]. Other settings, including a GeLUU activation function and an activity momentum of O, constant. All remaining parameters, such the number of epochs, batch size, and number of inference steps during training and evaluation, are kept consistent with the experiments described in Sections 4. and
and 4.2
In the supervised case, generation RMSE remains stable across ratios of Oldise/ but classi- fication accuracy declines the discriminative energy down-scaled. We suspect this is due to mismatch in energy magnitudes. A typical discPC model has energy of around 0.1 for test data samples after training, while genPC has an energy of approximately 50. Thus, the generative energy dominates when Odise is not significantly larger than Ogen, leading to a poor classification like genPC models. contrast, bPC's unsupervised learning performance is relatively robust across wide range of ratios, but degrades sharply when the discriminative energy is too large. We also observed increased training instability when Ocjisc >> a'gen-
These results highlight the importance of appropriately scaling the energy terms based on the task. Hybrid tasks such as those in Section 4.3 require tailored weighting across the models to ensure effective learning. Future work could make the energy scaling learnable parameter. This change could make bPC learn optimal scalings autonomously.
E EFFECT OF MAX-POOLING ON GENERATIVE ARTIFACTS
In this experiment, we modify the VGG-5 architecture of the models from Section 4.3 by removing max-pooling layers and increasing the convolutional strides from one two. We also increase the
27

---

reconstructions RMSE
0.50
Classification error
96
0.06
RonnnRRMM RMSE
0.45
94
0.40
0.04
92
90
0.02
0.30
88
0.25
0.00
10
10-3
10-2
10-1
100
10
102
10-
100
102
103
agen/agic
agen/asic
Figure 10: bPC's supervised (left) and unsupervised (right) learning performance on MNIST de- pending on the relative weighting of its discriminative and generative energy term.
base
bPC
hybridPC
AE
hybridBP
Figure 11:
CIFAR-10 image reconstruction from latent representation and labels for models of without max-pooling.
Section 4.3
inference time during training 48 inference steps. All other aspects of the experiment remain unchanged.
As shown in Figure this change eliminates the checkerboard artefacts in the reconstructed images and improves reconstruction quality for both bPC and hybridPC. This improvement is reflected in the lower RMSE values reported Table However, removing max pooling reduces classification accuracy across all models, with bPC experiencing a drop of approximately 25%. Despite this, the overall trends remain consistent: bPC show a substantially better discriminative performance than hybridPC for comparable reconstruction RMSE.
Table 17: Classification accuracy and image reconstruction RMSE from latent representations for CIFAR-10 models considered in Section 4.3 without max-pooling. Higher accuracy and lower RMSE are better. We report the mean +/- sem over five seeds.
Model
Aec
RMSE
bPC
61.67..0.29
0.2178-10.004
hybridPC
37.36+0018
0.2196=0.0010
AE
84.58=0.11
0.1777=000000
hybridBP
8..29
0.1095-40.0018
F SAMPLE GENERATION OF COMBINED DISCBP + GENBP MODELS
In this experiment, we repeat the combined model image generation procedure from Section 4.4 using discBP and genBP. The two models share input layer, which is iteratively updated to min- imise their energy functions until the energy falls below the 10th percentile of each model's energy distribution the test set. We generate samples using the same procedure described previously
28

---

0
3
3
3
3
3
5 5
5
7
7
Figure 12: Samples generated for the combined discBP + genBP model.
for the discPC + genPC model. We compute the FID and Inception scores to quantify the generation quality.
Figure 12 the generated samples for different classes. While the images resemble the class averages, they also exhibit noise across pixels and less visual diversity than bPC samples. As shown in Table 18 the backpropagation baagd based combined model yields higher FID and lower Inception scores, indicating that bPC generates samples more consistent with the MNIST distribution than the backpropagation-basee approach.
Table 18: Fréchet inception distance and inception score for samples generated for the combined discBP genBP model compared to bPC and discPC + genPC. Results given for mean +/- sem for three seeds.
Model
Inception
FID
bPC
6.05=0.17
44.4+2.2
discPC + genPC 3.62-0..03
140.5
discBP + genBP
5.00
99.1
G PARAMETER COUNT MATCHED EXPERIMENTS
In this section, repeat both the supervised and unsupervised experiments for genPC and discPC, but with parameter counts increased to match bPC. Initially, we maintained identical neural layer dimensions across all models, causing genPC and discPC have approximately half the number of trained parameters (weight matrices and biases) compared to bPC and hybridPC.
Here, replicate the MNIST and Fashion-MNIST experiments described in the main paper, ad- justing genPC and discPC to match the parameter count of bPC. Specifically, for supervised exper- iments, the hidden layers of genPC and discPC are expanded to 437 neurons, compared to 256 in bPC. For the unsupervised experiments, genPC layers have 439 neurons. These neuron counts were determined by finding the hidden layer sizes (rounded to the nearest integer) that yield the same parameter count bPC.
The results, presented in Tables 19 and 20 show that the previous findings remain consistent when parameter counts are matched. GenPC continues exhibit lower discriminative performance com- pared to discPC and bPC, as demonstrated by its poorer classification accuracy. Additionally, genPC's unsupervised learning performance remains inferior to that of bPC and hybridPC. DiscPC, similarly, maintains poor generative performance in supervised tasks.
Overall, these experiments confirm that the main paper's conclusion remains valid: bPC effectively integrates the strengths of both genPC and discPC, even when genPC and discPC are scaled to match bPC's parameter count.
H RELATIONSHIP BETWEEN HYBRIDPC AND BPC
In this work, we benchmark bPC model primarily against hybrid predictive coding (Tscshantz ct al. 2023). hybridPC is the only plausible PC model that incorporates both slow, iterative inference and fast, feedforward inference, to date. In HybridPC, fast inference is enabled by bottom-up
29

---

Table 19: Classification accuracy and class average image generation for models considered in Sec- tion 4.1 All models have approximately the same number of trained parameters. Higher accuracy and lower RMSE better. We report the mean +/- sem over five seeds.
Acc %
RMSE
Model
MNIST
Fashion-MNIST
MNIST
Fashion-MNIST
bPC
98. 08.10
89. $9.24#0.12
0.0581=0.000
0.0415#0.000
hybridPC
86. 86.22-10.15
80.34 80.34#0.11
0.0612=0.0000
0.0480 0000016
genPC
63.09-00.8
78. 78.28=00.23
0.0196=0.0002
0.0142=00.000z
discPC
98 60.00
89.30=0.26
0.4085=0.0040
0.3209
BP
98. 98.66-10.06
89.84 10.08
0.0197-0.0000
0.0133 0.013340.00m 1:00001
Table 20: Image reconstruction RMSE from latent representations for models considered in Section
4.2
All models have approximately the same number of trained parameters. Lower RMSE better.
We report the mean +/- sem over five seeds.
Model
MNIST
Fashion-MNIST
bPC
0.2320=0.0010
0. 0.2497
hybridPC
0.2401 0.2401110.0012 110.0012
0.2508 L0.0007
genPC 0.2453+00.000
0.2861 0.2861+000016 +0.0016
network added to the genPC network architecture, which serves solely to initialise neural activities and does not impact their dynamics.
HybridPC was originally defined with two separate objective functions. One of these is the genPC energy function. The other is loss for the bottom-up initialisation parameters. However, the learn- ing process of hybridPC can also be expressed as a single unified loss function. This formulation analogous to the energy function in bPC. The neural dynamics and weight updates in hybridPC minimise the following energy function:
(9) Unlike bPC, a stop gradient sg(.) operation applied. This change ensures that only the top- down generative predictions drive the neural dynamics. HybridPC can perform both supervised and unsupervised learning. However, the supervised learning performance
Enybria W,V) = Σ t=1
E' -
Vi- (V-i)))-Vi-if((aaa
1-
hybridPC is much poorer than that of discPC. This is due to the stop gradient operation in the bottom-up stream of this model that prevents from learning a discriminative model that maps well from Ø1 to xy (Tscshantz et al. 2023). which caps its supervised learning performance to that of genPC (Tscshantz et al.) 2023).
In bPC, bottom-up predictions to contribute directly to the inference process, ensuring that the learned neural activity patterns incorporate discriminative signals from the bottom-up pathway, thereby significantly enhancing discriminative performance while retaining strong generative ca- pabilities.
With this reformulation of hybridPC, and given the similarity in objective functions, the bottom- up weight updates in bPC can be interpreted as learning inversion of the activity updates. This inversion provides mechanism for fast initialisation in bPC, implementing amortised inference.
I SCALING BPC TO DEEPER MODELS
To investigate whether bPC scales effectively to deeper models and more complex datasets, we trained discPC. discBP, and bPC using the following architecture-dataee combinations: VGG-5 on CIFAR-10, VGG-9 on CIFAR-100, and VGG-16 on ImageNe.
After training, evaluated their discriminative performance by measuring classification accuracy. Additionally, we assessed generative capabilities by evaluating classification accuracy when 30% and 50% of the input pixels were missing, following the methodology outlined in Section
30

---

Data Preparation We used the CIFAR-10 and CIFAR-100 100 datasets (Krizhevsky 2009b). For disePC and discBP, images were rescaled to the range [0. 1] and normalized using the mean and standard deviation shown in Table [1] consistent with Pincheeti al. (2025b). For bpC, images were rescaled to the range [-1,1] to align with the effective output range of the tanh activation function, which served the output activation for the top-down predictions across all bPC models.
Table 21: Data normalization
Mean (n)
Std (a)
CIFAR-10 CIFAR-100
[0.4914, 0.4822, 0.4465)
[0.2023, 0.1994, 0.2010]
[0.5071, 0.4867, 0.4408]
[0.2675, 0.2565, 0.2761]
Tiny-ImageNET
[0.485, 0,456, 0.406]
[0.229, 0.224, 0.225]
VGG Architectures We utilized deep convolutional neural network architectures from the VGG family (Simonyan and Zisserman 2014). Table 22 summarises the specific architectures for VGG-5, VGG-9, and VGG-16. Following convolutional layers, single linear layer was used for classifica- tion. We used the GeLU activation function. Batch normalisation was also used in the VGG-9 and VGG-16 models after each convolutional layer to stabilise training.
In the bPC models, the top-down architecture mirrored the discriminative (bottom-up) layers. Each convolutional layer was paired with a transposed convolution sharing identical parameters. How- ever, when convolutional layers were immediately followed by max-pooling operations, the corre- sponding transposed convolutional layers used stride of two and a padding of one (instead of zero) to compensate for the change in channel width and height introduced by max pooling. No batch normalisation was used in the top-down predictions.
Table 22: Detailed architectures of VGG models. The locations of the pooling layers correspond to the indices of the convolutional layers after which the max-pooling operations are applied.
VGG-5
VGG-9
VGG-16
Channel Sizes
[128, 256, 512, 512J
[128, 128, 256, 256, 512, 512, 512, 512]
[64, 64, 128, 128, 256,
256, 256, 512, 512, 512, 512, 512, 512]
Kernel Sizes
3,
3,
3,
33,
3. 3.
Strides
[1.
Paddings
1, 1.
1.
1. 1,
1.
1.
Pool location
[0, 2.
2.4.
I1A.
7.10.131
2x2
2x2
Pool stride
2
2
Learning Rate Schedule
The learning rate schedule was structured as follows:
1. During the initial 10% of training, the learning rate linearly increased from w to 1.1 x w w.rr.
2. Subsequently, cosine decay schedule reduced the learning rate smoothly to 0.1 wibr over the remaining epochs.
where refers the tuned weight learning rate.
Simulating PC using Error optimisation To efficiently simulate the predictive coding models, we employed error optimisation as described in Goemmere et al. (2025). This approach prevents ex- ponential energy decay in predictive coding models and enables the training of larger architectures. discPC directly follows the formulation introduced in the paper. bpC can likewise be expressed using an error reparametrisation: this is done by rewriting the inference energy function of its discrimi- native component in the same way as for discPC, while leaving the top-down generative prediction error loss unchanged, i.c., computed directly from the layer activities. We validated this reformu- lation of bPC by confirming that its iterative inference converges to the same equilibrium point as the neural dynamics of bPC described in the main paper, but does so more quickly when using error reparametrisation.
Model Hyperparameters
The hyperparameters discPC and discBP for the VGG5 and VGG9 model were adopted from Goemaere et al. (2025). where they were tuned using Hyperhand Bayesian
31

---

optimization (via the Weights & Biases platform) across the combinations listed in Table 23 For the VGGI6 models, we repeated the tuning summarised in 23
For bPC models, additional hyperparameter tuning was performed. Instead, directly applied the optimal hyperparameters obtained for discPC, while setting bPC's adise = 1, and agen for VGG-5, and le for VGG-9 and VGG-16.
Table 23: Summary of hyperparameter search space from Goemaere et al. 2025
Method
Tuned hyperparameter range
Optimizer
Optim steps (T)
Epochs (sweep/final)
e_ir: fixed at 0.001
m.mometum: fixed at 0.0
SGD (error)
discPC
5 (all models)
25/25
Ir: log-uniform [le-5, le-2]
Adam (weights)
decay: log-uniform [1e-6, 1o-31
Ir: log-unilorm [le-5, 1e-2]
discBP
Adam (weights)
NA
25/25
w_decay: log-uniform [le-6, 1e-3]
Glossary: w w.lr: base weight learning rate (see learning rate schedule below), w w.decay: weight decay, {c,s
error / state learning rate, (e.s) _momentum: error state momentum, T: nr. of optimization steps
Evaluation We evaluated the models under three conditions: with 0%, 30%, and 50% of input pixels set to zero across all image channels. Missing pixels were selected uniformly at random, independent of their spatial locations.
For discPC and discBP models, we obtained classification accuracy directly from a single feedfor- ward pass, as these models inherently yield zero reconstruction loss for any given input.
For bPC models, missing pixel values in x were initialised to zero and left unclamped, allowing the network iteratively infer their values. While 600,000 inference steps were previously used to guarantee convergence for MNIST-trained MLPs, this approach is computationally infeasible for our larger models. To accelerate convergence, we adopted two-stage inference process. First, we performed 1,000 warm-up inference steps with Odiss = O, facilitating faster completion of missing pixel values. Subsequently, we restored Odise 1 and carried out an additional 2,000 inference steps to determine final classification accuracy.
Besides measuring classification accuracy, we qualitatively assessed the ability of bPC models to reconstruct missing inputs by examining post-inference neuron activity in ø1.
Results Table 24 shows that bPC achieves classification accuracy comparable to discPC and dis- cBP when full images presented, even in deeper VGG architectures.
Tables 25 and 26 demonstrate that bPC significantly outperforms disePC and discBP on images with missing inputs. This improvement arises from bpC's iterative inference, which fills in missing information. Figure 13 illustrates this effect: after inference, the activity of the input layer & reveals that neurons lacking sensory input have been updated to predict the missing values. Once the missing information is inferred, bPC classifies images accurately. This result holds for both 30% and 50% missing inputs, highlighting bPC's robustness incomplete data.
Overall, even in larger models, bPC effectively balances discriminative and generative performance by jointly minimising bottom-up and top-down prediction errors.
Model
bPC
discPC
discBP
VGGS/CIFAR10
87.3 37.3=003/99.99-
87 29. 8
$7.8+002/98 8
VGG9/CIFAR 100
61.2 84.3
61.3=0.5 5 84.6=02
64.4
VGG16/Tiny Miii///Tn--maaeeNe
10.30.1/73.6t00
50.0+0.3/72.6
47.5+0.6 / 70.3
Table 24: Classification accuracy of bPC, discPC and discBP when whole images are presented.
32

---

Model
bPC
discPC
discBP
VGGS/CIFARIO
80.8 10.3 /97.2
18.4 / /67.8115
16.910.4 / 63. 63.88118
VGG9/CIFARI0L 100
43.9+13 /6..4-11
4.9=0.3 18. 1 18.6*0
4.5 12.8
VGG16/Tiny-Image
15..3
17.3 366.5
Table 25: Classification accuracy of bPC, discPC and discBP when 30% of presented images are missing. Best result shown in bold.
Model
bPC
discPC
discBP
VGGS/CIFARIO VGG9/CIFAR 100
10.1 73.4 /95.6 6..11
12.3 / 46.0
12.2 54.91117
29.4 /48.0 0..8
2.5 10.4
2.3 /8.240.2
VGG16/Tiny-Image
27.8=0055 /41.7#1.1
7.3#0.3/19.3
7.1 19.2
Table 26: Classification accuracy of bPC, discPC and discBP when 50% of the presented images are missing. Best result shown in bold.
REFERENCES
Luca Pinchetti, Chang Qi, Oleh Lokshyn, Cornelius Emde, Amine M' C'aarra,,, Mufeng Tang. Simon Frieder, Bayar Menzat, Gaspard Oliviers, Rafal Bogacz, Thomas Lukasiewicz, and Tommaso Salvatori. Benchmarking predictive coding networks made simple. In Proceedings of the 13th International Conference on Learning Representations, ICLR 2025, Singapore, 24-28 April 2025, 2025a.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann.lecur wwtp:....m//ed////issss.oo 2010.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for bench-
marking machine learning algorithms. arXiv preprint 07X8../708.000 2017. URL https:
//axxiv.org/////oo..vv...... hat///bss/1/0 0170770777777
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical
port, University of Toronto, 2009a. URL. https://www cs wttootto.eedueo........
learning- featuuuuuueeeeeee feettrres- centturrs-000--110 20.9-TR.PO
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint
11X05:171..05001, 2017.
James CR Whittington and Rafal Bogacz. An approximation of the error backpropagation algorithm in predictive coding network with local hebbian synaptic plasticity. Neural Computation, 29(5):
URL http:/::///oi..r//11... 1229 1262, 2017. doi: 10.1162/NECO) .000999
.00949.
NECO 00949
Joseph Marino, Yisong Yue, and Stephan Mandt. Iterative amortized inference. In Proceedings of
the 35th International Conference on Machine Learning, pages 3403-3412. PMLR, 2018. URL
https://proces: wwt..:a................
ml..rr........ press/ /ateess/////// wwt/marinoola........aaaam html
Lukas Biewald. Experiment tracking with weights and biases, 2020. URL https://vei
wandb. com/ Software available from wandb.com.
Gaspard Oliviers, Rafal Bogacz, and Alexander Meulemans. Learning probability distributions of sensory inputs with monte carlo predictive coding. PLOS Compulational Biology, 20(10): 1-34, 10 2024. doi: /ooum//j.....bb..1113555555 URL https:://ci..oq// 1371// Fournal.pohe 1012532
Luca Pinchetti, Chang Qi, Oleh Lokshyn, Cornelius Emde, Amine M' Charrak, Mufeng Tang, Simon Frieder, Bayar Menzat, Gaspard Oliviers, Rafal Bogacz, Thomas Lukasiewicz. and Tommaso Salvatori. Benchmarking predictive coding networks made simple. In Proceedings of the 13th International Conference on Learning Represemaations, ICLR 2025, Singapore, 24-28 April 2025, 2025b.
33

---

VGG5 30% missing inputs
original
missing
reconstructed
50% missing inputs
VGG9 30% missing inputs
original
missing
reconstructed
50% missing inputs
VGG16 30% missing inputs
original
missing
reconstructed
50% missing inputs
Figure 13: Image reconstruction through iterative inference by bPC.
Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. https://github.com wwwptthub..coommmeeeeei.e. Pytorch-fid, fid August 2020a. Version 0.3.0.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Gunter Klambauer, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to nash equilibrium. CoRR, abs/1706.08800, 2017.
Maximilian Seitzer. pytorch-fid: FID Score for PyTorch. httpp:://gittub.commmmss /zer/ pytorch- fidd August 2020b. Version 0.3.0.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Im- proved techniques for training gans, 2016. URL UBL::ttte..//arrxvvoor///////bbb
03498
Jinlin Chen. Inception Score. httpss//gthabb.ccm//snyeeeeess tober 2020.
MS $S. qit Oc-
Mufeng Tang, Tommaso Salvatori, Beren Millidge, Yuhang Song, Thomas Lukasiewicz, and Rafal Bogacz. Recurrent predictive coding models for associative memory employing covariance learn ing. PLoS computational biology, 1914)ee010779, 2023.
Tommaso Salvatori, Luca Pinchetti, Beren Millidge, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz. Learning on arbitrary graph topologies via predictive coding. CoRR, abs/2201.13180,, 2022. JRLss/////a wtt.o/////..........eoo
Senhui Qiu, Saugat Bhattacharyya, Damien Coyle, and Shirin Dora. Deep predictive coding with bi-directional propagation for classification and reconstruction. arXiv preprint arXiv:2305. 18472, 2023.
Wei Sun and Jeff Orchard. A predictive-coding network that is both discriminative and generative. Neural Computation, 32:1836-1866 2020.
//api. semanticscholar: org/
CorpustD:221117744
34

---

Alexander Tscshantz, Beren Millidge, Anil K. Seth, and Christopher L. Buckley. Hybrid pre- dictive coding: Inferring, fast and slow. PLOS Computational Biology, 19(8):131,, 08 2023. doi: 10. 10.377//oummal..bb.00111 1011i.11112: 1280. htthttss: /tt.pp////bb..ogg.....1///////// pabi 1011280
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009b.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint vrXiv:/409.11. 1556, 2014.
Cédric Goemaere, Gaspard Oliviers, Rafal Bogacz, and Thomas Demeester. Error optimization: Overcoming exponential signal decay in deep predictive coding networks, 2025. URL https: /aaxxii.org/aas//555500000
35


Reviews:
[{"review_id": "rniSykoUFM", "reviewer": "Reviewer_QuQM", "summary": "This paper proposes a bidirectional predictive coding (PC) model. Unlike Rao and Ballard’s formulation, where feedforward signals primarily carry prediction errors and feedback connections carry predictions, this model allows both feedforward and feedback pathways to carry predictions or expectations. The latent activity at each level is jointly constrained by top-down expectations and bottom-up proposals, with the energy function minimizing both discrepancies. This formulation brings predictive coding closer to classical interactive activation models (McClelland and Rumelhart), adaptive resonance theory (Grossberg and Carpenter), and hierarchical Bayesian inference via bottom-up and top-down belief propagation (Lee and Mumford). While the conceptual shift is incremental, the formulation is clean and biologically motivated, and the empirical comparisons establish the model’s competitiveness relative to other predictive coding models, particularly the hybrid predictive coding schemes.", "questions": "1. How do the authors position this model relative to classical bidirectional inference frameworks such as interactive activation, adaptive resonance, hierarchical Bayesian inference, and encoder-decoder networks with skip pathways (e.g., U-Net), all of which integrate bottom-up and top-down signals during inference?\n\n2. Prior work (e.g., PredNet) argued that whether feedforward activity represents proposals or prediction errors may not significantly change performance in video-prediction tasks. Have you evaluated a variant using pure error propagation feedforward, and how do the results compare?\n\n3. What are the main obstacles to scaling this architecture to large-scale tasks comparable to modern deep nets? Are the challenges computational, architectural, representational, or learning algorithms?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The bidirectional predictive coding formulation is conceptually simple, elegant, and aligns more closely with classical interactive cortical models. It is worth emphasizing, however, that in Rao and Ballard’s model, the initial feedforward sweep does convey bottom-up evidence (the bottom-up prediction), and only subsequent iterations carry the prediction residues or prediction error signals. Thus, the primary difference is in the steady-state treatment and symmetry of information flow.\n\n2. The inference and learning rules are derived from an energy function, and learning is Hebbian, giving this approach greater biological plausibility than backprop-based deep networks.\n\n3. In contrast to autoencoder architectures, encoding and decoding (recognition and generation) in this model are unified. The same circuitry supports inference, classification, generation, and cross-modal completion without explicit mode-switching.\n\n4. Demonstrating cross-modal pattern completion within a predictive-coding framework is novel and compelling.\n\n5. From a neuroscience perspective, the model avoids assuming separate error and representation neurons; instead, each unit receives both bottom-up and top-down drives and evolves dynamically to reduce error. This yields an elegant single-population interpretation consistent with cortical microcircuit ideas.", "weaknesses": "1. The conceptual innovation is modest and closely related to earlier models of bidirectional inference (interactive activation, adaptive resonance, hierarchical Bayesian models). The paper could better situate itself in this lineage.\n\n2. Although the results show benefits over prior predictive-coding variants, scalability to real-world architectures and large-scale deep learning benchmarks remains unclear; current experiments appear focused on moderate-scale or simplified tasks.\n\n3. Historically, biologically motivated predictive coding architectures such as PredNet (Lotter, Kreigman, Cox) achieved strong performance in video prediction. It would be useful to discuss the connection and differences, particularly regarding information flow vs. error flow debates.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YSuSCuhUqe", "reviewer": "Reviewer_Pgr9", "summary": "This paper proposes Bidirectional Predictive Coding (bPC), a biologically inspired architecture that aims to integrate discriminative and generative learning within a single predictive coding framework. The authors claim that bPC can simultaneously achieve high classification accuracy and realistic image generation, and present results across supervised, unsupervised, and multimodal settings using datasets such as MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100. \nAlthough the paper is clearly written and the experiments are extensive, the overall novelty and empirical significance of the work are limited. The methodological design and experimental choices do not convincingly demonstrate contributions beyond existing predictive coding or joint generative–discriminative approaches.", "questions": "See Weaknesses.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The idea of integrating bidirectional inference under a unified predictive coding framework is conceptually clear and aligns with prior theoretical literature.\n2. The paper reports a broad range of experiments, including classification, image generation, multimodal learning, and occlusion robustness, showing some effort toward comprehensive evaluation.", "weaknesses": "1. The central idea of combining discriminative and generative pathways within a shared latent representation is not novel in the context of modern machine learning. Many recent architectures, such as VAVAE[1] or VAR[2], already unify discriminative and generative learning with better theoretical grounding and empirical performance. The proposed bPC model appears to be a minor variation of existing predictive coding formulations (discPC, genPC, hybridPC) with shared weights, rather than a fundamentally new mechanism.\n2. The network design is extremely simple. Most experiments are conducted with two hidden layers of 256 neurons or shallow convolutional networks. This small scale makes it difficult to evaluate whether the claimed properties would generalize to larger or more realistic settings.\n3. The choice of benchmarks (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100) is outdated and insufficient to support strong claims of scalability or generality. While such datasets can be acceptable for conceptual neuroscience modeling, they are not appropriate for evaluating new machine learning architectures, especially in a venue like ICLR which emphasizes methodological innovation and empirical rigor in AI.\n4. Although biologically inspired models sometimes employ simplified architectures to study neural mechanisms, those works typically analyze real neural data or test neuroscientific hypotheses. This paper does neither. Without such grounding, the justification for using small-scale networks and toy datasets becomes unclear and weakens the relevance of the work to both neuroscience and AI audiences.\n5. In Figure 1, subfigures (A) and (B) show no visible difference between discPC and discBP, or between genPC and genBP, making it difficult for readers to understand the structural differences among these variants.\n6. In Figure 7(A), the authors mention using one-hot labels as text inputs. However, multimodal setups typically require a meaningful text encoder (e.g., CLIP). Using one-hot vectors does not constitute true multimodal learning, and the symbol shown in the figure (an ear icon) is also misleading in this context.\n7. Many mathematical expressions lack punctuation marks at the end of equations. \n\n[1] Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models. CVPR 2025\n\n[2] Visual autoregressive modeling: Scalable image generation via next-scale prediction. NeurIPS 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HLwmewaN4f", "reviewer": "Reviewer_UixL", "summary": "The brain is able to do both generative and discriminative tasks, however the current most influential model for how the brain does that, predictive coding, has great trouble doing both in the same network. The authors present a surprisingly simple bidirectional version of the classical Rao&Ballard predictive coding model and show how that then a single model is able to solve both generative and discriminative tasks.", "questions": "- beyond fig 4e, what is the influence of the number of inference steps for the various problems, and is inference done sequentially per layer or in parallel? \n- can you say something on whether/how the approach would extend to dynamic rather than static stimuli?\n- can you comment more on how the networks are initialized for unsupervised learning (l106/l302)?\n- the VAE may not be the best self-supervised model for both generative and discriminative tasks within the set framework, that seems to rather be Masked-Autoencoders. Can you comment on this? Ideally compare to MAEs?\n- In 4.3, it is noted that learning rarely takes place only in an unsupervised setting. I would argue however that supervised learning is much rarer than RL. Can you comment on whether the approach extends to RL?\n- the fact that Max-Pooling has such detrimental results is noteworthy (l 364), and would seem to argue for for example all-convolutional networks. Comment?", "limitations": "", "rating": 10, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "This is a great paper. Conceptually simple, though with many smaller innovations that are only mentioned in passing, the presented work marks a great step towards highly functional predictive coding networks, fixing previous issues with capabilities and, importantly, scaling. \nThe work is exceptionally thorough, well designed and -mostly- very clear (see below). \nNoteworthy is also the work in the appendix where the models are scaled up to large networks and many-class classification benchmarks. This solves another big problem with predictive coding networks, that is, the previous work simply does not scale (we, and others, tried). For example, the approach by Song et al mentioned fails for cifar-100 and also for deeper networks. The approach presented here does scale, and this should be mentioned more clearly in the final version of the paper.", "weaknesses": "Minor issues:\n- the issue of symmetric/shared bottom-up/top-down connections (l154) is a bit exaggerated, as it is easy to show that as long as identical local learning signals can be applied at either end, weight decay can result in symmetrical weights. (Tim Lillicrap wrote on this). \n- section 4.5 lacks clarity, in particular on what exactly is shown in Figure 7. What am I seeing here, what is the paradigm?\n- the issue of scaling, which this approach solves to a large extend, is only mentioned in the appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9H0ryxRdKf", "reviewer": "Reviewer_QM9z", "summary": "This paper proposes a hierarchical neural network architecture in which the same error-minimization process is used to reconcile both bottom-up and top-down signals. Neural activity is determined using an iterative process of gradient decent to minimize these errors. Weights are also updated to minimize theses errors.", "questions": "The proposed architecture seems to be very similar to that proposed in Qiu et al. (2023). Apart from the lack of tied weights (mentioned on l.155) are there any other differences? Why not compare the performance of your network to this one?\n\nCould the proposed method scale-up to deal with large data-sets, such as ImageNet? \n\nHow was the number of iterations chosen? How sensitive are the results to this choice? Would the number of inference steps need to increase with the size of the architecture? \n\nHow do the computational costs of the different methods compare?\n\nCould the proposed method be used for semi-supervised learning?\n\nIt is claimed (l.292) that the proposed architecture only requires half as many neurons as an alternative architecture using separate bottom-up and top-down networks. Is this claim ignoring the error-neurons? If so, why?\n\nWhat do the error bars on the results figures represent? Are the differences in performance of the compared methods statistically significant?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed architecture employs only local information for inference and learning.\n\nThe proposed architecture is very flexible and can be used for a range of tasks, such as image classification, image generation, supervised and unsupervised learning.", "weaknesses": "Fig. 1 is not very helpful in showing the differences between architectures. One issue is that arrows of the same colour seem to mean different things in different panels.\n\nThe method is only tested using small, simple, data-sets.\n\nMany inference steps are performed to process each sample. The number of steps seems too large for this to be a viable model of biological visual inference or a method that could have practical, machine learning, applications.\n\nThe relatively small differences in performance are difficult to see given the large scales used in the graphs showing the results (Figs. 3, 4, and 5).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)