Generalized Predictive Coding: Bayesian Inference in Static and Dynamic models
Abstract
Predictive coding networks (PCNs) have an inherent degree of biological plausibility and can perform approximate backpropagation of error in supervised learning settings.
However, it is less clear how predictive coding compares to state-of-the-art architectures, such as VAEs, in unsupervised and probabilistic settings.
We propose a PCN that, inspired by generalized predictive coding in neuroscience, parameterizes hierarchical distributions of latent states under the Laplace approximation and maximises model evidence via iterative inference using locally computed error signals.
Unlike its inspiration it uses multi-layer neural networks with nonlinearities between latent distributions.
We compare our model to VAE and VLAE baselines on three different image datasets and find that generalized predictive coding shows performance comparable to variational autoencoders trained with exact error backpropagation.
Finally, we investigate the possibility of learning temporal dynamics via static prediction by encoding sequential observations in generalized coordinates of motion.
Introduction
Predictive coding is an influential theory in neuroscience that describes brain function as learning and maintaining a generative model of the world by minimising prediction errors about sensory and internal states [3, 18].
Static PCNs are organised hierarchically, where top-down signals from higher layers predict the activity of the layer below and bottom-up signals convey prediction errors.
In dynamical predictive coding models each layer additionally predicts temporal changes of expected neural activity in the layer below.
Given these dynamics, Hebbian weight updates can be defined that minimize the prediction error at each layer of the network.
The weight and activity update dynamics of PCNs can be interpreted as performing variational inference (VI) by iteratively refining an inferred distribution over possible causes p ( z | o ) of observed sensory data o [6, 5, 21].
In variational inference, an approximate distribution q ( z ; λ ) is fit to the generally intractable posterior p θ ( z | o ) by optimizing the variational free energy F , also known as evidence lower-bound (ELBO) [6, 7]: F θ ( o ; λ ) =
E q ( z ; λ ) [ln p θ ( o, z ) − ln q ( z ; λ )]
In predictive coding, we define q ( z ; λ ) to be a simple diagonal or full-covariance Gaussian distribution with λ as the sufficient parameters, i.e. the mean and covariance.
Given the generative model θ (decoder) of a particular hierarchical layer, inference in predictive coding models proceeds by estimating the optimal variational parameters λ ∗ that maximize model evidence given observed data and current parameterization.
Learning of the parameters of the generative model θ can be achieved by performing a gradient descent on F θ ( o ; λ ∗ ) with respect to θ which results in Hebbian weight updates.
Crucially, learning and inference in PCNs is driven by locally generated predictions and prediction errors.
In hierarchical PCNs, the predicted distributions of higher layers foster empirical priors for the next lower layer: p ( z, o )
=
p ( o | z 1 ) p ( z 1 |
z 2 ) . . .
p ( z L − 1 | z L ) , such that a layer’s inference model can be interpreted as the next higher layer’s generative model.
The variational autoencoder (VAE) is a highly influential class of deep neural networks that performs amortized inference of λ using an inference model ϕ (encoder)
The inference model in VAEs learns to predict the approximate posterior q ϕ ( z | o ) by learning the parameters ϕ of the variational mapping over a dataset.
In contrast to the Hebbian updates in PCNs, VAEs are trained using exact backpropagation of error through the entire model [20].
More recently, the notion of iterative inference has been adapted for VAEs to improve the posterior distribution [16, 13].
While VAEs and static PCNs have striking similarities in terms of architecture and optimisation scheme, there is still a lack of quantitative comparisons in the literature.
Similarly, various deep recurrent models for predicting sequential stimuli have been developed, but lack exhaustive comparison to dynamical PCNs [2, 19, 8].
To start addressing these gaps the main contributions of this paper are:
• We compare a static PCN model with full covariance estimation via the Laplace approximation for nonlinear neural networks to VAE baselines on several image datasets •
We extend our PCN model to temporal predictions of video in generalized coordinates of motion in line with generalized predictive coding in Neuroscience [5]
Generalized predictive coding
Generalized predictive coding (GPC) describes a particularly influential class of PCNs that covers static and dynamic models in combination with generalized coordinates of state motion and the
Laplace approximation [5, 6, 4, 1].
Static GPC networks infer the conditional mean and covariance of cause states v and hidden states x .
Each hierarchical layers predicts the expected activity in the next lower layer using non-linear function g : y
= g ( x, v, θ ) + z .
Dynamical GPC networks additionally predict the motion of hidden states ˙ x = f ( x, v, θ ) + w using a non-linear transition function f 1 .
z and w denote observation noise and transition noise respectively.
While cause states are predicted hierarchically, hidden states are usually not observed by higher layers.
Under the assumption of local linearity, GPC uses states in generalized coordinates of motion ˜ y =
[ y, y ′ , y ′′ , . .
T , where y ′ denotes the temporal derivative at y .
Similarly, for cause and hidden states:
[Equation]
Using Gaussian priors p ( z )
=
N ( z ; ¯ µ, Σ) , GPC infers posterior distributions of the causes p (˜ x | ˜ v ) =
N (cid:16) D ˜ x : ˜ f, ˜Σ z (cid:17) and the hidden states p (˜ y | ˜ x, ˜ v ) = N (cid:16) ˜
y :
˜ g, ˜Σ z (cid:17) .
Here, D denotes a derivative operator that replaces each order of state motion with the next higher order: x ← x ′ , x ′ ← x ′′ , ... .
While conditional mean parameters µ are encoded explicitly, the covariance Σ is encoded implicitly as a function of the mean using the Laplace approximation (LA).
Under the LA, the covariance is determined by the local curvature of − log p θ ( y, v, x ) at the inferred mode of p θ ( v, x | o ) .
Figureshows dynamical and static GPC in comparison to a VAE.
GPC proceeds by expressing the free energy F of each hierarchical layer l as a function of precision weighted prediction errors ξ ( l,o )
= Σ ( l,o ) − 1 ϵ ( l,o ) for outgoing predictions and for top-down predictions ξ ( l,v ) = Σ ( l,v ) − 1 ϵ ( l,v ) from the next higher layer.
Here, precision is the inverse of the covariance Σ .
Depending on the chosen prior, the distance between prior and posterior distribution is measured by ξ ( l,p ) = Σ ( l,p ) − 1 ϵ ( l,p ) .
Here, ϵ p = µ − ¯ µ = µ − 0 , is the prediction error between posterior and prior and ϵ ( l ) v = µ ( l ) − g ( µ ( l +1) ) is the hierarchical prediction error between layers (or the sensory prediction error at the lowest layer).
For dynamical models, the generalized predictions ˜
y result in generalized errors ˜ ϵ =
˜ y − ˜ o
=
[ ϵ, ϵ ′ , ϵ ′′ , . . .
T .
Inference in each layer is done via gradient descent on ξ = Σ − 1 ϵ for cause states ˙˜ µ ( l ) v = ˜ µ ( l ) v − ˜ ε ( l ) T v ξ ( l ) − ξ ( l +1) v [5].
Within each hierarchical layer, the motion of hidden states is inferred as ˙˜ µ ( l ) x = D ˜ µ ( l ) x − ˜ ε
( l ) T x ξ ( l ) .
An appeal of the GPC model is that it provides a simple model inversion scheme that with a strong degree of neurobiological plausibility[5, 1].
Encoding state motion in generalized coordinates casts dynamical predictions (which normally require complex recurrent connectivity) as a static prediction mechanism.
Similarly, the Laplace approximation provides a simplified scheme to represent uncertainty that allows approximate inference to be performed with simple Hebbian updates.
[Figure]
Implemented models and baselines 3.1 VAE and VLAE baseline
We use the conventional VAE architecture with fully factorized normal distribution, reparameterization of the latent distribution and training via backpropagation of error [10].
It uses a single sample from the latent distribution to estimate the reconstruction error. The complexity is measured using the analytical KL divergence between inferred distribution and a standard normal prior.
The VLAE is a variant of the VAE with iterative mode seeking that defines a full-covariance Gaussian posterior at the mode using the Laplace approximation [16].
The VLAE uses a single sample from the latent distribution at the inferred mode as input to the decoder.
The model uses a decaying learning rate for mode seeking.
For the VAE and VLAE models, the encoder and decoder consist of two ReLU activated layers with 256 hidden units and parameterize 16 latent units.
3.2 Static GPC model
We implement a static GPC with two hierarchical layers and fix the mean of the second layer’s latents to the data.
In this setup, the output of the second hierarchical layer provides empirical priors via amortized inference on the first hierarchical layer’s cause states p ( v 1 ) 2 .
The resulting architecture resembles that of an autoencoder as it uses the second layer’s generative model as the first layer’s inference model.
Predictions between cause states are parameterized by a dense neural network with three layers. All generative networks have 256 hidden units and 16 latent units (causes).
In contrast to the VAE baseline, inference and learning in the presented network (GPC-M) does not involve a sampling step. For comparison we also implement a network (GPC-V) that is trained using stochastic updates with a single sample from the posterior distribution. Section A of the Appendix provides a detailed explanation of the objective used for learning and inference in the proposed model. 3.3 Laplace approximation with ReLU nonlinearity
Inspired by the work of Park et al.
[16], we employ ReLU non-linearity for the input and hidden layer, followed by a linear output layer.
For network weights W and Jacobian W z with respect to latent states z = ( v, x ) ReLU non-linearity allows to efficiently compute the precision of inferred posterior states Σ − 1 = − ∇ 2 z log p θ ( o, z ) (cid:12)(cid:12) z = µ = W zT
W z + I | z = µ by computing binary activation masks ReLU( Wz )
=
O ( Wz ) during the decoder’s forward pass and recursively multiplying with the decoder’s weights [16].
After a fixed amount of inference steps towards the posterior mode the approximate posterior distribution is q ( z | o ) = N ( µ, Σ) using the LA [16].
This distribution is then used to compute gradients for the weights. We perform exact error propagation to the weights strictly locally within each hierarchical layer using automatic differentiation in PyTorch [7, 17]. This is in contrast to a backpropagation pass over all parameters, such as in VAEs, where the decoder’s error directly drive updates of encoder parameters.
3.4 Dynamical GPC model
We also implement two variants of dynamical GPC models to test amortized inference and dynamical inference of causes respectively: A simplified model with two hierarchical layers without cause states that predicts hidden states top-down.
Again, the data serves as fixed input to the second hierarchical layer.
The second model consists of a single hierarchical layer that infers cause and hidden states with associated dynamics using multiple shooting, as explained in the next section.
We use 16 units for cause and hidden states in all models.
Dynamical GPC models are trained like static GPC via iterative inference.
However, in a addition to decoding the states y = g ( z ) , the explicitly represented state motion ˜ y
= g (˜ z ) is decoded and compared with the data ˜ o .
During decoding the states y = g ( z ) , the Jacobian W z at the currently inferred mode is computed by masking the decoder network. All other orders of state motion y ′ = g z z ′ , y ′′ =
g z z ′′ , ... are decoded through this masked decoder network W z .
Similarly, during the prediction of hidden state motion x ′ =
f ( z ) the Jacobian of the transition network f z is computed. This Jacobian is reused for all higher order hidden state motion predictions z ′′ =
f z ( z ′ ) , ..., z N = f z ( z N − 1 ) .
We use N = 2 for the simplified and N = 3 for the full GPC model.
Amortized inference on the hidden states and their motion is possible by repeating the process with the second hierarchical layer’s decoder.
We interpret the Taylor series expansion underlying the forward and inverse embedding of sequential data as a convolution operation along the temporal axis, which can efficiently be computed using convolutional kernels.
More details and examples for such kernels are shown in section B of the Appendix.
Figure 3 a) displays the dynamical connectivity in the hierarchical GPC layer.
3.5 Multiple shooting
Following related work on multiple shooting (MS) based training of Neural ODEs we train the model by splitting discrete sequences into multiple segments, which are optimised in parallel [23, 22].
We sample discrete sequences [ o t 1 , ...
o t n ] of length n at m shooting points [ o τ 1 , ... o τ m ] which are then embedded into generalized coordinates. For b sequences sampled at m point, the network is trained with a batch size of b ∗ m . In practise, this means that we can omit the term D ˜ µ ( l ) x for the dynamical predictions, which would be required for sequential filtering.
Figure 2 shows an example with two discrete sequences.
Crucially, while hidden states are inferred for each shooting point o τ i individually, the prediction errors for cause states are averaged over all m samples from a sequence.
We use MS with b = 4 , m = 8 for the dynamical GPC model that infers causes and b = 64 , m = 1 otherwise.
We train and evaluate static models on MNIST [12], FashionMNIST [24] and OMNIGLOT [11].
Unlike the VLAE baseline, we do not initialise the decoder output variance based on dataset statistics.
Instead we add noise from a uniform distribution and apply a logit transformation for all datasets.
Tables 1 and 2 shows test results on all datasets for 3 iterative inference steps using the conventional train and test splits.
Listed are mean and standard deviation across 5 runs.
We trained for 1 e +steps with the ADAM optimiser at a learning rate of 1e-2 [9] and a inference update rate of 0.01 for GPC-M and GPC-S. The VLAE is trained with the default inference update rate of 0.5 [16].
GPC-S and GPC-M slightly outperform the VAE on MNIST and OMNIGLOT, while the VLAE model consistently outperforms both PCNs and VAEs on all datasets.
This indicates that PCNs, despite lacking exact error signals for the inference network learn a generative model that is comparable in performance to VAEs.
The GPC-M model, without explicit sampling, outperformed the sample-based GPC-S model, except for one configuration on MNIST.
In terms of divergence from the prior, the GPC models consistently showed posterior complexity that is comparable to, but slightly higher than, VAE complexity.
For more details on model complexity and the influence of inference steps, see Section C of the Appendix.
For all tested models, increasing the number of inference steps is beneficial only for low numbers of steps.
We found that reducing the inference learning rate or adding a decay term can improve stability, but did not include it in our experiments. [Table]
[Table]
Dynamical predictive coding on the dSprites dataset
To assess the capabilities of dynamical PCNs we train a dynamical GPC model on a variant of the Disentanglement testing sprites dataset [15].
Most conventional video benchmarks have relatively low sampling rates, where the local linearity assumption does not hold.
We generate high resolution videos for a single direction of rotation (counterclockwise) and use random, but constant, values for the remaining latent factors.
We applied Gaussian blur to all images and cropped the videos to 32x pixel resolution, making sure that no sprites appear outside the area. [Figure]
[Table]
Table 2 shows mean and standard deviation of the mean squared error (MSE) for 10 runs over 3e+ updates using two different variants of the simplified dynamical model:
The GPC-all model was trained using the prediction error from both dynamical layers, while GPC-L1 only considers the error in the lowest dynamical layer.
Both models smoothly predict the constant rotation across latent factors.
GPC-all shows improved MSE in terms of total and per-layer prediction.
This indicates that including higher-order dynamical predictions errors propagated through the network’s Jacobian indeed improves accuracy.
We found that GPC-L1 reacts poorly to increased latent dimensionality and stops predicting any meaningful state motion when 32 or more latent units were used.
In contrast,
GPC-all showed meaningful transitions for larger embeddings.
We found that training a dynamical model that infers causes and hidden states simultaneously on the rotating dSprites dataset lead to a clear clustering of causes into the two directions of motion for the inferred cause states, as shown in Figure 4 b). The hidden states capture spatial aspects, such as sprite shape, which change in dependency of the inferred cause. After training the network and freezing weights, new generalized observations ˜ o can be encoded via iterative inference.
The inferred stated can then be used for dynamical prediction of future timesteps, by applying x ′ =
f ( z ) , x ′′ = f z ( z ′ ) , ... . Figure 4 a) shows a typical extrapolation for up to 50 with different step sizes dt scaling the predicted hidden state motion x ′ = f ( x, v ) ∗ dt .
Changing the applied step size allows to increase and decrease [Figure] ' v v'' [Figure]
' v v''
[Figure] [Figure] the speed of motion forward and backward in time. Figure 3 b) shows the discrete observations and their temporal embeddings.
Additionally shown are the model’s generalized prediction ˜
y from hidden states as well as the resulting inverse embedding to a discrete sequence.
Conclusion
We presented a generalized predictive coding network (GPC) that uses Hebbian updates and the
Laplace approximation with nonlinear neural networks to infer posterior distributions.
We have shown that the model performs comparably to VAEs trained with exact error backpropagation.
We extended the model to cover dynamical predictions of simple video sequences and demonstrated the possibility to learn dynamics using generalized coordinates of motion.
We found that in many cases, GPC still learns meaningful dynamics on video datasets with lower sampling rate.
For this work however, we focus on high resolution data.
Important steps for future work could be to use convolutional neural networks or a comparison to related dynamical models, such as Neural ODEs or RNNs [22, 19].
Acknowledgments
This research was funded by the Federal Ministry of Education and Research of Germany (BMBF) as part of the "Cognitive neuroscience Inspired techniques for eXplainable AI" (CogXAI) project.
A Inference and learning with precision weighted prediction errors
For a hierarchical layer l, the variational free energy F decomposes into an accuracy term, that measures the quality of the outgoing prediction g ( l ) ( µ ( l ) ) , and a complexity term between top down predicted state g ( l +1) ( µ ( l +1) ) and inferred state µ ( l ) . Since we want to compare the GPC model to variational autoencoders, we regularize by the distance between a standard normal prior distribution and the inferred posterior distribution.
[Equation]
Under the Laplace approximation, the divergence from a standard normal prior simplifies to the divergence from zero mean.
Hidden hierarchical layers predict the activity µ l − 1 in the layer below instead of sensory observations o .
A central idea in predictive coding is to approximated the KL divergences in 1 with respect to prediction errors ϵ :
[Equation]
where ϵ ( l,o ) measures the reconstruction accuracy, ϵ ( l,v ) measures the distance between inferred activity µ ( l ) and top-down predicted activity ˆ µ ( l ) and ϵ ( l,p ) measures the distance of inferred activity from the prior with zero mean.
To make use of amortized inference, at the start of iterative inference, the inferred posterior is initialized with its top-down prediction µ 0 = ˆ µ .
During inference, the optimal µ ( l ) ∗ with respect to complexity and accuracy is computed using a gradient descent on ϵ ( l,p ) and ϵ ( l,o ) .
After inference, the covariance parameters of the top-down predicted distribution ˆ q ( z ) and the inferred posterior distribution q ∗ ( z ) are inferred following the routine described in Park et al. [16] using Σ − 1 = − ∇ 2 z log p θ ( o, z ) (cid:12)(cid:12)
z = µ , which can be efficiently computed for ReLU activations. The, the weights of the model can be updated with respect to the distance between optimal inferred posterior distribution q ∗ ( l ) ( z ) and the top-down predicted distribution ˆ q ( l ) :
[Equation]
where the KL divergence is approximated by a precision weighted prediction error:
[Equation]
[Equation]
We train all predictive coding models using the weighted prediction error instead of the full KL divergence.
For model evaluation and the comparison to VAE and VLAE baselines, we use the full (analytical
) KL divergence between prior and posterior.
B Generalized coordinates from discrete sequential data
We compute temporal embeddings of observations according to a Taylor expansion of form
[Equation]
5) for points x ± dx around a point x assuming a fixed step size e.g. dx = 1 .
Since we observe discrete samples [ o 1 , ...
o n ] we approximate the instantaneous derivatives f ′ , f ′′ , ... up to desired order using a central finite difference operator δ d x n [ f ]( x ) .
We interpret the resulting differencing coefficients as convolutional kernels, which can be applied to any sequential data with sufficiently high sampling rate either online or during preprocessing.
Mapping back from the network’s states to sequential data can easily be done using the inverse kernel.
Figure 5 shows examples for forward and inverse embedding kernels.
[Figure]
C Static predictive coding C.1 Influence of iterative inference steps
MNIST
OMNIGLOT fMNIST
MNIST
OMNIGLOT fMNIST
Steps
PC_S 49.0±0.
44.9±1.
42.8±0.
48.4±0.
60.2±3. 36.3±0.
PC 48.9±0.
49.0±0.
37.1±0.
48.2±0.
56.1±1.
36.8±0.
VAE 46.7±0.
41.8±0.
36.0±
[Table]
Table 4 shows the posterior complexity of models trained on the static prediction task for MNIST,
OMNIGLOT and Fashion MNIST in terms of mean and standard deviation over five runs.
The predictive coding models GPC-S and GPC-M show complexity that is comparable to, but slightly higher than the complexity of the baseline VAE.
For VLAE, increasing the amount of inference steps from 3 to 6 slightly increases the complexity of encoded states. For GPC models, increasing the amount of inference steps sometimes leads to an improvement in terms of complexity. C.2 Influence of iterative inference steps We found that, in the current configuration, increasing the amount of inference steps is beneficial primarily for low amounts (1-5 steps).
Figure 6 shows test set results on the MNIST dataset with various amounts of iterative inference steps. Since the top-down prediction performas amortized inference of the inferred state for even a single inference step leads to meaningful results. We found that choosing a higher inference update rate (e.g. 0.01) often shows favorable performance over choosing a lower learning rate (e.g. 0.001) with more update steps.
[Figure] D Data sources
We employ three popular datasets for unsupervised learning on images: The MNIST dataset (Creative
Commons Attribution-Share Alike 3.0 license), Fashion MNIST (MIT license) and OMNIGLOT (MIT License).
Evaluation of the dynamical predictive coding model is based on the Disentanglement testing Sprites dataset (Apache License 2.0).
Table 5 shows sources and licenses for all datasets.
MNIST and FashionMNIST contain 60000 train and 10000 test images (with 28 × 28 pixels) while the OMNIGLOT dataset contains 24345 train and 8070 test images (also with 28 × 28 pixels). Dataset Source License
MNIST [12] yann.lecun.com/exdb/mnist CC BY-SA 3. OMNIGLOT [11] github.com/brendenlake/omniglot MIT license
Fashion MNIST [24] github.com/zalandoresearch/fashion-mnist
MIT license dSprites [14] deepmind.com/open-source/dsprites
Apache License 2.
To generate discrete video sequences with high temporal resolution for the dynamical GPC model we use a customized version of the dSprites dataset [14].
We generate 128000 random samples from the original dataset and apply Gaussian blur along both spatial axes with kernel size 3 and Standard deviation of 10 before applying normalization.
We restrict x and y positions to six values respectively, such that all sprites appear within a center crop of 32 x 32 pixels.
Starting with the noisy version of the sprite, we apply a single direction of rotation (counterclockwise) by rotating the sprite by a single degree.
All remaining aspects, such as shape, size, horizontal position and vertical position stay constant.
The Gaussian noise was applied only to the first frame of each sequence.
We then projected the resulting discrete video sequences into generalized coordinates using the embedding kernels discussed in Section B of the appendix.