Constrained Predictive Coding as a Biologically Plausible Model of the Cortical Hierarchy
Abstract
Predictive coding (PC) has emerged as an influential normative model of neural computation with numerous extensions and applications.
As such, much effort has been put into mapping PC faithfully onto the cortex, but there are issues that remain unresolved or controversial.
In particular, current implementations often involve separate value and error neurons and require symmetric forward and backward weights across different brain regions.
These features have not been experimentally confirmed.
In this work, we show that the PC framework in the linear regime can be modified to map faithfully onto the cortical hierarchy in a manner compatible with empirical observations.
By employing a disentangling-inspired constraint on hidden-layer neural activities, we derive an upper bound for the PC objective.
Optimization of this upper bound leads to an algorithm that shows the same performance as the original objective and maps onto a biologically plausible network.
The units of this network can be interpreted as multi-compartmental neurons with non-Hebbian learning rules, with a remarkable resemblance to recent experimental findings.
There exist prior models which also capture these features, but they are phenomenological, while our work is a normative derivation.
Notably, the network we derive does not involve one-to-one connectivity or signal multiplexing, which the phenomenological models required, indicating that these features are not necessary for learning in the cortex.
The normative nature of our algorithm in the simplified linear case allows us to prove interesting properties of the framework and analytically understand the computational role of our network’s components.
The parameters of our network have natural interpretations as physiological quantities
in a multi-compartmental model of pyramidal neurons, providing a concrete link between PC and experimental measurements carried out in the cortex
Introduction
Over the past decades, predictive coding (PC), a normative framework for learning representations that maximize predictive power, has played an important role in computational neuroscience [1,2].
Initially proposed as an unsupervised learning paradigm in the retina [3], it has since been expanded to the supervised regime [4] with arbitrary graph topologies [5, 6].
The PC framework has been analyzed in many contexts [7,8] and has found many applications, from clinical neuroscience [9,10] to memory storage and retrieval [11].
We refer the reader to [12,13] for recent reviews.
[Figure]
Predictive coding is viewed as a possible theory of cortical computation, and many parallels have been drawn with the known neurophysiology of cortex [12].
While the initial works proposed a biologically plausible network [1,2], the connection with cortex was more closely examined in [14], where the PC module was mapped onto a cortical-column microcircuit.
However, there are aspects of this mapping that have proved controversial.
Among these are the requirement of multiple redundant cortical operations, the symmetric connectivity pattern, the one-to-one connectivity of value and error neurons, and also the requirement that feedback connections be inhibitory [12,15,16], as sketched in
Fig.
1a .
The presence of separate error and value neurons has itself been called into question [15].
The PC-based neural circuits also do not account for more recent experimental findings highlighting the details of computation in the cortex [17–26].
For example, the learning dynamics of multicompartmental pyramidal neurons has been closely investigated.
In particular, it was observed that the plasticity of the synapses of the basal compartment is driven by the activity in the apical tuft of the neuron by so-called calcium plateau potentials [19,20,24], leading to non-Hebbian learning rules [25].
These experiments have motivated the development of several models of microcircuits with multi-compartmental neurons [27–33].
In a number of cases, it has been shown that these models can replicate learning similar to the backpropagation algorithm under specific assumptions [30,34,35].
However, because of their rather phenomenological nature, detailed analysis of these models is in many cases challenging and one must resort to numerical simulations, rendering the task of understanding the role of various neurophysiological quantities difficult.
To this date, a normative framework (PC or otherwise) that can explain these experimental findings is still lacking.
In this work, we show how to modify the PC framework to make it compatible with the aforementioned experimental observations while maintaining much of the analytical control afforded to use by a normative approach.
Inspired by prior work which explored the effects of finding decorrelated representations [36–41], we add a decorrelating inequality constraint to the covariance of the latent representations of PC.
Using this constraint, we derive an upper bound for the PC objective.
By working in the linear regime, we can prove several useful properties of our algorithm.
We show that the learning algorithm derived from this upper bound does not suffer from the issues of prior implementations and naturally maps onto the hierarchical structure of the cortical pyramidal neurons.
Contributions
• We introduce BioCCPC, a novel biologically plausible algorithm derived as a modification of predictive coding (PC), and show that it closely matches the known physiology of the cortical hierarchy.
• We find an analytic upper bound to the PC objective by imposing a decorrelation-inspired inequality constraint on the latent space.
• We interpret the different parameters of the algorithm in terms of the conductances and leaks of the separate compartments of the pyramidal neuron.
We find that the neural compartmental conductances encode the variances of the PC framework, and the somatic leak maps onto a thresholding mechanism of the associated eigenvalue problem.
Related work and review of predictive coding
The backpropagation algorithm [34, 42] is the predominant tool in the training of deep neural networks but is generally not considered biologically plausible [43].
Over the years, many authors have explored biologically plausible approximations or alternatives to backpropagation [30,31,44–58] (for a more complete review see [59]).
These approaches generally fall into two categories.
First are normative approaches, such as Predictive Coding [1–3], Target Propagation and variations [44,45,47],
Equilibrium Propagation [49, 53] and others [51, 54–56], where one starts from a mathematically motivated optimization objective.
These methods, by virtue of their normative derivation, have a firm grounding in theory; however, they do not fully conform to the experimental observations in the brain (see below and Sec. 4).
The second approach is driven by biology, with network structures and learning rules inspired by experimental findings [28,30,31,48,60].
While these works mostly conform to the experimentally observed findings, they are more challenging to analyze because of their conjectured phenomenological nature.
The goal of the present paper is not to propose yet another biologically plausible alternative to backpropagation.
It is rather to demonstrate that the normative framework of predictive coding, when combined with a constraint, can indeed closely match experimental observations.
For this reason, in this work, we focus on comparing our method with previous implementations of predictive coding and do not concern ourselves with other biologically plausible alternatives to backpropagation.
The relationship between the PC framework and backpropagation was explored in [4,61–64].
The advantages of PC over backpropagation were highlighted in [13].
We now turn to a brief review of the predictive coding framework from [4] before delving into the details of our method.
Notation.
Bold upper case M and lower case variables v denote matrices and vectors, respectively.
By upper case letter X , Y , Z , we denote data matrices in R d × T , where d and T are the dimensions of the relevant variable and the number of samples.
Lower case x , y , z , denote the relevant quantities of a single sample, and x t , y t , z t , denote the t th sample.
∥ M ∥ 2 F denotes the Frobenius norm of M .
2.1 Review of predictive coding
Probabilistic model.
In this section, we review the supervised predictive coding algorithm from [4].
The derivation starts from a probabilistic model for supervised learning, which parallels the architecture of an artificial neural network (ANN) with n + 1 layers.
In this model, the neurons of each layer are random variables z ( l ) (denoting the vector of activations in the l th layer) with layers 0 and n , respectively, denoting the input and output layers of the network.
We assume that the joint probability of the latent variables factorizes in a Markovian structure
[Equation]
with the relationship between the random variables of adjacent layers given by:
[Equation]
with
[Equation]
The mean of the probability density on layer l mirrors the activity of the analogous ANN given by µ ( l ) =
W ( l − 1) f
( z ( l − 1) ) , where W ( l − 1) are the weights connecting layers l − 1 and l .
The objective function is then given by the negative log-likelihood of the joint distribution function:
[Equation]
where we have switched to the data matrix notation for brevity and assumed the variances σ ( l )2 are fixed hyperparameters.
In the following, we refer to L , Eq.
(2), without the constant term.
Learning.
Learning in the Whittington and Bogacz framework takes place in two steps [4].
First, the values of the random variables are determined by finding the most probable configuration of the joint distribution function when both the input and output layers are conditioned on the given input and output ( z (0) =
x and z ( n )
[Equation]
y ):
[Equation]
The solution to this minimization problem can be found via gradient descent, which we evaluate component-wise for clarity as:
[Equation]
with
[Equation]
σ
[Equation] [Equation]
where η is the gradient descent step size.
The second step is then to minimize the objective with respect to the weights while keeping the previously obtained neuron values fixed.
This corresponds to optimizing the value of the loss at the MAP estimate and can also be carried out by gradient descent, leading to:
[Equation]
where the star superscript denotes equilibrium values of the latent variable dynamics obtained in the first step (Eq. (3))
This algorithm can be implemented by a biologically plausible network as described in [4]; see Figure 1a.
However, as discussed in Section 1, its mapping onto the cortex has proved controversial.
For further details regarding these steps see [4].
A constrained predictive coding framework
In this section, we introduce and discuss our novel covariance-constrained predictive coding (CCPC) model that builds upon the supervised PC paradigm of [4].
Our model also straightforwardly extends to the unsupervised PC framework of [1,2].
For simplicity, we work in the linear regime ( f ( x ) = x ), which allows us to prove different properties of our framework.
3.1 Derivation of an upper bound objective
Reduction to a sum of objectives.
We start by reducing the optimization problem, Eq.
(2), into a set of overlapping sub-problems, which will allow us to break the symmetry between feedforward and feedback weights.
To do so, we first introduce a copy of the terms containing the weights W (1) to W ( n − 2) , denoted by W ( l ) a and W ( l ) b respectively, as min Z , W L =
min Z , W
[Equation]
For consistency, we rename W (0) , W ( n − 1) to W (0) b , W ( n − 1) a , respectively.
Introducing these copies does not change the optimization 2 but will help us avoid weight sharing in the steps below.
We now pair the terms two by two as
[Equation]
− W ( l ) a Z ( l ) (cid:13)(cid:13) 2 F (cid:105) , (7) with g ( n − 1) a = 1 /σ ( n )2 , g (1) b = 1 /σ (1)2 , and g ( l − 1) a = g ( l ) b = 1 / (2 σ ( l )2 ) for l = 2 , . . .
, n − 1 .
Weight sharing occurs here from terms like z ( l +1)
⊤ Wz ( l ) , obtained from expanding the squared norms in Eq.
Indeed, the gradient descent dynamics with respect to z ( l +1) (resp.
z ( l ) ) leads to terms of the form
Wz ( l ) in ˙ z
( l +1)
(resp.
W ⊤ z ( l +1) in ˙
z ( l ) ), which use the same weights W .
Thanks to the doubling of the weights, we can avoid this problem by optimizing each term in the sum in
Eq.
(7) separately.
In other words,
[Equation]
where ˆ L ( l ) ≡
[Equation]
This inequality holds simply because we are no longer finding the minimum of the full objective ˆ L .
We are instead finding the minimum of each component separately and then evaluating (cid:80) l ˆ L ( l ) =
ˆ L .
This splits the ( n + 1) -layer optimization problem into a set of 3 -layer optimizations, in each of which only the middle layer is being optimized.
Note, however, that these are overlapping, so the different optimization problems need to be solved self-consistently.
We make this precise in the supplementary materials section and show that it provides an upper bound for our objective L (SM
Sec.
A).
Separating the objective function in this manner eliminates the weight sharing problem for
W b , but the problem remains for W a .
This can be seen by noting that the gradient descent equations only depend on W b but depend on both W a and W ⊤ a .
We address this problem next.
Whitening constraint.
The idea of decorrelating internal representation has been widely used for unsupervised tasks, often motivated by neuroscience [36–38].
In the case of deep learning, the main motivations were improved convergence speed and generalization [39–41].
Decorrelation has also been used to circumvent the weight transport problem [55].
Inspired by these observations, we introduce the inequality constraint 1 T Z ( l ) Z ( l ) ⊤ ⪯
I , imposing an upper bound on the eigenvalues of the covariance matrix.
In Sec. 5 and SM Sec.
B we show that under some assumptions this inequality constraint leads to whitening the latent variables.
Adding this inequality constraint to the objective we have:
[Equation]
The first inequality results from the fact that the parameters of the new optimization problem are a subset of the parameters of the original optimization problem.
In the second inequality we have expanded the second term of ˆ L ( l ) and used the constraint 1 T Z ( l ) Z ( l ) ⊤ ⪯ I to replace Z ( l ) Z ( l ) ⊤ with T , making the term larger, resulting again in an upper bound.
This replacement eliminates the weight sharing problem in W a , as the gradient descent equations now only depend on W ⊤ a and not on W a (cf.
Eqs.
(10) and (11)).
The inequality can be implemented by using a positive-definite Lagrange multiplier Q ( l ) ⊤ Q ( l ) :
[Equation]
where we have added an additional quadratic term in Z as a regularizer.
This can also be interpreted as an extra term in the prior of the latent space.
Equation (9) is the final form of the objective, which we then optimize using gradient descent/ascent.
3.2 Neural dynamics and learning rules.
Similar to the PCN of [4], the dynamics of our network during learning proceeds in two steps.
First the neural dynamics is derived by taking gradient steps of ˆ L ( l ) from Eq.
(9) with respect to z ( l ) :
[Equation]
where we have defined the variables n ( l )
= (1 /g ( l ) a ) Q ( l )
z ( l ) for each layer of the network.
The weight updates are derived via stochastic gradient descent of the loss given in Eq.
(9) after the neural dynamics have reached equilibrium.
These are given by
[Equation]
We used the neural dynamics equilibrium equation for z ( l ) to simplify the weight update for W ( l ) b .
This yields our online algorithm (Alg. 1 in SM Sec.
E), with the architecture shown in Fig.
1b.
The algorithm can be implemented in a biologically plausible neural network as in Fig.
2; see Sec. 4.
Biological implementation and comparison with experimental observations
In this section, we introduce a biologically plausible neural circuit that implements the CCPC algorithm, denoted by BioCCPC.
We also demonstrate that the details of this circuit resemble the neurophysiological properties of pyramidal cells in the neocortex and the hippocampus.
Neural architecture.
The algorithm for BioCCPC (Alg. 1) summarized by the neural dynamics from Eq.
(10) and weight update rules from Eqs.
(11) can be implemented by a neural circuit with schematic shown in Fig.
The activity of the n − 1 hidden layers, { z ( l ) } n − 1 l =1 , is encoded as the outputs of n − 1 sets of neurons, representing pyramidal neurons of different cortical regions.
The matrices W ( l ) a and W ( l ) b are encoded as the weights of synapses between the pyramidal neurons of adjacent layers.
Explicitly, the matrix W ( l ) a (resp.
W ( l − 1) b ) is the efficacy of the synapse connecting z ( l +1) (resp.
z ( l − 1) ) to the pyramidal neurons z ( l ) .
Because of the disjoint nature of these two inputs, we model these as synapsing respectively onto the distal (apical tuft) and proximal (mostly basal) dendrites of the pyramidal neurons, respectively; see Fig.
This is reminiscent of cortical pyramidal neurons, which also have two integration sites, the proximal compartment comprised of the basal and proximal apical dendrites providing inputs to the soma, and the distal compartment comprised of the apical dendritic tuft [22,65].
These two compartments receive excitatory inputs from two separate sources [18,66].
Similarly, the auxiliary variables n ( l ) are represented by the activity of interneurons in each cortical region.
3 The Q ( l ) synaptic weights are encoded in the weights of synapses connecting n ( l ) to z ( l ) , while Q ( l ) ⊤ models the weights of synapses from z ( l ) to n ( l ) .
In a biological setting, the implied equality of weights of synapses from z ( l ) to n ( l ) and the transpose of those from n ( l ) to z ( l ) can be guaranteed by applying the same Hebbian learning rule [55].
However, unlike previous work, we do not require interneurons of one layer to be connected to the pyramidal neurons of another layer.
Neural dynamics and canonical components.
The neural dynamics of the pyramidal neurons of our circuit given by Eq.
(10) can be recast as the dynamics of a three compartment neuron:
[Equation]
where we have defined the apical and basal compartmental membrane potentials
[Equation]
[Figure]
Here we defined the leak conductance g ( l ) lk = c ( l ) − g ( l ) a .
Interestingly, this is the same neural dynamics of a three-compartment neuron posited previously in [28, 30], which we derived in a normative way.
Here, g a (resp.
g b ) are conductances between the apical (resp. basal) compartments and the soma, and g lk is the somatic leak.
Our derivation clarifies the relationship between the physiological quantities and the objective function.
Indeed, the basal and apical conductances play the role of the inverse variances in the predictive coding objective.
These inverse variances, generally referred to as ‘precisions’ in the PC literature, have previously been argued to be encoded in lateral inhibitions [2] or top-down attention [69,70].
In our model, they are simply encoded in the compartmental conductances.
In Sec.
5 we show that the leak term also has a functional role and determines a threshold for dynamically learning more diverse variables [71,72].
Synaptic weight updates.
Matching observations in the cortex, in our circuit [18,22,65,73,74], the basal and apical weights ( W b and W a ) of our algorithm are updated differently.
The basal synaptic weights of the pyramidal neurons, given by the elements of W ( l ) b , are updated by the product of two factors represented in the corresponding post- and pre-synaptic neurons (Eq. 11a):
[Equation]
The first term in parentheses, g ( l ) a v ( l ) a , is proportional to the total apical current, given by the difference between the excitatory synaptic current in the apical tuft, W ( l )
⊤ a z ( l +1) , and the inhibitory current induced by interneurons synapsing onto the distal compartment, Q ( l ) ⊤ n ( l ) .
Biologically, this factor can be approximated by the calcium plateau potential traveling down the apical shaft.
The calcium plateau potential has been experimentally seen to drive plasticity of the basal synapses matching the derived update rule of our circuit [18,20,23–25].
The second factor is simply the neural output.
Because the total update is not purely dependent on the action potentials of the pre- and postsynaptic neurons, such plasticity is called non-Hebbian [25].
The synaptic learning rule for the apical weights W ( l ) a (Eq. 11b) and the synapses connecting the interneurons and the pyramidal neurons
Q ( l ) (Eq. 11c) are simply Hebbian, also matching experimental observations in the cortex [19,75].
Features and limitations.
Here we summarize the main differences between the biological implementations of CCPC and PC.
In PC, the forward and backward weights (in our notation W b and
W ⊤ a ) are symmetric, whereas, in CCPC, they are not constrained to be symmetric and are generically not so (see Sec. 6).
Another difference is that in PC, the connection between value neurons and error neurons is one-to-one and fixed; this constraint does not exist between the pyramidal and interneurons of our model, which are no longer one-to-one, and have plastic connectivity.
While we have shown the improvement in biological realism of our model over the traditional PC network, this was done in the linear case.
However, recent work on deep linear networks [76] has provided many insights into the learning dynamics of deep networks.
Some of the properties discovered in the deep linear network, like “balancedness" of weights [77,78], generalize for certain nonlinear networks [78].
We believe several of our observations will generalize in a similar fashion.
There are other aspects that can also be improved.
For example, the connectivity between pyramidal neurons and interneurons of CCPC is required to be symmetric.
This symmetry can be achieved via Hebbian learning [55]; however, it would be interesting to explore if such symmetry is indeed required.
In CCPC, symmetric connectivity is only required between neurons of the same cortical region, which is a much less stringent biological requirement than that of symmetric weights between different cortical regions, as in PC.
Another shortcoming of our model is that while the teaching signal for the basal synapses in CCPC is signed and graded, in the cortex these signals are generally believed to be stereotypical [18].
Theoretical arguments and interpretation
In this section, we summarize some of the salient theoretical features of our framework and describe their biological significance.
Further details and proofs are given in SM Sec.
B.
5.1 Implicit error computation
In backpropagation, as well as in PC, the learning algorithm computes a loss (or a local error) which is then used to compute updates to the weights.
In CCPC, no error or loss is explicitly computed.
So how does CCPC learn?
In the following proposition, we show that in the g a → 0 limit, the algorithm implicitly computes an error in its weight updates.
This quasi feed-forward or weak nudging limit has been used to explore the learning dynamics of biologically plausible networks [30] and is related to the ‘fixed prediction’ assumption [4,61,62].
Proposition 1.
Assume that the learning rules (Eqs. 11) are at equilibrium
and we receive a new datapoint given by x T +1 , y T
+1 .
For c ( l ) = 0 and in the limit of ϵ ≡ g a /g b → 0 , the leading term in ϵ for the forward weight updates δ W ( l − 1) b for this new sample is given by
[Equation]
where ˜ y T +1 =
YZ ( l − 1) ⊤ (cid:0) Z ( l − 1)
Z
( l − 1) ⊤ (cid:1) − 1 z ( l − 1) T +1 is the optimal linear inferred value for y T +1 .
Biological interpretation Recall that the learning rule for the basal synapses is proportional to v ( l ) a (Eq.
We argued in the previous section that this quantity can be implemented as the calcium plateau potential.
In this light, Prop. 1 shows that the calcium plateau potential is implicitly encoding the prediction error.
This is remarkable as in our algorithm, neither the prediction nor the prediction error is explicitly computed.
However, via the combined learning dynamics of the synapses and the neural dynamics of the interneurons, the final apical signal implicitly encodes the prediction error.
This is in spirit similar to difference target propagation (DTP) [47], however, in DTP, the difference between the forward pass and the target is explicitly computed and backpropagated.
5.2 Adaptive latent dimension
For optimization within a single hidden layer, we show that the system performs adaptive latent dimension discovery by thresholding eigenvalues of a Gramian including contributions from both adjacent layers.
If we take the loss function in (9), setting g a = 1 , g b = ϵ , and optimizing over
W a , W b , we get the objective
Tr[( c + ϵ ) Z ⊤ Z − Z ⊤ Z (cid:2) ϵ X ⊤ ( XX ⊤ ) − 1 X + Y ⊤ Y (cid:3) for Z .
The next theorem indicates the solution:
Theorem 1.
Let the concatenated matrix Ξ =
[ ϵ 1 / 2 C − 1 /
[Equation]
x
X , Y
[Equation]
R
[Equation]
T have SVD
[Equation]
=
[Equation]
u α λ α v ⊤ α with { u α ∈ R ( d X + d Y ) } and { v α ∈ R T } both being sets of orthonormal vectors, with the convention that ( λ α ) are sorted in decreasing order.
We consider the minimization
[Equation]
Then one of the optimal solutions is given by ˆ Z = (cid:80)
Dα =
1 w α 1 ( λ α > c + ϵ ) v
⊤ α where { w α ∈ R d } is an arbitrary set of orthonormal vectors, with D = min( d, d X + d Y ) .
Thus, the latent modes are found by thresholding the eignevalues [72] of the sum of Gramians [71] in Eq.
From this theorem, we derive the computational limits of ϵ → ∞ and ϵ → 0 in the supplementary materials.
We also show that the neurons of the narrowest layer get whitened and the other layers become low-rank.
For further details of these statements and all proofs see SM Sec.
B.
Biological interpretation In our model, the number of neurons in the intermediate layers is fixed.
However, the dimensionality of the representation encoded in these intermediate layers is determined dynamically according to the spectrum of the eigenvalue problem specified in Thm.
In particular, it is the quantity c ( l ) , which encodes the somatic compartment leak conductance, that biologically sets an adaptive threshold for the dimensionality of the latent representation.
In other words, lower (resp.
higher) somatic leak corresponds to higher (resp.
lower) sensitivity to the small eigenvalues of the above eigenvalue problem.
For details of the interpretation of the eigenvalue problem in different limits ( ϵ → ∞ and ϵ → 0 ), see SM Sec.
B.
Numerical experiments
[Figure]
We tested our BioCCPC algorithm on the MNIST [79] as well as other datasets.
4 We compared our results with a standard predictive coding network [4] with linear activation (PC) and with a version that includes a covariance constraint (PC+cons; Sec. C).
We used unit variances, σ ( l )2 = 1 , in the PC networks, and matching conductances in our algorithm, g (1) b
[Equation]
for
[Equation]
Our method finds solutions that are almost optimal in terms of predictive-coding loss on a held-out validation set (see Fig. 3a) and are generally better than the solutions obtained from simply adding the covariance constraint to a predictive-coding network.
Learning is slower in our network than in unconstrained PCN, as it takes time for the covariance constraint to be enforced.
It is possible that different initialization schemes could improve the learning speed, but we leave this for future work.
Note that since we focus on linear networks, metrics based on prediction accuracy are not very informative for the datasets we consider, which are not linearly separable.
The covariance constraint is saturated in the narrowest layer—in our example, the second hidden layer, of size 5—see Fig. 3c.
The activation in the other hidden layer is rank-restricted by the narrowest layer, as described above, and so it is only whitened within a 5D subspace, Fig.
3b.
In SM Sec.
C, we provide further numerical experiments, exploring the effects of thresholding, the size of the network, and performance on other datasets.
In all tested scenarios, BioCCPC achieves the same validation loss as PC (Figs. 5 and 6 of SM Sec. C).
We also show that the forward and backward weights, W b and
W ⊤ a , are not symmetric, in accordance with biological observations, and explore their relationship.
In SM Sec.
D we compare PC and BioCCPC with regards to generalization.
We find that there is very little generalization gap for either case (Fig. 9).
This is due to the linearity of our model, which acts as a strong regularizer.
Because of this, the potential generalization benefits of whitening the latent variables are not observed.
Therefore, the primary benefit of the covariance constraint in this work is the elimination of weight sharing in W a (cf.
Sec. 3.1).
Conclusion
We have derived a biologically plausible algorithm for covariance-constrained predictive coding and have shown that it avoids the criticisms of PC and has many features in common with recent experimental observations.
Our algorithm does not have symmetric forward and backward connectivity matrices, and does not need one-to-one connectivity between any neurons.
Furthermore, we showed that the learning rules of our algorithm closely match experimental observations, and we connected our parameters to neurophysiological quantities.
Using the simplicity afforded to us because of the linearity, we showed a number of interesting properties of our algorithm, including the effect of the somatic leak term, which acts as an adaptive thresholding mechanism.
We also showed how our algorithm implicitly computes a difference term that it uses for learning.
We argued that this term could be encoded in the calcium plateau potential, which is known to affect the learning of basal synapses.
We hope that this concrete connection between our normative model and observation can further our understanding of cortical computation by guiding future experiments.
Acknowledgments and Disclosure of Funding
We would like to thank Colin Bredenberg for guidance and helpful comments throughout the project.
SG, TT, YB, and DC were internally funded by the Simons Foundation.
AS was partly funded by
Simons Foundation Neuroscience grant SF 626323 during this work.
Broader impact
We do not foresee any potential societal harm caused by our work.
We hope our contribution moves forward the discussion on the relationship between multi-layer networks and cortical structure.
A Derivation of upper bound
In this section we give a more detailed derivation of our framework, using a slightly modified order of steps compared to the main text.
This should help clarify some aspects of the final algorithm.
Similar to the main text
[Equation]
we start with the predictive coding objective and introduce the weight doubles.
We arrive at
[Equation]
where we have also introduced a quadratic cost (a prior on Z ) with coefficient c ( l ) .
We now impose the constraint on the empirical covariance
[Equation]
1 T Z ( l ) Z ( l ) ⊤ ⪯ ρ ( l ) I , of the internal variables Z ( l ) 5 : min Z , W
[Equation]
≤
[Equation]
Expanding this and using the constraint we again upper bound the objective:
[Equation]
We next implement the inequality constraint using positive-definite Lagrange multipliers:
[Equation]
where
[Equation] [Equation]
where we have added a positive constant c ( l ) q defining the coefficient of the inequality constraint.
This constant does not affect the solution of the optimization problem but does affect the details of the neural dynamics.
Motivated by biological arguments 6 , we set this constant equal to g ( l ) a .
In doing so, the interneuron variables no longer need a (1 /g ( l ) a ) in their definition, as in the main text, and are instead given by n ( l )
=
Q ( l ) z ( l ) .
A final upper bound to (19) can be found if each ˜ L ( l ) is optimized individually.
Explicitly, we perform gradient descent of each variable, Z ( l ) , W ( l ) a , W ( l ) b , only with respect to ˜ L ( l ) and not with respect to the entire objective L :
[Equation]
This affords us an upper bound because we are no longer finding the actual minimum of L .
Symbolically this can be written as in (9):
[Equation]
where the precise definition of this equation is given by the update equations (21).
Note that these updates are no longer curl-free and hence cannot be found as the gradient of a single objective function.
B Theoretical section proofs
Note that the definition of ϵ in Proposition 1 is different from the definition in other sections.
B.1 Proof of Proposition
To prove Proposition 1, we first show that the latent variables z ( l ) have a specific form.
As in the main text, we define ϵ ( l ) =
g ( l ) a /g ( l ) b and, for simplicity, we assume ϵ ( l ) =
ϵ is the same for all layers.
For simplicity also, we assume c ( l ) (equivalently d ( l ) in previous section) is equal to zero, and use c ( l ) q = g ( l ) a as in the previous section.
This choice makes the notation simpler without changing our results.
Equation (10) becomes
[Equation]
Proposition 2.
Assume the neural dynamics given in Eq.
(23) are at equilibrium.
Then the latent variables take the form
[Equation]
where B k and A k are independent of y and defined recursively via
[Equation]
with F ( l ) =
(cid:0) I + ϵ Q ( l ) ⊤ Q ( l ) (cid:1) − 1 and
[Equation]
Proof.
We prove this via induction.
Starting from the neural dynamics Eq.
(23), at equilibrium ( ˙ z ( l ) = 0 )
we have:
[Equation]
Setting l = n − 1 we get:
[Equation]
which satisfies the proposed form in Eq
(24) for k = 1 .
Now, we assume that the form in Eq.
(24) holds for z ( n − k ) , and show that it also holds for z ( n − k − 1) .
We have
[Equation]
Collecting z ( n − k − 1) and multiplying by the inverse on the left we arrive at:
[Equation]
as proposed.
Note that, whereas Proposition 1 holds only in the ϵ → 0 limit, Proposition 2 holds for general ϵ .
Using this result we show that the relationship between the activity, feedback and feedforward inputs of the l th module has a simple form given in the following corollary:
Corollary 1.
The leading order in ϵ of the dependence of z ( l ) and z ( l +1) on z ( l − 1) and y , which has non-zero dependence on y , is given by
[Equation]
where C ( l ) 1 and C ( l ) 2 are respectively the expansions of B n − l W ( l − 1)
b
[Equation]
n
[Equation]
up to order ϵ n − l − 1 .
Proof.
The result follows directly from applying Eq.
(24) twice and then dropping all but the leading dependence on ϵ in the coefficient of the y term.
Now, we restate and prove Proposition 1, which can be viewed as a generalization of the results in [55].
Note that in this proposition, the exact form of C ( l ) 1 and C ( l ) 2 is not important.
All that matters is that both z ( l ) and z ( l +1) have a linear dependence on z ( l − 1) .
Since the dependence of z ( l ) on y is subleading compared to the dependence of z ( l +1) on y , this can be dropped in the limit of ϵ → 0 .
Proposition 1.
Assume that the learning rules (Eqs. 11) are at equilibrium
and we receive a new datapoint given by x T +1 , y T
+1 .
For c ( l ) = 0 and in the limit of ϵ ≡ g a /g b → 0 , the leading term in ϵ for the forward weight updates δ W ( l − 1) b for this new sample is given by
[Equation]
where ˜
[Equation]
1 z ( l − 1) T +1 is the optimal linear inferred value for y T .
Proof.
The statement follows from two applications of the update rule for W b given in Eq.
(11a) for
W ( l − 1) b .
Putting c ( l ) =
we have:
[Equation]
The equilibrium condition given by δ W b = 0 is given by averaging over the entire dataset:
[Equation]
Now we plug the derived values in Corollary 1.
After right-multiplying by ( Z ( l − 1)
Z ( l − 1) ⊤ ) − 1 we get:
[Equation]
Now, we look at the update for a new sample T + 1 .
Plugging the above as well as the equations from Corollary 1 into the update rule we see that the C ( l ) 2 dependent term cancels and we are only left with the term depending on y :
[Equation]
with ˜ y T +1 as defined in the proposition.
B.2 Proof of Theorem 1 and consequences
In this section we look at some properties of the within-layer computations of CCPC.
We will focus on the computations within the l th layer and for clarity of notation, for this subsection only, we rename
Z ( l ) → Z , Z ( l +1)
→ Y , and Z ( l − 1)
→ X .
We will assume ρ ( l ) = 1 for brevity.
The results have straight-forward generalizations for the case where ρ ( l ) ̸ = 1 .
Theorem 1.
Let the concatenated matrix Ξ = UΛV ⊤ =
[ ϵ 1 / 2 C − 1 / 2 x
X , Y ] ∈ R ( d X + d Y ) × T have singular value decomposition
Ξ = (cid:80) ( d X + d Y ) α =
u
α λ α v ⊤ α , with { u α ∈ R ( d X + d Y ) } and { v α ∈ R T } both being sets of orthonormal vectors, and singular values ( λ α ) sorted in decreasing order.
We consider the minimization
[Equation]
Then one of the optimal solutions is given by ˆ Z = (cid:80)
Dα =
1 w α 1 ( λ α > c + ϵ ) v
⊤ α where { w α ∈ R d } is an arbitrary set of orthonormal vectors, with D = min( d, d X + d Y ) .
Proof.
Let the SVD of Z be WΛ Z V ⊤ Z .
Minimizing over Z amounts to finding the matrices W , Λ Z , and V Z that give the smallest value of the objective.
Because of the constraint ZZ ⊤ ≼ I d , Λ Z satisfies 0 ≤ λ Z α ≤ 1 .
Plugging this form into the optimization objective we have
[Equation]
where ˜ V = V ⊤ V Z .
Note that W does not appear in this expression and therefore remains undetermined in this optimization objective.
V only appears in the second term and hence can be found by optimizing the second term alone.
This term is of the form − (cid:80) α,β λ 2 Z α | ˜
V αβ | 2 λ 2 β .
The ˜ V that minimizes the term is the one that pairs λ β and λ Z α , in the sorted order.
Assuming ( λ Z α ) are also sorted in the decreasing order, this is achieved by setting V Z = V .
Plugging this in, the entire expression simplifies to
[Equation]
Since 0 ≤ λ Z α ≤ 1 , it is clear that, for a minimum, if ( λ α − c − ϵ ) < 0 , λ Z α = 0 , while ( λ α − c − ϵ ) > 0 implies λ Z α = 1 .
In case of equality, the choice of λ Z α does not matter.
Hence λ Z α = 1 ( λ α > c + ϵ ) is a valid minimum.
The vectors { w α } are the columns of W , while { v α } are the columns of V .
Hence, the result.
From this theorem, we derive two corollaries, describing the different limits of ϵ → 0 and ϵ → ∞ .
The proofs of these corollaries follow directly from Theorem 1 which maps the within-layer computation of CCPC onto a PCA problem, and the known properties of PCA.
Corollary 2.
If the input sample covariance matrix, C x , is full rank, the internal representation Z in the limit ϵ → 0 is driven by the principal components (scores) of Y , i.e.,
[Equation]
(29) with W a the top principal directions of Y and W b the maximally correlated directions between X and the columns of W a .
Corollary 3.
If the input sample covariance matrix, C x , is full rank, the internal representation Z in the limit ϵ → ∞ is driven by the maximally correlated directions between C − 1 / 2 x
X and Y , i.e.,
[Equation]
30) with W a , W b projecting onto a common lower-dimensional subspace so that the projections are maximally correlated.
B.3 Rank structure of CCPC
In this section again we set c ( l ) = 0 and ρ ( l ) = 1 , and assume g a = 1 , g b =
ϵ is the same for all layers.
The generalization to other cases is straightforward.
We demonstrate a series of rank constraints on the intermediate layers of CCPC.
The first of these is very general and requires no assumptions.
The following statements become gradually more tight but also require more assumptions.
In this section, we define r x and r y to be the ranks of the the covariances of X and Y , and r ( l ) to be the rank of the covariance of Z ( l ) at the equilibrium of the CCPC update equations.
We take r (0) =
r x and r ( n ) =
r y .
Corollary 4.
The rank of the l th layer is bounded by r ( l +1)
+ r ( l − 1) :
[Equation] [Equation]
Proof.
This follows directly from Theorem 1 which describes Z ( l ) as the whitened PCA of the concatenation of Z ( l − 1) and Z ( l +1) .
When the adjacent layers are whitened, i.e., the non-zero eigenvalues of Z ( l +1)
Z ( l +1)
⊤ and
Z ( l − 1) Z
( l − 1)
⊤ are all one, we can prove a tighter form of the rank constraint.
This is relevant for the intermediate layers of CCPC other than the final hidden layer.
This proposition is relevant for the final hidden layer only if the supervisory input Y is white.
Proposition 3.
Let r ( l ) c be the rank of the covariance matrix Z ( l − 1)
Z ( l +1)
⊤ .
The rank of intermediate layers l ≤ n − 2 is given by:
[Equation]
Proof.
This is a slight generalization of [71].
It follows from the eigenvalue structure of the concatenated data matrix given in Theorem 1, where only a subset of the eigenvalues exceed the threshold given by ϵ .
The number of eigenvalues that exceed this threshold is given by Eq.
Applying this proposition to all the layers of a CCPC network, we arrive at the following corollary:
Corollary 5.
Assume that Y is white and ϵ ≥ 1 .
In a CCPC network with c ( l ) ≥ 0 , ρ ( l ) = 1 , and layer widths w ( l ) , the rank of the latent variables z ( l ) is less than or equal to min { r x , r y , w (1) , · · · , , w ( n − 1) } .
In other words, the rank of the latent variables is bounded by the minimum of the input/output ranks and the bottleneck of the network.
B.4 Alignment and relation to CCA
In the limit where the internal loss terms in Eq.
(6) are zero, we effectively have a deep linear network [76].
A lot is known about deep linear networks, trained by gradient descent of weights.
One of the remarkable properties of such networks is that the weights remain balanced, namely
W ( l ) ⊤ W ( l ) − W ( l − 1) W ( l − 1) ⊤ remains a constant, under gradient descent.
In the case this difference is zero, the left singular vectors of W ( l − 1) match the right singular vectors of W ( l ) [80].
In addition, for two class linear discriminants, W ( l − 1) ultimately become rank 1, under gradient descent [80].
[Figure]
C Numerical simulations C.1 Effect of constraint scale
The BioCCPC network uses an inequality constraint on the covariance of the hidden variables,
T Z ( l ) Z ( l ) ⊤ ⪯ ρ ( l ) I , and this constraint is saturated at convergence.
The scale, ρ ( l ) , is therefore important: a scale that is too small or too big forces the hidden-layer activities into a range that can hinder learning, as seen in Fig. 4a.
Interestingly, the BioCCPC algorithm with a well-chosen constraint scale converges faster than a plain PC network, that is, it can reach a lower PC loss given the same number of training samples (Fig. 4a).
The predictive-coding loss, Eq.
(6), penalizes prediction errors in all layers.
If we think about the task as supervised learning, however, it might make more sense to focus only on the accuracy of the prediction in the last layer.
The best output prediction error may require a different constraint scale compared to the lowest PC loss, cf.
Fig.
4b and Fig.
4d.
C.2 Effect of dataset choice
While we derived our algorithm by starting from the supervised PC framework of [4], the same derivation would hold for the unsupervised or self-supervised PC networks.
In these settings,
PC can be thought of as a framework which combines information from the inputs at the first and last layer, rather than a method for predicting an output from an input.
In the linear case, this has much in common with biologically plausible circuits that perform canonical correlation
[Figure]
analysis (CCA) [55,71].
We therefore tested the algorithm on Mediamill [81], a dataset consisting of two views of the same scene, one given by the video and the other by text annotation.
The results (Figs.
4b-d and Fig.
5) show similar trends as those obtained on MNIST.
In particular, an appropriate choice of constraint scale leads to results that are competitive with less biologically plausible implementations of PC.
In Fig.
6 we show the evolution of the validation-set loss for four other datasets: FashionMNIST,
CIFAR10, CIFAR100, and the LFW (Labeled Faces in the Wild).
The first three are supervisedlearning scenarios and we use a network similar to the one we employed for MNIST.
The LFW dataset contains pairs of views of the same subject, and so we employ a symmetric autoencoder-like architecture for this self-supervised task.
As before, we see that the BioCCPC algorithm converges to a solution that has predictive-coding loss comparable to the minimum obtained from traditional PC implementations (see Fig. 6).
[Figure]
C.3 Network architecture
We tested BioCCPC across different architectures, such as a shallow network with a small hidden-layer dimension (Fig. 4) and a deeper network with two hidden layers, one of which is large-dimensional (Fig. 4a,b,d).
In both cases BioCCPC compares well to PC.
C.4 Lack of symmetry in feedforward and feedback weights
One of the difficulties with a biological implementation of predictive coding is that it requires symmetric weights across layers.
This is not so much an issue of plausibility—indeed, symmetric weights can be obtained simply by using the same Hebbian rules to train both feedforward and feedback connections—but rather a problem with realism: weights in many areas of the brain are not in fact seen to be symmetric.
[Figure]
[Figure]
Our algorithm explicitly breaks the symmetry between the weights, leading to solutions that generically alleviate this difficulty.
Indeed, in some cases there is very little correlation between feedforward and feedback weights; see Fig.
However, in some cases we see that the forward and backward weights converge to similar values even though the learning rules are not symmetric.
We suspect that this can occur in cases where there is a large imbalance between the width of adjacent layers, the larger the imbalance (i.e. the lower rank the wider layer is) the more correlation we see in our experiments; see Fig. 8.
In biologically relevant neural networks, we do not generally see this large imbalance in the width of adjacent layers and therefore we do not expect to see high correlation between the forward and backward weights.
C.5 Predictive coding with constraint
In the main text we considered a predictive-coding network that imposes a covariance constrained but we did not make explicit what this entails.
Explicitly, we augmented the Whittington-Bogacz objective, Eq.
(2), with a Lagrange multiplier enforcing an inequality constraint on the covariance of hidden variables, similar to what we do in BioCCPC:
[Equation]
and optimized using stochastic gradient descent.
This lets us distinguish the effect of the constraint from the effect of the approximations we used in the derivation.
C.6 Details of simulation runs
We initialized our the weights in our networks using uniform random numbers in the range [ − a, a ] , where a = 1 / √ number of columns.
This is similar to Xavier initialization [82] except the scale of initial weights depends only on the number of columns instead of both rows and columns.
We have found this initialization to provide slightly better results.
We optimized hyperparameters using Optuna [83] with the tree-structured Parzen Estimator (TPE) algorithm
We ran the optimization for 100 trials and repeated it 10 times with different random seeds for every combination of dataset, network architecture, and constraint scale ρ (with the exception of the plain PC algorithm which does not have a constraint).
In each case we optimized the learning rate η used in the weight updates and the learning rate τ − 1 of the fast dynamics (see
Algorithm 1).
For networks that have a constraint, we allowed a different learning rate fη for the Q dynamics, and optimized for the factor f .
Finally, for BioCCPC network, we allowed for different learning rates for the forward and backward weights, W b and W a , and optimized for the ratio between these learning rates.
Hyperparameter optimization runs were trained for 500 batches with batch size 100 and were evaluated on a held-out validation set comprising 500 samples.
Each run was repeated 4 times with different random initialization to account for stochasticity in the learning dynamics.
The simulation runs that were used to make the figures in the paper were based on the hyperparameter optimization results that yielded the lowest predictive-coding loss.
Since hyperparameter optimization tends to push the learning rate up to the brink of instability, the learning rate used in the simulations was lowered by multiplying by a certain “safety” factor α .
We generally used α = 0 . 8 .
We ran our simulations on the CPU.
Each run took from 1 to 4 minutes when run single-threaded on a 24-core Intel Xeon CPU E5-2643 at 3.40GHz running Linux.
We also used a cluster to run the hyperparameter optimization for different architectures in parallel.
D Generalization error
CIFAR
CIFAR
FashionMNIST
LFW
MediaMill MNIST
[Figure]
The linear constraint that we impose on our network is a strong regularizer.
Figure 9 compares the training and validation losses during training on six different datasets (as described above).
We see that there is very little overfitting in all cases.
E BioCCPC Pseudocode
input: x t , y t
▷ new sample and previous weight matrices
[Equation]
z
[Equation]
▷ initialize latents via forward pass run until convergence for all l ∈ { 1 ,n − 1 } ▷ neural dynamics n
[Equation]
l
[Equation]
l
[Equation]
v
[Equation]
z
[Equation]
l
[Equation]
z
[Equation]
update for all l ∈ { 1 ,n − 1 } ▷ synaptic weight updates
[Equation]
W
[Equation]
Q
[Equation] [Equation]
output: z ( l ) t ∀ l
[Equation] [Equation]
,n } ▷ internal representations and updated weights
W ( l ) a , W ( l − 1) b , Q ( l ) ∀ l ∈ { 1 ,...,n − 1 }